Dell R640, R740 User manual

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

February 2021

H18217.5

Design Guide

Abstract

This design guide describes how to design and specify a Dell Technologies server and switch infrastructure for validated hardware configurations, facilitating deployment of Red Hat OpenShift Container Platform 4.6 following a Dell Technologies infrastructure deployment.

Dell Technologies Solutions

Copyright

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2021 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA 02/21 Design Guide H18217.5.

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

2 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

		Contents
	Contents
Chapter 1	Introduction	5
Solution overview and key benefits ........................................................................		6
Document purpose .................................................................................................		7
Audience.................................................................................................................		7
We value your feedback .........................................................................................		7
Chapter 2 Technology and Deployment Process Overview		9
Introduction...........................................................................................................		10
OpenShift Container Platform...............................................................................		10
Cloud-native infrastructure ...................................................................................		13
Deployment process.............................................................................................		16
Infrastructure requirements ..................................................................................		19
Chapter 3 Networking Infrastructure and Configuration		21
Introduction...........................................................................................................		22
OpenShift network operations ..............................................................................		22
Physical network design .......................................................................................		25
Chapter 4	Storage Overview	30
OpenShift Container Platform storage..................................................................		31
Container Storage Interface (CSI) external storage .............................................		34
Chapter 5 Cluster Hardware Design		39
Introduction...........................................................................................................		40
Cluster scaling ......................................................................................................		40
Requirements planning.........................................................................................		40
Cluster hardware planning....................................................................................		42
Validated hardware configuration options ............................................................		44
Chapter 6	Use Cases	48
Introduction...........................................................................................................		49
Enterprise applications .........................................................................................		49
Telecommunications industry ...............................................................................		52
Data analytics and artificial intelligence ................................................................		54
Chapter 7	References	57
Dell Technologies documentation ........................................................................		58
Red Hat documentation........................................................................................		58
	Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6		3

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Contents

Other resources....................................................................................................	58
Appendix A Dell EMC PowerEdge BOMs	59
Dell EMC PowerEdge R640 node BOM ...............................................................	60
Dell EMC PowerEdge R740xd node BOM ...........................................................	62
Dell EMC Unity 380F BOM...................................................................................	64
Dell EMC PowerMax BOM ...................................................................................	64

4 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 1: Introduction

Chapter 1 Introduction

This chapter presents the following topics:
Solution overview and key benefits....................................................................	6
Document purpose...............................................................................................	7
Audience ...............................................................................................................	7
We value your feedback.......................................................................................	7

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

5

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 1: Introduction

Solution overview and key benefits

Ready Stack solution for OpenShift Container Platform 4.6

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 is a flexible infrastructure that has been designed, optimized, and validated for an OpenShift Container Platform 4.6 on-premises bare-metal deployment. The deployment that this guide describes does not require a hypervisor.

The Dell EMC Ready Stack solution consists of the following documents:

•Dell EMC Ready Stack design guide (this document)

•Dell EMC Ready Stack deployment guide

(Both documents are available at the Dell Technologies Info Hub for Containers.)

This Ready Stack solution provides:

•A detailed overview of validated OpenShift Container Platform hardware designs

•A scalable hardware platform of up to 210 compute nodes spread across seven racks

•Rapid implementation and time-to-value

The solution includes the following components:

•Red Hat OpenShift Container Platform 4.6 for application development and deployment

•Dell EMC PowerEdge R640 and R740xd servers for compute and storage

•Dell EMC PowerSwitch S5200 series switches for infrastructure network enablement

•Dell EMC PowerSwitch S3048 switch for out-of-band (OOB) management of the cluster

Note: While you can rely on Red Hat Enterprise Linux security and container technologies to prevent intrusions and protect your data, some security vulnerabilities might persist. For information about security vulnerabilities in OpenShift Container Platform, see OCP Errata. For a general listing of Red Hat vulnerabilities, see the RH Security Home Page.

OpenShift

Container

Platform and

Kubernetes

OpenShift Container Platform 4.6 consists of many open-source components that have been carefully integrated to provide a consistently dependable platform on which you can develop and deploy scalable containerized applications. OpenShift Container Platform provides great flexibility for accommodating platform deployment preferences. For more information, see OpenShift Container Platform 4.6 Documentation.

At the heart of OpenShift Container Platform is Kubernetes container orchestration software. For more information, see What Kubernetes is.

6 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 1: Introduction

Document purpose

Dell EMC Ready Stack for Red Hat OpenShift Container Platform is a proven design to help organizations accelerate their container deployments and cloud-native adoption. This guide provides information for building an on-premises infrastructure solution to host OpenShift Container Platform 4.6. The guide describes the Dell Technologies design decisions and configurations that enable solution architects to:

•Design and deploy a container platform solution.

•Extend or modify the design as necessary to meet customer requirements.

This guide includes:

•Container ecosystem design overview

•Network infrastructure design guidance

•Container and application storage design guidance

•Server requirements to support OpenShift Container Platform node roles

•Hardware platform configuration recommendations

•Rack-level design and power configuration considerations

A companion deployment guide provides information about automation-assisted deployment of the solution. This guide is available at the Dell Technologies Solutions Info Hub for Containers.

For information about the manual installation and deployment of Red Hat software products, see OpenShift Container Platform 4.6 Documentation.

Note: This guide may contain language from third-party content that is not under Dell's control and is not consistent with Dell's current guidelines for Dell's own content. When this content is updated by the relevant third parties, this guide will be revised accordingly.

Audience

This design guide is for system administrators and system architects. Some experience with Docker, Kubernetes, and OpenShift Container Platform technologies is recommended.

We value your feedback

Dell Technologies and the authors of this document welcome your feedback on the solution and the solution documentation. Contact the Dell Technologies Solutions team by email or provide your comments by completing our documentation survey.

Author: Piyush Tandon

Contributors: John Terpstra, Umesh Sunnapu, Scott Powers, Aighne Kearney

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

7

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 1: Introduction

Note: For additional information about this solution, see the Dell Technologies Solutions Info Hub for Containers.

8 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Chapter 2 Technology and Deployment

Process Overview

This chapter presents the following topics:
Introduction.........................................................................................................	10
OpenShift Container Platform ...........................................................................	10
Cloud-native infrastructure ...............................................................................	13
Deployment process ..........................................................................................	16
Infrastructure requirements ..............................................................................	19

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

9

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Introduction

OpenShift Container Platform 4.6 can host the development and runtime execution of containerized applications. The platform is continuing to mature and expand rapidly, providing you with access to the tools your team needs so that your business can grow. OpenShift Container Platform is based on Kubernetes, the de facto container automation and life cycle management platform for containerized workloads and services. Ready Stack for OpenShift Container Platform 4.6 includes Dell EMC hardware (servers, switches, and storage) to enable you to develop, validate, and deploy your containerized applications.

This chapter describes the OpenShift Container Platform architecture, infrastructure components, and requirements for a viable Ready Stack for OpenShift Container Platform 4.6 cluster, which can drive the core of modern telecommunications practices, multimedia operations, service provider infrastructure operations, the demands of the gaming industry, and financial transaction workloads.

OpenShift Container Platform

Overview	OpenShift Container Platform is an enterprise-grade declarative state machine that has
	been designed to automate application workload operations based on the upstream
	Kubernetes project. In a Kubernetes context, “declarative” means that developers can
	specify, in code, a configuration for an application or workload without knowing how that
	application is going to deployed. OpenShift Container Platform uses the enterprise-grade
	Kubernetes distribution, called the OpenShift Kubernetes Engine, to provide production-
	oriented container and workload automation. OpenShift Container Platform 4.6 is based
	on Kubernetes version 1.19, which includes native support for cluster snapshots, enabling
	cluster backup and recovery. On top of the Kubernetes Engine, OpenShift Container
	Platform provides administrators and developers with the tools they require to deploy and
	manage applications and services at scale, as shown in the following figure.
	Note: OpenShift Container Platform is a certified Kubernetes distribution.

10 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

OpenShift Container Platform architecture

What Kubernetes is

Kubernetes provides an abstraction layer for application containers, deployments, and services and automates all container operations. Developers and administrators manipulate Kubernetes object declarations and abstractions to achieve the desired state of operations. Developers and administrators can specify the needs of an application in a declarative manner, and Kubernetes automatically deploys, terminates, or restarts containers to converge on this desired state.

What Kubernetes is not

Why OpenShift?

Kubernetes is not just an “orchestration” platform for containers, which implies imperative, sequential actions. There is no imperative management of containers in Kubernetes. Rather, Kubernetes consists of independent control processes (state transition machines) that move the current state of the cluster towards the desired state. This mechanism has fundamental implications for how cluster operations, application middleware, and more can be managed automatically (see Cluster automation).

Upstream Kubernetes has some fundamental limitations in that it does not build or deploy applications, does not provide logging, monitoring, or alerting mechanisms, and is not a self-healing, self-managing system. As an open-source project, Kubernetes must support a variety of use cases and enable users to use a wide variety of projects that are compatible with Kubernetes.

OpenShift Container Platform fills the gaps that Kubernetes leaves open:

•Platform-level services including building and packaging applications

•Integrated logging and monitoring solutions (Prometheus and Grafana)

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 11

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

• Integrated web console

OpenShift Container Platform is intended as a turnkey solution for production-grade environments. Among other benefits, OpenShift Container Platform:

• Eliminates the complexity of installing Kubernetes and of adding authentication,

	management, logging, security, and networking.
	• Provides additional self-management capabilities that are not found in Kubernetes
	due to the tightly coupled toolchain: the default containers-first operating system
	(Red Hat CoreOS), a Kubernetes-first container runtime (CRI-O), and a rigorous
	testing and certification process for additional Red Hat and vendor middleware.
Kubernetes	In Kubernetes, everything is an object. Every object has a current state, a desired state,
concepts	and a specification of how a state transition can be achieved. This specification includes
	everything from applications, deployments, and services to machine configuration and
	management of specific hardware resources. When a Kubernetes object is created, the
	cluster uses the object to transition towards the desired state for the cluster. Custom
	Resource Definitions (CRDs) can be used to specify new resource types, which can then
	be used to create Custom Resources (CRs). Middleware (typically, operators) can use
	this extensible mechanism to create resource types that Kubernetes and other
	middleware with appropriate access can manage and use.
Cluster	The Operator Framework gives vendors the ability to manage the life cycle of the
automation	middleware they provide—for example, the Dell CSI Operator provides drivers for Dell
	EMC storage products. Operators attempt to encode the operational knowledge that is
	required for various stateful applications. Like Helm, an Operator can be used to configure
	and install middleware; however, depending on the complexity of the Operator, the
	Operator can fully automate an application’s life cycle management. Operators are
	application-specific, and therefore an Operator must be installed to manage each
	middleware application. In contrast, Helm is a universal package manager for Kubernetes.
	The following figure shows the benefits that Operators can provide, depending on the
	complexity of the Operator:

Operator maturity

12 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Operators are designed to simplify Day-2 operations by automatically deploying, updating, and maintaining specific application deployments. This simplification is achieved through the creation of CRDs that are managed through a control loop that is embedded in the Operator. More complex Operators can be used to fully automate the life cycle management of various applications and middleware, scaling, and handling abnormalities gracefully.

Cloud-native infrastructure

A cloud-native infrastructure must accommodate a large, scalable mix of service-oriented applications and their dependent components. These applications and components are generally microservice-based. The The key to sustaining their operation is to have the right platform infrastructure and a sustainable management and control plane. This reference design helps you specify infrastructure requirements for building an onpremises OpenShift Container Platform 4.6 solution.

The following figure shows the solution design:

OpenShift Container Platform 4.6 cluster design

Terminology

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 13

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

	This Ready Stack design recognizes four host types that make up every OpenShift
	Container Platform cluster: the bootstrap node, control-plane nodes, compute nodes, and
	storage nodes.
	The deployment process also requires a node called the Cluster System Admin Host
	(CSAH). A description of the process is available in the Ready Stack for Red Hat
	OpenShift Container Platform 4.6 Deployment Guide at the Dell Technologies Solutions
	Info Hub for Containers.
	Note: Red Hat official documentation does not refer to a CSAH node in the deployment process.
CSAH node
	The CSAH node is not part of the cluster, but it is required for OpenShift cluster
	administration. Dell Technologies strongly discourages logging in to a control plane node
	to manage the cluster. The OpenShift CLI administration tools are deployed onto the
	control plane nodes, while the authentication tokens that are required to administer the
	OpenShift cluster are installed on the CSAH node only as part of the deployment process.
	Note: Control-plane nodes are deployed using immutable infrastructure, further driving the
	preference for an administration host that is external to the cluster.

Bootstrap node (VM)

The CSAH node manages the operation and installation of the container ecosystem cluster. Installation of the cluster begins with the creation of a bootstrap VM on the CSAH node, which is used to install control-plane components on the controller nodes. Delete the bootstrap VM after the control plane is deployed. Dell Technologies recommends provisioning a dedicated host for administration of the OpenShift Container cluster. The initial minimum cluster can consist of three nodes running both the control plane and applications, or three control-plane nodes and at least two compute nodes. OpenShift Container Platform requires three control-plane nodes in both scenarios.

Basic node configuration

Node components are installed and run on every node within the cluster; that is, on controller nodes and compute nodes. The components are responsible for all node runtime operations. Key components consist of:

•Kubelet: An agent that runs on each node to perform declarations or actions that are provided to the cluster-API. Kubelet performs node service functions to ensure that running pods are compliant with PodSpecs and remain healthy. Kubelet does not manage containers or pods that were not created by Kubernetes.

•Kube-proxy: An instance of kube-proxy runs on every node of the cluster. It implements Kubernetes network services that run on the node. It also manages network connectivity and traffic route management based on host operating system packet filtering.

•Container Runtime: The chosen container runtime engine must be deployed on each node in a Kubernetes cluster. The Container Runtime Engine must comply with the Kubernetes Container Runtime Interface (CRI) specifications. OpenShift Container Platform defaults to the CRI-O container runtime and cannot be changed.

14 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Control plane

Compute plane

Nodes that implement control plane infrastructure management are called controller nodes. Three controller nodes establish the control plane for the operation of an OpenShift cluster. The control plane operates outside the application container workloads and is responsible for ensuring the overall continued viability, health, availability, and integrity of the container ecosystem. Removing controller nodes is not allowed. OpenShift Container Platform also deploys additional control-plane infrastructure to manage OpenShift-specific cluster components.

The control plane provides the following functions:

•API Server: The API server exposes the Kubernetes control plane API for other platform services (such as a web console) to consume and has API endpoints to manage cluster resources.

•Etcd: Highly available and consistent key-value store used to maintain Kubernetes cluster data. The etcd daemon is run on each control plane node and requires at least two running daemons to achieve quorum. For production clusters, at least three control-plane nodes are therefore required, each running an etcd daemon.

•Scheduler: The Kubernetes scheduler assigns new pods to a node based on the resource requirements (for CPU, RAM, and GPU, for example), and the affinity and anti-affinity mechanisms.

•Controller manager: The controller managers run all controller processes. While each controller process is independent, the processes are run as a single process to reduce complexity. The controllers include the node, replication, endpoints, service, and token controllers.

•OpenShift API server: The OpenShift API server validates and configures the data for OpenShift resources such as projects, routes, and templates. The OpenShift API server is managed by the OpenShift API Server Operator.

•OpenShift controller manager: The OpenShift controller manager watches etcd for changes to OpenShift objects such as project, route, and template controller objects, and then uses the API to enforce the specified state. The OpenShift controller manager is managed by the OpenShift Controller Manager Operator.

•OpenShift OAuth API server: The OpenShift OAuth API server validates and configures the data to authenticate to OpenShift Container Platform, such as users, groups, and OAuth tokens. The OpenShift OAuth API server is managed by the Cluster Authentication Operator.

•OpenShift OAuth server: Users request tokens from the OpenShift OAuth server to authenticate themselves to the API. The OpenShift OAuth server is managed by the Cluster Authentication Operator.

In an OpenShift cluster, application containers are deployed to run on compute nodes, by default. The term “compute node” is arbitrary; nothing specific is required to run compute nodes and, therefore, applications can be run on control plane nodes. Cluster nodes advertise their resources and resource utilization so that the scheduler can allocate containers and pods to these nodes and maintain a reasonable workload distribution. The Kubelet service runs on each compute node. This service receives container deployment requests and ensures that the requests are instantiated and put into operation. The

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 15

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

	Kubelet service also starts and stops container workloads and manages a service proxy
	that handles communication between pods that are running across compute nodes.
	Logical constructs called MachineSets define compute node resources. MachineSets can
	be used to match requirements for a pod deployment to a matching compute node.
	OpenShift Container Platform supports defining multiple machine types, each of which
	defines a compute node target type.
	Compute nodes can be added to or deleted from a cluster if doing so does not
	compromise the viability of the cluster. If the control plane nodes are not designated as
	schedulable, at least two viable compute nodes must always be operating. Further,
	enough compute platform resources must be available to sustain the overall cluster
	application container workload.
Storage nodes	Storage can be either provisioned from dedicated nodes or shared with compute services.
	Provisioning occurs on disk drives that are locally attached to servers that have been
	added to the cluster as compute nodes.
	OpenShift Container Storage (OCS), which is deployed after the cluster deployment,
	simplifies and automates the deployment of storage for cloud-native container use. To
	integrate Ceph OCS storage into the container ecosystem infrastructure, administrators
	must provision appropriate storage nodes. It is also possible to use existing compute
	nodes if they meet OpenShift Container Storage hardware requirements.
	You can initiate the deployment of OCS from the embedded OperatorHub when you are
	logged into OpenShift Container Platform as the cluster administrator. For more
	information, see OpenShift Container Platform 4.6 Documentation.

Deployment process

Dell Technologies has simplified the process of bootstrapping the OpenShift Container

Platform 4.6 cluster. To use the simplified process, ensure that:

•The cluster is provisioned with network switches and servers.

•Network cabling is complete.

•Internet connectivity has been provided to the cluster. Internet connectivity is necessary to install OpenShift Container Platform 4.6.

The deployment procedure begins with initial switch provisioning. This step enables preparation and installation of the CSAH node, involving:

•Installing Red Hat Enterprise Linux 7

•Subscribing to the necessary repositories

•Creating an Ansible user account

•Cloning a GitHub Ansible playbook repository from the Dell ESG container repository

•Running an Ansible playbook to initiate the installation process

16 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Dell Technologies has generated Ansible playbooks that fully prepare the CSAH node. Before the installation of the OpenShift Container Platform 4.6 cluster begins, the Ansible playbook sets up a PXE server, DHCP server, DNS server, HAProxy, and HTTP server. The playbook also creates ignition files to drive installation of the bootstrap, control plane, and compute nodes. It also starts the bootstrap VM to initialize control plane components. The playbook presents a list of node types that must be deployed in top-down order.

Note: For enterprise sites, consider deploying appropriately hardened DHCP and DNS servers. Similarly, consider using resilient multiple-node HAProxy configuration. The Ansible playbook for this design deploys a single HAProxy instance. This guide provides CSAH Ansible playbooks for reference only at the implementation stage.

The Ansible playbook creates an install-config.yaml file that is used to control deployment of the bootstrap node. For more information, see the Dell EMC Ready Stack: Red Hat OpenShift Container Platform 4.6 Deployment Guide at the Dell Technologies Solutions Info Hub for Containers. An ignition configuration control file starts the bootstrap node, as shown in the following figure:

Installation workflow: Creating the bootstrap, control-plane, and compute nodes

Note: An installation that is driven by ignition configuration generates security certificates that expire after 24 hours. You must install the cluster before the certificates expire, and the cluster must operate in a viable (nondegraded) state so that the first certificate rotation can be completed.

The cluster bootstrapping process consists of the following phases:

1.After startup, the bootstrap VM creates the resources that are required to start the control-plane nodes. Do not interrupt this process.

2.The control-plane nodes pull resource information from the bootstrap VM to bring them up into a viable state. This resource information is used to form the etcd control plane cluster.

3.The bootstrap VM instantiates a temporary Kubernetes control plane that is under etcd control.

4.A temporary control plane loads the application workload control plane to the control-plane nodes.

5.The temporary control plane is shut down, handing control over to the now viable control-plane nodes.

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 17

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

6.OpenShift Container Platform components are pulled into the control of the control-plane nodes.

7.The bootstrap VM is shut down.

The control-plane nodes now drive creation and instantiation of the compute nodes.

8.The control plane adds operator-based services to complete the deployment of the OpenShift Container Platform ecosystem.

The cluster is now viable and can be placed into service in readiness for Day-2 operations. You can expand the cluster by adding compute nodes.

18 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Infrastructure requirements

Basic guidance		The following table provides basic cluster infrastructure guidance. For detailed
		configuration information, see Cluster Hardware Design. Administrators can build a
		container cluster to be deployed quickly and reliably when each node is within the
		validated design guidelines.
		Table 1.		Hardware infrastructure for OpenShift Container Platform 4.6 cluster deployment
	Type	Description			Count	Notes
	CSAH node	Dell EMC PowerEdge R640			1	Creates a bootstrap VM.
		server				CSAH runs a single instance of HAProxy. For
						enterprise high availability (HA) deployment
						of OpenShift Container Platform 4.6, Dell
						Technologies recommends using a
						commercially supported L4 load-balancer or
						proxy service or system. Options include
						commercial HAProxy, Nginx, and F5.
	Controller nodes	Dell EMC PowerEdge R640			3	Deployed using the bootstrap node.
		server
	Compute nodes	Dell EMC PowerEdge R640 or			Minimum 2,*	No compute nodes are required for a three-
		R740xd server			maximum 30	node cluster.
					per rack	A standard deployment requires a minimum
						of two compute nodes (and three controller
						nodes).
						To expand a three-node cluster, you must
						add two compute nodes at the same time.
						After the cluster is operational, you can add
						more compute nodes to the cluster through
						the Cluster Management Service.
	Data switches	Either of the following switches:			2 per rack	Autoconfigured at installation time.
		•	Dell EMC PowerSwitch			Note:
			S5248-ON			• HA network configuration requires two
		•				data path switches per rack.
			Dell EMC PowerSwitch			• Multirack clusters require network
			S5232-ON
						topology planning. Leaf-spine network
						switch configuration may be necessary.
	iDRAC network	Dell EMC PowerSwitch S3048-			1 per rack	Used for OOB management.
		ON
	Rack	Selected according to site			1–3 racks	For multirack configurations, consult your Dell
		standards				Technologies or Red Hat representative
						regarding custom engineering design.

*A three-node cluster does not require any compute nodes. To expand a three-node cluster with additional compute machines, you must first expand the cluster to a five-node cluster using two additional compute nodes.

Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 19

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide

Chapter 2: Technology and Deployment Process Overview

Minimum viable solution requirements

Installing OpenShift Container Platform requires, at a minimum, the following nodes:

•One CSAH node, which is used to run the bootstrap VM. The CSAH node is used later to manage the cluster while the cluster is in production use.

•Three nodes running both the control plane and data plane, enabling customers to develop OpenShift 4.6 POCs using only four nodes. The cluster can be expanded with additional compute nodes as needed. However, an initial expansion beyond three nodes requires two compute nodes. A four-node cluster (three controllers, one compute) is not supported. The minimum viable solution options are a threenode cluster (three control-compute nodes) or a five-node cluster (three controller nodes, two compute nodes) plus the CSAH node for cluster administration with either option.

HA of the key services that make up the OpenShift Container Platform cluster is necessary to ensure run-time integrity. Redundancy of physical nodes for each cluster node type is an important aspect of HA for the bare-metal cluster.

In this design guide, HA includes the provisioning of at least two network interface controllers (NICs) and two network switches that are configured to provide redundant pathing. The redundant pathing provides for network continuity if a NIC or a network switch fails.

OpenShift Container Platform 4.6 must use Red Hat Enterprise Linux CoreOS (RHCOS) for the control-plane nodes and can use either RHCOS or Red Hat Enterprise Linux 7.6 for compute nodes. Using Red Hat Enterprise Linux 7 on the compute nodes is now deprecated, and the ability to use Red Hat Enterprise Linux 7 compute nodes in OpenShift will be removed in a future release of OpenShift. The bootstrap and control-plane nodes must use RHCOS as their operating system. Each of these nodes must be immutable.

The following table shows the minimum resource requirements:

Table 2.	Minimum resource requirements for OpenShift Container Platform 4.6 nodes
Node type		Operating system	Minimum CPU	RAM	Storage
			cores
CSAH		Red Hat Enterprise	4	32 GB	200 GB
		Linux 7.6+
Bootstrap		RHCOS 4.6	4	16 GB	120 GB
Controller		RHCOS 4.6	4	16 GB	120 GB
Compute		RHCOS 4.6 or Red	2	8 GB	120 GB
		Hat Enterprise Linux
		7.6 (deprecated)

Network connectivity requirements

The RHCOS nodes must fetch ignition files from the Machine Config server. This operation uses an initramfs-based-node startup for the initial network configuration. The startup requires a DHCP server to provide a network connection giving access to the ignition files for that node. Subsequent operations can use static IP addresses.

20 Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6

Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage

Design Guide