Dell R640, R740 User manual
Dell EMC Ready Stack for Red Hat OpenShift
Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers;
PowerSwitch Networking; PowerMax, PowerScale, Unity XT
Storage
February 2021
H18217.5
Abstract
This design guide describes how to design and specify a Dell
Technologies server and switch infrastructure for validated hardware
configurations, facilitating deployment of Red Hat OpenShift Container
Platform 4.6 following a Dell Technologies infrastructure deployment.
Dell Technologies Solutions
Design Guide
Copyright
2
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular
purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2021 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other
trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks
of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners.
Published in the USA 02/21 Design Guide H18217.5.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change
without notice.
Contents
3
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Contents
Chapter 1 Introduction 5
Solution overview and key benefits ........................................................................ 6
Document purpose ................................................................................................. 7
Audience ................................................................................................................. 7
We value your feedback ......................................................................................... 7
Chapter 2 Technology and Deployment Process Overview 9
Introduction ........................................................................................................... 10
OpenShift Container Platform ............................................................................... 10
Cloud-native infrastructure ................................................................................... 13
Deployment process ............................................................................................. 16
Infrastructure requirements .................................................................................. 19
Chapter 3 Networking Infrastructure and Configuration 21
Introduction ........................................................................................................... 22
OpenShift network operations .............................................................................. 22
Physical network design ....................................................................................... 25
Chapter 4 Storage Overview 30
OpenShift Container Platform storage .................................................................. 31
Container Storage Interface (CSI) external storage ............................................. 34
Chapter 5 Cluster Hardware Design 39
Introduction ........................................................................................................... 40
Cluster scaling ...................................................................................................... 40
Requirements planning ......................................................................................... 40
Cluster hardware planning .................................................................................... 42
Validated hardware configuration options ............................................................ 44
Chapter 6 Use Cases 48
Introduction ........................................................................................................... 49
Enterprise applications ......................................................................................... 49
Telecommunications industry ............................................................................... 52
Data analytics and artificial intelligence ................................................................ 54
Chapter 7 References 57
Dell Technologies documentation ........................................................................ 58
Red Hat documentation ........................................................................................ 58
Contents
4
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Other resources .................................................................................................... 58
Appendix A Dell EMC PowerEdge BOMs 59
Dell EMC PowerEdge R640 node BOM ............................................................... 60
Dell EMC PowerEdge R740xd node BOM ........................................................... 62
Dell EMC Unity 380F BOM ................................................................................... 64
Dell EMC PowerMax BOM ................................................................................... 64
Chapter 1: Introduction
5
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Chapter 1 Introduction
This chapter presents the following topics:
Solution overview and key benefits .................................................................... 6
Document purpose ............................................................................................... 7
Audience ............................................................................................................... 7
We value your feedback ....................................................................................... 7
Chapter 1: Introduction
6
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Solution overview and key benefits
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6 is a flexible
infrastructure that has been designed, optimized, and validated for an OpenShift
Container Platform 4.6 on-premises bare-metal deployment. The deployment that this
guide describes does not require a hypervisor.
The Dell EMC Ready Stack solution consists of the following documents:
• Dell EMC Ready Stack design guide (this document)
• Dell EMC Ready Stack deployment guide
(Both documents are available at the Dell Technologies Info Hub for Containers.)
This Ready Stack solution provides:
• A detailed overview of validated OpenShift Container Platform hardware designs
• A scalable hardware platform of up to 210 compute nodes spread across seven
racks
• Rapid implementation and time-to-value
The solution includes the following components:
• Red Hat OpenShift Container Platform 4.6 for application development and
deployment
• Dell EMC PowerEdge R640 and R740xd servers for compute and storage
• Dell EMC PowerSwitch S5200 series switches for infrastructure network
enablement
• Dell EMC PowerSwitch S3048 switch for out-of-band (OOB) management of the
cluster
Note: While you can rely on Red Hat Enterprise Linux security and container technologies to
prevent intrusions and protect your data, some security vulnerabilities might persist. For
information about security vulnerabilities in OpenShift Container Platform, see OCP Errata. For a
general listing of Red Hat vulnerabilities, see the RH Security Home Page.
OpenShift Container Platform 4.6 consists of many open-source components that have
been carefully integrated to provide a consistently dependable platform on which you can
develop and deploy scalable containerized applications. OpenShift Container Platform
provides great flexibility for accommodating platform deployment preferences. For more
information, see OpenShift Container Platform 4.6 Documentation.
At the heart of OpenShift Container Platform is Kubernetes container orchestration
software. For more information, see What Kubernetes is.
Ready Stack solution for OpenShift Container Platform 4.6
OpenShift Container Platform and Kubernetes
Chapter 1: Introduction
7
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Document purpose
Dell EMC Ready Stack for Red Hat OpenShift Container Platform is a proven design to
help organizations accelerate their container deployments and cloud-native adoption. This
guide provides information for building an on-premises infrastructure solution to host
OpenShift Container Platform 4.6. The guide describes the Dell Technologies design
decisions and configurations that enable solution architects to:
• Design and deploy a container platform solution.
• Extend or modify the design as necessary to meet customer requirements.
This guide includes:
• Container ecosystem design overview
• Network infrastructure design guidance
• Container and application storage design guidance
• Server requirements to support OpenShift Container Platform node roles
• Hardware platform configuration recommendations
• Rack-level design and power configuration considerations
A companion deployment guide provides information about automation-assisted
deployment of the solution. This guide is available at the Dell Technologies Solutions Info
Hub for Containers.
For information about the manual installation and deployment of Red Hat software
products, see OpenShift Container Platform 4.6 Documentation.
Note: This guide may contain language from third-party content that is not under Dell's control and
is not consistent with Dell's current guidelines for Dell's own content. When this content is updated
by the relevant third parties, this guide will be revised accordingly.
Audience
This design guide is for system administrators and system architects. Some experience
with Docker, Kubernetes, and OpenShift Container Platform technologies is
recommended.
We value your feedback
Dell Technologies and the authors of this document welcome your feedback on the
solution and the solution documentation. Contact the Dell Technologies Solutions team by
email or provide your comments by completing our documentation survey.
Author: Piyush Tandon
Contributors: John Terpstra, Umesh Sunnapu, Scott Powers, Aighne Kearney
Chapter 1: Introduction
8
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Note: For additional information about this solution, see the Dell Technologies Solutions Info Hub
for Containers.
Chapter 2: Technology and Deployment Process Overview
9
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Chapter 2 Technology and Deployment
Process Overview
This chapter presents the following topics:
Introduction ......................................................................................................... 10
OpenShift Container Platform ........................................................................... 10
Cloud-native infrastructure ............................................................................... 13
Deployment process .......................................................................................... 16
Infrastructure requirements .............................................................................. 19
Chapter 2: Technology and Deployment Process Overview
10
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Introduction
OpenShift Container Platform 4.6 can host the development and runtime execution of
containerized applications. The platform is continuing to mature and expand rapidly,
providing you with access to the tools your team needs so that your business can grow.
OpenShift Container Platform is based on Kubernetes, the de facto container automation
and life cycle management platform for containerized workloads and services. Ready
Stack for OpenShift Container Platform 4.6 includes Dell EMC hardware (servers,
switches, and storage) to enable you to develop, validate, and deploy your containerized
applications.
This chapter describes the OpenShift Container Platform architecture, infrastructure
components, and requirements for a viable Ready Stack for OpenShift Container Platform
4.6 cluster, which can drive the core of modern telecommunications practices, multimedia
operations, service provider infrastructure operations, the demands of the gaming
industry, and financial transaction workloads.
OpenShift Container Platform
OpenShift Container Platform is an enterprise-grade declarative state machine that has
been designed to automate application workload operations based on the upstream
Kubernetes project. In a Kubernetes context, “declarative” means that developers can
specify, in code, a configuration for an application or workload without knowing how that
application is going to deployed. OpenShift Container Platform uses the enterprise-grade
Kubernetes distribution, called the OpenShift Kubernetes Engine, to provide productionoriented container and workload automation. OpenShift Container Platform 4.6 is based
on Kubernetes version 1.19, which includes native support for cluster snapshots, enabling
cluster backup and recovery. On top of the Kubernetes Engine, OpenShift Container
Platform provides administrators and developers with the tools they require to deploy and
manage applications and services at scale, as shown in the following figure.
Note: OpenShift Container Platform is a certified Kubernetes distribution.
Overview
Chapter 2: Technology and Deployment Process Overview
11
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
OpenShift Container Platform architecture
Kubernetes provides an abstraction layer for application containers, deployments, and
services and automates all container operations. Developers and administrators
manipulate Kubernetes object declarations and abstractions to achieve the desired state
of operations. Developers and administrators can specify the needs of an application in a
declarative manner, and Kubernetes automatically deploys, terminates, or restarts
containers to converge on this desired state.
Kubernetes is not just an “orchestration” platform for containers, which implies imperative,
sequential actions. There is no imperative management of containers in Kubernetes.
Rather, Kubernetes consists of independent control processes (state transition machines)
that move the current state of the cluster towards the desired state. This mechanism has
fundamental implications for how cluster operations, application middleware, and more
can be managed automatically (see Cluster automation).
Upstream Kubernetes has some fundamental limitations in that it does not build or deploy
applications, does not provide logging, monitoring, or alerting mechanisms, and is not a
self-healing, self-managing system. As an open-source project, Kubernetes must support
a variety of use cases and enable users to use a wide variety of projects that are
compatible with Kubernetes.
OpenShift Container Platform fills the gaps that Kubernetes leaves open:
• Platform-level services including building and packaging applications
• Integrated logging and monitoring solutions (Prometheus and Grafana)
What Kubernetes is
What Kubernetes is not
Why OpenShift?
Chapter 2: Technology and Deployment Process Overview
12
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
• Integrated web console
OpenShift Container Platform is intended as a turnkey solution for production-grade
environments. Among other benefits, OpenShift Container Platform:
• Eliminates the complexity of installing Kubernetes and of adding authentication,
management, logging, security, and networking.
• Provides additional self-management capabilities that are not found in Kubernetes
due to the tightly coupled toolchain: the default containers-first operating system
(Red Hat CoreOS), a Kubernetes-first container runtime (CRI-O), and a rigorous
testing and certification process for additional Red Hat and vendor middleware.
In Kubernetes, everything is an object. Every object has a current state, a desired state,
and a specification of how a state transition can be achieved. This specification includes
everything from applications, deployments, and services to machine configuration and
management of specific hardware resources. When a Kubernetes object is created, the
cluster uses the object to transition towards the desired state for the cluster. Custom
Resource Definitions (CRDs) can be used to specify new resource types, which can then
be used to create Custom Resources (CRs). Middleware (typically, operators) can use
this extensible mechanism to create resource types that Kubernetes and other
middleware with appropriate access can manage and use.
The Operator Framework gives vendors the ability to manage the life cycle of the
middleware they provide—for example, the Dell CSI Operator provides drivers for Dell
EMC storage products. Operators attempt to encode the operational knowledge that is
required for various stateful applications. Like Helm, an Operator can be used to configure
and install middleware; however, depending on the complexity of the Operator, the
Operator can fully automate an application’s life cycle management. Operators are
application-specific, and therefore an Operator must be installed to manage each
middleware application. In contrast, Helm is a universal package manager for Kubernetes.
The following figure shows the benefits that Operators can provide, depending on the
complexity of the Operator:
Operator maturity
Kubernetes concepts
Cluster automation
Chapter 2: Technology and Deployment Process Overview
13
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Operators are designed to simplify Day-2 operations by automatically deploying, updating,
and maintaining specific application deployments. This simplification is achieved through
the creation of CRDs that are managed through a control loop that is embedded in the
Operator. More complex Operators can be used to fully automate the life cycle
management of various applications and middleware, scaling, and handling abnormalities
gracefully.
Cloud-native infrastructure
A cloud-native infrastructure must accommodate a large, scalable mix of service-oriented
applications and their dependent components. These applications and components are
generally microservice-based. The The key to sustaining their operation is to have the
right platform infrastructure and a sustainable management and control plane. This
reference design helps you specify infrastructure requirements for building an onpremises OpenShift Container Platform 4.6 solution.
The following figure shows the solution design:
OpenShift Container Platform 4.6 cluster design
Terminology
Chapter 2: Technology and Deployment Process Overview
14
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
This Ready Stack design recognizes four host types that make up every OpenShift
Container Platform cluster: the bootstrap node, control-plane nodes, compute nodes, and
storage nodes.
The deployment process also requires a node called the Cluster System Admin Host
(CSAH). A description of the process is available in the Ready Stack for Red Hat
OpenShift Container Platform 4.6 Deployment Guide at the Dell Technologies Solutions
Info Hub for Containers.
Note: Red Hat official documentation does not refer to a CSAH node in the deployment process.
The CSAH node is not part of the cluster, but it is required for OpenShift cluster
administration. Dell Technologies strongly discourages logging in to a control plane node
to manage the cluster. The OpenShift CLI administration tools are deployed onto the
control plane nodes, while the authentication tokens that are required to administer the
OpenShift cluster are installed on the CSAH node only as part of the deployment process.
Note: Control-plane nodes are deployed using immutable infrastructure, further driving the
preference for an administration host that is external to the cluster.
The CSAH node manages the operation and installation of the container ecosystem
cluster. Installation of the cluster begins with the creation of a bootstrap VM on the CSAH
node, which is used to install control-plane components on the controller nodes. Delete
the bootstrap VM after the control plane is deployed. Dell Technologies recommends
provisioning a dedicated host for administration of the OpenShift Container cluster. The
initial minimum cluster can consist of three nodes running both the control plane and
applications, or three control-plane nodes and at least two compute nodes. OpenShift
Container Platform requires three control-plane nodes in both scenarios.
Node components are installed and run on every node within the cluster; that is, on
controller nodes and compute nodes. The components are responsible for all node
runtime operations. Key components consist of:
• Kubelet: An agent that runs on each node to perform declarations or actions that
are provided to the cluster-API. Kubelet performs node service functions to ensure
that running pods are compliant with PodSpecs and remain healthy. Kubelet does
not manage containers or pods that were not created by Kubernetes.
• Kube-proxy: An instance of kube-proxy runs on every node of the cluster. It
implements Kubernetes network services that run on the node. It also manages
network connectivity and traffic route management based on host operating system
packet filtering.
• Container Runtime: The chosen container runtime engine must be deployed on
each node in a Kubernetes cluster. The Container Runtime Engine must comply
with the Kubernetes Container Runtime Interface (CRI) specifications. OpenShift
Container Platform defaults to the CRI-O container runtime and cannot be changed.
CSAH node
Bootstrap node (VM)
Basic node configuration
Chapter 2: Technology and Deployment Process Overview
15
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Nodes that implement control plane infrastructure management are called controller
nodes. Three controller nodes establish the control plane for the operation of an
OpenShift cluster. The control plane operates outside the application container workloads
and is responsible for ensuring the overall continued viability, health, availability, and
integrity of the container ecosystem. Removing controller nodes is not allowed. OpenShift
Container Platform also deploys additional control-plane infrastructure to manage
OpenShift-specific cluster components.
The control plane provides the following functions:
• API Server: The API server exposes the Kubernetes control plane API for other
platform services (such as a web console) to consume and has API endpoints to
manage cluster resources.
• Etcd: Highly available and consistent key-value store used to maintain Kubernetes
cluster data. The etcd daemon is run on each control plane node and requires at
least two running daemons to achieve quorum. For production clusters, at least
three control-plane nodes are therefore required, each running an etcd daemon.
• Scheduler: The Kubernetes scheduler assigns new pods to a node based on the
resource requirements (for CPU, RAM, and GPU, for example), and the affinity and
anti-affinity mechanisms.
• Controller manager: The controller managers run all controller processes. While
each controller process is independent, the processes are run as a single process
to reduce complexity. The controllers include the node, replication, endpoints,
service, and token controllers.
• OpenShift API server: The OpenShift API server validates and configures the data
for OpenShift resources such as projects, routes, and templates. The OpenShift
API server is managed by the OpenShift API Server Operator.
• OpenShift controller manager: The OpenShift controller manager watches etcd
for changes to OpenShift objects such as project, route, and template controller
objects, and then uses the API to enforce the specified state. The OpenShift
controller manager is managed by the OpenShift Controller Manager Operator.
• OpenShift OAuth API server: The OpenShift OAuth API server validates and
configures the data to authenticate to OpenShift Container Platform, such as users,
groups, and OAuth tokens. The OpenShift OAuth API server is managed by the
Cluster Authentication Operator.
• OpenShift OAuth server: Users request tokens from the OpenShift OAuth server
to authenticate themselves to the API. The OpenShift OAuth server is managed by
the Cluster Authentication Operator.
In an OpenShift cluster, application containers are deployed to run on compute nodes, by
default. The term “compute node” is arbitrary; nothing specific is required to run compute
nodes and, therefore, applications can be run on control plane nodes. Cluster nodes
advertise their resources and resource utilization so that the scheduler can allocate
containers and pods to these nodes and maintain a reasonable workload distribution. The
Kubelet service runs on each compute node. This service receives container deployment
requests and ensures that the requests are instantiated and put into operation. The
Control plane
Compute plane
Chapter 2: Technology and Deployment Process Overview
16
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Kubelet service also starts and stops container workloads and manages a service proxy
that handles communication between pods that are running across compute nodes.
Logical constructs called MachineSets define compute node resources. MachineSets can
be used to match requirements for a pod deployment to a matching compute node.
OpenShift Container Platform supports defining multiple machine types, each of which
defines a compute node target type.
Compute nodes can be added to or deleted from a cluster if doing so does not
compromise the viability of the cluster. If the control plane nodes are not designated as
schedulable, at least two viable compute nodes must always be operating. Further,
enough compute platform resources must be available to sustain the overall cluster
application container workload.
Storage can be either provisioned from dedicated nodes or shared with compute services.
Provisioning occurs on disk drives that are locally attached to servers that have been
added to the cluster as compute nodes.
OpenShift Container Storage (OCS), which is deployed after the cluster deployment,
simplifies and automates the deployment of storage for cloud-native container use. To
integrate Ceph OCS storage into the container ecosystem infrastructure, administrators
must provision appropriate storage nodes. It is also possible to use existing compute
nodes if they meet OpenShift Container Storage hardware requirements.
You can initiate the deployment of OCS from the embedded OperatorHub when you are
logged into OpenShift Container Platform as the cluster administrator. For more
information, see OpenShift Container Platform 4.6 Documentation.
Deployment process
Dell Technologies has simplified the process of bootstrapping the OpenShift Container
Platform 4.6 cluster. To use the simplified process, ensure that:
• The cluster is provisioned with network switches and servers.
• Network cabling is complete.
• Internet connectivity has been provided to the cluster. Internet connectivity is
necessary to install OpenShift Container Platform 4.6.
The deployment procedure begins with initial switch provisioning. This step enables
preparation and installation of the CSAH node, involving:
• Installing Red Hat Enterprise Linux 7
• Subscribing to the necessary repositories
• Creating an Ansible user account
• Cloning a GitHub Ansible playbook repository from the Dell ESG container
repository
• Running an Ansible playbook to initiate the installation process
Storage nodes
Chapter 2: Technology and Deployment Process Overview
17
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Dell Technologies has generated Ansible playbooks that fully prepare the CSAH node.
Before the installation of the OpenShift Container Platform 4.6 cluster begins, the Ansible
playbook sets up a PXE server, DHCP server, DNS server, HAProxy, and HTTP server.
The playbook also creates ignition files to drive installation of the bootstrap, control plane,
and compute nodes. It also starts the bootstrap VM to initialize control plane components.
The playbook presents a list of node types that must be deployed in top-down order.
Note: For enterprise sites, consider deploying appropriately hardened DHCP and DNS servers.
Similarly, consider using resilient multiple-node HAProxy configuration. The Ansible playbook for
this design deploys a single HAProxy instance. This guide provides CSAH Ansible playbooks for
reference only at the implementation stage.
The Ansible playbook creates an install-config.yaml file that is used to control
deployment of the bootstrap node. For more information, see the Dell EMC Ready Stack:
Red Hat OpenShift Container Platform 4.6 Deployment Guide at the Dell Technologies
Solutions Info Hub for Containers. An ignition configuration control file starts the bootstrap
node, as shown in the following figure:
Installation workflow: Creating the bootstrap, control-plane, and compute
nodes
Note: An installation that is driven by ignition configuration generates security certificates that
expire after 24 hours. You must install the cluster before the certificates expire, and the cluster
must operate in a viable (nondegraded) state so that the first certificate rotation can be completed.
The cluster bootstrapping process consists of the following phases:
1. After startup, the bootstrap VM creates the resources that are required to start the
control-plane nodes. Do not interrupt this process.
2. The control-plane nodes pull resource information from the bootstrap VM to bring
them up into a viable state. This resource information is used to form the etcd
control plane cluster.
3. The bootstrap VM instantiates a temporary Kubernetes control plane that is under
etcd control.
4. A temporary control plane loads the application workload control plane to the
control-plane nodes.
5. The temporary control plane is shut down, handing control over to the now viable
control-plane nodes.
Chapter 2: Technology and Deployment Process Overview
18
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
6. OpenShift Container Platform components are pulled into the control of the
control-plane nodes.
7. The bootstrap VM is shut down.
The control-plane nodes now drive creation and instantiation of the compute
nodes.
8. The control plane adds operator-based services to complete the deployment of
the OpenShift Container Platform ecosystem.
The cluster is now viable and can be placed into service in readiness for Day-2
operations. You can expand the cluster by adding compute nodes.
Chapter 2: Technology and Deployment Process Overview
19
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Infrastructure requirements
The following table provides basic cluster infrastructure guidance. For detailed
configuration information, see Cluster Hardware Design. Administrators can build a
container cluster to be deployed quickly and reliably when each node is within the
validated design guidelines.
Table 1. Hardware infrastructure for OpenShift Container Platform 4.6 cluster deployment
Type
Description
Count
Notes
CSAH node
Dell EMC PowerEdge R640
server
1
Creates a bootstrap VM.
CSAH runs a single instance of HAProxy. For
enterprise high availability (HA) deployment
of OpenShift Container Platform 4.6, Dell
Technologies recommends using a
commercially supported L4 load-balancer or
proxy service or system. Options include
commercial HAProxy, Nginx, and F5.
Controller nodes
Dell EMC PowerEdge R640
server
3
Deployed using the bootstrap node.
Compute nodes
Dell EMC PowerEdge R640 or
R740xd server
Minimum 2,*
maximum 30
per rack
No compute nodes are required for a threenode cluster.
A standard deployment requires a minimum
of two compute nodes (and three controller
nodes).
To expand a three-node cluster, you must
add two compute nodes at the same time.
After the cluster is operational, you can add
more compute nodes to the cluster through
the Cluster Management Service.
Data switches
Either of the following switches:
• Dell EMC PowerSwitch
S5248-ON
• Dell EMC PowerSwitch
S5232-ON
2 per rack
Autoconfigured at installation time.
Note:
• HA network configuration requires two
data path switches per rack.
• Multirack clusters require network
topology planning. Leaf-spine network
switch configuration may be necessary.
iDRAC network
Dell EMC PowerSwitch S3048ON
1 per rack
Used for OOB management.
Rack
Selected according to site
standards
1–3 racks
For multirack configurations, consult your Dell
Technologies or Red Hat representative
regarding custom engineering design.
*A three-node cluster does not require any compute nodes. To expand a three-node
cluster with additional compute machines, you must first expand the cluster to a five-node
cluster using two additional compute nodes.
Basic guidance
Chapter 2: Technology and Deployment Process Overview
20
Dell EMC Ready Stack for Red Hat OpenShift Container Platform 4.6
Enabled by Dell EMC PowerEdge R640 and R740xd Servers; PowerSwitch Networking; PowerMax, PowerScale, Unity XT Storage
Design Guide
Installing OpenShift Container Platform requires, at a minimum, the following nodes:
• One CSAH node, which is used to run the bootstrap VM. The CSAH node is used
later to manage the cluster while the cluster is in production use.
• Three nodes running both the control plane and data plane, enabling customers to
develop OpenShift 4.6 POCs using only four nodes. The cluster can be expanded
with additional compute nodes as needed. However, an initial expansion beyond
three nodes requires two compute nodes. A four-node cluster (three controllers,
one compute) is not supported. The minimum viable solution options are a threenode cluster (three control-compute nodes) or a five-node cluster (three controller
nodes, two compute nodes) plus the CSAH node for cluster administration with
either option.
HA of the key services that make up the OpenShift Container Platform cluster is
necessary to ensure run-time integrity. Redundancy of physical nodes for each cluster
node type is an important aspect of HA for the bare-metal cluster.
In this design guide, HA includes the provisioning of at least two network interface
controllers (NICs) and two network switches that are configured to provide redundant
pathing. The redundant pathing provides for network continuity if a NIC or a network
switch fails.
OpenShift Container Platform 4.6 must use Red Hat Enterprise Linux CoreOS (RHCOS)
for the control-plane nodes and can use either RHCOS or Red Hat Enterprise Linux 7.6
for compute nodes. Using Red Hat Enterprise Linux 7 on the compute nodes is now
deprecated, and the ability to use Red Hat Enterprise Linux 7 compute nodes in OpenShift
will be removed in a future release of OpenShift. The bootstrap and control-plane nodes
must use RHCOS as their operating system. Each of these nodes must be immutable.
The following table shows the minimum resource requirements:
Table 2. Minimum resource requirements for OpenShift Container Platform 4.6 nodes
Node type
Operating system
Minimum CPU
cores
RAM
Storage
CSAH
Red Hat Enterprise
Linux 7.6+
4
32 GB
200 GB
Bootstrap
RHCOS 4.6
4
16 GB
120 GB
Controller
RHCOS 4.6
4
16 GB
120 GB
Compute
RHCOS 4.6 or Red
Hat Enterprise Linux
7.6 (deprecated)
2
8 GB
120 GB
Network connectivity requirements
The RHCOS nodes must fetch ignition files from the Machine Config server. This
operation uses an initramfs-based-node startup for the initial network configuration.
The startup requires a DHCP server to provide a network connection giving access to the
ignition files for that node. Subsequent operations can use static IP addresses.
Minimum viable solution requirements
Loading...