This
sample configurations for a converged infrastructure that is based on
VMware Cloud Foundation and Dell EMC PowerEdge servers, Open
Networking, and all
on PowerEdge Servers and Unity Storage
February 2021
H17667.2
Design Guide
design guide provides architectural guidance, best practices, and
-flash storage.
Dell EMC Solutions
Copyright
2
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular
purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
We value your feedback ..................................................................................... 7
Chapter 1: Introduction
5
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Ready Stack overview
As IT infrastructure in the data center becomes more complex, managing the
infrastructure becomes increasingly difficult. Making changes to the traditional data center
infrastructure can significantly affect stability, performance, costs, and upgradability.
The Dell EMC Ready Stack certified reference system for VMware Cloud Foundation on
PowerEdge Servers and Unity Storage helps organizations meet long-term data center
needs for various mixed workloads. Ready Stack provides the simplicity of a complete, yet
flexible, validated converged infrastructure (CI) that is based on the following components:
• VMware Cloud Foundation virtualization software stack
This Ready Stack provides:
•One trusted vendor for all CI stack components—compute, storage, networking,
virtualization software and management platform, and data protection
•Design and deployment guidance that incorporates scale, flexibility, high
availability, and best practices
Ready Stack key benefits
This Ready Stack provides the following key benefits:
•Resiliency—The Ready Stack architecture ensures that no single point of failure
exists. Redundancy is incorporated in the critical aspects of the Ready Stack, which
includes server high-availability features, redundant networking, and multipath
storage.
•Virtualization support—We designed this Ready Stack for general-purpose
virtualized workloads. Each server is equipped with processing power, memory,
and converged network adapters that support Ethernet and Fibre Channel (FC), as
required for virtualization.
•Scalability—You can configure this Ready Stack to suit your needs for a virtualized
infrastructure. The Ready Stack provides flexibility by supporting various options
including server model, number of compute servers, server processor model, server
memory capacity, type of FC storage, and FC storage capacity.
•Powerful all-flash storage—This Ready Stack includes the Unity x50 All Flash
storage platform. With all-inclusive software, these all-flash systems deliver
consistent performance with fast response times and are ideal for mixed virtual
workload requirements.
•Integrated data protection—The IDPA preconfigured backup solution combines
protection storage and software, search, analytics, and simplified management. The
Design Guide
Chapter 1: Introduction
6
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
IDPA DP4400 appliance that is included in this Ready Stack is optimized for VMware
environments.
•Integrated Dell EMC management—This Ready Stack includes the following Dell
EMC management tools:
Dell EMC OpenManage Integration for VMware vCenter (OMIVV)—A virtual
appliance that enables administrators to view physical host details in VMware
vSphere.
Dell EMC Virtual Storage Integrator (VSI)—A vCenter plug-in that enables
administrators to view, manage, and optimize storage for vSphere servers and
hosts.
band (OOB) management tool for Dell EMC PowerEdge rack servers. iDRAC9
has its own HTML5-based UI and can be accessed by using various methods
including SSH, RACADM, and the Redfish API. iDRAC9 Group Manager
provides a one-to-many console for accessing the details of multiple servers.
As needed, administrators can also use Dell EMC OpenManage Enterprise
(OME) and OME Modular Edition to manage server content that is accessible
through iDRAC9.
Brocade Web Tools—An embedded UI on Dell EMC Connectrix switches that
enables administrators to monitor and manage single or small fabrics, switches,
and ports. Web Tools is launched directly from a web browser or from the
Brocade Network Advisor.
IDPA System Manager—An HTML5-based UI that provides administrators
with configurable dashboards to simplify and automate data protection
management, monitoring, and reporting.
VMware Cloud Foundation
VMware Cloud Foundation is a unified software-defined data center (SDDC) platform for
the hybrid cloud. Composed of VMware vSphere, VMware vSAN, VMware NSX, and
VMware SDDC Manager, VMware Cloud Foundation delivers a natively integrated cloudinfrastructure software stack coupled with a cloud management platform.
SDDC Manager features life-cycle management capabilities that automate critical day 0 to
day 2 operations for the cloud infrastructure stack, enable rapid provisioning and one-click
patching and upgrades, and use policy-based provisioning to simplify resource allocation.
Document purpose
This design guide describes how VMware Cloud Foundation can use Dell EMC compute
and Dell EMC Open Networking with Unity storage. It provides design principles, best
practices, architectural guidance, and sample configurations for compute, management,
networking, and storage.
Chapter 1: Introduction
7
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Audience
This guide is for Dell EMC personnel, channel partners, and end users.
We value your feedback
Dell EMC and the authors of this document welcome your feedback. Contact the Dell
EMC Solutions team by email
documentation survey.
Authors: Robert Percy, Don Pisinski, David Hartman, Karen Johnson
Note: For additional Ready Stack documentation, see the Ready Stack Info Hub.
or provide your comments by completing our
Design Guide
Chapter 2: Architecture and Components
8
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Introduction
Installing, configuring, and running an on-premises CI cloud solution involves multiple
considerations, including:
• Operating system and virtualization software distributions
• Selection of monitoring and management software
• Allocation of cluster services and data storage to physical nodes and arrays
• Selection of appropriate server hardware
• Design of the network fabric
• Sizing and scalability
• Performance requirements
Complicating these considerations is the need to understand the types of workloads that
will run on the cluster and the challenges of managing a system that is designed to
accommodate multiple general-purpose, virtualized workloads.
Dell EMC’s customer-centered approach creates rapidly deployable and highly available
CI systems that run on enterprise-class hardware. Ready Stack designs include all the
hardware, software, resources, and best practices that are needed to run a scalable,
highly available CI environment. With this approach, organizations can be operational in a
shorter time than is typically possible with homegrown solutions.
Infrastructure requirements
Note: Ensure that you have enough VMware licenses for the VMware Cloud Foundation package
being installed and for the number of servers, CPUs, VMs, and vCenter instances that are planned
for the environment. Work with your local Dell EMC or VMware sales team to find the most
appropriate VMware Cloud Foundation package for your solution.
Implementing Ready Stack for VMware vCloud Foundation requires that the following
infrastructure elements are present in the existing data center or configured at the time of
deployment:
• A Domain Name System (DNS) server on the management network
• A Network Time Protocol (NTP) server on the management network
• An Ethernet infrastructure
Dell EMC recommends a 10/25 GbE or 40/100 GbE infrastructure. Additional
components, such as Dell Networking cables and transceivers, are required to
uplink the Ready Stack to the network. The specific component requirements
depend on your networking and uplink requirements.
•Sufficient power and cooling to support the Ready Stack components
Contact your Dell EMC Sales representative for more information about the power
and cooling requirements.
Design Guide
Chapter 2: Architecture and Components
10
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Compute
Ready Stack architecture
This Ready Stack architecture includes compute servers, data protection, management
servers, LAN switches, SAN switches, storage arrays, and an OOB switch, as shown in
the following figure.
Figure 1. Ready Stack design architecture
Key components
The Dell EMC portfolio of PowerEdge rack servers is designed to optimize application
performance. The servers support intuitive tools to simplify and automate tasks over the
entire server life cycle. Every data center and virtual environment has unique
requirements, and PowerEdge technology provides the flexibility that is needed to build a
scalable vSphere infrastructure.
This Ready Stack supports the following PowerEdge servers:
• PowerEdge R640―A 1U 2-socket server platform that supports up to 24 DIMM
• PowerEdge R740/740xd― A 2U, 2-socket server platform that supports up to 24
slots, ten 2.5 in. drives, and three PCIe cards. The scalable business architecture of
the R640 server maximizes application performance and provides the flexibility to
optimize configurations that are based on the application and use case.
DIMM slots, thirty-two 2.5 in. drives, and eight PCIe cards and delivers a balance
between storage scalability and performance. This platform is ideal for softwaredefined storage (SDS), service providers, and virtual desktop infrastructures. The
Chapter 2: Architecture and Components
11
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Networking
PowerEdge R740 server accelerates application performance by using accelerator
cards and storage scalability. The versatile R740xd server supports a mix of any
drive types to create the optimum configuration of NVMe drives, SSDs, and HDDs
for either performance or capacity, or both.
Embedded in each PowerEdge server in Ready Stack for VMware vCloud Foundation is
iDRAC9, which provides secure and remote server access for a multitude of common
management functions. iDRAC with Lifecycle Controller operates regardless of the
operating system state or the presence of a hypervisor. It also offers a complete set of
server management features, including configuration, operating system deployment,
firmware updates, health monitoring, and maintenance.
iDRAC9 supports various remote connectivity interfaces and protocols, including IPMI,
Redfish, SMASH-CLP, and WS-Man, enabling administrators to securely configure,
deploy, manage, monitor, and update the server. The iDRAC9 HTLM5 Web UI provides
secure connectivity through HTTPS. The iDRAC9 Enterprise license includes Group
Manager, which provides a one-to-many console experience. With Group Manager,
administrators can view and manage a set of servers rather than visually inspect the
servers for faults and manage them manually.
The Ready Stack architecture network design consists of these functional groups:
• OOB network
• LAN
• SAN
OOB network
The key building block of the OOB network is the Dell EMC S4148T-ON management
switch, which provides forty-eight 10GBase-T ports and multiple uplink port options.
LAN
The Dell EMC S5248-ON switch provides LAN connectivity. This switch provides up to
forty-eight 25 GbE ports and six 100 GbE ports. Using two S5248-ON switches—referred
to as top-of-rack (ToR) switches—provides redundancy. Built-in NICs directly connect the
ToR switches to the rackmount servers.
SAN
For FC storage traffic, Ready Stack uses Dell EMC Connectrix DS-6620B switches. The
DS-6620B switch delivers up to 32 Gb/s FC performance in a 1U form factor and, by
default, has twenty-four 32 Gb/s ports enabled. More ports can be added, through Ports
on Demand upgrade kits, up to 64 ports per chassis. Brocade Web Tools, the embedded
UI on the Connectrix DS-6620B switches, facilitates the monitoring and management of
single or small fabrics, switches, and ports.
Design Guide
Chapter 2: Architecture and Components
12
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Storage
Dell EMC Unity All Flash storage systems offer simplicity, modern design, and flexible
deployments. Unity All Flash storage systems implement an integrated architecture for
block, file, and virtual hard disks, with concurrent support for native NAS, iSCSI, and FC
protocols. Each system uses dual storage processors, full 12 Gb SAS back-end
connectivity, and Dell EMC’s patented multicore architected operating environment to
deliver performance and efficiency. Disk array enclosures (DAEs) provide additional
storage capacity, and online controller upgrades are available for additional performance.
Unity all-flash technology supports:
• File and block environments
• Point-in-time snapshots
• Thin clones
• Synchronous and asynchronous replication
• Integrated encryption
• Tiering to the cloud
• Deep ecosystem integration with VMware, Microsoft, and OpenStack software
Although this guide describes VMware Cloud Foundation with external FC-attached Unity
storage, you can also use an FC-attached external Dell EMC PowerMax array or
externally attached NFS Unity storage. For information about these options, see
Storage with VMware Cloud Foundation.
Dell EMC
Data reduction
Dell EMC data reduction includes deduplication and compression logic. Unity data
reduction features are available for block LUNs and datastores in an all-flash pool.
Administrators can enable Unity data reduction at the resource level so that storage
resources with and without enabled data reduction can exist in the same pool on a
system. Data reduction savings also can be set at the system level, pool level, and other
levels.
Dynamic pools
Dynamic pools present an entirely redesigned pool structure that increases the flexibility
of configuration options within a Unity system. Dynamic pools replace the existing pool
technology as the default pool type within the Unisphere management platform for Unity
All Flash systems. As with traditional pools, dynamic pools can be created, expanded, and
deleted, but they also offer improvements. Expansion of a dynamic pool can be as
granular as adding a single drive to the pool to increase its capacity. This feature provides
a fully flexible deployment model, which improves the planning and provisioning process
and reduces the total cost of ownership. For more information, see the
Dynamic Pools White Paper.
Dell EMC Unity:
Host I/O activity limits
The Host I/O Limits feature of the Unity array restricts the amount of I/O activity that the
Unity system services. Limits can be applied on LUNs, vSphere VMFS datastores, and
thin clones and their associated attached snapshots. Use the Host I/O Limits feature to
limit incoming host activity based on IOPS, bandwidth, or both.
Chapter 2: Architecture and Components
13
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
VMware Cloud
Foundation
You can configure Host I/O Limits on LUNs or datastores that are consuming a large
portion of the system’s resources and reducing the performance of other system
resources. A burst policies option, which is available for any Host I/O Limits policy,
enables host activity or traffic to exceed the base policy limit by a user-specified amount.
Specify the amount as a percentage along with the duration and frequency of the I/O limit
burst allowance.
Unisphere storage management
For vSphere hosts, Unisphere storage management for Unity arrays uses integration
points to reduce required steps and performs several host-side functions. Unisphere
storage management for Unity arrays includes an HTML5 UI that displays NFS, vSphere
VMFS, and vSphere Virtual Volumes datastore storage and enables administrators to
present storage to all host servers. The UI also displays system status and performance
information, including a graphical representation of the Unity system, highlighting areas of
interest such as drive faults and network link failures.
VMware Cloud Foundation is an integrated software stack that bundles compute
virtualization (VMware vSphere), storage virtualization (VMware vSAN), network
virtualization (VMware NSX), and cloud management and monitoring (VMware vRealize
Suite) into a single platform. The platform also includes SDDC Manager, which helps
make the SDDC possible.
VMware Cloud Foundation can be deployed on the premises as a private cloud or run as
a service within a public cloud, enabling a consistent operating model from the onpremises private cloud to the public cloud. With on-premises deployment of VMware
Cloud Foundation, you can use existing infrastructure investments to enhance your
private-cloud capabilities.
VMware Cloud Foundation uses on-board disks from the PowerEdge servers in a vSAN
configuration as the primary storage. In this Ready Stack certified reference system,
VMware Cloud Foundation uses Unity resources as secondary, FC-attached storage for
its workload domains. By supporting external storage, VMware Cloud Foundation enables
you to meet the demands of applications that require more storage resources but do not
necessarily require more compute resources. VMware Cloud Foundation support for Unity
storage enables you to use existing storage assets as you deploy applications in new
workload domains.
VMware Cloud Foundation includes the following components:
•vSphere—A suite of server virtualization tools that includes the VMware ESXi
hypervisor and vCenter management software.
•NSX—Networking and security software that provisions virtual networking
environments by abstracting network operations from the underlying hardware onto
a distributed virtualization layer.
•vSAN—A distributed layer of software that is included in the ESXi hypervisor and
fully integrated in vSphere. vSAN pools direct-attached storage devices across a
vSphere cluster to create a distributed shared datastore.
•SDDC Manager—A multithreaded execution engine that provides built-in
automated life-cycle management of resources.
Design Guide
Chapter 2: Architecture and Components
14
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Resource requirements
In this Ready Stack certified reference system, a vSAN ReadyNode is a validated server
configuration in a tested, certified hardware form factor for vSAN deployment, jointly
recommended by Dell EMC and VMware. Each vSAN ReadyNode profile specifies a
server's size; type and quantity of CPUs, memory, network, and I/O controllers; and type
of disks. Thus, you can choose the vSAN ReadyNode profiles that best suit your
application workloads.
Although vSAN ReadyNodes can help accelerate vSAN deployment, they are optional.
Dell EMC servers can be customized to meet your requirements. You can add resources
such as more powerful CPUs, RAM, and storage, provided that the resources meet the
minimum requirements for similar vSAN Ready Nodes.
For information about the minimum hardware requirements (server and networking
requirements, for example) for running the individual software products in VMware Cloud
Foundation, see the VMware Compatibility Guide
Note: This Ready Stack certified reference system supports VMware Cloud Foundation 3.5. The
VMware Compatibility Guide does not list vSAN ReadyNodes for VMware Cloud Foundation 3.x;
therefore, refer to the individual components along with the corresponding VMware software
product.
.
Licensing
Ensure that you have the required number of software licenses. You cannot create or
expand workload domains if you do not enter license keys for the following products:
• VMware vCloud Suite
• vSAN
• vCenter Server
• NSX for vSphere
• SDDC Manager
• vRealize Log Insight
Ensure that you have enough VMware licenses for the VMware Cloud Foundation
package being installed and for the number of servers, CPUs, VMs, and vCenter
instances that are planned for the environment. Work with your Dell EMC or VMware
sales team to find the most appropriate VMware Cloud Foundation package for your
solution and to ensure that you have the required number of licenses.
Chapter 2: Architecture and Components
15
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Virtualization software requirements
To support VMware Cloud Foundation, the components in this Ready Stack certified
reference system require the virtualization software that is listed in the following table.
Table 1. Ready Stack components and their VMware Cloud Foundation roles
Dell EMC component Purpose
4–32 x PowerEdge R740
servers
1 x S4148T-ON switch OOB management switch NSX-V 6.4.4
4 (min.) x PowerEdge R640
servers
2 x S5248-ON switches ToR leaf switches NSX-V 6.4.4
1 (min.) x Z9264F-ON
switch
2 x Connectrix DS-6620B
switches
Unity storage arrays Secondary storage for
VMware Cloud Foundation
workload domain servers
(per rack)
VMware Cloud Foundation
management domain
servers
Spine switches for multirack scale-out
FC SAN switches for
workload domains
workload domains
Minimum virtualization
software required
vSphere 6.7u1
vSphere 6.7u1
NSX-V 6.4.4
Not applicable
vSphere 6.7u1
For additional details, see the following VMware knowledge base article:
Correlating VMware Cloud Foundation version with the versions of its constituent products
(52520)
vSAN
vSAN is an object-based, in-kernel solution that uses vSphere Storage Policy Based
Management (SPBM) to provide storage on a per-VM basis. vSAN enables you to
address the needs of specific VMs and their application workload requirements without
directly modifying or enhancing the services that are provided by the underlying storage
array. vSAN runs in the kernel of each ESXi host in a cluster, and all hosts in the cluster
share a single vSAN datastore.
As shown in the following figure, each cluster in VMware Cloud Foundation is a domain.
One cluster is a dedicated management domain, and one or more clusters are workload
domains that contain deployed applications. Workload domains are policy-based resource
containers with specific availability and performance attributes that combine compute
(vSphere), storage (vSAN), and networking (NSX) into a single consumable entity.
Design Guide
Chapter 2: Architecture and Components
16
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Figure 2. VMware Cloud Foundation with Dell EMC Storage
Each workload domain instantiates a dedicated vSAN cluster with its own performance
and availability characteristics, which enables multiple workload domains to be deployed
in the cloud, each supporting different service-level agreements.
The local storage on each host is then configured into a pair of disk groups. These disk
groups from all servers in the workload domain are then aggregated into a single vSAN
datastore.
You can extend storage capacity by expanding a workload domain. As SDDC Manager
adds more hosts to the cluster, it automatically extends the VMware vSAN datastore. The
external Unity storage is managed by using native storage tools (Dell EMC Unisphere) or
the VSI plug-in to vCenter; however, it does not participate in the automated functions of
VMware Cloud Foundation SDDC Manager.
VMware Cloud Foundation networking
VMware Cloud Foundation is seamlessly deployed into your existing data center and
requires no modification to the physical network. Its software-defined networking
subsystem provides network virtualization with VMware NSX. VMware Cloud Foundation
supports both L2 and L3 uplink connectivity.
When creating the private cloud (that is, the management domain), VMware Cloud
Foundation SDDC Manager configures a link aggregation group (LAG) to provide
bandwidth aggregation and to protect against NIC failures. Then, when creating workload
domains, SDDC Manager:
•Allocates cloud capacity into workload domains. Each workload domain consists of
a dedicated vSphere cluster and is managed by its own vCenter Server.
•Creates a vSphere Distributed Switch that has multiple VLAN-backed port groups to
logically separate and isolate the different traffic types.
•Automatically deploys and configures NSX, which includes modern virtual
networking and advanced security services such as virtual extensible LAN
(VXLAN), virtual-to-physical routing, Distributed Firewall (DFW), and distributed
Chapter 2: Architecture and Components
17
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
routing for east-west routing and security. NSX edge service gateways can also be
deployed to manage north-south routing along with load balancing and NAT
services for workloads running inside the workload domain.
You can increase capacity later by adding servers within a rack (scale up) or adding racks
(scale out). In larger environments where multiple racks are deployed, racks are
connected to each other from the ToR switches to a pair of redundant inter-rack switches
in a dual LAG configuration, using separate 25 Gb or 40 Gb uplinks depending on the
switch port types available. The network automatically scales, to the maximum number of
connections available per spine pair, as you increase capacity. The result is a highperformance, resilient, and scalable network.
Unity integration with VMware Cloud Foundation
The Unity family of storage arrays provides numerous native features and tools within the
Unisphere platform, as well as integration with various VMware APIs for enhanced
storage resource management and utilization. Unity arrays support vSphere Storage APIs
– Array Integration (VAAI), which defines a set of storage primitives that enable the ESXi
hosts to offload certain storage operations to the array. VAAI reduces resource overhead
on the ESXi hosts and can significantly improve performance for storage-intensive
operations such as storage cloning and zeroing.
Unity arrays also support VMware vSphere APIs – Storage Awareness (VASA). VASA is a
set of VMware APIs that enables storage arrays to integrate with vCenter for management
operations. Storage monitoring and reporting activities that are typically done within the
Unisphere platform are available within vCenter, providing administrators with a single
interface from which to manage Unity storage within the VMware environment. vCenter
captures information regarding the features of the physical Unity storage arrays, including
health status, capacity utilization, and configuration capabilities, which can significantly
simplify storage administration and workflow provisioning within a VMware environment.
VMware Cloud Foundation with FC storage
Consider the following information when using FC storage with VMware Cloud
Foundation:
•Presenting FC datastores to the management domain is not supported. Present FC
datastores to workload domains only.
•When external storage is presented to the vSphere hosts, FC datastores are
considered ancillary/secondary storage for workload domains. Therefore, vSAN or
NFS storage is required for the initial deployment of a workload domain. After initial
deployment, you can make the FC datastores available to the workload hosts.
•Although vSAN and NFS storage can be managed from within the SDDC Manager
interface, all FC datastores must be managed from the vCenter instance in the
workload domain through the VSI plug-in. This external storage does not participate
in the automated SDDC Manager functions.
This design guide describes how to use vSAN as primary storage with VMware Cloud
Foundation. If you prefer to use NFS storage as primary storage for workload domains,
see Dell EMC Storage with VMware Cloud Foundation
.
Design Guide
Chapter 2: Architecture and Components
18
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Private cloud deployment process overview
The following steps outline the procedure for deploying a private cloud, including
assembling the physical Ready Stack components, imaging the servers, and creating the
management domain. For detailed information, see
Architecture and Deployment Guide in the VMware Docs library.
1. Deploy the VMware Cloud Foundation Builder VM.
The VMware Cloud Foundation Builder VM includes the VMware Imaging
Appliance that you can use to install ESXi on your servers.
2. Install ESXi software on VMware Cloud Foundation servers.
To image servers for use in the management domain and workload domains, do
one of the following:
Use the VMware Imaging Appliance (VIA) that is included with the VMware
Cloud Foundation Builder VM.
Image the servers with existing tools or processes you have, provided that the
documented host prerequisite configurations are already in place.
3. Download the Deployment Parameter Sheet from the VMware Cloud Foundation
Builder VM and complete it.
VMware Cloud Foundation
The Deployment Parameter Sheet is a multi-tab spreadsheet in which you specify
the required deployment information for your environment. This information
includes hosts, license keys, DNS, network (VLANS, IPs, MTUs), and credentials.
The completed spreadsheet is converted to a JSON file, and the VMware Cloud
Foundation Builder VM then validates the network information that you provide.
You can reuse the Deployment Parameter Sheet to deploy multiple VMware
Cloud Foundation instances of the same version.
4. Initiate the VMware Cloud Foundation bring-up process.
During the bring-up process, the management domain is created on the ESXi
hosts that are specified in the Deployment Parameter Sheet. The VMware Cloud
Foundation software components are automatically deployed, configured, and
licensed using the information that you provided.
When the bring-up process is complete, the VMware Cloud Foundation Builder VM
transfers inventory and control to SDDC Manager and you can begin creating workload
domains.
Creating workload domains
In SDDC Manager, you can select from various workload domain types, depending on the
requirements of the applications you intend to deploy on your VMs. During this process,
you also select from among the available licenses for NSX and vSphere.
After initial deployment, VMware Cloud Foundation makes it easy to add capacity to the
private cloud by adding new ReadyNodes, or their equivalent, as you scale up resources
within existing racks or scale out into multiple racks.
Chapter 3: Configuration and Specifications
19
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Introduction
This chapter provides details about the recommended platforms for Ready Stack for
VMware Cloud Foundation, component specifications for three sample configurations, and
information about scaling Ready Stack.
Recommended components
Ready Stack for VMware Cloud Foundation uses the latest Dell EMC PowerEdge rack
servers, LAN switches, SAN switches, and all-flash Unity storage arrays. It also includes
platform management software.
The following table lists the recommended components.
Table 2. Recommended components
Platform/category Components
Workload domain server platform Dell EMC PowerEdge R740/R740xd
LAN connectivity for Ethernet traffic
SAN connectivity
OOB connectivity 1 x Dell EMC Networking S4148T-ON switch
Storage array Unity 350F, 450F, 550F, or 650F
Management domain server platform Minimum of 3 x PowerEdge R640
Data protection IDPA DP4400
Server and storage management software
Virtualization software
• For leaf: 2 x Dell EMC Networking S5248ON switches
• For spine (if needed): Dell EMC
Networking Z9264F-ON switch
2 x Dell EMC Connectrix DS-6620B 32 Gb/s
FC switches
• VMware vCenter
• Dell EMC OpenManage Integration for
VMware vCenter
• Dell EMC Virtual Storage Integrator
• iDRAC9
VMware Cloud Foundation base, which
includes:
• SDDC Manager
• NSX Data Center Professional
• vSAN Advances
• vSphere Enterprise Plus
Chapter 3: Configuration and Specifications
21
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Overview
Sample
configuration
Design configuration and specifications
Ready Stack for VMware Cloud Foundation offers flexibility to address various
virtualization use cases and workloads. Sizing a physical infrastructure for all virtualized
workloads requires knowledge of all the individual workloads and their compute, memory,
storage, and network performance requirements. After you create one or more VM profiles
that meet the Ready Stack requirements, approximate the physical infrastructure that is
required to support VMs and account for expected future growth. When finalizing sizing,
ensure that you also consider server and storage maintenance requirements and
virtualization features such as high availability. If an approximation is not sufficient, we
recommend a site survey.
The following table details a medium-size configuration of Ready Stack for VMware Cloud
Foundation. The configuration is based on a balanced VM profile of 2 vCPUs, 6 GB of
memory, a 60 GB virtual disk, and 50 IOPS.
Table 3. Ready Stack sample configuration
Configuration
Enterprise Medium 240 480 1.44 TB 14.4 TB 12,000
Number of
VMs
Number of
vCPUs
vRAM size vDisk size IOPS
The balanced VM profile totals for VMs, vCPUs, vRAM, vDisk, and IOPS are based on an
Intel SP Skylake generation processor configuration of 10 cores per CPU and a 2:1
virtual-to-physical CPU utilization ratio. Depending on the VM profile, the required
compute resource values might be different from the values that are shown in the
preceding table. For more information about tuning the Ready Stack product configuration
and counts to meet infrastructure requirements, consult your Dell EMC Sales
representative.
The following table lists the component specifications for a sample Ready Stack
configuration.
Dell EMC Unity 450F with 25 x 3.84 TB SSDs (RAID 5, 6, 10)
secondary storage for
servers/VMs in workload
domain
Networking OOB 1 x Dell EMC Networking S4148T-ON switch
Licenses
ToR 2 x Dell EMC Networking S5248-ON switches
SAN fabric 2 x Connectrix DS-6620B 32 Gb/s FC switches
iDRAC9 Enterprise—1 per PowerEdge server
OpenManage Integration for VMware vCenter2—1 per PowerEdge server
1
Specifications are for a minimum configuration.
2
3-year license.
Note: Ensure that you have enough VMware licenses for the VMware Cloud Foundation package
being installed and for the number of servers, CPUs, VMs, and vCenter instances that are planned
for the environment. Work with your local Dell EMC or VMware sales team to find the most
appropriate VMware Cloud Foundation package for your solution.
Chapter 3: Configuration and Specifications
23
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Ready Stack scaling
Scale the Ready Stack by adding multiple compute nodes (pods) in the data center. You
can use Dell EMC Networking Z9264 switches to create a simple yet scalable network, as
shown in the following figure. The Z9264 switches serve as the spine switches in the leafspine architecture. The Z9264 is a multiline rate switch that supports 10/25/40/50/100
GbE connectivity and can aggregate multiple racks with little or no oversubscription.
Figure 3. Multiple compute pods scaled out by using leaf-spine architecture
When connecting multiple racks by using the 40/100 GbE uplinks from the rack, you can
build a large fabric that supports multi-terabit clusters. The density of the Z9264 allows
flattening of the network tiers, enabling you to create an equal-cost fabric from any point to
any other point in the network.
For large-domain Layer 2 requirements, use extended Virtual Link Trunking (eVLT) on the
Z9264, as shown in the following figure. The VLT pair that is formed can scale to
hundreds of servers inside multiple racks. Each rack has four 40/100 GbE links to the core
network, providing enough bandwidth for all the traffic between each rack.
Design Guide
Chapter 3: Configuration and Specifications
24
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Figure 4. Multiple compute pods scaled out by using eVLT
Chapter 4: Design
25
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Compute server
configuration
Introduction
This chapter summarizes the design of all the main components of a production Ready
Stack, including the physical compute server hardware, network fabric, storage
infrastructure, software layers, and ongoing management.
Compute design
The latest Intel SP Skylake generation processors power Dell EMC Ready Stack. With up
to 28 cores per CPU and clock speeds of up to 3.6 GHz in the PowerEdge rack servers,
the processors reduce CPU socket-based licensing costs and provide greater VM density.
Scaling compute resources with Intel Xeon scalable processors delivers a 27 percent
increase in processing cores and a 50 percent increase in bandwidth over previous
generations of Xeon processors.
The PowerEdge rack server platforms support the RDIMM and Load Reduced DIMM
(LRDIMM) memory types. LRDIMM uses an internal buffer to isolate electrical loading
from the host memory controller, enabling the use of quad-ranked DIMMs to increase
overall memory capacity. For general-purpose virtualization solutions, we recommend
2,666 MT/s RDIMMs. You can configure memory in various modes from within the BIOS.
To provide an optimized memory performance, the default Optimizer Mode works best for
most virtualization use cases. Mirror Mode, Single Rank Spare Mode, and Multi Rank
Spare Mode are available for improved reliability and resiliency.
The following table lists the recommended adapters.
Table 5. Recommended network and HBA adapters
Adapter Purpose
Intel X710 quad-port 10 GbE Ethernet Virtual workload and host communication
QLogic 2692 dual-port 16 Gb FC FC SAN connectivity for PowerEdge servers
in the workload domain that connect to Unity
storage
PowerEdge servers support various BIOS configuration profiles that control the processor,
memory, and other options. Dell EMC recommends that you:
• Enable the Performance Optimized virtualization profile.
• Disable the processor C States and C1E options to ensure the highest performance
in a virtualized environment.
•Enable Intel Hyper-Threading and virtualization features.
The Dell server platforms recommended for Ready Stack for VMware Cloud Foundation,
Dell PowerEdge R640 and R740, are configured with specifications similar to their vSAN
ReadyNode counterpart servers. If the Dell servers meet or exceed the specifications for
their counterpart vSAN ReadyNode server model types, the use of vSAN ReadyNodes is
not required. For specification details, see Table 4, Sample configuration: Component
specifications, on page 21.
Chapter 4: Design
27
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Dell EMC Open
Networking
Network traffic
Network design
This section provides an overview of the network architecture, including compute and
management server connectivity, and configuration details for the ToR and virtual
switches.
Open Networking is a core element of Dell EMC’s networking strategy and mission. Open
Networking disaggregates the hardware from the operating system, giving you the choice
of picking the operating system that best fits your unique network infrastructure needs.
Open Networking is achieved by using standards-based building blocks that use open
source where possible.
The Open Network Install Environment (ONIE) is an open-source project within the Open
Computer Project that provides an installation environment for bare-metal networking
switches. ONIE also provides the option to install different network operating systems on a
common set of networking hardware. ONIE is like any operating system boot loader for
Windows or Linux and uses the same GRand Unified Bootloader (GRUB) structure that is
commonly found in Linux distributions.
In the Dell EMC Networking portfolio, any switch model with an “-ON” suffix, such as the
Dell EMC Networking Z9264F-ON and the Dell EMC Networking S5248-ON switches, has
ONIE enabled.
For information about ONIE, see http://onie.org/
The network architecture employs a VLT connection between the two ToR switches. The
inherent redundancy of a non-VLT environment requires standby equipment, which
increases infrastructure costs and risks. In a VLT environment, all paths are active, adding
immediate value and throughput while protecting against hardware failures. VLT
technology enables a server or bridge to uplink a physical trunk into more than one
Networking S5248-ON switch by treating the uplink as one logical trunk. A VLT-connected
pair of switches acts as a single switch to a connecting bridge or server. Both links from
the bridge network can forward and receive traffic. VLT provides a replacement for
Spanning Tree Protocol (STP)-based networks by using multiple active paths to provide
both redundancy and full bandwidth utilization. The major benefits of VLT technology are:
• Dual control plane for highly available resilient network services
• Full utilization of the active LAG interfaces
• Active/active design for seamless operations during maintenance events
The Dell EMC Networking S5248-ON switches each provide six 40/100 GbE uplink ports.
The VLT interconnect (VLTi) configuration in this architecture uses two 40/100 GbE ports
from each ToR switch to provide a 200 GB data path between the switches.
.
Design Guide
Chapter 4: Design
28
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
vSphere
Distributed
Switch
configuration for
LAN traffic
vSphere
Distributed
Switch
configuration for
LAN traffic
The following figure illustrates the Networking S5248-ON VLTi configuration.
Figure 5. Networking S5248-ON VLTi configuration
The Dell EMC network switching infrastructure uses VMware NSX network virtualization,
which is part of the VMware Cloud Foundation software infrastructure.
NSX technology enables the decoupling of network services from the physical
infrastructure. Logical networks are created on top of a basic Layer 2 (switched) or Layer
3 (routed) physical infrastructure to decouple the physical and virtual environments. This
decoupling enables agility and security in the virtual environment while allowing the
physical environment to focus on throughput.
The NSX platform also provides for network services in the logical space, including
switching, routing, firewalls, load balancing, and VPN services.
NSX provides:
• Simplified network service deployment, migration, and automation
• Reduced provisioning and deployment time
• Scalable multi-tenancy across one or more data centers
• Distributed routing and a distributed firewall at the hypervisor, to enable better east-
to-west traffic flow and an enhanced security model
•Solutions for traditional networking problems, such as limited VLANs, MAC
addresses, and FIB and ARP entries
•Normalization of underlying hardware, enabling easier hardware migration and
interoperability
In addition, application requirements do not require modification to the physical network.
You can achieve bandwidth prioritization for different traffic classes such as host
management, vSphere vMotion, and the VM network by using vSphere Distributed
Switch (VDS). VDS, which can be configured, managed, and monitored from a central
interface, provides:
• Simplified VM network configuration
• Enhanced network monitoring and troubleshooting capabilities
• Support for network bandwidth partitioning
Chapter 4: Design
29
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Unity All Flash
array models
The following figure shows the VDS configuration for the management and compute servers.
Figure 6. vSphere Distributed Switch for dual-port configuration
Storage design
Ready Stack for VMware Cloud Foundation uses the Unity x50F storage platform that is
integrated with vSphere and connected to a 32 Gb FC storage network. The Unity x50F
platform is a flexible storage product that is designed for all-flash needs. Unity All Flash
systems deliver consistent performance with fast response times and are ideal for mixed
virtual workload requirements. Two Connectrix DS-6620B switches comprise the FC
fabrics.
VMware Cloud Foundation enables you to deploy and manage the on-board vSAN-based
storage. When you create a workload domain, SDDC Manager virtualizes the on-board
storage resources of the PowerEdge servers and combines them into a single vSAN. The
storage resources in this single vSAN can then be apportioned according to the needs of
specific applications in the workload domain. After the workload domain is deployed, all
ESXi hosts in the cluster share a single vSAN datastore.
Accordingly, VMware Cloud Foundation uses on-board storage for its management
domain and for the workload domains on which applications are deployed. For
secondary/ancillary storage, VMware Cloud Foundation supports either on-board storage
or external storage. This guide describes using VMware Cloud Foundation with Unity
storage arrays, but you can also use Dell EMC PowerMax arrays for external storage.
The following table shows a comparison of the Unity x50F storage arrays that the Ready
Stack supports. The Disk Processor Enclosure (DPE) contains up to 25 SSD drives and
includes two storage processors. You can use additional drive enclosures to expand
overall storage capacity.
Design Guide
Chapter 4: Design
30
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
CNA ports per array 4 additional ports per I/O module
Maximum raw capacity
(drive type dependent)
IOPS (100% reads, 8K
blocks)
I/O modules per array Up to 4 Up to 4 Up to 4 Up to 4
*PCIe lane. For more information, see the Dell EMC Unity Hybrid Storage Specification Sheet.
6/150 6/250 6/500 6/1000
4 x embedded
4-lane* 12 Gb/s
SAS
2.4 PB 4.0 PB 8.0 PB 16.0 PB
Up to 130 K Up to 305 K Up to 395 K Up to 440 K
4 x embedded
4-lane 12 Gb/s
SAS
4 x embedded
4-lane 12 Gb/s
SAS
6 x 4-lane, or 2 x
4-lane with I/O
module
4 x embedded
4-lane 12 Gb/s
SAS
6 x 4-lane, or 2 x
4-lane with I/O
module
Ready Stack for VMware Cloud Foundation is configured with two FC fabrics for high
availability.
The following figure illustrates the storage fabric configuration. For the Unity x50F arrays,
FC port 0 from each controller connects to the FC fabric switch A, while port 1 connects to
FC fabric switch B. Unity x50F arrays have expansion slots (not shown in the following
figure) that can provide additional front-end (FC) or back-end (mini-SAS HD) ports. For
more information, see the Dell EMC Unity Family Hardware Information Guide
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Storage
connectivity for
compute servers
Unity x50F
enclosures
Each workload domain server that will have additional storage from the Unity array is
configured with a QLogic 2692 dual-port FC adapter for connecting to the storage fabrics,
with each port connecting to the Connectrix switch, as shown in the following figure.
Figure 8. PowerEdge rack server SAN connectivity
Unity x50F series storage arrays are equipped with two back-end buses that use mini-SAS
HD connectivity, as shown in the following figure. Connect any additional enclosures so that
the load is balanced equally between the available buses. Expansion slots on the Unity x50F
arrays (not shown in the following figure) can provide additional front-end or back-end (miniSAS HD) ports. The DPE is on bus 0. Therefore, the first expansion enclosure must be on bus
1 while the second expansion enclosure must be on bus 0, and so on.
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Storage
configuration
Unity FC storage
with VMware
Cloud Foundation
Unity arrays and
VMware vSphere
Management
infrastructure
Management
server
For both rack server platforms, each server’s HBA port 1 connects to Connectrix switch 1,
while HBA port 2 connects to Connectrix switch 2. These ports are then zoned with the
Unity array target ports to enable storage access for the hypervisor hosts.
For important considerations when using FC storage with VMware Cloud Foundation, see
VMware Cloud Foundation with FC storageon page 17.
Unity All Flash systems support thin provisioning with VMware ESXi. For details about
how to take advantage of thin provisioning and about using vSphere with Unity storage,
see Dell EMC Unity Storage with VMware vSphere and
Guide.
Dell EMC Unity: Best Practices
Management design
The management of this Ready Stack design centers around VMware Cloud Foundation
management tools such as SDDC Manager and vCenter. Additional plug-ins such as Dell
EMC OpenManage Integration for VMware vCenter (OMIVV) and iDRAC provide ease of
management for the physical PowerEdge servers. Dell EMC Virtual Storage Integrator
(VSI) provides ease of management for the Unity storage.
The following figure illustrates the management infrastructure.
Figure 10. Management infrastructure
The management system consists of at least three PowerEdge R640 servers that are
configured as the management domain within VMware Cloud Foundation. The
management domain then deploys the VMs and provides ongoing life-cycle management
through the SDDC Manager.
Chapter 4: Design
33
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
VMware vCenter
Server Appliance
Dell EMC Virtual
Storage
Integrator
Dell EMC
Op
Integration for
VMware vCenter
vCenter Server Appliance 6.7 is a preconfigured Linux VM that is optimized for running
vCenter Server and its associated services. The appliance package contains the following
software:
• Project Photon OS 1.0
• PostgreSQL database
• vCenter Server 6.7 and vCenter Server 6.7 components
• VMware Platform Services Controller, which bundles all the necessary services for
running vCenter Server, such as vCenter Single Sign On, vSphere License Service,
and VMware Certificate Authority
vCenter Server provides management of virtualized hosts and VMs from a single console.
It gives administrators visibility into the configuration of the critical components of a virtual
infrastructure. vCenter Server enables key capabilities such as vSphere vMotion, vSphere
Distributed Resource Scheduler (DRS), vSphere High Availability (HA), and vSphere Fault
Tolerance (FT). The open plug-in architecture of vCenter Server supports a broad range
of additional capabilities from VMware and its partners. The vCenter Server APIs also
enable integration of physical and virtual management tools for maximum flexibility.
VSI for VMware vSphere Web Client is a plug-in for VMware vCenter. It enables
administrators to view, manage, and optimize storage for vSphere servers and hosts, and
then map that storage to the hosts. VSI consists of a UI and the Dell EMC Solutions
Integration Service (SIS), which provides communication and access to the storage
systems.
enManage
Using the SIS, a storage administrator can enable VM administrators to perform
management tasks on a set of storage pools. Tasks that you can perform with VSI include
provisioning datastores, reclaiming space, and setting multipathing policies.
During installation, VSI automatically registers Dell EMC PowerPath/VE software,
enabling you to set multipathing policies for all devices by using the VMware Native
Multipathing Plug-in (NMP) or PowerPath/VE software. After VIS is installed, you can
provision VMFS6 datastores in vCenter inventory lists in data centers, folders, clusters,
and hosts. Additionally, using the space reclamation feature in VSI, you can reclaim
unused storage on datastores, hosts, clusters, folders, and storage folders. Space
reclamation tasks are created per datastore. Scheduling from the host, cluster, folder, or
storage folder level results in one scheduled task per datastore under the target object.
Only one scheduled reclamation task can exist for each datastore.
OMIVV streamlines the management processes in the data center environment by
enabling the use of vCenter to manage the entire server infrastructure—both physical and
virtual. OMIVV expands data center management with PowerEdge servers through many
ways, including:
• Monitoring system-level information
• Producing system alerts
• Rolling out cluster-level BIOS and firmware updates for an ESXi cluster
• Supporting bare-metal deployments
Design Guide
Chapter 4: Design
34
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Management
software
resources
OMIVV provides deep-level details for inventory, monitoring, and alerting of Dell EMC
hosts within vCenter, and it recommends or performs vCenter actions based on Dell EMC
hardware events.
Proactive HA is a feature of vSphere HA that works with OMIVV. Proactive HA safeguards
workloads by proactively taking measures based on degradation of redundancy health of
supported components in a host. When OMIVV detects a change in the redundancy
health status of supported components, through either traps or polling, the health update
notification for the component is sent to the vCenter server. Polling runs every hour, and it
is available as a fail-safe mechanism to cover the possibility of a trap loss. After assessing
the redundancy health status of the supported host components, the OMIVV appliance
updates the vCenter server with the health status change.
Deployment templates contain a system profile, hardware profile, hypervisor profile, a
combination of system profile and hypervisor profile, or a combination of the hardware
profile and hypervisor profile. The OMIVV deployment wizard uses the template to
provision server hardware and deploy hosts within vCenter. Dell EMC recommends using
the system profile type for the servers that are described in this Ready Stack certified
reference system. For Dell EMC Internal Dual SDM Module (IDSDM) deployment, enable
the IDSDM from the BIOS before you deploy a hypervisor with OMIVV.
The management software components run on VMs that reside in the management
cluster. The following table shows the recommended VM sizing of the management
components.
Table 7. Recommended VM sizing of management components
Component
vCenter Server
Appliance
OMIVV 1 2 8 44 1
VSI 1 2 8 11 1
Number of
VMs
1 4 16 290 1
Number of
CPU cores
RAM (GB)
Disk space
(GB)
Number of
NICs
Chapter 5: References
35
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Dell EMC Ready Stack: VMware Cloud Foundation on PowerEdge Servers and Unity Storage
Design Guide
Dell EMC documentation
The following Dell EMC documentation provides relevant information. Access to these
documents depends on your login credentials. If you do not have access to a document,
contact your Dell EMC representative.
• Brocade Fabric OS Web Tools Administration Guide