Dell Dynamic Pools User manual
H16289
Technical White Paper
Dell EMC Unity: Dynamic Pools
Abstract
This white paper provides an overview of Dell EMC Unity dynamic pool
technology including information on the benefits, underlying structures, and
management methods.
November 2019
Revisions
2 Dell EMC Unity: Dynamic Pools | H16289
Revisions
Date
Description
November 2019
Template and format update; vVols branding update
Acknowledgements
Author: Ryan.Poulin@Dell.com
The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its
subsidiaries. Other trademarks may be trademarks of their respective owners. [11/7/2019] [Technical White Paper] [H16289.4]
Table of contents
3 Dell EMC Unity: Dynamic Pools | H16289
Table of contents
1 Overview ....................................................................................................................................................................... 7
2 Architecture .................................................................................................................................................................. 8
2.1 Drives .................................................................................................................................................................. 8
2.2 Drive partnership groups .................................................................................................................................... 8
2.3 Drive extents ....................................................................................................................................................... 9
2.4 Spare space extents ......................................................................................................................................... 10
2.5 RAID extents ..................................................................................................................................................... 11
2.6 Dynamic pool private RAID groups and LUNs ................................................................................................. 12
2.7 Expanding a dynamic pool ............................................................................................................................... 14
2.8 Proactive copy and drive rebuilds..................................................................................................................... 16
2.9 Wear balancing within a dynamic pool ............................................................................................................. 17
3 Supported configurations ........................................................................................................................................... 19
3.1 Supported drives............................................................................................................................................... 19
3.2 Supported RAID types ...................................................................................................................................... 20
3.3 Supported stripe widths .................................................................................................................................... 20
3.4 Minimum drive counts ....................................................................................................................................... 21
4 Traditional pools ......................................................................................................................................................... 23
5 Management ............................................................................................................................................................... 24
5.1 Viewing the drives within the system ................................................................................................................ 24
5.2 Creating a dynamic pool ................................................................................................................................... 25
5.3 Viewing the properties of a dynamic pool ......................................................................................................... 29
5.4 Expanding a dynamic pool ............................................................................................................................... 34
6 Interoperability ............................................................................................................................................................ 37
6.1 Data at Rest Encryption .................................................................................................................................... 37
6.2 Import ................................................................................................................................................................ 37
6.3 Local LUN move ............................................................................................................................................... 37
6.4 Replication ........................................................................................................................................................ 38
7 Conclusion .................................................................................................................................................................. 39
A Technical support and resources ............................................................................................................................... 40
A.1 Related resources............................................................................................................................................. 40
Executive summary
4 Dell EMC Unity: Dynamic Pools | H16289
Executive summary
The Dell EMC™ Unity product line is a simple, easy-to-use, and intelligent storage system which helps meet
the needs of IT professionals in large or small companies. The systems are designed with flash in mind and
leverage the latest cloud-based data management, efficiency, rich data services, and mobility technologies.
The Dell EMC Unity product line comes in a variety of deployment models from purpose-built all-flash or
hybrid configurations to converged infrastructure and even virtual storage appliances. To further increase the
flexibility of configuration options, the dynamic pools feature was introduced.
Dynamic pools is a pool technology introduced for all-flash systems in Dell EMC Unity OE version 4.2 and
later. Dynamic pools replaces the previous pool technology, now referred to as traditional pools, as the default
pool type for all-flash systems. Dynamic pools utilize advanced RAID techniques and distributed sparing to
offer better storage utilization and more simplified planning than traditional pools. Dynamic pools provides a
number of benefits over traditional pools, which are discussed throughout this document.
This document provides in-depth information about dynamic pools including the underlying architecture of and
all hidden components of the pool structure. Also discussed are the configurations supported by dynamic
pools, which outline the drives supported within a dynamic pool, the number of drives supported, and the
RAID types and widths supported. Information on creating and managing dynamic pools is also provided,
which can easily be achieved through Unisphere™, Unisphere CLI, and REST API. The interoperability of
dynamic pools with other software features is also discussed.
Audience
This white paper is intended for customers, partners, and employees who are planning to utilize dynamic
pools. It assumes familiarity with Dell EMC Unity storage and the management software.
Terminology
All-flash pool: A pool which contains only flash drives.
Block storage resources: LUNs, consistency groups and VMware VMFS datastores.
Compression: A data-reduction method which reduces the physical amount of storage required to save a
dataset.
Consistency group: A storage instance which contains one or more LUNs within a storage system.
Consistency groups help organize the storage allocated for a particular host or hosts. Data protection
configurations, such as replication and snapshot settings, on a consistency group affect all the LUNs
contained in the group, providing ease of management and crash consistency if the LUNs are dependent on
each other.
Data at Rest Encryption (D@RE): The process of encrypting data and protecting it against unauthorized
access unless valid keys are provided. This prevents data from being accessed and provides a mechanism to
quickly crypto-erase data.
Deduplication: A data -method which reduces the physical amount of storage required to save a data set.
Drive extent: A portion of a drive contained within a dynamic pool. Each drive extent is used to provide
usable capacity to the pool, or spare space.
Terminology
5 Dell EMC Unity: Dynamic Pools | H16289
Drive partnership group: A collection of drives of the same drive type which have been combined into a
hidden dynamic pool object. The drive partnership group contains a maximum of 64 drives, and is used to
provide space to the pool.
Dynamic pool: A pool type introduced in Dell EMC Unity OE version 4.2 to provide space to storage
resources. Dynamic pools allow for flexible deployment models, and reduced rebuild times and flash wear
compared to Traditional pools. A dynamic pool is created when creating a pool in Unisphere in Dell EMC
Unity OE version 4.2 and later. Dynamic pools are only supported on Dell EMC Unity all-flash systems.
Dynamic pool private RAID group: A hidden dynamic pool object created using RAID extents. A dynamic
pool private LUN is created on each private RAID group, which is then used to partition space to storage
resources created on the pool.
Dynamic pool private LUN: A hidden dynamic pool object created on a dynamic pool private RAID group.
The private LUN is later partitioned into 256 MB slices, which are used as the underlying storage objects for
storage resources created on the pool.
File storage resources: File systems (NFS, SMB) and VMware NFS datastores.
Flash drive (SSD): A flash-based storage device used to store data.
LUN: A block-based storage resource which a user provisions. It represents a SCSI logical unit.
Pool: A set of drives that provide specific storage characteristics for the resources that use them, such as
LUNs, VMware datastores, and file systems.
RAID extent: An object created using drive extents to provide RAID protection for the data within the pool.
Once the RAID type and stripe width (RAID width) for the pool is set, each RAID extent provides the RAID
protection for the pool. As an example, if the dynamic pool is created with RAID 5 (4+1) protection, a RAID
extent contains 5 drive extents to provide the 4+1 protection.
REST API: An application programming interface (API) that utilizes familiar HTTP operations like GET, POST,
and DELETE. REST architecture includes certain constraints that ensure that different implementations of
REST conform to the same guiding principles, thereby allowing developers the ease of application
development when working with different REST API deployments.
Snapshot: A snapshot, also called a Dell EMC Unity Snapshot, is a point-in-time view of a storage resource.
When a snapshot is taken, the snapshot is an exact copy of the source storage resource, and shares all
blocks of data with it. As data changes on the source, new blocks are allocated and written to. Snapshot
technology can be used to take a snapshot of a block or file storage resource.
Spare space extent: A drive extent reserved as spare space within the dynamic pool. Spare space extents
are only utilized to replace extents on a failing or failed drive within the same drive partnership group.
Storage resource: An addressable and configurable storage instance associated with a specific quantity of
storage. LUNs, file systems, and VMware datastores constitute storage resources.
Stripe width: The stripe width, also known as the stripe RAID width, is the number of drive extents used for
the RAID type chosen for the pool. For example, a RAID 5 (4+1) RAID extent has a stripe width of 5 (4+1),
and therefore contains 5 drive extents.
Terminology
6 Dell EMC Unity: Dynamic Pools | H16289
Traditional pool: The pool technology available within Dell EMC Unity systems before the dynamic pools
feature was released. Traditional pools continue to be supported, and can be managed through Unisphere,
Unisphere CLI, and REST API. To create a traditional pool, Unisphere CLI or REST API must be used.
Unisphere: A web-based management environment used to create storage resources, configure and
schedule protection for stored data, and manage and monitor other storage operations.
Unisphere CLI (UEMCLI): The command line interface (CLI) for managing Dell EMC Unity storage systems.
Overview
7 Dell EMC Unity: Dynamic Pools | H16289
1 Overview
A pool is a collection of drives within the system arranged into an aggregated group, with some form of RAID
protection applied to the drives to provide redundancy. The capacity within the pool, minus the overhead for
the RAID protection selected, can be used to provision block, file, or VMware® vSphere® Virtual Volumes™
(vVols) storage resources. Dynamic pools were released in Dell EMC Unity OE version 4.2 for all-flash
systems. This feature increases the flexibility of configuration options within the system with an entirely
redesigned pool structure. When created in Dell EMC Unisphere™, dynamic pools replaced traditional pools
as the default pool type for all-flash systems. Dynamic pools be created, expanded, and deleted like
traditional pools, but include other improvements as well.
Within a traditional pool, private pool RAID groups are created based on the RAID protection and width
selected at time of pool creation or expansion. These private RAID groups are fixed in width, and drives must
be added to the pool in drive-count multiples which match the stripe width selected. Dynamic pools eliminate
the need to design a pool based on a multiple of a stripe width. When creating a dynamic pool, once a
minimum drive count for a given RAID protection has been selected, the user can select almost any number
of drives to place within the pool. This allows planning a pool based on a specific capacity without concern for
stripe-width-based multiples of drives counts. When expanding a dynamic pool, since the stripe-width multiple
does not apply, the user can also expand by a specific target capacity. In most cases, the user can add a
single drive to the pool to increase its capacity. These features provide flexible deployment models which
improves the planning and provisioning process. The total cost of ownership of the configuration is also
reduced because extra drives needed to fulfill a stripe-width multiple are eliminated.
In systems that use traditional pools, hot spares must be included in the configuration. This requirement
increases the cost of a solution for drives that are left unused within the system, and are only used when a
drive is failing or has failed. For dynamic pools, space is reserved within the pool to replace drives which may
fail or have failed within the pool. This space is considered overhead, and is not part of the usable capacity of
the pool. With this, hot spares are not required for dynamic pools, and all drives within a storage system can
be placed within a dynamic pool. To fully use the drives within a dynamic pool, the user data and space
reserved for drive failures is spread across all drives within the pool. This helps to spread out application I/O
across the drives within the pool, and also helps to spread out and mitigate wear across the drives.
As dynamic pools are created without using discreet groups of drives, rebuild operations are different than
rebuilds occurring within a traditional pool. When a drive is failing or has failed with a traditional pool, a hot
spare is engaged and used to replace the bad drive. This replacement is one-to-one, and the speed of the
proactive copy or rebuild operation is limited by the fixed width of the private RAID group and the single
targeted hot spare. With dynamic pools, regions of RAID and regions within the pool used for drive
replacement are spread across the drives within the pool. In this design, multiple regions of a failing or failed
drive can be rebuilt simultaneously. As the space for the replacement is spread across the drives in the pool,
the proactive copy or rebuild operations are also targeted to more than one drive. With these features, the
amount of time to replace a failed drive is significantly reduced.
Architecture
8 Dell EMC Unity: Dynamic Pools | H16289
2 Architecture
This section covers the underlying architecture of dynamic pools, with each of the subsections outlining a
building block of the overall dynamic pool structure. While there are many elements to dynamic pools, most of
the following topics are hidden to the user in Unisphere, Unisphere CLI, and REST API. They are only
discussed to provide insight into the underlying components of dynamic pools. Creating or expanding a
dynamic pool involves determining only the number of drives to attain the usable capacity required.
2.1 Drives
A pool is a set of drives that provide specific storage characteristics for storage resources created on the pool
to utilize. A typical all-flash system contains either SAS flash 3 or SAS flash 4 drives of the same capacity
which are used to create pools within the system. With dynamic pools, a drive is the most basic part of the
pool. Once a pool has been created, other dynamic pool structures are created on the drives to provide the
usable capacity to the user, and provide fault tolerance.
2.2 Drive partnership groups
When a dynamic pool is created or expanded, the selected drives are placed into drive partnership groups. A
drive partnership group is a group of drives of the same drive type which are combined into a hidden dynamic
pool object. The number of drives placed in each group directly depends on the number and type of drives
selected at pool creation, or selected later when the pool is expanded. A dynamic pool always includes at
least one drive partnership group, and each drive within a dynamic pool can only be part of a single drive
partnership group. For the life of the pool, a drive never changes the drive partnership group that it is part of.
Each drive partnership group can only contain a single drive type, though different sizes of a particular drive
type can be mixed within the group. Figure 1 shows an example of a dynamic pool created with different drive
types and sizes. In this example, all SAS flash 3 drives are placed into a single drive partnership group, while
all SAS flash 4 drives are placed into their own drive partnership group. A typical all-flash configuration
contains a single drive type and size, but dynamic pools allow for different drive types and sizes to be mixed.
Drive partnership group example with SAS flash 3 and SAS flash 4 drives
The maximum number of drives contained within a drive partnership group is 64. When more drives of the
same type are included in the pool, more drive partnership groups are created. Each drive partnership group
is started with a minimum number of drives, which is controlled by the RAID type and stripe width of the pool.
Architecture
9 Dell EMC Unity: Dynamic Pools | H16289
Once the RAID type and stripe width are set for a particular drive type, all drive partnership groups containing
that type of drive will use those settings. When a drive partnership group is full, another minimum number of
drives is needed to create the next drive partnership group with the same RAID type and stripe width. This
topic is covered in further detail in section 3 on supported configurations.
Figure 2 shows an example of multiple dynamic pools and drive partnership groups. In the left image, the
dynamic pool has only six drives of the same type. All of these drives are placed into a single drive
partnership group at time of pool creation. The middle image is a pool which contains 64 drives of the same
type. The pool may have been created with 64 drives, or was created with less drives and later expanded.
Since the total number of drives is 64, they are all placed into a single drive partnership group, and the group
is considered full. The right image is an example of a dynamic pool which contains 70 drives. Since a drive
partnership group is considered full at 64 drives, the remaining six drives are placed in a new drive
partnership group. As with the middle pool, this pool may have been created using 70 drives, or was
expanded after creation to reach this drive count.
Drive partnership group example
The drive partnership group within a dynamic pool is used to limit the amount of drives other dynamic pool
objects can cross. These objects include drive extents, RAID extents, and spare space extents, and dynamic
pool private RAID groups, all of which are explained in the following subsections. This level of division
ensures the data is not spread across a large amount of drives, which can increase the risk of encountering
multiple drive failures and could degrade or break the RAID protection selected.
2.3 Drive extents
A drive extent within a dynamic pool is simply a portion of a drive that is gigabytes in size. The size of each
extent is fixed for each drive type, which can either be SAS flash 3 or SAS flash 4 within an all-flash system.
After a dynamic pool is created on the system, each drive within a dynamic pool is partitioned into drive
extents. The number of extents created on each drive directly depends on the drive type and size. Each
extent within a drive is either used to store user data, or reserved as space to replace a failing or faulted drive
within the pool.
Architecture
10 Dell EMC Unity: Dynamic Pools | H16289
Figure 3 shows two drives, a 400 GB SAS FLASH 3 drive and an 800 GB SAS flash 3 drive, partitioned into
drive extents. Since all drive extents for this drive type are equal in size, the number of extents created on the
800 GB drive is twice the number created on the 400 GB drive.
Drive extent example with SAS flash 3 drives
2.4 Spare space extents
Unlike traditional pools, dynamic pools do not require dedicated hot spare drives. Instead, space within each
pool is reserved and can only be used to replace a failing or failed drive. After a dynamic pool is created and
the drives are partitioned into drive extents, a number of drive extents are reserved and marked as spare
space extents. The number of spare space extents reserved within a dynamic pool directly depends on the
number and size of the drives within the pool. The amount of spare space is slightly higher than one drive’s
worth of spare-space extents for every 31 drives of a specific type within the pool. The amount of spare space
reserved always ensures that the drive with the largest usable capacity within the pool can be replaced with
the spare space extents remaining within the pool.
Spare space is counted as part of a pool’s overhead, as with RAID overhead, and is not reported to the user.
Spare space is also not part of the usable capacity within the pool for user data. If a pool were to become full
and a drive failure occurred, there would still be spare-space extents within the pool to complete the rebuild
operation. The amount of spare space within the pool is not user-configurable, and is reserved at the time of
pool creation and when expanding a pool. When a drive count crosses the 1-for-31 mark, a single drive’s
worth of extents is reserved from the additional capacity being added to the pool. When expanding a pool in
Unisphere in this instance, the amount of usable capacity would show less than that total amount of drive
capacity being added to the pool.
Architecture
11 Dell EMC Unity: Dynamic Pools | H16289
Figure 4 shows an example of a 6-drive dynamic pool. Each drive extent in this example is marked with a Dx
DE y label signifying which drive and which extent it is, such as D1 DE 1 (drive 1, drive extent 1). At time of
creation, several extents on each drive are reserved as spare space extents, denoted in the figure as Spare.
The figure shows that spare space extents are evenly spread across the drives within the pool. For
demonstration purposes, all spare space extents are shown at the end of usable capacity of each drive. This
is not necessarily the way spare space extents are allocated by the dynamic pool algorithms.
Spare space example within a 6-drive dynamic pool
2.5 RAID extents
After spare space extents are reserved within the dynamic pool, RAID extents are created with the remaining
drive extents. A RAID extent is a collection of drive extents which complete the stripe width chosen for a
particular RAID type. For example, if RAID 5 is selected as the RAID type, and 4+1 is selected as the stripe
width for the pool, the RAID extent would contain 5 drive extents (4+1), one for each element of the stripe
width. The RAID extent provides RAID protection for user data stored within the dynamic pool, and is later
used to provide usable capacity to the pool for storage resource creation. A single RAID extent cannot contain
two drive extents from a single drive for protection purposes. Also, RAID extents must contain drive extents
from only a single drive partnership group.
Architecture
12 Dell EMC Unity: Dynamic Pools | H16289
Figure 5 shows an example of a dynamic pool created with six drives, assuming that RAID 5 (4+1) has been
selected. In this example, a number of extents have already been reserved as spare space extents, and the
first three RAID extents are shown. Each RAID extent in this example contains five drive extents, due to the
4+1 stripe width. The five-drive extents are selected from the drives within the drive partnership group, and no
2 extents are selected from the same drive. For ease of illustration, the drive extents selected are in order
across the drives within the pool. Within a real system, the dynamic pool algorithm will select drive extents
from different drives seemingly at random within the drive partnership group.
RAID extent example
2.6 Dynamic pool private RAID groups and LUNs
Within a pool on a Dell EMC Unity system, whether Traditional or Dynamic, exists one or more private RAID
groups and a single private LUN created on each. These objects, as the names suggest, are underlying
components of the system that are not shown in Unisphere, CLI, or REST API. The private RAID group is
used to provide space to the private LUN, which provides space in the form of 256 MB slices to the user for
storage resource allocation. In a traditional pool, the private RAID group is created by combining a number of
drives into a physical RAID group within the system, whose RAID protection and width matches the settings
chosen at time of creation or pool expansion. In the case of dynamic pools, the private RAID group concept
and the creation of the private LUN and 256 MB slices on top of the usable capacity remain the same. What is
different is how space is provided to the private RAID group.
Within dynamic pools, a private RAID group is created using a combination of RAID extents. Depending on
the number of drives selected at time of pool creation or expansion, the dynamic pool software will determine
how many RAID extents and private RAID groups to create. Later, a single private LUN is created on each
private RAID group.
Loading...