Dell PowerScale SmartPools User Manual

1 |

Next Generation Storage Tiering with Dell EMC PowerScale SmartPools

STORAGE TIERING WITH DELL EMC POWERSCALE SMARTPOOLS

Abstract

This white paper provides a technical overview of Dell EMC PowerScale SmartPools software and how it provides a native, policy-based tiering capability, which enables enterprises to reduce storage costs and optimizing their storage investment by automatically moving data to the most appropriate storage tier within a OneFS cluster.

February 2021

WHITE PAPER

2 |

Revisions

Version

Date

Comment

1.0

November 2013

Initial release for OneFS 7.1

2.0

June 2014

Updated for OneFS 7.1.1

3.0

November 2014

Updated for OneFS 7.2

4.0

June 2015

Updated for OneFS 7.2.1

5.0

November 2015

Updated for OneFS 8.0

6.0

September 2016

Updated for OneFS 8.0.1

7.0

April 2017

Updated for OneFS 8.1

8.0

November 2017

Updated for OneFS 8.1.1

9.0

February 2019

Updated for OneFS 8.1.3

10.0

April 2019

Updated for OneFS 8.2

11.0

August 2019

Updated for OneFS 8.2.1

12.0

December 2019

Updated for OneFS 8.2.2

10.0

June 2020

Updated for OneFS 9.0

11.0

September 2020

Updated for OneFS 9.1

Acknowledgements

This paper was produced by the following: Author: Nick Trimbee

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Other trademarks may be trademarks of their respective owners.

3 |

TABLE OF CONTENTS

Executive Summary ............................................................................................................................................................... 5

Scale-out NAS Architecture ................................................................................................................................................... 5

Single File System ............................................................................................................................................................. 6

Data Layout and Protection ................................................................................................................................................ 6

Job Engine ......................................................................................................................................................................... 7

Data Rebalancing ............................................................................................................................................................... 8

Caching .............................................................................................................................................................................. 8

Smart Pools ............................................................................................................................................................................ 9

Overview ............................................................................................................................................................................ 9

Hardware Tiers ................................................................................................................................................................. 10

Storage Pools ................................................................................................................................................................... 11

Disk Pools ........................................................................................................................................................................ 11

Node Pools ....................................................................................................................................................................... 12

Tiers ................................................................................................................................................................................. 14

Node Compatibility and Equivalence ............................................................................................................................... 15

SSD Compatibility ............................................................................................................................................................ 15

Data Spill Over ................................................................................................................................................................. 15

Automatic Provisioning ..................................................................................................................................................... 16

Manually Managed Node Pools ....................................................................................................................................... 16

Global Namespace Acceleration ...................................................................................................................................... 16

Virtual Hot Spare .............................................................................................................................................................. 17

SmartDedupe and Tiering ................................................................................................................................................ 18

SmartPools Licensing ...................................................................................................................................................... 18

File Pools .......................................................................................................................................................................... 19

File Pool Policies .............................................................................................................................................................. 20

Custom File Attributes ...................................................................................................................................................... 22

Anatomy of a SmartPools Job.......................................................................................................................................... 22

File Pool Policy Engine .................................................................................................................................................... 23

FilePolicy Job ................................................................................................................................................................... 23

Data Location ................................................................................................................................................................... 24

Node Pool Affinity ............................................................................................................................................................. 27

Performance with SmartPools.......................................................................................................................................... 27

Using SmartPools to Improve Performance ..................................................................................................................... 27

Data Access Settings ....................................................................................................................................................... 28

Leveraging SSDs for Metadata & Data Performance ...................................................................................................... 29

Enabling L3 Cache (SmartFlash) ..................................................................................................................................... 30

4 |

Minimizing the Performance Impact of Tiered Data ......................................................................................................... 31

SmartPools Best Practices ............................................................................................................................................... 31

SmartPools Use Cases .................................................................................................................................................... 32

SmartPools Workflow Examples .......................................................................................................................................... 32

Example A: Storage Cost Efficiency in Media Post Production ....................................................................................... 32

Example B: Data Availability & Protection in Semiconductor Design .............................................................................. 33

Example C: Investment Protection for Financial Market Data ......................................................................................... 34

Example D: Metadata Performance for Seismic Interpretation ........................................................................................ 35

Conclusion ............................................................................................................................................................................ 36

TAKE THE NEXT STEP ....................................................................................................................................................... 37

5 |

Executive Summary

Dell EMC PowerScale SmartPools software enables multiple levels of performance, protection, and storage density to co-exist within the same file system and unlocks the ability to aggregate and consolidate a wide range of applications within a single extensible, ubiquitous storage resource pool. This helps provide granular performance optimization, workflow isolation, higher utilization, and independent scalability – all with a single point of management.

SmartPools allows you to define the value of the data within your workflows based on policies, and automatically aligns data to the appropriate price/performance tier over time. Data movement is seamless, and with file-level granularity and control via automated policies, manual control, or API interface, you can tune performance and layout, storage tier alignment, and protection settings – all with minimal impact to your end-users.

Storage tiering has a very convincing value proposition, namely separating data according to its business value, and aligning it with the appropriate class of storage and levels of performance and protection. Information Lifecycle Management techniques have been around for a number of years, but have typically suffered from the following inefficiencies: complex to install and manage, involves changes to the file system, requires the use of stub files, etc.

SmartPools is a next generation approach to tiering that facilitates the management of heterogeneous clusters. The SmartPools capability is native to the OneFS scale-out file system, which allows for unprecedented flexibility, granularity, and ease of management. In order to achieve this, SmartPools leverages many of the components and attributes of OneFS, including data layout and mobility, protection, performance, scheduling, and impact management.

Intended Audience

This paper presents information for deploying and managing a heterogeneous Dell EMC OneFS cluster. This paper does not intend to provide a comprehensive background to the OneFS architecture.

 Please refer to the OneFS Technical Overview white paper for further details on the OneFS architecture.

The target audience for this white paper is anyone configuring and managing data tiering in a OneFS clustered storage environment. It is assumed that the reader has a basic understanding of storage, networking, operating systems, and data management.

 More information on OneFS commands and feature configuration is available in the OneFS Administration Guide.

Scale-Out NAS Architecture

OneFS combines the three layers of traditional storage architectures—file system, volume manager, and data protection—into one unified software layer, creating a single intelligent distributed file system that runs on a Dell EMC PowerScale storage cluster.

Figure 1. OneFS combines file system, volume manager and data protection into one single intelligent, distributed system.

6 |

This is the core innovation that directly enables enterprises to successfully utilize scale-out NAS in their environments today. It adheres to the key principles of scale-out: intelligent software, commodity hardware, and distributed architecture. OneFS is not only the operating system, but also the underlying file system that stores and manages data in a cluster.

Single File System

OneFS provides a single file system that operates outside the constraints of traditional scale up constructs like RAID groups, allowing data to be placed anywhere on the cluster, thereby accommodating a variety of levels of performance and protection.

Each cluster operates within a single volume and namespace with the file system distributed across all nodes. As such, there is no partitioning, and clients are provided a coherent view of their data from any node in the cluster.

Because all information is shared among nodes across the internal network, data can be written to or read from any node, thus optimizing performance when multiple users are concurrently reading and writing to the same set of data.

From an application or user perspective, all capacity is available - the storage has been completely virtualized for the users and administrator. The file system can grow organically without requiring too much planning or oversight. No special thought has to be applied by the administrator about tiering files to the appropriate disk, because SmartPools will handle that automatically. Additionally, no special consideration needs to be given to how one might replicate such a large file system, because the OneFS SyncIQ service automatically parallelizes the transfer of the data to one or more alternate clusters.

 Please refer to the OneFS Technical Overview white paper for further details on the OneFS architecture.

Data Layout and Protection

OneFS is designed to withstand multiple simultaneous component failures (currently four per node pool) while still affording unfettered access to the entire file system and dataset. Data protection is implemented at the file level using Reed Solomon erasure coding and, as such, is not dependent on any hardware RAID controllers. This provides many benefits, including the ability to add new data protection schemes as market conditions or hardware attributes and characteristics evolve. Since protection is applied at the file-level, a OneFS software upgrade is all that’s required in order to make new protection and performance schemes available.

OneFS employs the popular Reed-Solomon erasure coding algorithm for its protection calculations. Protection is applied at the filelevel, enabling the cluster to recover data quickly and efficiently. Inodes, directories, and other metadata are protected at the same or higher level as the data blocks they reference. Since all data, metadata, and forward error correction (FEC) blocks are striped across multiple nodes, there is no requirement for dedicated parity drives. This guards against single points of failure and bottlenecks, allows file reconstruction to be highly parallelized, and ensures that all hardware components in a cluster are always in play doing useful work.

OneFS supports several protection schemes. These include the ubiquitous +2d:1n, which protects against two drive failures or one node failure.



The best practice is to use the recommended protection level for a particular cluster configuration. This recommended level of protection is clearly marked as ‘suggested’ in the OneFS WebUI storage pools configuration pages and is typically configured by default.

The hybrid protection schemes are particularly useful for high-density node configurations, such as the Isilon A2000 chassis, where the probability of multiple drives failing surpasses that of an entire node failure. In the unlikely event that multiple devices have simultaneously failed, such that the file is “beyond its protection level”, OneFS will re-protect everything possible and report errors on the individual files affected to the cluster’s logs.

OneFS also provides a variety of mirroring options ranging from 2x to 8x, allowing from two to eight mirrors of the specified content. This is the method used for protecting OneFS metadata. Metadata, for example, is mirrored at one level above FEC by default. For example, if a file is protected at +1n, its associated metadata object will be 3x mirrored.

The full range of OneFS protection levels are summarized in the following table:

7 |

Protection Level

Description

+1n

Tolerate failure of 1 drive OR 1 node

+2d:1n

Tolerate failure of 2 drives OR 1 node

+2n

Tolerate failure of 2 drives OR 2 nodes

+3d:1n

Tolerate failure of 3 drives OR 1 node

+3d:1n1d

Tolerate failure of 3 drives OR 1 node AND 1 drive

+3n

Tolerate failure of 3 drives or 3 nodes

+4d:1n

Tolerate failure of 4 drives or 1 node

+4d:2n

Tolerate failure of 4 drives or 2 nodes

+4n

Tolerate failure of 4 nodes

2x to 8x

Mirrored over 2 to 8 nodes, depending on configuration

Table 1: OneFS FEC Protection Levels

Additionally, a logical separation of data and metadata structures within the single OneFS file system allows SmartPools to manage data and metadata objects separately, as we will see.

OneFS stores file and directory metadata in inodes and B-trees, allowing the file system to scale to billions of objects and still provide very fast lookups of data or metadata. OneFS is a completely symmetric and fully distributed file system with data and metadata spread across multiple hardware devices. Data are generally protected using erasure coding for space efficiency reasons, enabling utilization levels of up to 80% and above on clusters of five nodes or more. Metadata (which generally makes up around 2% of the system) is mirrored for performance and availability. Protection levels are dynamically configurable at a per-file or per-file system granularity, or anything in between. Data and metadata access and locking are coherent across the cluster, and this symmetry is fundamental to the simplicity and resiliency of OneFS’ shared nothing architecture.

When a client connects to a OneFS-managed node and performs a write operation, files are broken into smaller logical chunks, or stripes units, before being written to disk. These chunks are then striped across the cluster’s nodes and protected either via erasurecoding or mirroring. OneFS primarily uses the Reed-Solomon erasure coding system for data protection, and mirroring for metadata. OneFS’ file level protection typically provides industry leading levels of utilization. And, for nine node and larger clusters, OneFS is able to sustain up to four full node failures while still providing full access to data.

OneFS uses multiple data layout methods to optimize for maximum efficiency and performance according to the data’s access pattern – for example, streaming, concurrency, random, etc. And, like protection, these performance attributes can also be applied per file or

per filesystem.

 Please refer to the OneFS Technical Overview white paper for further details on OneFS data protection levels.

Job Engine

The Job Engine is OneFS’ parallel task scheduling framework, and is responsible for the distribution, execution, and impact management of critical jobs and operations across the entire cluster.

8 |

The OneFS Job Engine schedules and manages all the data protection and background cluster tasks: creating jobs for each task, prioritizing them and ensuring that inter-node communication and cluster wide capacity utilization and performance are balanced and optimized. Job Engine ensures that core cluster functions have priority over less important work and gives applications integrated with OneFS – add-on software or applications integrating to OneFS via the OneFS API – the ability to control the priority of their various functions to ensure the best resource utilization.

Each job, for example the SmartPools job, has an “Impact Profile” comprising a configurable policy and a schedule which characterizes

how much of the system’s resources the job will take, plus an Impact Policy and an Impact Schedule. The amount of work a job has to

do is fixed, but the resources dedicated to that work can be tuned to minimize the impact to other cluster functions, like serving client data.

The specific jobs that the SmartPools feature comprises include:

Job

Description

SmartPools

Job that runs and moves data between the tiers of nodes within the same cluster. Also executes the CloudPools functionality if licensed and configured.

SmartPoolsTree

Enforces SmartPools file policies on a subtree.

FilePolicy

Efficient changelist-based SmartPools file pool policy job.

IndexUpdate

Creates and updates an efficient file system index for FilePolicy job.

SetProtectPlus

Applies the default file policy. This job is disabled if SmartPools is activated on the cluster.

 When a cluster running the FSAnalyze job is upgraded to OneFS 8.2 or later, the legacy FSAnalyze index and snapshots are

removed and replaced by new snapshots the first time that IndexUpdate is run. The new index stores considerably more file and snapshot attributes than the old FSA index. Until the IndexUpdate job effects this change, FSA keeps running on the old index and snapshots.

Data Rebalancing

Another key Job Engine task is AutoBalance. This enables OneFS to reallocate and rebalance, or restripe, data across the nodes in a cluster, making storage space utilization more uniform and efficient.

OneFS manages protection of file data directly, and when a drive or entire node failure occurs, it rebuilds data in a parallel fashion. OneFS avoids the requirement for dedicated hot spare drives and serial drive rebuilds, and simply borrows from the available free space in the file system in order to recover from failures; this technique is called Virtual Hot Spare. This approach allows the cluster to be self-healing, without human intervention, and with the advantages of fast, parallel data reconstruction. The administrator can create a virtual hot spare reserve, which prevents users from consuming capacity that is reserved for the virtual hot spare.

The process of choosing a new layout is called restriping, and this mechanism is identical for repair, rebalance, and tiering. Data is moved in the background with the file available at all times, a process that’s completely transparent to end users and applications.

 Further information is available in the OneFS Job Engine white paper.

Caching

SmartCache is a globally coherent read and write caching infrastructure that provides low latency access to content. Like other resources in the cluster, as more nodes are added, the total cluster cache grows in size, enabling OneFS to deliver predictable, scalable performance within a single filesystem.

9 |

A OneFS cluster provides a high cache to disk ratio (multiple GB per node), which is dynamically allocated for read operations as needed. This cache is unified and coherent across all nodes in the cluster, allowing a user on one node to benefit from I/O already transacted on another node. OneFS stores only

distinct

data on each node. The node’s RAM is used as a “level 2” (L2) cache of such

data. These distinct, cached blocks can be accessed across the backplane very quickly and, as the cluster grows, the cache benefit increases. For this reason, the amount of I/O to disk on a OneFS cluster is generally substantially lower than it is on traditional platforms, allowing for reduced latencies and a better user experience. For sequentially accessed data, OneFS SmartRead aggressively pre-fetches data, greatly improving read performance across all protocols.

An optional third tier of read cache, called SmartFlash or Level 3 cache (L3), is also configurable on nodes that contain solid state drives (SSDs). SmartFlash is an eviction cache that is populated by L2 cache blocks as they are aged out from memory. There are several benefits to using SSDs for caching rather than as traditional file system storage devices. For example, when reserved for caching, the entire SSD will be used, and writes will occur in a very linear and predictable way. This provides far better utilization and also results in considerably reduced wear and increased durability over regular file system usage, particularly with random write workloads. Using SSD for cache also makes sizing SSD capacity a much more straightforward and less error prone prospect compared to using use SSDs as a storage tier.

OneFS write caching uses write buffering to aggregate, or coalesce, multiple write operations to the NVRAM file systems journals so that they can be written to disk safely and more efficiently. This form of buffering reduces the disk write penalty which could require multiple reads and writes for each write operation.

Figure 2: OneFS Caching Hierarchy

 Further information is available in the OneFS SmartFlash white paper.

SmartPools

Overview

SmartPools is a next-generation data tiering product that builds directly on the core OneFS attributes described above. These include the single file system, extensible data layout and protection framework, parallel job execution, and intelligent caching architecture.

SmartPools was originally envisioned as a combination of two fundamental notions:

•

The ability to define sub-sets of hardware within a single, heterogeneous cluster.

10 |

•

A method to associate logical groupings of files with these hardware sub-sets via simple, logical definitions or rules.

Figure 3. SmartPools Tiering Model

The current SmartPools implementation expands on these two core concepts in several additional ways, while maintaining simplicity and ease of use as primary goals.

Hardware Tiers

Heterogeneous OneFS clusters can be architected with a wide variety of node styles and capacities, in order to meet the needs of a varied data set and wide spectrum of workloads. These node styles encompass Isilon Gen6 and PowerScale hardware generations and fall loosely into four main categories or tiers. The following table illustrates these tiers, and the associated hardware models:

OneFS with

Ethernet or Infiniband Network

Flash

I/Ops

Hybrid

Throughput