Dell Grid Management Platform Common Design Architecture User Manual

WHITE PAPER
Dell Technologies Grid Management Platform Common Design Architecture
CONTENTS
EXECUTIVE SUMMARY ........................................................................... 3
VIRTUALIZATION AND VXRAIL ................................................................ 4
REGULATORY COMPLIANCE ................................................................... 5
VMWARE VALIDATED DESIGN ................................................................. 6
GRID MANAGEMENT SYSTEM ................................................................ 6
A CASE STUDY ....................................................................................... 9
CONCLUSION AND NEXT STEPS ............................................................. 9
2 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel
®
Runs on current state VxRail
v4.7 and VVD for NIST v5.1
Runs on current state VxRail
v4.7 and VVD for NIST v5.1
and manual NERC CIP
Runs on current state
vSphere-vSAN-NSX
leveraging NIST
conguration guidance
Runs on current state
vSphere-vSAN leveraging
NIST and NERC CIP
conguration guidance
Journey to Future State
EXECUTIVE SUMMARY
The utility industry is seeing unprecedented change in the utility grid, which is driven by a global transition to renewable energy, faster uptake of Distributed Energy Resources (DERs) and the need to support worldwide sustainability goals. The industry is undergoing an IT (Information Technology)/OT (Operations Technology) convergence to support the migration from a rigid OT-centric model to a more dynamic software dened, data-driven model.
The foundation for this IT/OT convergence is a robust enterprise-grade software-dened data center (SDDC) that can extend from the data center (control center) to the near edge (substation), capable of supporting cloud-native and legacy (Windows/Linux) x86 applications on a single platform while providing intrinsic security, compliance, and lifecycle management.
The term SDDC or virtual infrastructure describes the decoupling of physical hardware from the operating system(s) and application(s) leveraging the physical hardware. This layer of abstraction between compute, storage and network hardware gives administrators the advantage of managing pooled resources across the enterprise, allowing IT to be more responsive and agile to organizational and business needs.
The journey to a virtualized Grid Data center (Control Center) and Edge (Substation) can be an incremental process across both Grid Transmission and Distribution Operations in the Data center or at the near Edge (Substation). Dell Technologies, VMware, and Intel are providing proven solutions to utility providers across four pillars (see Figure 1).
FIGURE 1. Transmission / Distribution Operations & Transmission / Distribution Substation
Virtualization of Grid
Virtualization of Grid Distribution in the Data Center (ADMS / OMS)
Transmission in the Data
Center (EMS)
Data center virtualization can be accomplished for both grid transmission and distribution operations. Dell Technologies, VMware, and Intel® are providing utility providers with both non-NERC CIP (Distribution Operations) and NERC CIP (Transmission Operations) regulated environments. The foundation for the data center deployment is a VMware Validated Design (VVD) on the Dell EMC Hyperconverged VxRail infrastructure. The Dell EMC VxRail infrastructure powered by Intel® is specically designed and built to fully leverage the VVD
Blueprint architecture. This can be further enhanced with the VVD for NIST 800-53 Compliance Toolkit as a cybersecurity foundational baseline. With a few additional technology control
settings, the environment can be further extended to meet NERC CIP (North American Energy Reliability Corporation – Critical Infrastructure Protection) compliance.
The product and services solution sets from Dell Technologies, VMware and Intel have been curated for optimized performance and deployment into the substations and grid data centers.
3 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel © 2020 Dell Inc. or its subsidiaries.
Technical Validation / Pilot
/ Deployment of Virtualization for
Distribution Substation
®
Technical Validation / Pilot
/ Deployment of Virtualization for
Transmission Substation
Joint partnerships with the leading utility software vendors who provide both EMS (Energy
Data Center (Control Center)
Substation (Edge)
Software Dened
(SD Layer)
Physical Layer
NSX
NSX
All-Flash-vSAN
All-Flash-vSAN
ESXi-Hypervisor
ESXi-Hypervisor
Software Dened Networking (SDN)
Software Dened Storage (SDS)
Software Dened Compute (SDC)
Management System) and ADMS (Advanced Distributed Management System) solutions have been formed to further validate their specic application portfolios on the Dell Technologies, VMware and Intel infrastructure platforms.
FIGURE 2. Dell Technologies and VMware Data Center and Edge Congurations
The remainder of this document is focused on the denition of a common design architecture for deployment of a hyperconverged infrastructure platform at the data center called a Grid Management Platform (GMP). This document describes the details of how Dell Technologies, Intel and VMware are partnering to deliver solutions to assist Electric Utilities with the challenges associated with implementing transformational grid management platform modernization projects and the associated business impact.
The utility grid has been undergoing a fundamental change from a rigid Operations Technology (OT) centric one-way energy ow model, to a more dynamic two-way, data-driven model supporting intermittent renewable resources and exible customer loads. As a result, managing energy on the grid has become increasingly complex, necessitating more ecient solutions to manage peak demand and maintain reliability. Importantly, they also need a modern distributed architecture and analytics capabilities to take full advantage of the data collected to deliver real-time insights that meet changing business demands. To manage the grid in this increasingly complex environment, utility leaders are adopting new digital transformation technologies. These transformation projects which in the past would have taken ve to seven years are being reduced to a year or two. This has resulted in a rapid convergence of operations technology (OT) and Information Technology (IT) organizations where innovative technologies brought forward from IT departments and other back oce functions are being leveraged by the business to become more responsive to the changing utilities market.
CURRENT CHALLENGES FACING UTILITIES
Modernize legacy infrastructure and
application silos
Improve cybersecurity capabilities
Meet regulatory compliance requirements
Improve reliability and resiliency
Automate grid process operations
Become more data driven
Adopt and integrate DERs (distributed energy
resources)
Reduce greenhouse gases
4 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel
VIRTUALIZATION AND VXRAIL
As the introduction section of this paper indicated, many utilities have turned to virtualization of critical grid management applications to help streamline operations and reduce expenses while enhancing reliability. The virtualization of these utility systems enables the decoupling
®
of application and hardware refresh cycles, delivering non-disruptive updates and reduced
Automated Deployment
Simple
Expansions
Automated
Upgrades
Built-In
Security
Application
Optimized
total cost of operations (TCO). The adoption of virtualized Hyperconverged Infrastructure (HCI) models removes the dependencies on Storage Area Networks (SAN) and Fiber Channel components using a software dened storage conguration integrated into and delivered by VxRail HCI nodes. Utilities can now eliminate the capital expense (CapEx) and operational expense (OpEx) spending associated with procuring and maintaining separate storage area networks (SAN), and redirect resources to more value-add activities.
The Dell Technologies VxRail HCI platform powered by Intel automates and eliminates many manual administrative and maintenance activities for a drastically improved lifecycle management capability, that lowers total cost of ownership while enhancing data security, compliance, and data protection services. Infrastructure patching is now automated, and system upgrades no longer require planned downtime.
The VxRail platform includes VMware’s software dened data center SDDC architecture. Specic system components included are VMware’s vSphere hypervisor with vCenter, VSAN software dened storage and VxRail Manager for low touch, remote administration. Automated deployment, operations and lifecycle management features makes the VxRail a simple and scalable platform for the next generation of grid management platform software. The VxRail solution includes several built-in security features (data at rest encryption, IDPA integration) to ensure a high level of reliability and protection from cyber-attacks. VxRail also oers a single support organization to handle all hardware and software support. (See Figure 3.)
FIGURE 3. VxRail Automated Infrastructue, the Next Generation of Grid Management Platform
5 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel
REGULATORY COMPLIANCE
The Utility Industry is a critical industry that requires Standards and Regulations to ensure safe, secure, and reliable Grid Operations. The North American Electric Reliability Corporation (NERC) is a not-for-prot authority governed by the Federal Energy Regulatory Commission (FERC). NERC’s primary goal is to ensure the reliability of the Bulk Electric System (BES) in North America. This is done through the identication and protection of approved Critical Infrastructure Protection (CIP) assets for improved regulatory accountability of Grid Operations.
The critical component in deploying infrastructure is a standardized methodology that provides an enhanced level of consistency to improve People / Process / Technology. Risk is mitigated by deploying in the environment leveraging a repeatable common design that incorporates internal guiding principles and external security and compliance sources like NIST and NERC CIP. This is enhanced with security constructs developed specically for virtualized environments to develop a highly secure, zero trust architecture for Critical Infrastructure Protection (CIP) and beyond.
®
VMWARE VALIDATED DESIGN
The Dell Technologies and VMware vision for the Utility Industry is a single infrastructure platform that can run in the control center and at the near-edge or substation. This provides a single platform capable of supporting both legacy (Windows / Linux) systems and cloud native container applications with intrinsic security, compliance, and lifecycle management.
VMware recognizes the need for cybersecurity hygiene and partnering with Regulatory bodies such as NIST (National Institute of Standards and Technology). VMware demonstrated the VMware Validated Design (think reference architecture) for NIST as the security baseline for Grid Transmission and Distribution. VMware is currently working to pivot NERC CIP compliance on top of NIST providing a strong security and compliance framework for utilities.
GRID MANAGEMENT SYSTEM
A new term that is emerging is Grid Management System.2 The term is meant to convey the concept of the “System of Systems.” The term Grid Management System (GMS) is meant to encompass EMS (Energy Management System), ADMS (Advanced Distributed Management System), OMS (Outage Management System) and other Grid Transmission and Distribution Operations Management solutions.
As the proliferation of distributed energy resources (DERs) continues to increase, modernization of the Grid Distribution Operations infrastructure is needed to provide operators with improved tools for ecient Grid Operations while meeting required regulatory and operational standards. This requires increased levels of grid automation technology and the integration of the grid systems with Information Technology (IT) systems.
The adoption of virtualization within the utility grid is at both the control center and the substation. Virtualization of the control center is focused on virtualizing Grid Management Systems that run in the data center. As ADMS grid technologies evolve, they will provide the next-generation control capabilities such as management of Distributed Energy Resources (DERs), feeder automation, edge IoT Analytics and battery storage optimization for improved eciency at the near edge (substation). The result of a single platform that can extend from data center to edge is tighter integration with utility tools to maintain overall grid reliability. This global ADMS market is being driven by several key factors such as the increase in demand for energy, the need for improved customer service and increased grid eciency. In addition, sustainability demands are driving the industry towards net-zero carbon footprint strategies, while consumers are demanding increased cost savings.
Dell Technologies and VMware are partnering with the leading GMS solution vendors to develop the industry’s rst turnkey GMS appliances engineered to address Grid Modernization. Based on the VxRail hyperconverged infrastructure platform and the VMware Validated Design (VVD) for NIST and our guidance on NERC-CIP regulatory compliance, this appliance is designed to perform optimally for the GMS solutions, speed the time to production, improve reliability and reduce overall total cost of ownership, while also improving the security prole and industry regulatory compliance of the IT Grid T&D (Transmission and Distribution) Operations infrastructure.
6 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel © 2020 Dell Inc. or its subsidiaries.
®
Quality Assurance Environment
Data Replication
Production Environment
Data Replication
Internal Firewalls
Internal Firewalls
Access Services (DMZ)
Access Services (DMZ)
Access Services (DMZ)
Access Services (DMZ)
QA Dev
QA Dev
Prod Staging
PDS/DTS
Prod Staging
External Firewalls
External Firewalls
Backup Data Center (BDC)
FIGURE 4. Standard VxRail Grid Management Platform Conguration
This all-in-one engineered appliance solution provides:
Corporate
WAN
Reliability: The GMS workloads are deployed on the hyper converged VxRail
infrastructure platform from Dell Technologies, and tightly integrated virtualization software components from VMware; designed exclusively for high performance and continuous availability of the GMS software portfolio.
Security: This common design architecture is based on VMware’s Validated Design for
NIST which provides an industry standard security baseline for the solution.
Compliance: This VMware Validated Design also provides the conguration settings
required to meet NERC CIP regulatory compliance.
Time to Production: Automated deployment and operations along with a standardized,
proven infrastructure platform drastically reduces the time to deploy new applications over legacy non virtualized application stacks.
Lower TCO: Historical analysis of IT organizations both large and small have gained 52%
reduction in cost of operations over traditional infrastructure deployments taken over a ve-year period.¹
Dell Technologies has adopted a standard approach to delivering a virtualized GMS architecture with VMWare and Intel. Characteristics of this approach include: VxRail Hyperconverged Infrastructure. VMware Validated Design for NIST and NERC-CIP guidance. Identical congurations are typically duplicated across a primary and secondary datacenter location for resiliency and to maintain business continuity in the event of a disaster. Optimized for GMS solution performance and throughput and reliability. A Dell Technologies Integrated Data Protection Appliance (IDPA) or Data Domain data protection conguration for data backup and recovery are also recommended. The installation of this conguration is managed and implemented with Dell ProDeploy professional services and ongoing support is provided with Dell Technologies Mission Critical hardware and software one call support. (see Figure 5 and Figure 6.)
7 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel © 2020 Dell Inc. or its subsidiaries.
®
Grid Management Application Workloads
ADMS - SCADA, DERMS Network Mngmt, OMS, Analytics
VMware Validated Design - SDDC
Dell ProDeploy Plus Service
Dell ProSupport Plus Support
High Level Data Center Architecture for GRID Modernization Workloads
Primary Data Center
Production
QA/Test
Production
QA/Test
Data
Application
FIGURE 5. Dell Technologies and VMware’s standard approach to GMP architecture
Grid Management Application Workloads
Platform Hardware
Platform Software
Advanced Distribution Manangement System (ADMS) Distribution SCADA System Outage Management System (OMS) Distributed Energy Resources Management System (DERMS) Advanced Analytics Energy Management System (EMS)
VxRail P-Series High Performance Nodes (2) Top of Rack Network Switches 10gB Out of Band Management Switch (IDRAC) Data Backup Device 24U Rack Power Distribution Units UPS (optional)
Compute and storage nodes Secure connection into corporate network Remote access for support and administration Data backup and recovery Server and equipment rack In-Rack power distribution Uninterrupted Power System
UPS
Dell Technologies
vSphere VSAN VxRail Manager Vrealize Suite
System Hypervisor Software Dened Storage Administration Self Service Automation and Operations
VMware
Grid Modernization Solution Platform
UPS
VxRail HCI Nodes
Data Backup
Top Rack & Management Switch
Secondary Data Center
FIGURE 6. Dell Technologies and VMware’s standard components for GMP architecture
The benets of the Dell Technologies turnkey solution, all in one Grid Management Platform are as follows:
Virtualization improves reliability and supports rapid recovery
Moving to a virtual HCI platform streamlines operations and reduces operational expense
Proven HCI technologies like VxRail drive utility companies to move to advanced deployment
and lifecycle management through one click upgrades and automated VxRail Manager capabilities
Virtualization enhances data security, protection and recovery
VMware Validated Design integrates best practices for design, setup and seamless migration
8 | Grid Management Platform Common Design Architecture | Dell Technologies, VMware, and Intel
to an HCI platform
®
A CASE STUDY
A large investor owned utility is on a journey to develop an electric grid that increases reliability and supports the transition to a clean and sustainable energy future. They reviewed their operations to identify opportunities to improve existing practices and leverage new and emerging energy technologies to strengthen and modernize the grid. One technology that oered them tremendous promise for increased grid reliability and system resilience is machine virtualization in the bulk power system. Working in collaboration with Dell Technologies, VMware and Intel they chose the VxRail HCI platform with VMware’s SDDC architecture designed for regulated utilities as the platform architecture to standardize on for their grid management application portfolio. They felt this technology oered a more resilient, cost-eective, and reliable solution. By using standard IT practices, these systems become more redundant, more automated with system and device backups, and more stable with minimal physical engineering. To this end, they have partnered with Dell Technologies and VMware as their infrastructure platform provider for their ongoing grid and substation modernization projects.
CONCLUSION AND NEXT STEPS
Transforming and modernizing the energy industry is at the forefront of utility operational objectives. Traditional architectures in electric utilities are typically costly to maintain and inexible. Utilities seeking to modernize the Grid and enhance situational awareness, require a new approach, one that leverages a common, virtual architecture from the data center to the edge providing more agility and higher levels of interoperability, security, and reliability.
This Common Design Architecture paper shows how the Dell Technologies, VMware and Intel standard Grid Management Platform is leveraging emerging technologies to strengthen and modernize the grid, to improve reliability, security and safety, and operational practices. The use of machine virtualization in the electric utility grid is having a major impact on electric utilities’ operations, and VMware machine virtualization is the foundation of the Grid Management Platform.
The Grid Management Platform solution and technology oers a more exible, cost-eective, and dependable solution to deliver increased grid reliability and system resilience. Through proven projects delivered to date, we have found there is clear ROI evidence in the electric utility industry for moving away from today’s stand-alone, traditional grid architectures to a virtualized architecture on a standardized Grid Management Platform. The benets are reduced overall costs, including hardware, installation, engineering, and maintenance; server redundancy and reliability; automated self-monitoring and alerting; and enhanced cybersecurity.
Please contact your Dell Technologies, VMware or Intel representative for more information on how we can provide a comprehensive portfolio of products, solutions, services, and partnerships for electric utility companies to become agile and dynamic producers and suppliers of increasingly clean, ecient resources.
1. IDC White Paper: Delivering Ecient Business Expansion with Dell EMC VMware based HCI, page 13; October 2018 authors: Eric Sheppard, Matthew Marden
2. Grid Management System – A Key Enabler of Grid Modernization https://smartgrid.ieee.org/newsletters/august-2019/grid-management-system-a-key-enabler-of­grid-modernization.
© 2020 Dell Inc. or its subsidiaries. All rights reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel®, the Intel logo and Xeon® are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Reference number: H18551.
Dell Technologies believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Loading...