Dell Chassis Management Controller Carrier Grade User Manual

Carrier Grade

Dell Chassis Management

Controller Firmware

User’s Guide Addendum

Notes and Cautions

NOTE: A NOTE indicates important information that helps you make better use of

your computer.

CAUTION: A CAUTION indicates potential damage to hardware or loss of data if

instructions are not followed.

____________________

Reproduction of these materials in any manner whatsoever without the written permission of Dell Inc. is strictly forbidden. Trademarks used in this text: Dell™, the DELL logo, FlexAddress™ PowerEdge™ Explorer trademarks of Microsoft Corporation in the United States and other countries. Red Hat Enterprise Linux Novell countries. Intel Open Group in the United States and other countries. Avocent OSCAR

Copyright 1998-2006 The OpenLDAP Foundation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License. A copy of this license is available in the file LICENSE in the top-level directory of the distribution or, alternatively , at OpenLDAP Foundation. Individual files and/or contributed packages may be copyrighted by other parties and subject to additional restrictions. This work is derived from the University of Michigan LDAP v3.3 distribution. This work also contains materials derived from public sources. Information about OpenLDAP can be obtained at Copyright 1998-2004 Net Boolean Incorporated. Portions Copyright 2001-2004 IBM Corporation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License. Portions Copyright 1999-2003 Howard Y.H. Chu. Portions Copyright 1999-2003 Symas Corporation. Portions Copyright 1998-2003 Hallvard B. Furuseth. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that this notice is preserved. The names of the copyright holders may not be used to endorse or promote products derived from this software without their specific prior written permission. This software is provided "as is'' without express or implied warranty. Portions Copyright (c) 1992-1996 Regents of the University of Michigan. All rights reserved. Redistribution and use in source and binary forms are permitted provided that this notice is preserved and that due credit is given to the University of Michigan at Ann Arbor. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission. This software is provided "as is'' without express or implied warranty.

Other trademarks and trade names may be used in this publication to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own.

2012 - 11

and PowerConnect™ are trademarks of Dell Inc. Microsoft®, Active Directory®,

, Windows®, Windows Server®,

is a registered trademark and SUSE ™ is a trademark of Novell Inc. in the United States and other

are registered trademarks of Red Hat, Inc. in the United States and other countries.

is a registered trademark of Intel Corporation. UNIX® is a registered trademark of The

is a registered trademark of Avocent Corporation or its affiliates.

http://www .Open L DA P.org/license.html

http://www.o p e nld a p.org /

and Windows Vista®

are either trademarks or registered

is a trademark of Avocent Corporation.

. OpenLDAP is a registered trademark of the

OpenManage™,

Internet

and Red Hat

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Power Management

Grid Redundancy Mode

Grid Redundancy Levels

Power Supply Redundancy Mode

. . . . . . . . . . . . . . . . . . . . 5

. . . . . . . . . . . . . . . 6

. . . . . . . . . . 7

No Redundancy Mode . . . . . . . . . . . . . . . . 8

Power Budgeting for Hardware Modules

Server Slot Power Priority Settings

. . . . . . 9

. . . . . . . . 12

Dynamic Power Supply Engagement. . . . . . . . 13

Redundancy Policies

. . . . . . . . . . . . . . . . 15

Power Conservation and Power

Budget Changes . . . . . . . . . . . . . . . . . . 16

Power Conservation and Max Conservation Mode

. . . . . . . . . . . . . . . . . 16

Server Performance Over Power

Redundancy . . . . . . . . . . . . . . . . . . . . 17

Remote Logging

Using Web Interface

Using RACADM

. . . . . . . . . . . . . . . . . . . . . 18

. . . . . . . . . . . . . . . . 19

. . . . . . . . . . . . . . . . . . . 19

PSU Failure With Degraded or No Redundancy Policy

. . . . . . . . . . . . . . . . . . . 20

New Server Engagement Policy

. . . . . . . . . . . . . 20

PSU Removals With Degraded or No Redundancy Policy

Limitations

. . . . . . . . . . . . . . . . . . . 22

. . . . . . . . . . . . . . . . . . . . . 22

Contents 3

Power Supply and Redundancy Policy Changes

in System Event Log . . . . . . . . . . . . . . . . . . . 23

Redundancy Status and Overall Power Health

Configuring and Managing Power

. . . . . . . . . . . . 25

Viewing the Health Status of the PSUs

Viewing Power Consumption Status

Viewing Power Budget Status

. . . . . . . . . . . 33

. . . . . 25

. . . . . . . 25

. . . . . . . . 28

Configuring Power Budget and

Redundancy. . . . . . . . . . . . . . . . . . . . . 38

Assigning Priority Levels to Servers

. . . . . . . . 43

Setting Power Budget . . . . . . . . . . . . . . . 44

Server Power Reduction to Maintain

Power Budget

. . . . . . . . . . . . . . . . . . . . 45

4 Contents

Overview

This document provides additional information for the Carrier Grade Dell Chassis Management Controller when running with DC input power supply units in a Network Equipment-Building Standards (NEBS) configuration. The information presented in this addendum supersedes the information as presented in the Dell CMC Controller Firmware Version 4.1 User's Guide.

For more information, see the CMC Online Help for Carrier Grade CMC.

Power Management

The Dell PowerEdge M1000e server enclosure is the most power-efficient modular server enclosure in the market. It is designed to include highly efficient power supplies and fans, has an optimized layout so that air flows more easily through the system, and contains power-optimized components throughout the enclosure. The optimized hardware design is coupled with sophisticated power management capabilities built into the Chassis Management Controller (CMC), power supplies, and iDRAC to allow you to further enhance power efficiency and to have full control over your power environment.

The PowerEdge M1000e modular enclosure takes in power and distributes the load across all active internal power supply units (PSUs). The system can deliver up to 16685 Watts of input power that is allocated to server modules and the associated enclosure infrastructure. You can also control Power management through the Power Measure, Mitigate, and Manage Console (PM3). When PM3 controls power externally, CMC continues to maintain:

• Redundancy Policy

• Remote Power Logging

• Server Performance Over Power Redundancy

• Dynamic Power Supply Engagement

PM3 then manages:

• Server power

• Server priority

• System Input Power Capacity

• Maximum Power Conservation Mode

Carrier Grade Chassis Management Controller User’s Guide Addendum 5

For more information, see the External Power Management section in the Chassis Management Controller Version 4.1 User’s Guide.

NOTE: Actual power delivery is based on configuration and workload.

The Power Management features of the M1000e help administrators configure the enclosure to reduce power consumption and to customize power management to their unique requirements and environments. You can configure the PowerEdge M1000e enclosure for any of three redundancy policies that affect PSU behavior and determine how chassis Redundancy state is reported to administrators.

Grid Redundancy Mode

The purpose of the Grid redundancy policy is to enable a modular enclosure system to operate in a mode in which it can tolerate input power failures. These failures may originate in the input power grid, the cabling and delivery, or a PSU itself.

When you configure a system for Grid redundancy, the PSUs are divided into grids: PSUs in slots 1, 2, and 3 are in the first grid while PSUs in slots 4, 5, and 6 are in the second grid. CMC manages power so that if there is a failure of either grid the system continues to operate without any degradation. Grid redundancy also tolerates failures of individual PSUs.

NOTE: Since one role of Grid redundancy is to provide seamless server operation

despite failure of a whole power grid, the most power is available to maintain Grid redundancy when the capacities of the two grids are approximately equal.

NOTE: Grid redundancy is only met when the load requirements do not exceed the

capacity of the weakest power grid.

Grid Redundancy Levels

One PSU in each grid is the minimum configuration necessary for use as grid redundant. Additional configurations are possible with every combination that has at least one PSU in each grid. However, to make the maximum power available for use, the total power of the PSUs in each grid should be as close to equal as practical. The upper limit of power available to the M1000e while maintaining grid redundancy is the power available on the weaker of the two grids. Figure 1-1 illustrates two PSUs per grid and a power failure on grid 1. If

6 Carrier Grade Chassis Management Controller User’s Guide Addendum

for some reason CMC is unable to maintain grid redundancy, then E-mail

Power Supply #1

Power Supply #2

Empty Slot #3

Power Supply #4

Power Supply #5

Empty Slot #6

DC Power Grid #1

DC Power Grid #2

Chassis DC Power Bus

and/or SNMP alerts are sent to administrators if the Redundancy Lost event is configured for alerting.

Figure 1-1. 2 PSUs per grid and a power failure on grid 1

DC Power Grid #1

NOTE: In the event of a single PSU failure in this configuration, the remaining PSUs

in the failing grid are marked as Online. In this state, any of the remaining PSUs can fail without interrupting operation of the system. If a PSU fails, the chassis health is marked non-critical. If the smaller grid cannot support the total chassis power allocations then grid redundancy status is reported as No Redundancy and Chassis health is displayed as Critical.

Power Supply Redundancy Mode

The power supply redundancy mode is useful when redundant power grids are not available, but you may want to be protected against a single PSU failure bringing down your servers in a modular enclosure. The highest capacity PSU is kept in online reserve for this purpose. This forms a Power Supply redundancy pool. Figure 1-2 illustrates power supply redundancy mode. PSUs beyond those required for power and redundancy are still available and is added to the pool in the event of a failure. Unlike grid redundancy, when power supply redundancy is selected CMC does not require the PSU units to be present in any specific PSU slot positions.

Carrier Grade Chassis Management Controller User’s Guide Addendum 7

NOTE: Dynamic Power Supply Engagement (DPSE) allows PSUs to be placed in

Power Supply #1

Power Supply #2

Power Supply #3

Power Supply #4

Empty Slot #5

Empty Slot #6

Chassis DC Power Bus

Dual or Single Power Grid: Power Supply Redundancy protects against failure of a single power supply.

standby. The standby state indicates a physical state: that of not supplying power. When you enable DPSE, the extra PSUs may be placed in Standby mode to increase efficiency and save power.

Figure 1-2. Power Supply Redundancy: Totally 4 PSUs with a failure of one PSU.

No Redundancy Mode

The no redundancy mode is the factory default setting for a 3 PSU configuration and indicates that the chassis does not have any power redundancy configured. In this configuration, the overall redundancy status of the chassis always indicates No Redundancy. Figure 1-3 illustrates no redundancy mode is the factory default setting for 3 PSU configuration. CMC does not require the PSU units to be present in any specific PSU slot positions when No Redundancy is configured.

NOTE: All PSUs in the chassis are Online if DPSE is disabled when in No

Redundancy mode. When DPSE is enabled all active PSUs in the chassis are listed as Online and additional PSUs may be turned to Standby to increase the system's power efficiency.

8 Carrier Grade Chassis Management Controller User’s Guide Addendum

Figure 1-3. No Redundancy with three PSUs in the chassis

Power Supply #1

Power Supply #2

Power Supply #3

Empty Slot #4

Empty Slot #5

Empty Slot #6

DC Power Grid #1

Chassis DC Power Bus

Single Power Grid: No protection against grid or power supply failure

A PSU failure brings other PSUs out of Standby mode, as needed, to support the chassis power allocations. If you have 4 PSUs, and require only three, then in the event that one fails, the fourth PSU is brought online. A chassis can have all 6 PSUs online.

When you enable DPSE, the extra PSUs may be placed in Standby mode to increase efficiency and save power. For more information, see

Supply Engagement

Dynamic Power

Power Budgeting for Hardware Modules

Figure 1-4 illustrates a chassis that contains a six-PSU configuration. The PSUs are numbers 1-6, starting on the left-side of the enclosure.

Carrier Grade Chassis Management Controller User’s Guide Addendum 9

Figure 1-4. Chassis With Six-PSU Configuration

PSU1 PSU2 PSU3 PSU4 PSU5 PSU6

CMC maintains a power budget for the enclosure that reserves the necessary wattage for all installed servers and components. CMC allocates power to the CMC infrastructure and the servers in the chassis. CMC infrastructure consists of components in the chassis, such as fans, I/O modules, and iKVM (if present). The chassis may have up to 32 servers that communicate to the chassis through the iDRAC. For more information, see the iDRAC User's Guide at support.dell.com/manuals.

iDRAC provides CMC with its power envelope requirements before powering up the server. The power envelope consists of the maximum and minimum power requirements necessary to keep the server operating. iDRAC's initial estimate is based on its initial understanding of components in the server. After operation commences and further components are discovered, iDRAC may increase or decrease its initial power requirements.

When a server is powered-up in an enclosure, the iDRAC software reestimates the power requirements and requests a subsequent change in the power envelope.

10 Carrier Grade Chassis Management Controller User’s Guide Addendum

CMC grants the requested power to the server, and the allocated wattage is subtracted from the available budget. Once the server is granted a power request, the server's iDRAC software continuously monitors the actual power consumption. Depending on the actual power requirements, the iDRAC power envelope may change over time. iDRAC requests a power step-up only if the servers are fully consuming the allocated power.

Under heavy load the performance of the server's processors may be degraded to ensure power consumption stays below the user-configured System Input Power Cap. The PowerEdge M1000e enclosure can supply enough power for peak performance of most server configurations, but many available server configurations do not consume the maximum power that the enclosure can supply. To help data centers provision power for their enclosures, the M1000e allows you to specify a System Input Power Cap to ensure that the overall chassis input power draw stays under a given threshold. CMC first ensures enough power is available to run the fans, IO Modules, iKVM (if present), and CMC itself. This power allocation is called the Input Power Allocated to Chassis Infrastructure. Following Chassis Infrastructure, the servers in an enclosure are powered up. Any attempt to set a System Input Power Cap below the actual consumption fails.

If necessary for the total power budget to stay below the value of the System Input Power Cap, CMC allocates servers a value less than their maximum requested power. Servers are allocated power based on their Server Priority setting, with higher priority servers getting maximum power, priority 2 servers getting power after priority 1 servers, and so on. Lower priority servers may get less power than priority 1 servers based on System Input Max Power Capacity and the user-configured setting of System Input Power Cap.

Configuration changes, such as an additional server in the chassis, may require the System Input Power Cap to be increased. Power needs in a modular enclosure also increase when thermal conditions change and the fans are required to run at higher speed, which causes them to consume additional power. Insertion of I/O modules and iKVM also increases the power needs of the modular enclosure. A fairly small amount of power is consumed by servers even when they are powered down to keep the management controller powered up.

Carrier Grade Chassis Management Controller User’s Guide Addendum 11

Additional servers can be powered up in the modular enclosure only if sufficient power is available. The System Input Power Cap can be increased any time up to a maximum value of 16685 watts to allow the power up of additional servers.

Changes in the modular enclosure that reduce the power allocation are:

•Server power off

•Server

•I/O module

• iKVM removal

• Transition of the chassis to a powered off state

You can reconfigure the System Input Power Cap when chassis is either ON or OFF.

NOTE: While inserting a server with geometry other than single height and if there

is insufficient power for the iDRAC, the server is displayed as multiple single-height servers.

Server Slot Power Priority Settings

CMC allows you to set a power priority for each of the sixteen server slots in an enclosure. The priority settings are 1 (highest) through 9 (lowest). These settings are assigned to slots in the chassis, and the slot's priority is inherited by any server inserted in that slot. CMC uses slot priority to preferentially budget power to the highest priority servers in the enclosure.

According to the default server slot priority setting, power is equally apportioned to all slots. Changing the slot priorities allows administrators to prioritize which servers are given preference for power allocations. If the more critical server modules are left at their default slot priority of 1, and the less critical server modules are changed to lower priority value of 2 or higher, the priority 1 server modules would be powered on first. These higher priority servers would then get their maximum power allocation, while lower priority servers may be not be allocated enough power to run at their maximum performance or they may not even power on at all, depending on how low the system input power cap is set and the server power requirements. If an administrator manually powers on the low priority server modules before the higher priority ones, then the low priority server modules are the first modules to have their power allocation lowered down to the minimum value, in order

12 Carrier Grade Chassis Management Controller User’s Guide Addendum

to accommodate the higher priority servers. So after the available power for allocation is exhausted, then CMC reclaims power from lower or equal priority servers until they are at their minimum power level.

NOTE: I/O modules, fans, and iKVM (if present) are given the highest priority. CMC

reclaims power only from lower priority devices to meet the power needs of a higher priority module or server.

Dynamic Power Supply Engagement

Dynamic Power Supply Engagement (DPSE) mode is disabled by default. DPSE saves power by optimizing the power efficiency of the PSUs supplying power to the chassis. This also results in increased PSU life, and reduced heat generation.

CMC monitors total enclosure power allocation, and moves the PSUs into Standby state, causing the total power allocation of the chassis to be delivered through fewer PSUs. Since the online PSUs are more efficient when running

at higher utilization, this improves their efficiency while also improving longevity of the standby PSUs.

To operate remaining PSUs at their maximum efficiency:

• No Redundancy mode with DPSE is highly power efficient, with optimal PSUs online. PSUs that are not needed are placed in standby mode.

• PSU Redundancy mode with DPSE also provides power efficiency. At least two supplies are online, with one PSU required to power the configuration and one to provide redundancy in case of PSU failure. PSU Redundancy mode offers protection against the failure of any one PSU, but offers no protection in the event of input power grid loss.

• Grid Redundancy mode with DPSE, where at least two of the supplies are active, one on each power grid, provides a good balance between efficiency and maximum availability for a partially-loaded modular enclosure configuration.

• Disabling DPSE provides the lowest efficiency as all six supplies are active and share the load, resulting in lower utilization of each power supply.

DPSE can be enabled for all three power supply redundancy configurations explained above - No Redundancy, Power Supply Redundancy, and Grid Redundancy.

Carrier Grade Chassis Management Controller User’s Guide Addendum 13

• In a No Redundancy configuration with DPSE, the M1000e can have up to five power supply units in Standby state. In a six PSU configuration, some PSU units are placed in Standby and are not utilized to improve power efficiency. Removal or failure of an online PSU in this configuration cause a PSU in Standby state to change to Online; however, standby PSUs can take up to two seconds to become active, so some server modules may lose power during the transition in the No Redundancy configuration.

NOTE: In a three PSU configuration, server load may prevent any PSUs from

transitioning to Standby.

• In a Power Supply Redundancy configuration, the enclosure always keeps an additional PSU powered on and marked Online in addition to the PSUs required to power the enclosure. Power utilization is monitored and up to four PSUs could be moved to Standby state depending on the overall system load. In a six PSU configuration, a minimum of two power supply units are always powered on.

Since an enclosure in the Power Supply Redundancy configuration always has an extra PSU engaged, the enclosure can tolerate the loss of one online PSU and still have enough power for the installed server modules. The loss of the online PSU causes a standby PSU to come online. Simultaneous failure of multiple PSUs may result in the loss of power to some server modules while the standby PSUs are powering up.

• In Grid Redundancy configuration, all power supplies are engaged at chassis power up. Power utilization is monitored, and if system configuration and power utilization allows, PSUs are moved to the Standby state. Since the Online status of PSUs in a grid mirrors that of the other grid, the enclosure can sustain the loss of power to an entire grid with no interruption of power to the enclosure.

An increase in power demand in the Grid Redundancy configuration causes the engagement of PSUs from the Standby state. This maintains the mirrored configuration needed for dual-grid redundancy.

NOTE: With DPSE Enabled, the Standby PSUs are brought Online to reclaim power

if power demand increases in all three Power Redundancy policy modes.

14 Carrier Grade Chassis Management Controller User’s Guide Addendum

+ 32 hidden pages