IBM SAN768B-2 Installation, Service And User Manual

IBM System Storage SAN768B-2



Installation, Service, and User Guide

Service information: 2499-816

Read Before Using

This product contains software that is licensed under written license agreements. Your use of such software is subject to the license agreements under which they are provided.

GA32-0893-00

IBM System Storage SAN768B-2



Installation, Service, and User Guide

Service information: 2499-816

GA32-0893-00

Note:

Before using this information and the product it supports, read the information in “Notices” on page 143.

The following paragraph does not apply to any country (or region) where such provisions are inconsistent with local law.

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states (or regions) do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you.

US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Read this first

Getting help

For the latest version of your product documentation, visit the web at www.ibm.com/shop/publications/order. Search by form number or title.

Redbooks®often provide in depth information about product best practices,

IBM configurations, and more technical information. For redbooks associated with this product, enter search terms on the following Web site: www.redbooks.ibm.com/.

For more information about IBM SAN products, see the following Web site: www.ibm.com/servers/storage/san/

For support information for this and other IBM products, see the IBM Support Portal, www.ibm.com/supportportal. Search for the product Machine type or product name.

For Fabric OS Release Notes and access to Fabric OS firmware downloads, go to the IBM Support Portal, www.ibm.com/supportportal. Search for the product Machine type or product name, and then follow links for Downloads. More detailed instructions are available through the Accessing firmware updates and OS documentation updates link on the product documentation CD that is shipped with this product.

You can also contact IBM within the United States at 1-800-IBMSERV (1-800-426-7378). For support outside the United States, you can find the service number at: www.ibm.com/planetwide/.

Visit www.ibm.com/contact for the contact information for your country or region.

For detailed information about the Fibre Channel standards, see the Fibre Channel Industry Association (FCIA) Web site at: www.fibrechannel.org/

For information about storage industry standards, see the Storage Networking Industry Association (SNIA) Web site at: www.snia.org/

Taiwan Contact Information

IBM Taiwan Product Service Contact Info: IBM Taiwan Corporation 3F, No 7, Song Ren Rd., Taipei Taiwan Tel: 0800-016-888

How to send your comments

Your feedback is important in helping us provide the most accurate and high-quality information. If you have comments or suggestions for improving this document, send us your comments by e-mail to starpubs@us.ibm.com . Be sure to include the following:

v Exact publication title v Publication form number (for example, GC26-1234-02) v Page, table, or illustration numbers v A detailed description of any information that should be changed

iv SAN768B-2 Installation, Service, and User Guide

Contents

Read this first ............iii

Getting help ..............iii

Taiwan Contact Information ........iii

How to send your comments ........iv

Figures ...............ix

Tables ...............xi

Safety and environmental notices . . . xiii

Safety notices and labels..........xiii

Danger notices ............xiii

Caution notices ............xvi

Safety labels ............xviii

Attention notices ...........xix

Rack safety ..............xx

Rack installation............xx

Rack relocation (19" rack) ........xxi

Safety inspections ............xxii

Removing ac power ..........xxii

External machine checks ........xxii

Internal machine checks .........xxii

Product recycling and disposal .......xxiii

About this document ........xxv

Who should read this document .......xxv

Product documents ...........xxv

Brocade documents ...........xxv

IBM and Brocade product matrix .....xxvi

Accessibility features for SAN768B-2 ....xxvi

Chapter 1. Introduction ........1

Overview of the SAN768B-2 .........1

Hardware components ...........3

SAN768B-2 blades ............5

High availability .............7

Reliability ...............8

Serviceability ..............8

Software features .............8

Security ...............9

Network Manageability .........10

Chapter 2. Installing and removing a

SAN768B-2 .............11

Ordering the lift tool ...........12

World trade locations ..........12

United States locations .........12

Installation guidelines ...........13

Installing a SAN768B-2 in a cabinet ......14

Time required ............14

Tools required ............15

Parts list ..............15

Unpacking the SAN768B-2 ........17

Attaching the shelf brackets ........18

Removing the chassis door and cable

management comb ...........19

Using the lift tool, the 24–inch load plate, and the

bridge tool .............20

Port numbering .............26

Chassis slots ..............27

Cable organization ............27

Removing a SAN768B-2 from the cabinet ....29

Chapter 3. Starting and configuring the

SAN768B-2 .............31

Providing power to the SAN768B-2 ......33

Establishing a serial connection and logging on to

the SAN768B-2 .............33

Logging in to the serial console port ......34

Configuring the IP addresses ........34

Logging off the serial console port and

disconnecting the serial cable ........35

Establishing an Ethernet connection ......36

Customize a switch name .........36

Customizing a chassis name .........36

Setting the Domain ID...........37

Setting the date and time ..........37

Setting the date ............37

Setting the time zone ..........37

Synchronizing local time .........38

Verifying the PID mode ..........39

Determining installed software licenses .....39

Installing transceivers and attaching cables ....39

Managing cables.............42

Verifying correct operation and backing up the

configuration ..............42

Fabric OS firmware updates .........43

Downloading and installing firmware ....43

Powering off the SAN768B-2 ........44

Chapter 4. Monitoring system

components ............45

Port or application blade status........51

Control processor blade (CP8) status ......57

Core switch blade (CR16-8) status .......59

Power supply status ...........61

Blower assembly status ..........62

WWN bezel and WWN card status ......64

Chapter 5. Removing and installing

components ............67

Removing and installing the chassis door ....67

Time and items required .........67

Removing the chassis door ........67

Removing and installing a cable management comb 68

Time and items required .........68

Removing a cable management comb.....68

Installing a cable management comb .....69

Removing and installing port and application blades 69

Time and items required .........70

Removing a blade ...........70

Installing a blade ...........71

Removing and installing blade filler panels ....72

Removing a filler panel .........72

Installing a filler panel..........73

Removing and installing a CP8 control processor

blade ................73

Time and items required .........74

Verifying the need for replacement .....74

Recording critical SAN768B-2 information . . . 75

Removing a control processor blade (CP8) . . . 76

Installing a control processor blade (CP8) . . . 77

Verifying operation of the new CP blade....78

Downloading firmware from an FTP server. . . 79

Downloading firmware from a USB drive . . . 81

Completing the CP8 blade replacement ....83

Removing and installing a core switch blade

(CR16-8) ...............83

Time and items required .........84

Verifying the necessity of installation .....84

Removing a core switch blade (CR16-8) ....84

Installing a core switch blade (CR16-8) ....85

Removing and installing a power supply ....86

Time and items required .........86

Removing a power supply ........86

Installing a power supply ........87

Removing and installing a blower assembly . . . 88

Time and items required .........88

Removing a blower assembly .......88

Installing a blower assembly........89

Removing and installing a WWN bezel and WWN

card.................89

Time and items required .........89

Verifying the need for replacement .....90

Removing the WWN bezel and WWN card. . . 90

Installing the WWN bezel and WWN card . . . 91

Removing and installing transceivers ......92

Time required ............92

Items required ............92

Removing and installing an SFP+ optical

transceiver..............92

Removing and installing an mSFP optical

transceiver..............94

Removing and replacing inter-chassis link (QSFP)

cables ................95

Time and items required .........97

Removing an inter-chassis link (QSFP) cable . . 97

Installing an ICL cable ..........97

Possible ICL configurations ........98

Removing the batteries ..........100

Removing and replacing the chassis ......101

Verifying necessity of replacement .....102

Preparing for a chassis replacement .....102

Disconnecting the SAN768B-2 from the network

and fabric .............105

Removing components from the chassis . . . 106

Removing and replacing the chassis from the

cabinet ..............106

Installing components into the new chassis . . 107

Downloading the configuration ......108

Verifying the correct operation of the system 108 Reconnecting the system to the network and

fabric ...............110

Verifying the correct configuration of the fabric 110

Cable routing tables ..........111

Chapter 6. Installing new features . . 115

FC3632, FC3648- Installing a 16-Gb port port blade

(32 or 48 ports) .............115

Time required ............115

Items required ............115

Procedures .............116

FC3864 - Installing an 8-Gb 64-port port blade . . 117

Time required ............117

Items required ............117

Procedures .............117

FC3890 - Installing an FX8-24 extension blade . . 119

Time required ............119

Items required ............119

Procedures .............119

FC7871 - Installing a QSFP and inter-chassis license 120

Time required ............120

Items required ............120

Procedures .............120

Appendix A. Product specifications 123

General specifications ..........123

System architecture ...........123

System size and weight ..........126

Facility requirements ...........126

Power specifications ...........127

Environmental requirements ........128

Fibre channel port specifications .......128

Data transmission ranges .........129

Powercords..............129

Appendix B. Application blades . . . 131

FX8-24 blade .............131

Appendix C. Diagnostics and

troubleshooting ..........133

Obtaining chassis and component status ....133

Interpreting POST and boot results ......134

POST...............134

Boot ...............135

Diagnostics ..............135

Troubleshooting ............135

Appendix D. Blade port numbering 139

Notices ..............143

Trademarks ..............145

Electronic emission notices .........146

Federal Communications Commission (FCC)

Class A Statement ...........146

Industry Canada Class A Emission Compliance

Statement .............146

vi SAN768B-2 Installation, Service, and User Guide

Avis de conformité à la réglementation

d'Industrie Canada ..........146

European Union EMC Directive Conformance

Statement .............146

Germany Electromagnetic Compatibility

Directive ..............147

People's Republic of China Class A Electronic

Emission Statement ..........148

Japan Voluntary Control Council for Interference

(VCCI) Class A Statement ........148

Japan Electronics and Information Technology Industries Association (JEITA) Statement . . . 148 Korea Communications Commission (KCC)

Class A Statement ...........148

Russia Electromagnetic Interference (EMI) Class

A Statement .............149

Australia and New Zealand Class A Statement 149

Index ...............151

Contents vii

viii SAN768B-2 Installation, Service, and User Guide

Figures

1. Port side of the SAN768B-2 (sample

configuration) ............4

2. Blower (non-port) side of the SAN768B-2

(sample configuration ) .........5

3. Parts in the 14U rack mount kit ......17

4. Left and right shelf brackets installed on the

cabinet rails ............18

5. Installing clip or retainer nuts on the vertical

cabinet rails ............19

6. Chassis door removal .........20

7. Chassis installation from rear (exhaust aisle

side) of the cabinet ..........22

8. Locking the shelf into position ......23

9. Attaching the port side of the chassis to the

cabinet rails ............24

10. Aligning the chassis door with the ball studs

on the chassis ............25

11. Details of the chassis and door, showing the ball studs on the chassis and the mounting

hardware on the door .........26

12. Cable design for the mSFP patch cables for the

FC8-64 high density port blade ......28

13. Flow of basic configuration tasks .....32

14. FC16-32 port blade ..........52

15. FC16-48 port blade ..........53

16. FC8-64 port blade ..........54

17. FX8-24 extension blade .........55

18. Control processor blade (CP8) components 58

19. Core switch blade (CR16-8), LEDs and QSFP

connectors .............60

20. Power supply LEDs ..........62

21. Blower assembly LEDs .........63

22. LEDs on the WWN card and bezel .....65

23. Chassis door ............68

24. Cable management comb ........69

25. Port and application removal and installation (FC16-48 port blade shown as example) . . . 71

26. Filler panel removal and installation ....73

27. Removing the control processor (CP8) blade 77

28. Removing the core switch blade......85

29. Power supply locations ........86

30. Removing and installing a power supply 87

31. Removal and replacement of the blower

assembly .............89

32. WWN bezel and card removal and installation 91

33. Optical transceiver (SFP+) extraction tool 93

34. Installing and removing a 16 Gbps SFP+ with pull tab (shown without cable attached) . . . 93

35. Removing and replacing an SFP+ optical

transceiver with a wire bail .......94

36. Optical mSFP transceiver ........95

37. QSFP cable and transceiver .......97

38. QSFP cable connections–sample configuration,

parallel type ............99

39. Battery locations on the CP8 blade ....101

40. FC16-32 port blade ..........139

41. FC16-48 port blade ..........140

42. FC8-64 port blade ..........141

43. FX8-24 extension blade ........142

x SAN768B-2 Installation, Service, and User Guide

Tables

1. Brocade and IBM product and model

number matrix ...........xxvi

2. Blades available for the SAN768B-2 .....5

3. Installation tasks, time, and items required 14

4. Items supplied with the 14U rack mount kit 15

5. Port and application blades available on the

SAN768B-2.............26

6. Port and application blade LED descriptions 56

7. CP8 blade LED descriptions .......59

8. CR16-8 blade LED descriptions ......60

9. Power supply LED descriptions......62

10. Blower assembly LED descriptions.....63

11. Messages that may indicate WWN card failure 64

12. WWN LED patterns..........65

13. Steps for upgrading through multiple versions

of Fabric OS ............79

14. ICL connector port LEDs ........96

15. Critical information checklist example 103

16. Sample cable routing table for SAN768B-2 112

17. Cable routing table for SAN768B-2 (64 ports

shown)..............112

18. General specifications.........123

19. System architecture .........123

20. System sizes and weights .......126

21. System FRU weights .........126

22. Power specifications .........127

23. Environmental requirements ......128

24. Supported optics, speeds, cables, and

distances .............129

25. Environmental status and maintenance

commands ............134

26. Troubleshooting...........135

xii SAN768B-2 Installation, Service, and User Guide

Safety and environmental notices

This section contains information about:

v “Safety notices and labels” v “Rack safety” on page xx v “Safety inspections” on page xxii v “Product recycling and disposal” on page xxiii

Safety notices and labels

When using this product, observe the danger, caution, and attention notices contained in this guide. The notices are accompanied by symbols that represent the severity of the safety condition. The danger and caution notices are listed in numerical order based on their IDs, which are displayed in parentheses, for example (D004), at the end of each notice. Use this ID to locate the translation of these danger and caution notices in the IBM System Storage and TotalStorage b-type Directors and Cabinets Safety Notices publication, which is shipped with this product.

The following notices and statements are used in IBM documents. They are listed below in order of increasing severity of potential hazards. Follow the links for more detailed descriptions and examples of the danger, caution, and attention notices in the sections that follow.

v Note: These notices provide important tips, guidance, or advice. v “Attention notices” on page xix: These notices indicate potential damage to

programs, devices, or data.

v “Caution notices” on page xvi: These statements indicate situations that can be

potentially hazardous to you.

v “Danger notices”: These statements indicate situations that can be potentially

lethal or extremely hazardous to you. Safety labels are also attached directly to products to warn of these situations.

v In addition to these notices, “Safety labels” on page xviii may be attached to the

product to warn of potential hazards.

Danger notices

A danger notice calls attention to a situation that is potentially lethal or extremely hazardous to people. A lightning bolt symbol accompanies a danger notice to represent a dangerous electrical condition. Read and comply with the following danger notices before installing or servicing this device.

DANGER

To prevent a possible shock from touching two surfaces with different protective ground (earth), use one hand, when possible, to connect or disconnect signal cables. (D001)

DANGER

Overloading a branch circuit is potentially a fire hazard and a shock hazard under certain conditions. To avoid these hazards, ensure that your system electrical requirements do not exceed branch circuit protection requirements. Refer to the information that is provided with your device or the power rating label for electrical specifications. (D002)

DANGER

If the receptacle has a metal shell, do not touch the shell until you have completed the voltage and grounding checks. Improper wiring or grounding could place dangerous voltage on the metal shell. If any of the conditions are not as described, STOP. Ensure the improper voltage or impedance conditions are corrected before proceeding. (D003)

DANGER

An electrical outlet that is not correctly wired could place hazardous voltage on metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (D004)

A general electrical danger notice provides instructions on how to avoid shock hazards when servicing equipment. Unless instructed otherwise, follow the procedures in the following danger notice.

xiv SAN768B-2 Installation, Service, and User Guide

DANGER

When working on or around the system, observe the following precautions:

Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard:

v Connect power to this unit only with the IBM provided power

cord. Do not use the IBM provided power cord for any other product.

v Do not open or service any power supply assembly.

v Do not connect or disconnect any cables or perform installation,

maintenance, or reconfiguration of this product during an electrical storm.

v The product might be equipped with multiple power cords. To

remove all hazardous voltages, disconnect all power cords.

v Connect all power cords to a properly wired and grounded

electrical outlet. Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate.

v Connect any equipment that will be attached to this product to

properly wired outlets.

v When possible, use one hand only to connect or disconnect

signal cables.

v Never turn on any equipment when there is evidence of fire,

water, or structural damage.

v Disconnect the attached power cords, telecommunications

systems, networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures.

v Connect and disconnect cables as described below when

installing, moving, or opening covers on this product or attached devices.

To Disconnect:

1. Turn off everything (unless instructed otherwise).

2. Remove the power cords from the outlets.

3. Remove the signal cables from the connectors.

4. Remove all cables from the devices.

To Connect:

1. Turn off everything (unless instructed otherwise).

2. Attach all cables to the devices.

3. Attach the signal cables to the connectors.

4. Attach the power cords to the outlets.

5. Turn on the devices.

(D005)

The weight of the SAN cabinet (2109-C36) with two SAN768B-2s is greater than 227 kg (500 lb), and has a fully loaded capacity of 816 kg (1795 lb). Under these conditions, the following statement and notice apply.

Safety and environmental notices xv

Delivery and subsequent transportation of the equipment

The customer should prepare his environment to accept the new product based on the installation planning information provided, with assistance from an IBM Installation Planning Representative (IPR) or IBM authorized service provider. In anticipation of the equipment delivery, the final installation site should be prepared in advance such that professional movers/riggers can transport the equipment to the final installation site within the computer room. If for some reason, this is not possible at the time of delivery, the customer will need to make arrangements to have professional movers/riggers return to finish the transportation at a later date. Only professional movers/riggers should transport the equipment. The IBM authorized service provider will only perform minimal frame repositioning within the computer room, as needed, to perform required service actions. The customer is also responsible for using professional movers/riggers in the case of equipment relocation or disposal.

>(>)500 lbs. 227 kg.

Caution notices

A caution notice calls attention to a situation that is potentially hazardous to people because of some existing condition. A caution notice can be accompanied by different symbols, as in the examples below:

DANGER

Heavy equipment—personal injury or equipment damage might result if mishandled. (D006)

a69i0333

If the symbol

is... It means....

A hazardous electrical condition with less severity than electrical danger.

A generally hazardous condition not represented by other safety symbols.

A specification of product weight that requires safe lifting practices. The weight range of the product is listed below the graphic, and the wording of the caution varies, depending on the weight of the device.

55 kg ( 121.2 lbs)

svc00169

>55kg (121.2 lb)

A potential hazard of pinching the hand or other body parts between parts.

P/N 18P5850-B

SJ000752

A hazardous condition due to moving parts nearby.

A hazardous condition due to the use of a laser in the product. Laser symbols are always accompanied by the classification of the laser as defined by the U. S. Department of Health and Human Services (for example, Class I, Class II, and so forth).

xvi SAN768B-2 Installation, Service, and User Guide

Read and comply with the following caution notices before installing or servicing this device.

CAUTION: Energy hazard present. Shorting may result in system outage and possible physical injury. Remove all metallic jewelry before servicing. (C001)

CAUTION: The weight of this part or unit is more than 55 kg (121.2 lb). It takes specially trained persons, a lifting device, or both to safely lift this

55 kg ( 121.2 lbs)

>55kg (121.2 lb)

part or unit. (C011)

svc00169

CAUTION: The system contains circuit cards, assemblies, or both that may contain lead solder. To avoid the release of lead (Pb) into the environment, do not burn. Discard the circuit card as instructed by local regulations. (C014)

CAUTION: This product is equipped with a 3-wire (two conductors and ground) power cable and plug. Use this power cable with a properly grounded electrical outlet to avoid electrical shock. (C018)

CAUTION: This product might contain one or more of the following devices: CD-ROM drive, DVD-ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information:

v Do not remove the covers. Removing the covers of the laser product

could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device.

v Use of the controls or adjustments or performance of procedures

other than those specified herein might result in hazardous radiation exposure.

(C026)

CAUTION: The power-control button on the device does not turn off the electrical current supplied to the device. The device might also have more than one connection to dc power. To remove all electrical current from the device, ensure that all connections to dc power are disconnected at the dc power input terminals. (C031)

CAUTION: Servicing of this product or unit is to be performed by trained service personnel only. (C032)

Safety and environmental notices xvii

CAUTION: For CA residents only: IBM recommends installing this product in a room size of 62 cubic meters (2190 cubic feet) or larger at 0.4 ACH ventilation rate to reduce the concentrations of any chemicals emitted by the product.

Safety labels

As an added precaution, safety labels are often installed directly on products or product components to warn of potential hazards. These can be either danger or caution notices, depending upon the level of the hazard.

The actual product safety labels may differ from these sample safety labels:

DANGER

Hazardous voltage, current, or energy levels are present inside any component that has this label attached. Do not open any cover or barrier that contains this label. (L001)

DANGER

Rack-mounted devices are not to be used as a shelf or work space. (L002)

DANGER

Multiple power cords. The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords. (L003)

DANGER

Hazardous voltage present. Voltages present constitute a shock hazard, which can cause severe injury or death. (L004)

CAUTION: Hazardous energy present. Voltages with hazardous energy might cause heating when shorted with metal, which might result in splattered metal, burns, or both. (L005)

xviii SAN768B-2 Installation, Service, and User Guide

CAUTION: Hazardous moving parts nearby (L008)

CAUTION: Pinch hazard. (L012)

P/N 18P5850-B

SJ000752

Attention notices

An attention notice indicates the possibility of damage to a program, device, or system, or to data. An exclamation point symbol may accompany an attention notice, but is not required. A sample attention notice follows:

Attention: Do not bend a fibre cable to a radius less than 5 cm (2 in.); you can damage the cable. Tie wraps are not recommended for optical cables because they can be easily overtightened, causing damage to the cable.

ESD precautions

Attention: Many of the field replaceable units (FRUs) are sensitive to electrostatic discharge (ESD), and can potentially be damaged by improper handling. Wear a wrist grounding strap connected to chassis ground (if the SAN768B-2 is plugged in) or a bench ground. Store all ESD-sensitive components in antistatic packaging.

Safety and environmental notices xix

Rack safety

Rack installation

DANGER

Observe the following precautions when working on or around your IT rack system:

v Heavy equipment—personal injury or equipment damage might result if

mishandled.

v Always lower the leveling pads on the rack cabinet.

v Always install stabilizer brackets on the rack cabinet.

v To avoid hazardous conditions due to uneven mechanical loading, always install the

heaviest devices in the bottom of the rack cabinet. Always install servers and optional devices starting from the bottom of the rack cabinet.

v Rack-mounted devices are not to be used as shelves or work spaces. Do not place

objects on top of rack-mounted devices.

v Each rack cabinet might have more than one power cord. Be sure to disconnect all

power cords in the rack cabinet when directed to disconnect power during servicing.

v Connect all devices installed in a rack cabinet to power devices installed in the

same rack cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power device installed in a different rack cabinet.

v An electrical outlet that is not correctly wired could place hazardous voltage on the

metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock.

(R001 part 1 of 2)

CAUTION:

v Do not install a unit in a rack where the internal rack ambient temperatures will

exceed the manufacturer ’s recommended ambient temperature for all your rack-mounted devices.

v Do not install a unit in a rack where the air flow is compromised. Ensure that air flow

is not blocked or reduced on any side, front, or back of a unit used for air flow through the unit.

v Consideration should be given to the connection of the equipment to the supply circuit

so that overloading of the circuits does not compromise the supply wiring or overcurrent protection. To provide the correct power connection to a rack, refer to the rating labels located on the equipment in the rack to determine the total power requirement of the supply circuit.

v (For sliding drawers) Do not pull out or install any drawer or feature if the rack stabilizer

brackets are not attached to the rack. Do not pull out more than one drawer at a time. The rack might become unstable if you pull out more than one drawer at a time.

v (For fixed drawers) This drawer is a fixed drawer and must not be moved for servicing

unless specified by the manufacturer. Attempting to move the drawer partially or completely out of the rack might cause the rack to become unstable or cause the drawer to fall out of the rack.

(R001 part 2 of 2)

xx SAN768B-2 Installation, Service, and User Guide

Rack relocation (19" rack)

CAUTION: Removing components from the upper positions in the rack cabinet improves rack stability during relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a room or building:

v Reduce the weight of the rack cabinet by removing equipment starting at the

top of the rack cabinet. When possible, restore the rack cabinet to the configuration of the rack cabinet as you received it. If this configuration is not known, you must do the following:

– Remove all devices in the 32U position and above.

– Ensure that the heaviest devices are installed in the bottom of the rack

cabinet.

– Ensure that there are no empty U-levels between devices installed in the

rack cabinet below the 32U level.

– If the rack cabinet you are relocating is part of a suite of rack cabinets,

detach the rack cabinet from the suite.

– Inspect the route that you plan to take when moving the rack to eliminate

potential hazards.

– Verify that the route that you choose can support the weight of the loaded

rack cabinet. Refer to the documentation that came with your rack cabinet for the weight of a loaded rack cabinet.

– Verify that all door openings are at least 760 x 2030 mm (30 x 80 in.).

– Ensure that all devices, shelves, drawers, doors, and cables are secure.

– Ensure that the four leveling pads are raised to their highest position.

– Ensure that there is no stabilizer bracket installed on the rack cabinet

during movement.

– Do not use a ramp inclined at more than 10 degrees.

– Once the rack cabinet is in the new location, do the following:

- Lower the four leveling pads.

- Install stabilizer brackets on the rack cabinet.

- If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest position to the highest position.

– If a long distance relocation is required, restore the rack cabinet to the

configuration of the rack cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent. Also, lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the pallet.

(R002)

Safety and environmental notices xxi

Safety inspections

Perform the following safety checks to identify unsafe conditions. Be cautious of potential safety hazards that are not covered in the safety checks. If unsafe conditions are present, determine how serious the hazards are and whether you should continue before you correct the problem.

Removing ac power

Perform the following steps to remove the alternating current (ac) power:

1. Perform a controlled system shutdown.

2. Set the power switch on the product to the off position.

3. Disconnect the power cables from the power source.

External machine checks

Perform the following external machine checks:

1. Verify that all external covers are present and are not damaged.

2. Ensure that all latches and hinges are in correct operating condition.

3. Check the power cable for damage.

4. Check the external signal cable for damage.

5. Check the cover for sharp edges, damage, or alterations that expose the internal

6. Check that any unused serial ports are covered for dust and ESD protection.

7. Correct any problems that you find.

DANGER

Multiple power cords. (L003)

parts of the device.

The cover should be kept on the serial port whenever it is not being used.

Internal machine checks

Perform the following internal machine checks:

1. Check for any non-IBM changes that might have been made to the machine. If any are present, obtain the “Non-IBM Alteration Attachment Survey” form, number R009, from the IBM branch office. Complete the form and return it to the branch office.

2. Check the condition of the inside of the machine for:

v Metal or other contaminants v Indications of water or other fluid v Fire v Smoke damage

3. Check for any obvious mechanical problems, such as loose components.

4. Check any exposed cables and connectors for wear, cracks, or pinching.

xxii SAN768B-2 Installation, Service, and User Guide

Product recycling and disposal

Refer to the IBM Systems Environmental Notices and User Guide (Z125-5823) on the CD shipped with the product for translated environmental statements and information regarding product recycling and disposal. This document may be provided either in printed version or on a documentation CD. See “Removing the batteries” on page 100 for battery removal instructions, if needed to meet environmental regulations.

Safety and environmental notices xxiii

xxiv SAN768B-2 Installation, Service, and User Guide

About this document

This document describes how to install and service the IBM System Storage SAN768B-2 fabric backbone. Throughout this document, the product is referred to as the SAN768B-2, or more generically as system, device,orchassis, where appropriate.. This document includes information specific to the SAN768B-2 running Fabric OS version 7.0. and later. For information about a Fabric OS version other than 7.0, refer to the documentation specific to your Fabric OS version.

The sections that follow provide information about:

v “Who should read this document” v “Product documents” v “Brocade documents”

Who should read this document

This document is intended for trained service representatives who are responsible for installing and servicing the SAN768B-2, and for network administrators responsible for maintaining and using the SAN768B-2.

Product documents

The following documents contain information related to this product. The documentation may be printed material, on the documentation CD that is shipped with the product, or available on the web through the IBM Support Portal or IBM Publications Center.

v IBM System Storage SAN768B-2 Installation, Service, and User Guide, GA32-0893

(this document, which is also available in accessible HTML format on the documentation CD).

v IBM System Storage and TotalStorage b-type Director and Cabinet Safety Notices v IBM Systems Environmental Notices and User Guide, Z125-5823 v IBM Warranty v Implementing an IBM/Brocade SAN with 8 Gbps Directors and Switches, (an IBM

Redbook), SG24-6116

Note: This publication was written for earlier products, but many of the

Brocade documents

IBM switches use software licensed from Brocade Communications Systems, Inc. You can find information related to the software that supports the SAN768B-2 in the following documents on the CD-ROM supplied with this product:

Brocade Fabric OS

v Fabric OS Administrator's Guide

v Fabric OS Command Reference Manual

v Fabric OS MIB Reference

v Fabric OS Message Reference

procedures still apply.

v Fabric OS Troubleshooting and Diagnostics Guide

v Fabric OS FCIP Administrator's Guide

v Fabric OS Encryption Administrator’s Guide Supporting Tivoli Key Lifecycle Manager

(TKLM) Environments

Brocade Fabric OS optional features

v Fabric Watch Administrator's Guide

v Web Tools Administrator's Guide

IBM and Brocade product matrix

When you use any of the Brocade documents, such as Fabric Operating System (FOS) publications, you will notice that the model numbers reflect the corresponding Brocade products. Table 1 provides a product matrix for you to use to correlate the Brocade products and models to the IBM product names and machine types and model numbers. Products withdrawn from marketing are not listed.

Table 1. Brocade and IBM product and model number matrix

Brocade product name IBM product name

Brocade 6510 SAN48B-5 2498 Model F48

Brocade DCX 8510-4 SAN384B-2 2499 Model 416

Brocade DCX 8510-8 SAN768B-2 2499 Model 816

Brocade DCX-4S SAN384B 2499 Model 192

Brocade DCX SAN768B 2499 Model 384

Brocade Encryption Switch SAN32B-E4 2498 Model E32

Brocade 8000 IBM Converged Switch B32 3758 Models B32 and L32

Brocade 7800 SAN06B-R 2498 Model R06

Brocade 5300 SAN80B-4 2498 Model B80

Brocade 5100 SAN40B-4 2498 Models B40 and 40E

Brocade 300 SAN24B-4 2498 Models B24 and 24E

IBM machine type and model number

Accessibility features for SAN768B-2

Accessibility features help users who have a disability, such as restricted mobility or limited vision, to use information technology products successfully.

Accessibility features

Use and operation of this device is accomplished primarily through external devices which may provide different accessibility features.

The following list includes the major accessibility features in the SAN768B-2 either directly or through external devices or interfaces:

v Keyboard-only operation v Interfaces that are commonly used by screen readers v Keys that are discernible by touch but do not activate just by touching them v Industry-standard devices for ports and connectors v The attachment of alternative input and output devices

xxvi SAN768B-2 Installation, Service, and User Guide

Keyboard navigation

This product uses standard Microsoft Windows navigation keys.

Vendor software

The SAN768B includes certain vendor software that is not covered under the IBM license agreement. IBM makes no representation about the accessibility features of these products. Contact the vendor for the accessibility information about its products.

Related accessibility information

You can view the publications for the SAN768B-2 in Adobe Portable Document Format (PDF) using the Adobe Acrobat Reader. The PDFs are provided on a CD that is packaged with the product.

IBM and accessibility

See the IBM Human Ability and Accessibility Center for more information about the commitment that IBM has to accessibility: www.ibm.com/able.

About this document xxvii

xxviii SAN768B-2 Installation, Service, and User Guide

Chapter 1. Introduction

This chapter introduces the features and components of the IBM System Storage SAN768B-2 fabric backbone. Throughout the remainder of this document, the product will be referred to as the SAN768B-2, or more generically as system, device, or chassis, where appropriate. This chapter contains the following information:

v “Overview of the SAN768B-2” v “Hardware components” on page 3 v “High availability” on page 7 v “Reliability” on page 8 v “Serviceability” on page 8 v “Software features” on page 8

Overview of the SAN768B-2

The SAN768B-2 is part of IBM's industry-leading backbone-class product line, a highly robust class of network switching platform that combines breakthrough performance, scalability, and energy efficiency with long-term investment. Supporting open systems and System z the data growth and application demands of evolving enterprise data centers, achieve server, SAN, and data center consolidation, and reduce infrastructure and administrative costs. The IBM backbone-class product line provides the most reliable, scalable, high performance foundation for private cloud storage and highly virtualized environments. They are designed to increase business agility by adapting to dynamic growth and change, providing high availability access to information, and reducing infrastructure and administrative costs.

, SAN backbones are designed to address

Key features of the SAN768B-2 include: v Up to 384 16-Gbps external ports in a single chassis, enabling high density SAN

configurations with reduced footprint.

v Support for 2, 4, 8, and 16 Gbps auto-sensing Fibre Channel ports. Trunking

technology groups up to eight ports to create high performance 64-Gbps ISL trunks between switches.

v Support for 10 Gbps FC-type SFPs in 16 Gbps port blades only, and also

supports 10 GbE SFPs in the FX8-24 extension blades. The two types of SFPs are not interchangeable.

v The 10 Gbps ports can be configured manually on only the first eight ports of

the 16 Gbps port blades.

v Support for all of the application, port blade, and control processor (CP) blades

supported in the SAN384B-2 (with the exception of the Core Switch Blade), providing flexible system configurations and fewer types of new blades.

v Up to five chassis can be connected with the use of 4x16 Gbps quad SFP (QSFP)

inter-chassis links (ICLs).

v Support for high-performance port blades running at 2, 4, 8, 10, or 16 Gbps,

enabling flexible system configuration.

v Redundant and hot-swappable control processor and core switch blades, power

supplies, blower assemblies, and WWN cards that enable a high availability platform and enable nondisruptive software upgrades for mission-critical SAN applications.

v Universal ports that self-configure as E_Ports, F_Ports, EX_Ports and M_Ports

(mirror ports). 10 Gbps ports are E_Ports only.

v Diagnostic port (D_Port) functionality. v Fibre Channel over IP (FCIP) functionality through the FX8-24 blade.

Attention: This product is not intended to be connected directly or indirectly by any means whatsoever to interfaces of public telecommunications networks.

2 SAN768B-2 Installation, Service, and User Guide

Hardware components

The SAN768B-2 features a modular and scalable mechanical construction that allows a wide range of flexibility in installation, fabric design, and maintenance. The chassis may be mounted with the cables facing the front of the equipment rack or to the rear, and consists of the following:

v Up to eight hot-swappable port blade assemblies can be configured in a single

chassis, delivering up to 384 16 Gbps Fibre Channel ports.

v Two slots for control processor blades (CP8):

– A single active CP8 blade can control all 384 ports in the chassis. – The standby CP8 blade assumes control of the SAN768B-2 if the active CP

fails.

v Two slots for core switch blades (CR16-8):

– CR16-8 blade interconnects all port blades – Inter-chassis link (ICL) connectors to connect to as many as four neighboring

chassis

– Both CR16-8 blades are active

v Modular hot-swappable port blades:

– 64-port, 8-Gbps blades (FC8-64) – 32-port, 16-Gbps blades (FC16-32) – 48-port, 16-Gbps blades (FC16-48)

v Modular hot-swappable application blades:

– FX8-24: 24-port (12 FC, 10 GbE, 2 10GbE) FCIP extension blade enabling long

distance communication over existing IP infrastructure.

v Modular hot-swappable field replaceable units (FRUs):

– Three blower assemblies – Up to four power supplies – Two WWN cards – Blades use small form-factor pluggable (SFP+, and mSFP) optical transceivers.

SFP+ and mSFP transceivers support speeds of 2, 4, 8, 10, or 16 Gbps.

Note: The 8-Gbps SFP+s and mSFPs auto-negotiate at 2, 4, and 8 Gbps. The

– QSFP-based inter-chassis link (ICL) cabling running at 64 Gbps (four 16 Gbps

clustered in a single quad connector and cable).

v Blades are serviced from the port side of the SAN768B-2. Blowers, power

supplies, and power cables are serviced from the nonport side.

v World Wide Name (WWN) cards on the nonport side, to maintain

chassis-specific information such as WWNs, IP addresses, and summary status information of each port blade and power supply through LEDs

v Constant intake and FRU temperature monitoring v Redundant AC primary power connections to ensure high availability. Each

power supply has its own connector.

Note: Airflow in the SAN768B-2 is from the non-port (non-cable) side to the port

(cable) side and out the exhaust vent. Figure 1 on page 4 shows a sample configuration of the port side of the SAN768B-2, with the chassis door removed.

16-Gbps SFP+s auto-negotiate at 4, 8, and 16 Gbps. The 10 Gbps speeds must be manually set and require special 10 Gbps SFP+ transceivers.

Chapter 1. Introduction 3

b768ig062

Figure 1. Port side of the SAN768B-2 (sample configuration)

1 Exhaust vent 4 FC16-32 port blade

2 Core switch blade (CR16-8) 5 Cable management comb

3 Control processor blade (CP8)

Figure 2 on page 5 shows a sample configuration of the non-port side of the SAN768B-2 with 4 power supplies.

4 SAN768B-2 Installation, Service, and User Guide

Figure 2. Blower (non-port) side of the SAN768B-2 (sample configuration )

1 WWN bezel 3 Blower assembly

2 Power supply

SAN768B-2 blades

Table 2 summarizes the port, application, control processor, and core switch blades that are available for the SAN768B-2.

Table 2. Blades available for the SAN768B-2

Description Feature Code Name Function

Control processor blade

b768ig014

N/A CP8 The CP8 blade

contains the control plane for the chassis. There are two CP8 blades for redundancy. This control processor blade is compatible with the SAN768B-2, SAN384B-2, SAN768B, and SAN384B.

Chapter 1. Introduction 5

Table 2. Blades available for the SAN768B-2 (continued)

Description Feature Code Name Function

Core switch blade N/A CR16-8 The CR16-8 blade

contains the ASICs for switching between port blades. Every port blade connects to every core switch blade. There can be up to 384 16-Gbps (512 8-Gbps) total ports for port blades. Each core switch blade connects to 128 backplane ports. Core switch blades have additional front port connectivity to connect multiple chassis and backplane connections for the storage server blade. This core switch blade is compatible only with the SAN768B-2.

32-port 16-Gbps port blade

48-port 16-Gbps port blade

FC3632 FC16-32 A 32-port port blade

supporting 2, 4, 8, 10, and 16 Gbps Fibre Channel port speeds. The blade also supports port-based in-flight encryption / decryption and compression / decompression. This port blade is compatible with the SAN384B-2 and SAN768B-2.

FC3648 FC16-48 A 48-port Brocade

port blade supporting 2, 4, 8, 10, and 16 Gbps Fibre Channel port speeds. The blade also supports port-based in-flight encryption / decryption and compression / decompression. This port blade is compatible with the SAN384B-2 and SAN768B-2.

6 SAN768B-2 Installation, Service, and User Guide

Table 2. Blades available for the SAN768B-2 (continued)

Description Feature Code Name Function

64-port 8-Gbps port blade

FCIP extension blade FC3890 FX8-24 The FX8-24 blade

FC3864 FC8-64 A 64-port port blade

supporting 2, 4, and 8 Gbps port speeds with mSFPs. This port blade is compatible with the SAN768B-2, SAN384B-2, SAN768B, and SAN384B

enables FCIP functionality over existing IP infrastructure. It has 12 FC ports, ten 1 GbE ports, and two 10 GbE ports available. This application blade is compatible with the SAN768B-2, SAN384B-2, SAN768B, and SAN384B

High availability

The following features contribute to the SAN768B-2's high-availability design: v Redundant, hot-swappable FRUs, including blades, power supplies, blowers,

WWN cards

v Enhanced data integrity on all data paths v Fabric Shortest Path First (FSPF) rerouting around failed links v Integration with Simple Network Management Protocol (SNMP) managers v Automatic control processor failover v Nondisruptive "hot" software code loads and activation v Easy configuration, save, and restore

The high-availability software architecture of the SAN768B-2 provides a common framework for all applications that reside on the system, allowing global and local status to be maintained through any component failure. High-availability elements consist of the High Availability Manager, the heartbeat, the fault/health framework, the replicated database, initialization, and software upgrade.

The High Availability Manager controls access to the standby control processor, facilitates software upgrades, prevents extraneous CP failover activity, closes and flushes streams as needed, provides flow control and message buffering, and supports a centralized active and standby state.

Chapter 1. Introduction 7

Reliability

Serviceability

The SAN768B-2 uses the following error detection and correction mechanisms to ensure reliability of data:

v Error Detection and Correction over main control processor memory v Error Detection and Correction mechanism, which checks for encoder errors and

fault isolation (EDFI), such as cyclic redundancy checking (CRC), parity checking, checksum, and illegal address checking

v Power-on self test (POST) v Dual control processors that enable hot, nondisruptive fast firmware upgrades v One serial port and two Ethernet ports on each control processor for

management and for service. Offline control processor diagnostics and remote diagnostics simplify troubleshooting. The standby control processor monitors diagnostics to ensure the system is operational should a failover be necessary.

v Bus monitoring and control of blades and other field-replaceable units (FRUs).

The SAN768B-2 provides the following features to enhance and ensure serviceability:

v Modular design with hot-swappable components v Flash memory that stores two firmware images per control processor v USB port on control processor blades for most tasks that formerly required an

FTP/SCP server, including software and firmware upgrades

v Nonvolatile random-access memory (NVRAM), containing the OEM serial

number, IBM serial number, revision information, and part number information

v Background health-check daemon v Memory scrubber, self test, and bus ping to determine if a bus is not functioning v RASlog messages v SMI-S compliant v Hardware and software watchdog timers. v Status LEDs v Predictive diagnostics analysis through Fabric Watch v SNMP (including version 3) integration with higher-layer managers

Software features

The Fabric OS allows any Fibre Channel-compliant device to attach to the switches as long as it conforms to the device login, name service, and related Fibre Channel standards. Each operating environment requires that a Fibre Channel host bus adapter (HBA) be available with a standards-compliant driver for correct interface to the fabric.

Fabric OS consists of a set of embedded applications running on top of an embedded Linux operating system kernel. Some of these applications include:

v Name server v Alias server v Zone server v Simple Network Management Protocol (SNMP) agent v SMI-S compliant API

8 SAN768B-2 Installation, Service, and User Guide

v Syslog auditing v RCS (Reliable Commit Service) v NTP v Tasks to manage address assignment, routing, link initialization, fabric

Security

The list below highlights some of the key security features available for the SAN384B-2 and for other switches running Fabric OS 7.0 or later.

v DH-CHAP v SSHv2 (using AES, 3DES, RSA) v HTTPS (using AES) v SNPMv3 v FC-SP v Secure RPC v Secure file copy (SCP) v Telnet disable v Telnet timeout v IP filters (block listeners) v Secure passwords (centralized control through RADIUS/CHAP) v Multiple user accounts (MUAs). Up to 255. v Role-based access controls (RBACs) v Administrative domains/Virtual fabrics v Boot PROM password reset v Password hardening policies v Upfront login in Web Tools v Login banner v Monitoring of attempted security breaches (through audit logging) v Monitoring of attempted security breaches (through Fabric Watch Security Class) v Fibre Channel security policies: DCC and SCC v Trusted Switch (FCS) for central security management v Management access controls (SNMPv3, Telnet, FTP, serial port, front panel) v Hardware-enforced zoning by WWN, domain/port ID, or both v Default zoning v RSCN suppression and aggregation v Configurable RSCN suppression by port v NTPv3 (to synchronize timestamps) v Event auditing v Change tracking v Firmware change alerts in Fabric Manager v Persistent port disable v Persistent domain ID v E_Port disable

initialization, link shutdown, chassis shutdown, and the user interface.

Chapter 1. Introduction 9

Network Manageability

The SAN768B-2 has a single domain and is managed as a single element with Network Advisor. The SAN768B-2 responds to its own IP address and appears as a separate entity to the Telnet protocol and SNMP.

All management interfaces, such as Telnet, Web Tools, standards compliant SMI-S, and Management Server, support a “port N within blade M” naming scheme.

The SAN768B-2 supports SNMPv1 and SNPMv3. When SNMP devices send SNMP messages to a management console running SAN management software, the information is stored in a management information base (MIB). Fabric OS 7.0 and later supports the latest Fibre Alliance Fibre Channel Management (FCMGMT) and Storage Management Initiative (SMI) MIBs, which allow common information necessary for management software to provide information to a SAN administrator. Refer to the Fabric OS MIB Reference for additional MIB information.

10 SAN768B-2 Installation, Service, and User Guide

Chapter 2. Installing and removing a SAN768B-2

The SAN768B-2 is only available for installation in the IBM TotalStorage 2109 SAN Switch Cabinet C36. This SAN768B-2 is to be installed and serviced only by qualified IBM service representatives. The SAN768B-2 can be ordered pre-installed in the cabinet, or the SAN768B-2 can be added to an existing C36 cabinet. The cabinet is designed to support a total of two SAN256Bs or SAN768B-2s. The SAN384B chassis can also be installed in the C36 cabinet.

Attention: The SAN768B-2 is only available for installation in the IBM TotalStorage 2109 SAN Switch Cabinet C36. The SAN768B-2 can be pre-installed in the cabinet, or it can be added to an existing IBM C36 cabinet. For installation into an existing C36 cabinet, a special bridge tool ships with the SAN768B-2 which is designed to attach to the IBM C36 cabinet. This bridge tool supports the weight of the director as it is moved into the cabinet. This tool may not attach to non-IBM cabinets. There is a risk of personal injury or machine damage during installation into any cabinet other than the approved IBM cabinets .

This chapter describes how to install the SAN768B-2 into a C36 cabinet, and how to remove the SAN768B-2 from the cabinet in the event you need to move or replace the SAN768B-2. One SAN768B-2 is installed in the cabinet at the factory. If the SAN768B-2 is already installed in the cabinet, continue on to Chapter 3, “Starting and configuring the SAN768B-2,” on page 31.

Note: Some illustrations in this chapter display different models of IBM SAN

directors. Details, such as the cable management comb and blades may

differ, but the procedures remain the same for installing the SAN768B-2.

Attention: Refer to “Safety and environmental notices” on page xiii for general safety instructions, and “Rack safety” on page xx before performing any installation or service procedures.

CAUTION: The weight of this part or unit is more than 104 kg (228 lb). It takes specially trained persons with a lifting device to safely lift this part or unit. (C011)

55 kg ( 121.2 lbs)

svc00169

CAUTION: A fully populated (384 port) SAN768B-2 weighs approximately 159.2 kg (351 lb). Before you install it, verify that the additional weight of the chassis does not exceed the cabinet's weight limit or unbalance the cabinet. When you calculate the additional weight, include the weights of all components that can potentially be added, to avoid overloading in the future.

CAUTION: For CA residents only: IBM recommends installing this product in a room size of 2190 cubic feet (62 cubic meters) or larger at 0.4 ACH ventilation rate to reduce the concentrations of any chemicals emitted by the product.

Before you install a SAN768B-2 into the cabinet, perform the following tasks:

v Read “Power specifications” on page 127 to plan for meeting power supply

standards before installing the chassis.

v To ensure adequate cooling, plan to install the chassis with the port side facing

the aisle where exhaust air is released (usually called the service aisle). This prevents the fans from pulling in heated exhaust air.

v Plan for managing the cables before you install the chassis. A fully populated

SAN768B-2 will have a large number of cables that must be carefully routed to minimize problems with installation and replacement of components, and in order to maintain the minimum bend radius required for optical cables. Refer to “Managing cables” on page 42 for more specific information. You can manage the cables in a variety of ways, such as:

– Routing the cables down through the cable management tray – Routing the cables out either side of the chassis – Routing the cables through the cable channels on the sides of the cabinet – Using patch panels

v Ensure that the airflow in the vicinity of the SAN768B-2 is a minimum of 595

cubic meters per hour (350 cubic feet per minute).

v Ensure that the air temperature that is measured at the blower inlet does not

exceed 40°C (104°F ) during operation.

Ordering the lift tool

Important

The lift tool is required only when you install an additional SAN768B-2 or when you install or remove the SAN768B-2 from the cabinet.

The ordering procedures for the lift tool vary depending on your location. You should direct questions about these procedures to your regional representative.

Note: The bridge tool (PN 18P5855), which is required for installation, is shipped

with a new or a replacement SAN768B-2. The bridge tool, when installed, serves as a temporary ramp between the lift tool and the SAN768B-2 rails that are installed in a C36 cabinet. You do not need to return the bridge tool when you are finished with it.

World trade locations

The following ordering procedures are for world trade locations:

v Order the lift tool by using the parts order system, like any other part. v Use the following part numbers when you order:

– Lift tool: PN 09P2481 – 24–inch load plate: PN 11P4369

v You do not record parts usage. v Return the lift tool and the 24–inch load plate to the parts center after you

complete the installation or removal of the SAN768B-2.

United States locations

In the United States, call UPS Logistics at 800–528–6070 to order the lift tool and the 24–inch load plate.

12 SAN768B-2 Installation, Service, and User Guide

Note: For the SSR branch and territory, the United States cannot order the lift tool

or 24–inch load plate through the parts order system. UPS Logistics are used

to ship and return the lift tool and 24–inch load plate.

Use the following part numbers when you order:

v Lift tool: PN 09P2481 v 24–inch load plate: PN 11P4369

Note: When you order the lift tool, you will receive an 18–inch load plate. Do not

use the 18–inch load plate. You must order and use a 24–inch load plate

when you install the SAN768B-2.

You must provide the following information when you order the lift tool. This information is necessary to ensure that the lift tool is delivered when you need it. Failure to provide this information might delay the completion of the order request and the shipping request. It might also result in a time and date to return the tool that is different from what you need.

v Phone number and customer contact v Account code: 98577 v Time and date of delivery v Accurate destination address with zip code v Time and date of return pickup

You must return the lift tool at the time that was scheduled with UPS Logistics. If you need to change the scheduled return time or date, contact UPS Logistics.

You are responsible for ensuring that all of the paperwork and components are packed and restored in the arbocrate (shipping container) of the lift tool. Ensure that the lift tool is functioning properly before you release the tool to UPS Logistics for return. You are accountable for the lift tool until UPS Logistics picks up the lift tool for return delivery to their parts storage facility.

Contact your branch office tools coordinator or your region specialist if you have any questions or concerns.

Installation guidelines

Follow these general installation guidelines:

1. Provide a space that is 14 rack units (14U) high, 61.29 cm (24.09 in.) deep, and

43.74 cm (17.22 in.). 1U is equal to 4.45 cm (1.75 in.).

2. Plan to install the SAN768B-2 with the non-port side facing the air-intake aisle.

The SAN768B-2 can be installed facing either direction, if serviceability and cooling requirements are met.

3. Plan for cable management before installing the chassis. See “Managing cables”

on page 42. Cables can be managed in a variety of ways, such as by routing cables below

the chassis, to either side of the chassis, through cable channels on the sides of the cabinet, or by using patch panels.

4. Ensure that dedicated electrical branch circuits with the following

characteristics are available: v 200 – 240 VAC, 50–60 Hz (two branch circuits) when configured with 192

16-Gbps ports (four circuits required for high availability if configured with 384 16-Gbps ports)

Chapter 2. Installing and removing a SAN768B-2 13

v Two or four cables for 200 - 240 VAC service v Protected by a circuit breaker in accordance with local electrical codes v Supply circuit, line fusing, and wire size adequate to the electrical rating on

the chassis nameplate

v Location close to the chassis and easily accessible v Grounded outlets installed by a licensed electrician, compatible with the

power cords Attention: To maximize fault tolerance, connect each power cord to a

separate power source.

5. Ensure that the following are available for configuration of the chassis:

v Workstation with an installed terminal emulator, such as HyperTerminal v Serial cable (provided) v Three Ethernet cables (including one spare) v Access to an FTP server for backing up the switch configuration or collecting

supportsave output data (optional)

v SFPs (provided) and compatible cables

6. Ensure that the air intake and exhaust vents have a minimum of 2 in. of airspace.

7. Ensure that the air temperature on the air intake side is less than 40 degrees Celsius (104 degrees Fahrenheit) during operation.

CAUTION: Use safe lifting practices when moving the product. (C015)

Note: A fully populated SAN768B-2 (eight FC8-64 port cards, 512 ports) weighs

approximately 159.2 kg (351 lb) and requires the specified lift tool to install it.

Installing a SAN768B-2 in a cabinet

This section describes how to install the SAN768B-2 in the Model C36 cabinet. These procedures use parts that are included in the 14U rack mount kit.

Time required

Allow approximately 4 hours to complete this procedure. This includes unpacking, assembling, and repacking the lift tool. This is for the physical installation of the SAN768B-2 only. It does not include installing Ethernet cables, SFPs, attaching and routing the fiber optic cables, or completing the initial configuration.

Table 3 provides a summary of time estimates for the different installation tasks and the tools required for each task. Times may differ, depending on experience level and your specific configuration.

Table 3. Installation tasks, time, and items required

Installation task Time estimate Items required

Unpacking and assembling the lift tool

Site preparation and unpacking the SAN768B-2

15 minutes No special tools required.

30 minutes #2 Phillips screwdriver (for

cable management comb) Pallet jack (if the shipment has not been positioned near the installation site)

14 SAN768B-2 Installation, Service, and User Guide

Table 3. Installation tasks, time, and items required (continued)

Installation task Time estimate Items required

Installing the rack mount kit 30 minutes Torque wrench with #2

Phillips screwdriver tip Flathead screwdriver

Mounting and securing the SAN768B-2 in the rack

Installing power cables and powering on the SAN768B-2

Establishing serial connection, logging on to the SAN768B-2, and configuring IP addresses.

Installing an Ethernet cable, opening a Telnet session, and configuring the SAN768B-2 domain ID, date and time, and additional system parameters. Verifying and backing up the configuration.

Installing transceivers as needed

Attaching fiber optic cables, cable wraps, and cable guides

Disassembling and packing the lift tool

30 minutes Torque wrench with #2

Phillips screwdriver tip Lift tool and load plate

20 minutes Power cables

20 minutes Serial cable (provided in the

accessory kit). Workstation computer with a serial port or terminal server port and a terminal emulator application (such as HyperTerminal). Ethernet IP addresses for the chassis and for both control processor blades (total of 3 IP addreses).

20 minutes Ethernet cable for Telnet

access. Refer to the Fabric OS Administrator’s Guide.

30 minutes (longer if using high density port blades)

2-3 hours Fiber optic cables and cable

20 minutes No special tools required.

SFP+, mSFP, and QSFP optical transceivers as needed.

wraps.

Note: Assembly instructions for the lift tool are in the shipping container.

Tools required

You will require the following items to install the SAN768B-2 in the cabinet:

v Torque wrench with #2 Phillips screwdriver tip v Flathead screwdriver v Lift tool

Parts list

The following parts list refers to items illustrated in Figure 3 on page 17

Table 4. Items supplied with the 14U rack mount kit

Item Description Quantity

For all types of installations

Chapter 2. Installing and removing a SAN768B-2 15

Table 4. Items supplied with the 14U rack mount kit (continued)

Item Description Quantity

A Left rack mount shelf bracket (rail brackets may differ from the

illustration)

B Right rack mount shelf bracket (rail brackets may differ from the

illustration)

C 10-32 x 5/8 inch (1.58 cm) panhead Phillips screw, washer 6

For cabinets that have vertical rails with round holes

D 10-32 clip nut 6

E 1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with lock

washer

For cabinets that have vertical rails with square holes

F 10-32 retainer nut 6

G 1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with glue 16

H 0.375 in. (0.953 cm) alignment washer 16

16 SAN768B-2 Installation, Service, and User Guide

Item A

Item B

C (6x)

10-32 x 5/8"

F (6x)

10-32 retainer nut

Figure 3. Parts in the 14U rack mount kit

1/4-20 x 1/2"

Unpacking the SAN768B-2

Attention: If the product needs to be transported a long distance, or if there are uneven floors to cross, keep the product in its original packaging, and relocate the entire shipment close to the installation location, using a pallet jack to lift and move it.

To unpack the SAN768B-2, follow these steps:

1. Cut the bands that encircle the packaging.

2. Remove the lid, the 14U rack mount kit, the accessory kit, and the packing

foam from the top of the chassis. Set the kits aside for later use.

3. Lift the cardboard box off the chassis and remove the plastic bag from around the chassis. Save the packing materials if you are returning an old chassis.

4. Leave the chassis on top of the plastic shipping tray if the chassis must be transported to the installation location.

D (6x)

clip nut

G (16x)

E (16x)

1/4-20 x 1/2"

H (16x)

alignment washer

b768ig027

Chapter 2. Installing and removing a SAN768B-2 17

Note: The standard packaging does not incorporate a wood pallet and pallet

brackets. The chassis sits on top of a plastic shipping tray.

5. Use the lift tool to transport the new chassis to the installation area. Doorways must be wider than 91 cm (36 in.) to accommodate the chassis on the shipping tray.

Attaching the shelf brackets

Complete the following steps to install the shelf brackets to the cabinet rails:

1. Find the left and right rack mount shelf brackets (A and B in Figure 4) in the 14U rack mount kit. These brackets are adjustable to allow for different cabinet depths.

2. Loosen the adjusting screws on both brackets enough to allow the brackets to slide easily to adjust the length.

Rail

Setup for rails with square holes

Intake Air

Aisle

Figure 4. Left and right shelf brackets installed on the cabinet rails

(4x)

Shown with

Round Hole Rails

(4x)

Adjustable ends

3. Position the shelf brackets (A and B) with the adjustable ends at the intake aisle side of the cabinet (see Figure 4) and attach them to the cabinet rails.

v For rails with round holes: Use eight screws with lock washers (E) for each

bracket, four at each end. Tighten the screws to a torque of 92 cm-kg (80 in.-lb).

v For rails with square holes: Use eight screws (G) eight alignment washers

(H) for each bracket, four at each end. Tighten the screws to a torque of 92 cm-kg (80 in.-lb).

(4x)

b768ig028

Aisle

Exhaust Air

18 SAN768B-2 Installation, Service, and User Guide

Note: Standard EIA rails have holes in sets of three; spaces between the holes

are 1.58 cm, 1.58 cm, 1.27 cm (5/8 in., 5/8 in., and 1/2 in.). If cables are to be routed down through the cable management comb, allow at least 1U space below the brackets for cable management.

4. Tighten the adjusting screws on the shelf brackets to a torque of 37 cm-kg (32 in.-lb).

5. Attach the clip nuts D (for rails with round holes) or retainer nuts F (for rails with square holes) to the vertical rails on the exhaust aisle side of the cabinet (see Figure 5). These nuts are used for securing the port side of the chassis to the rack rails using 10-32 x 5/8-inch screws. Use three nuts on each rail. Place the nuts in optimum locations for securing the chassis to the rails. See Figure 7 on page 22 for approximate positions.

Note: Do not align the clip or retainer nuts with the top or bottom holes of the

SAN768B-2 mounting brackets because the door will interfere with the screw heads.

Attaching clip nuts

Setup for rails

with Square Holes

Retainer Nut (Item F)

Rail

Top screws in shelf bracket

Hole 1

Figure 5. Installing clip or retainer nuts on the vertical cabinet rails

Bottom screws in shelf bracket

Shown with clip nuts for round hole rails

Removing the chassis door and cable management comb

The chassis door must be removed before the chassis can be installed into the cabinet. The door is not hinged. It is held to the chassis by mounting hardware on the door that snaps onto eight ball studs on the chassis. To remove the door, grip both sides of the door, and gently pull the door straight out until it releases from the ball studs on the chassis.

Rail

1/2"

5/8" 5/8"

b768ig029

Chapter 2. Installing and removing a SAN768B-2 19

Set the door aside for later reinstallation.

Note: Reinstall the chassis door once the chassis is installed and all cabling and

configuration is complete. The chassis door must be installed to ensure the SAN768B-2 meets EMI and other regulatory certifications.

Figure 6. Chassis door removal

Remove the cable management comb (see “Removing and installing a cable management comb” on page 68), and set the parts aside for later reinstallation.

Using the lift tool, the 24–inch load plate, and the bridge tool

Step 1 on page 21 through step 9 on page 22 describe how to use the lift tool (PN 09P2481) and the 24–inch load plate (PN 11P4369) to install a SAN768B-2 in the cabinet. Step 10 on page 22 through step 23 on page 24 describe how to install the bridge tool (PN 18P5855) in the cabinet for the lower SAN768B-2 or the upper SAN768B-2. This procedure uses parts from the 14U chassis mount kit. See “Items required” on page 102 for a list of these parts.

CAUTION: A fully populated SAN768B-2 weighs approximately 159.2 kg (351 lb) and requires a minimum of two people and a lift tool to install it.

55 kg ( 121.2 lbs)

20 SAN768B-2 Installation, Service, and User Guide

(C011)

svc00169

b768ig063

Attention: Before you install it, verify that the additional weight of the chassis does not exceed the cabinet's weight limits or unbalance the cabinet. When you calculate the additional weight, include the cards or power supplies that partially extend out of the chassis.

CAUTION: A pinch point exists between the load plate and the bridge tool.

P/N 18P5850-B

SJ000752

Perform the following steps to install the SAN768B-2:

1. Assemble the lift tool if it is not assembled. The assembly and disassembly instructions are included with the lift tool.

2. Attach the 24–inch load plate to the lift tool.

3. Although not required, you can remove the power supplies and blower

assemblies from the SAN768B-2 to reduce the weight. This can reduce the weight by up to 27 kg (60 lb).

4. Move the lift tool next to the pallet that contains the SAN768B-2. a. Adjust the load platform to the same height. b. Slide the SAN768B-2 from the pallet onto the load platform. c. Center the switch on the load platform.

5. Attach the retention straps around the drawer.

6. If the load platform is not at its lowest position, lower the load platform to its

lowest position.

7. Move the lift tool to a position near the rear of the cabinet. See Figure 7 on page 22.

Chapter 2. Installing and removing a SAN768B-2 21

Air Intake Side

of Chassis

Direction

Rack Cabinet

(shown without sheet metal)

Figure 7. Chassis installation from rear (exhaust aisle side) of the cabinet

Install

Note: If you are passing through a narrow passage or door, it might be

necessary to rotate the drawer on the load platform.

8. Set the wheel brake by pressing down on the wheel brake pedal.

9. Locate the anti-tips bars. If they are not attached, attach them to the lift tool

by performing step 9a through step 9d. If the anti-tip bars were previously attached, go to step 10.

a. Pull up on the leg lock pins and insert the anti-tip bars inside the legs. b. Pull on the legs and anti-tip bars to ensure that they are locked into

position.

c. Raise the platform to allow enough room to attach the anti-tip bracket. d. Secure the anti-tip bars by using the anti-tip bracket and bolt.

10. If you are installing the lower SAN768B-2, go to step 11. If you are installing

the upper SAN768B-2, go to step 12 on page 23.

11. For the lower SAN768B-2, install the bridge tool by completing the following steps:

a. Remove the bridge tool from the box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 3,

and move the two supports another 45°. Make sure that the following conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

c. Align the two-pin section of the support on the back of the vertical rail

with the bottom and middle holes in EIA unit 3.

b768ig030

22 SAN768B-2 Installation, Service, and User Guide

d. Align the one-pin section of the support on the front of the vertical rail

with the middle hole in EIA unit 1.

Note: Tilt the bridge tool towards the back of the cabinet approximately

45° to make it easier to align the two-pin and the one-pin sections with the vertical support.

e. Lock the shelf into position. See Figure 8.

Note: The shelf in the picture below is shown with a different director.

Sj000680

Figure 8. Locking the shelf into position

12. For the upper SAN768B-2, install the bridge tool by completing the following steps:

a. Remove the bridge tool from the box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 20,

and move the two supports another 45°. Make sure that the following conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section is on the front of the vertical support.

Note: Tilt the bridge tool towards the back of the cabinet approximately

45° to make it easier to align the two-pin and one-pin sections with the vertical support.

c. Align the two-pin section of the support on the front of the vertical rail

with the bottom hole and the middle hole in EIA unit 20.

d. Align the one-pin section of the support on the front of the vertical rail

with the middle hole in EIA unit 18.

13. Turn the winch crank counterclockwise, 1/4 turn, to set the winch brake.

14. Set the wheel brake and place the wheel chocks around the wheels.

Chapter 2. Installing and removing a SAN768B-2 23

15. Remove the retention straps.

16. Slide the SAN768B-2 into the cabinet until it rests securely on the shelf

brackets.

17. Remove the wheel chocks, release the wheel brake, and move the lift tool away from the cabinet.

18. Uninstall the bridge tool and return it to its original box.

19. Reinstall the power supplies and blower assemblies if any were removed.

20. Fasten the port side of the chassis to the cabinet rails. Use three 10–32 x

5/8-in. Phillips panhead screws with washers C on each side to attach to the clip nuts (D) or retainer nuts (F) installed on the rails earlier. Tighten the screws to a torque of 37 cm-kg (32 in-lb). See Figure 9 for the approximate location of the screws.

Note: Do not use the top or bottom holes of the SAN768B-2 mounting

brackets because the screw heads will interfere with the chassis door.

Figure 9. Attaching the port side of the chassis to the cabinet rails

21. Reinstall the cable management comb, if it was removed (see “Removing and installing a cable management comb” on page 68).

22. If ICL cables will not be used, ensure that the ICL sockets in the core switch blades have EMI plugs inserted.

23. Reinstall the chassis door. Align the door with the chassis and push it into place, as shown in Figure 10 on page 25 and Figure 11 on page 26.

24 SAN768B-2 Installation, Service, and User Guide

b768ig031

Figure 10. Aligning the chassis door with the ball studs on the chassis

b768ig032

Chapter 2. Installing and removing a SAN768B-2 25

Inside view

of door

Outside view

of door

Ball stud

on chassis

Mounting

hardware

Figure 11. Details of the chassis and door, showing the ball studs on the chassis and the mounting hardware on the door

Ball stud

on chassis

Note: For turning on the power and for the configuration instructions, see

Chapter 3, “Starting and configuring the SAN768B-2,” on page 31.

Attention: Do not connect the SAN768B-2 to the network until the IP addresses are correctly set.

Port numbering

Table 5 has a list of available port blades for the SAN768B-2.

Table 5. Port and application blades available on the SAN768B-2

Blade label IBM blade name Feature code

FC16-32 32 port 16Gbps Switch Blade 3632

FC16-48 48 port 16Gbps Switch Blade 3648

FC8-64 64 port 8Gbps Switch Blade 3864

FX8-24 FCIP 8 Gbps extension blade 3890

b768ig033

The SAN768B-2 uses the following port numbering scheme for each of the supported blades (see also Appendix D, “Blade port numbering,” on page 139 for sample configurations).

v CR16-8 core blade - Inter-chassis link connectors are numbered from 0 through 7

from bottom to top. Each connector is actually a group of four 16 Gbps ports.

v FC16-32 port blade - Ports are numbered from 0 through 15 from bottom to top

on the left set of ports and 16 through 31 from bottom to top on the right set of ports.

v FC16-48 port blade - Ports are numbered from 0 through 23 from bottom to top

on the left set of ports and 24 through 47 from bottom to top on the right set of ports.

v FC8-64 port blade - Ports are numbered from 0 through 31 from bottom to top

on the left set of ports and 32 through 63 from bottom to top on the right set of ports. See Figure 42 on page 141 for more information. Trunking groups are

26 SAN768B-2 Installation, Service, and User Guide

Chassis slots

permitted with up to eight ports per group. trunking groups are as follows: 0-7, 8-15, 16-23, 24-31, 32-39, 40-47, 48-55, and 56-63*. (* - Octet 56-63 E-port trunks for the SAN768B-2 are permitted on the logical or base switch only)

v FX8-24 FCIP extension blade - The port numbering scheme is fairly complex for

this blade, but is displayed for reference on the blade itself in a diagram towards the lower end of the blade. (See Figure 43 on page 142 for a description and illustration of the port layout.) There are twenty-four physical ports. Twelve FC ports are located towards the lower end of the blade, with ports 0 through 5 on the left side, and ports 6 through 11 on the right side. Two 10 Gigabit Ethernet (10GbE or XGE) ports are located in the middle of the blade on the left side, and are numbered 0 and 1, from bottom to top. Ten GbE ports are located towards the top of the blade, with eight clustered at the top, and the remaining two located to the right of the two 10GbE ports. The GbE ports 0 through 3, numbered from bottom to top are located on the left side at the top of the blade. Ports 4 and 5 are located above the two 10GbE ports, and ports 6 through 9 (numbered bottom to top) are at the top on the right side, to the right of ports 0 through 3. Up to three FC trunking groups. The three groups are defined as:

– Trunk group 0: FC ports 0, 1 – Trunk group 1: FC ports 6, 7 – Trunk group 2: FC ports 2, 3, 4, 5, 8, 9, 10, 11

Slots are numbered 1 through 12, from left to right when facing the port side of the SAN768B-2. Control processor blades (CP8) can be installed only in slots 6 and 7. Core switch blades (CR16-8) can be installed only in slots 5 and 8. Port or application blades can be installed in slots 1–4 and 9–12. Unused slots must be filled with blade filler panels to maintain adequate cooling.

See Appendix D, “Blade port numbering,” on page 139 for port numbering details of the SAN768B-2 port blades. Also refer to the IBM Redbook, Implementing an IBM/Brocade SAN with 8 Gbps Directors and Switches, SG24-6116, for more comprehensive information on port numbering and area port numbering, and other technical information. This publication is available through www.redbooks.ibm.com/. Search by title or publication number.

Note: This publication was written for earlier products, but many of the

Cable organization

The cable management comb (5 in Figure 1 on page 4) is attached to the chassis under the chassis door and allows for simple cable management. The comb can be installed without service disruption.

Attention: Do not use tie wraps with optical cables because they are easily overtightened and can damage the optic fibers.

Cables can be organized and managed in a variety of ways, for example, using cable channels on the sides of the cabinet or patch panels to minimize cable management. Following is a list of recommendations:

v Plan for rack space required for cable management before installing the chassis

procedures still apply.

Chapter 2. Installing and removing a SAN768B-2 27

v Leave at least 1 m (3.28 ft) of slack for each port cable. This provides room to

remove and replace the switch, allows for inadvertent movement of the rack, and helps prevent the cables from being bent to less than the minimum bend radius.

v If you are using ISL Trunking, consider grouping cables by trunking groups. The

cables used in trunking groups must meet specific requirements, as described in the Fabric OS Administrator’s Guide.

v For easier maintenance, label the fiber optic cables and record the devices to

which they are connected.

v Keep LEDs visible by routing port cables and other cables away from the LEDs. v Use hook and loop type straps to secure and organize fiber optic cables.

Attention: The minimum bend radius for a 50 micron cable is 2 in. under full tensile load and 1.2 in. with no tensile load.

Attention: Do not route the cables in front of the air exhaust vent, which is located at the top of the port side of the chassis.

The FC8-64 high density port blade cannot use the standard LC cables because the pitch between optics in the new mSFP transceiver is smaller than in standard SFPs. Patch cables and panels can be used to attach standard size cabling to the blade if necessary. Figure 12 illustrates the mSFP to SFP patch cable. The mSFP transceivers are used only with the FC8-64 port blade. Narrower OM-3 LC cables are used to connect the FC8-64. These cables are offered by several major manufacturers. Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8-64 port blades.

2 3

Figure 12. Cable design for the mSFP patch cables for the FC8-64 high density port blade

1 mini-SFP connector 3 1.6 mm cable

2 Duplex clip (black) 4 SFP connector

Note that the duplex clip on the mSFP end of the cable is black for easier recognition. See Table 24 on page 129 for a listing of the qualified mSFP optical cables for the FC8-64 port blade.

If ISL Trunking is in use, group the cables by trunking group. The ports are color-coded to indicate which ports can be used in the same ISL Trunking group: eight ports marked with solid black ovals alternate with eight ports marked with oval outlines.

b768ig045

28 SAN768B-2 Installation, Service, and User Guide

Table 24 on page 129 provides a listing of supported cable speeds and distances. Optional inter-chassis link cables (ICL) can also be installed between SAN768B-2 chassis. See “Removing and replacing inter-chassis link (QSFP) cables” on page 95 for instructions on installing these cables.

Removing a SAN768B-2 from the cabinet

If you need to remove a SAN768B-2 chassis from a cabinet to move it to a new location, follow the instructions below to remove the chassis, and then follow the installation instructions (“Installing a SAN768B-2 in a cabinet” on page 14) to install the chassis in the new cabinet. If you are replacing the chassis, follow the complete instructions in “Removing and replacing the chassis” on page 101.

Attention: Refer to “Safety notices and labels” on page xiii before starting any service procedure. Due to the weight of the product, this procedure requires a lift tool. See “Ordering the lift tool” on page 12 for information on ordering the lift tool.

Step 1 through step 5 describes how to use the lift tool (PN 09P2481) and the 24–inch load plate (PN 11P4369) to remove a SAN768B-2 from the cabinet. Step 6 through step 17 on page 30 describes how to remove the bridge tool (PN 18P5855) from the cabinet.

Perform the following steps to remove a SAN768B-2:

1. Assemble the lift tool if it is not assembled. The assembly and disassembly instructions are included with the lift tool.

2. Remove both power cords from the SAN768B-2.

3. Optionally, remove the power supplies and blower assemblies to reduce the

weight.

4. Remove the chassis door from the chassis. See “Removing the chassis door and cable management comb” on page 19.

5. Remove the three 10-32 x 5/8 in. Phillips panhead screws with washers on each side. See Figure 9 on page 24 for the location of the screws.

6. If you are replacing the lower SAN768B-2, go to step 7. If you are replacing the upper SAN768B-2, go to step 8 on page 30.

7. Install the bridge tool by completing the following steps: a. Remove the bridge tool from its box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 3. c. Move the two supports another 45°. Make sure that the following

conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

d. Align the two-pin section of the support on the back of the vertical rail

with the bottom and middle holes in EIA unit 3.

e. Align the one-pin section of the support on the front of the vertical rail

with the middle hole in EIA unit 1.

f. Lock the shelf into position.

Note: Tilt the bridge tool towards the back of the cabinet approximately

45° to make it easier to align the two-pin and one-pin sections with the vertical support.

Chapter 2. Installing and removing a SAN768B-2 29

8. For the upper SAN768B-2, install the bridge tool by completing the following steps:

a. Remove the bridge tool from its box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 20. c. Move the two supports another 45°. Make sure that the following

conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

d. Align the two-pin section of the support on the back of the vertical rail

with the bottom and middle holes in EIA unit 20.

e. Align the one-pin section of the support on the front of the vertical rail

with the middle hole in EIA unit 18.

f. Lock the shelf into position.

9. Locate the anti-tip bars and if they are not attached, attach them to the lift

tool. If anti-tip bars were previously attached, go to step 10.

a. Pull up on the leg-lock pins and insert the anti-tip bars inside the legs. b. Pull on the legs and anti-tip bars to ensure that they are locked into

position.

c. Raise the platform to allow enough room to attach the anti-tip bracket. d. Secure the anti-tip bars with the anti-tip bracket and bolt.

10. Move the lift tool to a position near the rear of the cabinet.

11. Turn the winch crank clockwise to raise the switch. Move the lift tool into

position in front of the cabinet while you align the 24-inch load plate with the line on the bridge tool.

12. Turn the winch crank counterclockwise 1/4 turn to set the winch brake.

13. Set the wheel brake. Place the wheel chocks around the wheels.

CAUTION: A fully populated SAN768B-2 weighs approximately 159.2 kg (351 lb) and requires a minimum of two people and a lift tool to install it.

55 kg ( 121.2 lbs)

P/N 18P5850-B

SJ000752

(C011)

svc00169

CAUTION: A pinch point exists between the load plate and the bridge tool.

14. Slide the SAN768B-2 from the cabinet onto the 24–inch load plate. Center the SAN768B-2 on the platform.

15. Install the retention straps.

16. Remove the wheel chocks, release the wheel brake, and move the lift tool

away from the cabinet.

17. Lower the load platform to its lowest position.

18. Remove the bridge tool.

30 SAN768B-2 Installation, Service, and User Guide

Chapter 3. Starting and configuring the SAN768B-2

Attention: Refer to the latest Fabric OS Release Notes or Interoperability Matrix for information regarding Fabric OS compatibility. This is particularly important for fabrics with 1-Gbps switches using earlier Fabric OS versions. To access the appropriate matrix:

v Go to the IBM SAN web page www.ibm.com/systems/storage/san v Select the IBM System Storage SAN768B-2 and SAN384B-2 product link v On the displayed product page, click the Interoperability Matrix link in the

Learn more section.

The SAN768B-2 must be configured before it is connected to the fabric, and all of the configuration commands must be entered through the active CP blade. The SAN768B-2 configuration includes the following parameters:

v IP address and subnet mask for the chassis v IP addresses, host names, subnet masks, and gateway addresses for both CP

blades

v Switch name v Domain ID for the SAN768B-2 (optional) v WWN for the SAN768B-2

Note: The SAN768B-2 WWN is initially set by the factory to match the license

ID (which is based on the chassis serial number). The WWN can be changed, but the license ID cannot be modified.

The configuration information is mirrored to the standby CP blade, which allows the most current configuration to remain available even if the active CP blade fails. The configuration information for the SAN768B-2 is stored in the WWN cards and the flash memory of the CP blades. The configuration can be backed up to a workstation (uploaded) and then downloaded to the active CP blade if necessary.

Figure 13 on page 32 provides an overview of the sequence of tasks to configure the SAN768B-2.

Configuration Tasks

Connect Serial Cable

between Switch and Host

Set IP Address

Log off Serial Console Port

and Disconnect Serial Cable

Connect Ethernet Cable

and Open Telnet Session

Set Domain ID

Set Date and Time

Install SFPs and

Attach Cables

Manage Cables

Verify Operation and

Back Up Configuration

Figure 13. Flow of basic configuration tasks

384b004

The initial physical hardware installation continues with the following steps:

v “Providing power to the SAN768B-2” on page 33 v “Establishing a serial connection and logging on to the SAN768B-2” on page 33 v “Configuring the IP addresses” on page 34

The following additional steps are customer responsibilities, generally completed during the initial configuration:

1. Establish an Ethernet connection. See “Establishing an Ethernet connection” on

page 36.

2. Specify a switch name. See “Customize a switch name” on page 36.

3. Specify a domain ID for the SAN768B-2. See “Setting the Domain ID” on page

37.

4. Verify the PID mode and connect the SAN768B-2 to the fabric. See “Verifying

the PID mode” on page 39.

5. Enable software licenses. See “Determining installed software licenses” on page

39.

6. Back up the configuration. See “Verifying correct operation and backing up the

configuration” on page 42.

32 SAN768B-2 Installation, Service, and User Guide

Providing power to the SAN768B-2

DANGER

Use the supplied power cords. Ensure the facility power receptacle is the correct type, supplies the required voltage, and is properly grounded. (D004)

To provide power to the SAN768B-2:

1. Connect the AC power cords to the power supply assemblies. Two or four power cords are required depending on electrical service.

Attention: A chassis fully loaded with 16 Gbps port blades (384 ports total) should be supplied with four power supplies connected to 200-240 VAC lines.

2. Connect the power cords to power sources with voltage of 200 to 240 VAC, 47 to 63 Hz (two or four power cords).

3. Turn the AC power switches on the power supplies to ON (|). The AC power switches light green when switched on and power is supplied.

4. The SAN768B-2 performs a power-on self-test (POST) each time it is powered on. POST takes approximately 10 minutes and is complete when indicator light activity indicates the operational state. For information about LED patterns, see Chapter 4, “Monitoring system components,” on page 45.

You can bypass POST by using the fastBoot command. You can also disable POST for successive reboots on the SAN768B-2 using the diagDisablePost command.

Attention: To prevent a potential IP address conflict, do not connect the SAN768B-2 to the network until the IP addresses are configured.

Note: To power off the SAN768B-2, see “Powering off the SAN768B-2” on page

44.

Establishing a serial connection and logging on to the SAN768B-2

To establish a serial connection and log on to the SAN768B-2:

1. Verify that the SAN768B-2 is powered on and that POST is complete by verifying that all power LED indicators on the port, control processor, and core switch blades display a steady green light.

2. Remove the shipping cap from the CONSOLE port on the active CP. The active CP blade is indicated by an illuminated (blue) LED. Use the serial cable provided with the SAN768B-2 to connect the CONSOLE port on the active CP to a computer workstation.

Attention: The CONSOLE port is intended primarily for the initial setting of the IP address and for service purposes.

3. Access the SAN768B-2 using a terminal emulator application (such as HyperTerminal in a Windows environment or TERM in a UNIX environment).

4. Disable any serial communication programs running on the workstation (such as synchronization programs).

5. Open the terminal emulator application and configure as follows: v For most Microsoft Windows systems, use the following configuration

parameters.

Bits per second: 9600 Data bits: 8

Chapter 3. Starting and configuring the SAN768B-2 33

Parity: None Stop bits: 1 Flow control: None

v For most UNIX systems, type the string at the prompt:

term /dev/ttyb -9600

If ttyb is already taken, use ttya instead and enter the following string at the prompt:

term /dev/ttya -9600

When the terminal emulator application stops reporting information, press Enter. You will receive the following login prompt:

CP0 Console Login:

Logging in to the serial console port

To log in to the SAN768B-2 through the serial connection, follow these steps.

1. Log in to the SAN768B-2 as admin. The default password is "password." At the initial login, you are prompted to change the default passwords. Record the new passwords and keep this information in a secure location. The example below shows the login without changing the password.

Fabric OS (swDir)

swDir login: admin Password: Please change your passwords now. Use Control-C to exit or press ’Enter’ key to proceed.

Password was not changed. Will prompt again at next login until password is changed. swDir:admin>

2. (Optional) Modify passwords. To skip modifying the password, press Ctrl+C . Passwords can be 8 to 40 characters long. They must begin with an alphabetic character. They can include numeric characters, the dot (.), and the underscore (_). Passwords are case-sensitive, and they are not displayed when you enter them on the command line. For more information on passwords, refer to the Fabric OS Administrator’s Guide.

Configuring the IP addresses

The SAN768B-2 requires three IP addresses, which are set using the ipAddrSet command. IP addresses are required for both CP blades (CP0 and CP1) and for the chassis management IP (shown as SWITCH under the ipAddrShow command). These IP addresses are set using the ipAddrSet command through the serial or Ethernet connection to the active CP blade.

Note: The default IP addresses and host names for the SAN768B-2 are as follows:

v 10.77.77.75 for CP0 (the CP blade in slot 6 at the time of configuration) v 10.77.77.74 for CP1 (the CP blade in slot 7 at the time of configuration)

Attention: Resetting an IP address while the SAN768B-2 has active IP traffic such as Fabric Manager, Fabric Watch, SNMP, or other applications can cause traffic to be interrupted or stopped.

Complete the following steps to set the IP addresses for the SAN768B-2.

1. Log in to the active CP as admin using the serial connection.

34 SAN768B-2 Installation, Service, and User Guide

2. Set up the SAN768B-2 IP address by entering the ipaddrset -chassis command at the prompt.

swDir:admin> ipaddrset -chassis

Enter the information at the prompts. Specify the -chassis IP address. The -sw 0 IP address is not valid on this chassis.

Note: The addresses 10.0.0.0 through 10.0.0.255 are reserved and used

internally by the SAN768B-2. External IPs must not use these addresses.

3. Set up the CP0 blade IP address by entering the ipaddrset -cp 0 command at the prompt.

swDir:admin> ipAddrSet -cp 0

Enter the requested information at the prompts.

4. Set up the CP1 blade IP address by entering the ipaddrset -cp 1 command at the prompt.

swDir:admin> ipAddrSet -cp 1

Enter the requested information at the prompts. The following is a sample IP configuration for the SAN768B-2 (chassis) and the

two CP blades (cp0 and cp1).

swDir:admin>ipaddrset -chassis Ethernet IP Address [0.0.0.0]: 123.123.123.120 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Fibre Channel IP Address [0.0.0.0]: Fibre Channel Subnetmask [0.0.0.0]: Issuing gratuitous ARP...Done. Committing configuration...Done.

swDir:admin> ipaddrset -cp 0 Host Name [cp0]: Ethernet IP Address [10.77.77.75]: 123.123.123.121 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Gateway IP Address [0.0.0.0]: 123.123.123.124 IP address is being changed...Done. Committing configuration...Done.

swDir:admin> ipaddrset -cp 1 Host Name [cp1]: Ethernet IP Address [10.77.77.74]: 123.123.123.122 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Gateway IP Address [0.0.0.0]: 123.123.123.124 IP address of remote CP is being changed...Done. Committing configuration...Done. swDir:admin> reboot

Logging off the serial console port and disconnecting the serial cable

You can use the serial port to monitor error messages through the serial connection. If the serial port is no longer required, use the logout command to log out of the serial console, remove the serial cable, and replace the plug in the serial port.

Chapter 3. Starting and configuring the SAN768B-2 35

Establishing an Ethernet connection

Note: Connecting the CP blades to a private network/VLAN is recommended.

After using a serial connection to configure the IP addresses for the SAN768B-2, you can connect the active CP blade to the local area network (LAN) if desired.

By establishing an Ethernet connection, you can complete the SAN768B-2 configuration using a serial session, Telnet, or through a management application such as Network Advisor.

To establish an Ethernet connection to the SAN768B-2, follow these steps:

1. Remove the shipping plug from the Ethernet port on the active CP blade.

2. Insert one end of an Ethernet cable into the Ethernet port.

3. Connect the other end to an Ethernet 10/100/1000 Base-T LAN.

The SAN768B-2 can now be accessed by remote connection using the command line via Telnet or any of the management tools, such as Web Tools or Network Advisor.

4. To complete any additional SAN768B-2 configuration procedures through a Telnet session, log in to the SAN768B-2 by Telnet, using the admin login. The default password is password.

Customize a switch name

The switch name of the SAN768B-2 can be up to 30 characters long; can include letters, numbers, and underscore characters; and must begin with a letter.

Note: Changing the name causes a domain address format RSCN to be issued.

To customize the name, follow these steps:

1. Enter the switchName command with the new name in quotes.

swDir:admin> switchName "ModularSwitch5" Committing configuration... Done. ModularSwitch5:admin

2. Record the new name for future reference.

Customizing a chassis name

The chassis name of the SAN768B-2 can be up to 15 characters long; can include letters, numbers, hyphens, and underscore characters; and must begin with a letter.

1. Enter chassisName followed by the new name in double quotes.

switch:admin> chassisname "SAN768B-2_chassis_1" Committing configuration... Done.

2. To display the new chassis name, enter the chassisName command.

switch:admin> chassisname SAN768B-2_chassis_1

3. Record the new name for reference.

36 SAN768B-2 Installation, Service, and User Guide

Setting the Domain ID

Each switch in the fabric must have a unique Domain ID. The Domain ID can be set using the configure command or can be automatically set. The default Domain ID for the SAN768B-2 is "1".

To set the Domain ID, follow these steps:

1. Enter the switchDisable command to disable the SAN768B-2.

2. Enter the configure command.

3. Enter y at the "Fabric parameters" prompt:

Fabric parameters (yes, y, no, n): [no] y

4. Enter a unique Domain ID:

Domain: (1..239) [1] 3

5. Complete the remaining prompts or press Ctrl+D to accept the settings and to exit.

6. Enter the switchEnable command to re-enable the SAN768B-2.

Setting the date and time

The date and time settings are used for logging events. Switch operation does not depend on the date and time; a switch with an incorrect date and time value still functions properly. However, because the date and time are used for logging, error detection, and troubleshooting, they should be set correctly.

Setting the date

To set the date, follow these steps.

1. If necessary, log on to the SAN768B-2 by Telnet, using the admin account. The default password is password.

2. Enter the date command, using the following syntax:date "mmddHHMMyy" Where:

v mm is the month; valid values are 01 through 12. v dd is the date; valid values are 01 through 31. v HH is the hour; valid values are 00 through 23 v MM is minutes; valid values are 00 through 59. v yy is the year; valid values are 00 through 99 (values greater than 69 are

interpreted as 1970 through 1999, and values less than 70 are interpreted as 2000-2069).

switch:admin> date Fri Sep 24 17:01:48 UTC 2010 switch:admin> date "0927123010" Mon Sep 27 12:30:00 UTC 2010 switch:admin>

Setting the time zone

You must perform the procedure on all chassis for which the time zone must be set. However, you only need to set the time zone once on each switch, because the value is written to nonvolatile memory. Use one of the two following procedures to set the time zone.

Chapter 3. Starting and configuring the SAN768B-2 37

The following procedure describes how to set the current time zone using timezone_fmt mode to Central Standard time.

1. Log on to the chassis using the admin account. The default password is

password

2. Enter the tsTimeZone command as follows:

switch:admin> tstimezone [--interactive]/ [, timezone_fmt]

Use timezone_fmt to set the time zone by Country/City or by time zone ID, such as PST. The following example shows how to change the time zone to US/Central.

switch:admin> tstimezone_fmt Time Zone : US/Pacific switch:admin> tstimezone US/Central switch:admin> tstimezone Time Zone : US/Central

The following procedure describes how to set the current time zone using interactive mode to Pacific Standard Time.

1. Enter the tsTimeZone command as follows:

switch:admin> tstimezone --interactive

2. You are prompted to select a general location.

Please identify a location so that time zone rules can be set correctly.

3. Enter the appropriate number or Ctrl+D to quit.

4. At the prompt, select a country location.

5. At the prompt, enter the appropriate number to specify the time zone region or

Ctrl+D to quit.

Synchronizing local time

To synchronize the local time of the principal or primary switch with that of an external NTP server, follow these steps.

1. If necessary, log on to the switch by Telnet, using the admin account.

2. Enter the tsClockServer command:

switch:admin> tsclockserver "<ntp1;ntp2>"

where ntp1 is the IP address or DNS name of the first NTP server, which the switch must be able to access. The variable ntp2 is the second NTP server and is optional. The operand "<ntp1;ntp2>" is optional; by default, this value is LOCL, which uses the local clock of the principal or primary switch as the clock server.

The tsClockServer command accepts multiple server addresses in either IPv4, IPv6, or DNS name formats. When multiple NTP server addresses are passed, tsClockServer sets the first obtainable address as the active NTP server. The others will be stored as backup servers that can take over if the active NTP server fails. The principal or primary FCS switch synchronizes its time with the NTP server every 64 seconds.

switch:admin> tsclockserver LOCL switch:admin> tsclockserver "132.163.135.131" switch:admin> tsclockserver

132.163.135.131 switch:admin>

The following example shows how to set up more than one NTP server using a DNS name:

38 SAN768B-2 Installation, Service, and User Guide

switch:admin> tsclockserver "10.32.170.1;10.32.170.2;ntp.localdomain.net" Updating Clock Server configuration...done. Updated with the NTP servers

Changes to the clock server value on the principal or primary FCS switch are propagated to all switches in the fabric

Verifying the PID mode

Before connecting the SAN768B-2 to the fabric, verify that the port identifier (PID) mode on the SAN768B-2 matches the other switches in the fabric. This parameter must be set identically on all switches in the fabric. This parameter is set using the

configure command. For information on PID mode, refer to the Fabric OS Administrator's Guide.

Determining installed software licenses

Certain licenses may be factory installed on the SAN768B-2. To determine which licenses are currently enabled, use the licenseShow command.

swDir:admin> licenseshow AAbbccDDeeFFeeGG:

Web license Zoning license Extended Fabric license Fabric Watch license Performance Monitor license Trunking license Security license

In the example above, the license key is AAbbccDDeeFFeeGG. Keep a copy of the license key for reference.

The 64-bit chassis ID is required to obtain and activate licenses for the SAN768-2. The chassis ID is available through the licenseIdShow command.

The licenseShow and licenseIdShow commands must be entered on the active CP blade.

Refer to the Fabric OS Administrator's Guide for more information.

Installing transceivers and attaching cables

The following two sets of steps cover the installation of transceivers and cables for most SFPs and for QSFPs.

Note: For a listing of transceivers compatible with this product:

the Learn more section.

v In the Interoperability matrix, click the Transceivers link.

Follow the first set of steps to install SFP+s and mSFPs (FC8-64 port blade only) and cables in the SAN768B-2 (for more details, see “Removing and installing transceivers” on page 92). Follow the second set of steps to install the QSFP

Chapter 3. Starting and configuring the SAN768B-2 39

transceivers and cables in the core blades for inter-chassis link connections (for more details, see “Removing and replacing inter-chassis link (QSFP) cables” on page 95).

Attention: mSFP transceivers are compatible only with the FC8-64 port blade. While they will fit in other blades, this configuration is unsupported and will generate an error.

The ports are color-coded to indicate which can be used in the same port group for trunking (trunking port groups can be up to eight ports). The ports and cables used in trunking groups must meet specific requirements. Refer to the Fabric OS Administrator’s Guide for more information.

Installing SFP+ and mSFP transceivers and cables

Follow these steps to add the optical transceivers and cables to the Fibre Channel ports.

1. Position one of the optical transceivers so that the key is oriented correctly to the port. Insert the transceiver into the port until it is firmly seated and the latching mechanism clicks.

Transceivers are keyed so that they can only be inserted with the correct orientation. If a transceiver does not slide in easily, ensure that it is correctly oriented.

2. Position a cable so that the key (the ridge on one side of the cable connector) is aligned with the slot in the transceiver; then, insert the cable into the transceiver until the latching mechanism clicks.

Cables are keyed so that they can only be inserted with the correct orientation. If a cable does not slide in easily, ensure that it is correctly oriented.

3. Repeat steps 1 and 2 for the remaining ports.

4. Organize the cables as required. For recommendations regarding cable

management, refer to “Managing cables” on page 42.

5. Verify the chassis and port status using the switchShow command.

6. Verify fabric connectivity using the fabricShow command.

Installing QSFP transceivers and ICL cables

Follow these steps to install the QSFPs and cables in the 16 Gbps core blades. These transceivers and cables are used to form the inter-chassis links (ICL) with neighboring SAN768B-2 and SAN384B-2 backbones. The transceivers should be installed in the blades before connecting the cables. Because each QSFP contains four 16 Gbps ports, be aware that any problems with one port could affect all four ports in the quad if the QSFP has to be replaced.

1. Position one of the QSFP transceivers so that the key is oriented correctly to the port. Insert the transceiver into the port until it is firmly seated. Transceivers are keyed so that they can only be inserted with the correct orientation. If a transceiver does not slide in easily, ensure that it is correctly oriented. When the transceiver is correctly seated, the status LED will flash amber several times and then turn solid amber.

2. Remove the protective cap from the special QSFP cable and insert it into the transceiver until it is firmly seated. The cables are also keyed to fit into the transceivers correctly. When the cable is correctly seated, the status LED will change from amber to green.

3. Repeat steps 1 and 2 for the remaining ICL.

40 SAN768B-2 Installation, Service, and User Guide

4. Organize the cables (see “Managing cables” on page 42).

5. Verify the chassis and connector and port status using the switchShow -qsfp

command. A sample of the command output is shown below. The example is from a SAN384B-2 with a core blade installed in slot 3. Some details for the SAN768B-2 will be different (such as the slot numbers) , but the reported information for the QSFPs will be similar. Note that the State reported for an unconnected QSFP (shown QSFP 0 and Ports 0-3 below) is No_SigDet. This is different from the State of No_Synch that is reported for regular SFPs that are unconnected. QSFP 7 (ports 3/28-3/31, Index 748-751) in the following example shows the results for a connected QSFP.

switch:FID128:admin> switchshow -qsfp switchName: switch name switchType: 121.3 switchState: Online switchMode: Native switchRole: Subordinate switchDomain: 75 switchId: fffc4b switchWwn: 10:00:00:05:1e:4f:eb:00 zoning: ON (zoning name) switchBeacon: OFF FC Router: OFF Allow XISL Use: OFF LS Attributes: [FID: 128, Base Switch: No, Default Switch: Yes, Address Mode 0] Index Slot Port QSFP Address Media Speed State Proto ============================================================= 256300------ id 16G No_SigDet FC 257310------ id 16G No_SigDet FC 258320------ id 16G No_SigDet FC 259330------ id 16G No_SigDet FC 260341------ -- 16G No_Module FC 261351------ -- 16G No_Module FC 262361------ -- 16G No_Module FC 263371------ -- 16G No_Module FC 264382------ -- 16G No_Module FC 265392------ -- 16G No_Module FC 266 3 10 2 ------ -- 16G No_Module FC 267 3 11 2 ------ -- 16G No_Module FC 268 3 12 3 ------ -- 16G No_Module FC 269 3 13 3 ------ -- 16G No_Module FC 270 3 14 3 ------ -- 16G No_Module FC 271 3 15 3 ------ -- 16G No_Module FC 736 3 16 4 ------ -- 16G No_Module FC 737 3 17 4 ------ -- 16G No_Module FC 738 3 18 4 ------ -- 16G No_Module FC 739 3 19 4 ------ -- 16G No_Module FC 740 3 20 5 ------ -- 16G No_Module FC 741 3 21 5 ------ -- 16G No_Module FC 742 3 22 5 ------ -- 16G No_Module FC 743 3 23 5 ------ -- 16G No_Module FC 744 3 24 6 ------ -- 16G No_Module FC 745 3 25 6 ------ -- 16G No_Module FC 746 3 26 6 ------ -- 16G No_Module FC 747 3 27 6 ------ -- 16G No_Module FC 748 3 28 7 ------ id 16G Online FC E-Port 10:00:00:05:1e:39:e4:5a "trunkmaster name" (Trunk master) 749 3 29 7 ------ id 16G Online FC E-Port 10:00:00:05:1e:39:e4:5a "trunkmaster name" (Trunk master) 750 3 30 7 ------ id 16G Online FC E-Port 10:00:00:05:1e:39:e4:5a "trunkmaster name" (Trunk master) 751 3 31 7 ------ id 16G Online FC E-Port 10:00:00:05:1e:39:e4:5a "trunkmaster name" (Trunk master)

Chapter 3. Starting and configuring the SAN768B-2 41

Managing cables

Attention: The minimum bend radius should be no smaller than 10x the cable radius. The minimum bend radius for a 50 micron cable is 5 cm (2 in.) under full tensile load and 3 cm (1.2 in.) with no tensile load.

Note: Contact your IBM representative for options regarding different cable and

Cable management recommendations:

v Leave at least 1 m (3.28 ft) of slack for each port cable. This provides room to

v If ISL Trunking is in use, group the cables by trunking group. The ports are

v Use only hook-and-loop fasteners for securing optical cables. Do not use cable

v For easier maintenance, label the fiber optic cables and record the devices to

v Use the cable management fingers attached to the rack rails to organize the

v Keep LEDs visible by routing port cables and other cables away from the LEDs.

patch panel configurations to simplify cable management with higher density FC8-64 port blades.

remove and replace blades in the chassis, allows for inadvertent movement of the rack, and helps prevent the cables from being bent to less than the minimum bend radius.

color-coded to indicate which ports can be used in the same ISL Trunking group: eight ports marked with solid black ovals alternate with eight ports marked with oval outlines.

ties. They can be easily overtightened and can damage the optical cables.

which they are connected.

cables off to the side of the chassis, and away from the chassis exhaust vents.

For the procedure to install the ICL cables see “Removing and replacing inter-chassis link (QSFP) cables” on page 95.

Verifying correct operation and backing up the configuration

To verify correct operation and to back up the system, follow these steps.

1. Check the LEDs to verify that all components are functional. For information about LED patterns, see Chapter 4, “Monitoring system components,” on page

45.

2. If necessary, log on to the switch by Telnet, using the admin account.

3. Verify the correct operation of the system by entering the switchShow

command from the workstation. This command provides information about switch and port status.

4. Verify the correct operation of the SAN768B-2 in the fabric by entering the fabricShow command from the workstation. This command provides general information about the fabric.

5. To back up the configuration, run the following two steps: a. Enter the configupload -vf command. This command uploads the system

virtual fabric data.

b. Enter the configupload command. This command uploads the system

configuration.

6. Run the following commands to see additional configuration information that you can then copy to a file to save:

v configShow

v ipAddrShow

42 SAN768B-2 Installation, Service, and User Guide

v licenseShow

v switchShow

Alternatively, you can save the configuration file to the USB device by using the usbstorage command.

Note: Passwords are not saved in the configuration file, and are not uploaded

during a configUpload. Password information should be saved in hardcopy in a secure location for emergency reference, subject to your security and password management policy.

Note: It is recommended that the configuration be backed up on a regular basis to

ensure that a complete configuration is available for downloading to a replacement system.

Fabric OS firmware updates

To ensure optimum functioning of your system, IBM recommends operating your system with the latest Fabric OS version and firmware updates. Refer to the latest Fabric OS Release Notes and Interoperability Matrix for information regarding Fabric OS compatibility. This is particularly important for fabrics with 1-Gbps switches using earlier Fabric OS versions. Detailed information on downloading and installing firmware are provided in “Downloading firmware from an FTP server” on page 79 and “Downloading firmware from a USB drive” on page 81. More information can also be found in the Fabric OS Administrator's Guide.

To access the appropriate matrix:

v Go to the IBM SAN web page www.ibm.com/systems/storage/san v Select the SAN768B-2 and SAN384B-2 product link v On the displayed product page, click the Interoperability Matrix link in the

Learn more section.

Firmware release notes and download files are available through the IBM Support Portal.

To view release notes and to download firmware:

v Go to the IBM Support Portal www.ibm.com/supportportal. v Search for the product Machine type, 2499 or the product name. v On the next page that is displayed, select Download under the Task section. v From the displayed page, select the Release notes link. Review the release notes

of the appropriate release version before installing the firmware update.

v Click the Release Firmware link and follow the online prompts to navigate to

the Brocade Downloads for IBM End Users page.

v Follow the links and instructions on the Brocade pages. Note that associated

FOS documentation can also be downloaded from this site.

Downloading and installing firmware

See “Downloading firmware from an FTP server” on page 79 and “Downloading firmware from a USB drive” on page 81 for detailed steps to download and install firmware on the CP blades.

1. The firmware version on both the active and backup CP blade should match, and should be at the latest level. If High Availability is enabled (haEnable), the system will automatically update the firmware level on both CP blades.

Chapter 3. Starting and configuring the SAN768B-2 43

Attention: Firmware downloads require Ethernet connections to both CP blades. Procedures for downloading and installing firmware are fully described in the Fabric OS Administrator's Guide.

2. When you have navigated to the appropriate download site, follow the online instructions on the displayed Fabric OS download pages, referring to the Fabric OS Administrator's Guide as needed.

3. Enter firmwareDownload -s on your to download the firmware to one of the CP blades from your server.

Powering off the SAN768B-2

If you need to power off the system, follow these steps to avoid loss of data.

1. Shut down the system using the sysShutdown command.

switch::admin> sysshutdown This command will shutdown the operating systems on your switch. You are required to power-cycle the switch in order to restore operation. Are you sure you want to shutdown the switch [y/n]? y HA is disabled Stopping blade 1

Shutting down the blade....

Stopping blade 4

Shutting down the blade....

Stopping blade 10

Shutting down the blade....

Broadcast message from root (pts/1) Tue Aug 23 14:23:06 2011... The system is going down for system halt NOW !!

2. Power off the chassis by flipping the AC power switches on the power supplies to "O". The LEDs inside AC power switches should turn off. To maintain the ground connections, leave the power cords connected to the power supplies and to the electrical outlets.

44 SAN768B-2 Installation, Service, and User Guide

Chapter 4. Monitoring system components

The SAN768B-2 is engineered for reliability and requires no routine operational steps or maintenance. This chapter provides information about determining the status of the following components using LEDs and CLI commands. Refer to the Fabric OS Administrator’s Guide and the Web Tools Administrator’s Guide for additional information.

There are two commands that can be especially helpful in monitoring the health of the SAN768B-2. These commands are switchShow and chassisShow. Examples of these commands are shown below. Note in the switchShow command the new switchType for the SAN768B-2 as well as the 16 Gbps speed identification for capable ports. The output has been truncated to reduce information duplication.

SAN768B-2_8 :root> switchshow switchName: SAN768B-2_8 switchType: 120.3 switchState: Online switchMode: Native switchRole: Subordinate switchDomain: 120 switchId: fffc78 switchWwn: 10:00:00:05:1e:d2:64:00 zoning: ON (BB) switchBeacon: OFF FC Router: ON FC Router BB Fabric ID: 10 Address Mode: 0 Index Slot Port Address Media Speed State Proto =======================================================

0 1 0 780000 id N8 No_Light FC Disabled