IBM SAN768B Installation, Service And User Manual

Download

Page 1

IBM System Storage SAN768B



Installation, Service, and User’s Guide

Service information: 2499-384

Read Before Using

This product contains software that is licensed under written license agreements. Your use of such software is subject to the license agreements under which they are provided.

GA32-0574-01

Page 2

Page 3

IBM System Storage SAN768B



Installation, Service, and User’s Guide

Service information: 2499-384

GA32-0574-01

Page 4

Note:

Before using this information and the product it supports, read the information in “Notices” on page 117.

The following paragraph does not apply to any country (or region) where such provisions are inconsistent with local law.

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states (or regions) do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you.

US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Page 5

Read this first

Summary of changes

| | | | |

Second edition

| |

Getting help

This is the second edition of the IBM System Storage SAN768B Installation, Service, and User's Guide. The content changes since the last edition of this

publication are noted by a vertical line placed in the left margin beside each change. Minor edits are not identified by this mark. A summary of the changes for each edition are listed below.

The following changes were made in the second edition:

v The section on power cords was corrected. v The rack installation instructions were modified to reflect a new rack installation

kit.

v The Integrated Routing feature was added. v Minor changes were made to the environmental, safety, and other notices. v Minor changes to Fabric OS publication titles.

For the latest version of your product documentation, visit the web at http://www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi. Search by form number or title.

For more information about IBM® SAN products, see the following Web site: http://www.ibm.com/servers/storage/san/

For support information for this product and other SAN products, see the following Web site: http://www.ibm.com/servers/storage/support/san

You can also contact IBM within the United States at 1-800-IBMSERV (1-800-426-7378). For support outside the United States, you can find the service number at: http://www.ibm.com/planetwide/.

Visit www.ibm.com/contact for the contact information for your country or region.

For detailed information about the Fibre Channel standards, see the Fibre Channel Industry Association (FCIA) Web site at: www.fibrechannel.org/

For information about storage industry standards, see the Storage Networking Industry Association (SNIA) Web site at: http://www.snia.org/

Taiwan Contact Information

IBM Taiwan Product Service Contact Info: IBM Taiwan Corporation 3F, No 7, Song Ren Rd., Taipei Taiwan Tel: 0800-016-888

Page 6

How to send your comments

Your feedback is important in helping us provide the most accurate and high-quality information. If you have comments or suggestions for improving this document, send us your comments by e-mail to starpubs@us.ibm.com or use the Readers’ Comments form at the back of this publication. Be sure to include the following:

v Exact publication title v Form number (for example, GC26-1234-02) v Page numbers to which you are referring

the Reader Comment Form in the back of this manual is missing, you can direct

If your mail to:

International Business Machines Corporation Information Development Department GZW 9000 South Rita Road Tucson, Arizona 85744-0001 U.S.A.

When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

iv SAN768B Installation, Service, and User’s Guide

Page 7

Read this first . . . . . . . . . . . . . . . . . . . . . . . . . iii

Summary of changes . . . . . . . . . . . . . . . . . . . . . . . iii

Second edition . . . . . . . . . . . . . . . . . . . . . . . . iii

Getting help . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Taiwan Contact Information . . . . . . . . . . . . . . . . . . . . iii

How to send your comments . . . . . . . . . . . . . . . . . . . .iv

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . .xi

Safety and environmental notices . . . . . . . . . . . . . . . . . xiii

Safety notices and labels . . . . . . . . . . . . . . . . . . . . . xiii

Danger notices . . . . . . . . . . . . . . . . . . . . . . . . xiii

Caution notices . . . . . . . . . . . . . . . . . . . . . . . . xvi

Safety labels . . . . . . . . . . . . . . . . . . . . . . . . xvii

Attention notices . . . . . . . . . . . . . . . . . . . . . . . xviii

Rack safety . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Rack installation . . . . . . . . . . . . . . . . . . . . . . . xix

Rack relocation (19″ rack) . . . . . . . . . . . . . . . . . . . .xx

Safety inspections . . . . . . . . . . . . . . . . . . . . . . . xxi

Removing ac power . . . . . . . . . . . . . . . . . . . . . . xxi

External machine checks . . . . . . . . . . . . . . . . . . . . xxi

Internal machine checks . . . . . . . . . . . . . . . . . . . . xxi

Product recycling and disposal . . . . . . . . . . . . . . . . . . . xxii

Battery return program . . . . . . . . . . . . . . . . . . . . . xxiii

For the European Union: . . . . . . . . . . . . . . . . . . . . xxiii

About this document . . . . . . . . . . . . . . . . . . . . . xxv

Who should read this document . . . . . . . . . . . . . . . . . . xxv

Product documents . . . . . . . . . . . . . . . . . . . . . . . xxv

Brocade documents . . . . . . . . . . . . . . . . . . . . . . . xxv

IBM and Brocade product matrix . . . . . . . . . . . . . . . . . xxv

Chapter 1. Introduction . . . . . . . . . . . . . . . . . . . . . .1

Overview of the SAN768B . . . . . . . . . . . . . . . . . . . . .1

Hardware components . . . . . . . . . . . . . . . . . . . . . .2

SAN768B blades . . . . . . . . . . . . . . . . . . . . . . . .4

High availability . . . . . . . . . . . . . . . . . . . . . . . . .6

Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . .6

Serviceability . . . . . . . . . . . . . . . . . . . . . . . . . .6

Software features . . . . . . . . . . . . . . . . . . . . . . . .7

Security . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Network Manageability . . . . . . . . . . . . . . . . . . . . .8

Port numbering . . . . . . . . . . . . . . . . . . . . . . . . .8

Managing cables . . . . . . . . . . . . . . . . . . . . . . . . .9

Chapter 2. Installing and removing a SAN768B . . . . . . . . . . . .11

Ordering the lift tool . . . . . . . . . . . . . . . . . . . . . . .12

World trade locations . . . . . . . . . . . . . . . . . . . . .12

United States locations . . . . . . . . . . . . . . . . . . . . .12

Installation guidelines . . . . . . . . . . . . . . . . . . . . . .13

Installing a SAN768B in a cabinet . . . . . . . . . . . . . . . . . .14

Page 8

Time required . . . . . . . . . . . . . . . . . . . . . . . .14

Tools required . . . . . . . . . . . . . . . . . . . . . . . .14

|| || || ||

Parts list . . . . . . . . . . . . . . . . . . . . . . . . . .14

Unpacking the SAN768B . . . . . . . . . . . . . . . . . . . .15

Attaching the shelf brackets . . . . . . . . . . . . . . . . . . .16

Removing the chassis door . . . . . . . . . . . . . . . . . . .17

Using the lift tool, the 24–inch load plate, and the bridge tool . . . . . . .17

Removing a SAN768B from the cabinet . . . . . . . . . . . . . . . .23

Chapter 3. Starting and configuring the SAN768B . . . . . . . . . . .25

Providing power to the SAN768B . . . . . . . . . . . . . . . . . .26

Establishing a serial connection and logging on to the SAN768B . . . . . .26

Configuring the SAN768B IP addresses . . . . . . . . . . . . . . . .27

Establish an Ethernet connection . . . . . . . . . . . . . . . . . .29

Customize a switch name . . . . . . . . . . . . . . . . . . . . .30

Set the Domain ID . . . . . . . . . . . . . . . . . . . . . . .30

Verifying the PID mode and connecting to the fabric . . . . . . . . . . .30

Enabling software licenses . . . . . . . . . . . . . . . . . . . .31

Backing up the configuration . . . . . . . . . . . . . . . . . . . .32

Chapter 4. Monitoring system components . . . . . . . . . . . . .33

Control processor blade (CP8) . . . . . . . . . . . . . . . . . . .33

Core switch blade (CR8) . . . . . . . . . . . . . . . . . . . . .35

FR4-18i router blade . . . . . . . . . . . . . . . . . . . . . . .37

Port or application blades . . . . . . . . . . . . . . . . . . . . .40

Power supply . . . . . . . . . . . . . . . . . . . . . . . . .45

Blower assembly . . . . . . . . . . . . . . . . . . . . . . . .46

WWN bezel and WWN card . . . . . . . . . . . . . . . . . . . .48

Chapter 5. Removing and installing components . . . . . . . . . . .51

Removing and installing the chassis door . . . . . . . . . . . . . . .51

Time and items required . . . . . . . . . . . . . . . . . . . .51

Removing the chassis door . . . . . . . . . . . . . . . . . . .51

Removing and installing a cable management comb . . . . . . . . . . .52

Time and items required . . . . . . . . . . . . . . . . . . . .52

Removing a cable management comb . . . . . . . . . . . . . . .52

Installing a cable management comb . . . . . . . . . . . . . . . .53

Removing and installing port blades . . . . . . . . . . . . . . . . .53

Time and items required . . . . . . . . . . . . . . . . . . . .54

Removing a port blade . . . . . . . . . . . . . . . . . . . . .54

Installing a port blade . . . . . . . . . . . . . . . . . . . . .55

Removing and installing port blade filler panels . . . . . . . . . . . . .56

Removing a filler panel . . . . . . . . . . . . . . . . . . . . .56

Installing a filler panel . . . . . . . . . . . . . . . . . . . . .57

Removing and installing a control processor blade (CP8) . . . . . . . . .57

Time and items required . . . . . . . . . . . . . . . . . . . .58

Verifying the necessity of installation . . . . . . . . . . . . . . . .58

Recording critical SAN768B information . . . . . . . . . . . . . . .58

Removing a control processor blade (CP8) . . . . . . . . . . . . .60

Installing a control processor blade (CP8) . . . . . . . . . . . . . .61

Verifying operation of the new CP blade . . . . . . . . . . . . . .62

Removing and installing a core switch blade (CR8) . . . . . . . . . . .63

Time and items required . . . . . . . . . . . . . . . . . . . .64

Verifying the necessity of installation . . . . . . . . . . . . . . . .64

Removing a core switch blade (CR8) . . . . . . . . . . . . . . . .64

Installing a core switch blade (CR8) . . . . . . . . . . . . . . . .65

vi SAN768B Installation, Service, and User’s Guide

Page 9

Removing and installing an FR4-18i router blade . . . . . . . . . . . .66

Time required . . . . . . . . . . . . . . . . . . . . . . . .66

Items required . . . . . . . . . . . . . . . . . . . . . . . .66

Removing a router blade . . . . . . . . . . . . . . . . . . . .66

Installing a router blade . . . . . . . . . . . . . . . . . . . . .67

Configuring FCIP and Fibre Channel routing services and enabling the ports 68

Cabling the FR4-18i router blade . . . . . . . . . . . . . . . . .68

Removing and installing a power supply . . . . . . . . . . . . . . .69

Time and items required . . . . . . . . . . . . . . . . . . . .69

Removing a power supply . . . . . . . . . . . . . . . . . . . .69

Installing a power supply . . . . . . . . . . . . . . . . . . . .70

Removing and installing a blower assembly . . . . . . . . . . . . . .71

Time and items required . . . . . . . . . . . . . . . . . . . .71

Removing a blower assembly . . . . . . . . . . . . . . . . . .71

Installing a blower assembly . . . . . . . . . . . . . . . . . . .72

Removing and installing a WWN bezel and WWN card . . . . . . . . . .72

Time and items required . . . . . . . . . . . . . . . . . . . .72

Verifying necessity of installation . . . . . . . . . . . . . . . . .73

Removing the WWN bezel and WWN card . . . . . . . . . . . . .74

Installing the WWN bezel and WWN card . . . . . . . . . . . . . .75

Installing and removing FC transceivers in a port blade . . . . . . . . . .76

Chapter 6. Installing new features . . . . . . . . . . . . . . . . .79

FC3816, FC3832, FC3848 - Installing a 8-Gb port blade (16, 32, or 48 ports) 79

Time required . . . . . . . . . . . . . . . . . . . . . . . .79

Items required . . . . . . . . . . . . . . . . . . . . . . . .79

Procedures . . . . . . . . . . . . . . . . . . . . . . . . .79

FC3850 - Installing an FR4-18i routing blade . . . . . . . . . . . . . .80

Time required . . . . . . . . . . . . . . . . . . . . . . . .81

Items required . . . . . . . . . . . . . . . . . . . . . . . .81

Procedures . . . . . . . . . . . . . . . . . . . . . . . . .81

FC3870 - Installing an FC10-6 10 Gb blade . . . . . . . . . . . . . .82

Time required . . . . . . . . . . . . . . . . . . . . . . . .82

Items required . . . . . . . . . . . . . . . . . . . . . . . .82

Procedures . . . . . . . . . . . . . . . . . . . . . . . . .82

Chapter 7. Installing the chassis . . . . . . . . . . . . . . . . . .83

Verifying necessity of replacement . . . . . . . . . . . . . . . . . .83

Preparing for a chassis installation . . . . . . . . . . . . . . . . . .84

Time required . . . . . . . . . . . . . . . . . . . . . . . .84

Items required . . . . . . . . . . . . . . . . . . . . . . . .84

Recording critical SAN768B and SAN information . . . . . . . . . . .84

Chassis critical information checklist . . . . . . . . . . . . . . . .87

Disconnecting the SAN768B from the network and fabric . . . . . . . . .88

Removing components from the chassis . . . . . . . . . . . . . . .88

Removing and replacing the chassis . . . . . . . . . . . . . . . . .89

Installing components into the new chassis . . . . . . . . . . . . . .90

Verifying the correct operation of hardware . . . . . . . . . . . . . .90

Verifying the correct operation of the system . . . . . . . . . . . . . .91

Reconnecting the system to the network and fabric . . . . . . . . . . .92

Verifying the correct configuration of the fabric . . . . . . . . . . . . .93

Cable routing tables . . . . . . . . . . . . . . . . . . . . . . .93

Appendix A. Product specifications . . . . . . . . . . . . . . . .95

System architecture . . . . . . . . . . . . . . . . . . . . . . .95

Physical dimensions . . . . . . . . . . . . . . . . . . . . . . .97

Contents vii

Page 10

Facility requirements . . . . . . . . . . . . . . . . . . . . . . .97

Power specifications . . . . . . . . . . . . . . . . . . . . . . .98

Environmental requirements . . . . . . . . . . . . . . . . . . . .99

Data transmission ranges . . . . . . . . . . . . . . . . . . . . .99

Fibre channel port specifications . . . . . . . . . . . . . . . . . . 100

FR4-18i router Fibre channel port specifications . . . . . . . . . . . . 100

FR4-18i router GbE port specifications . . . . . . . . . . . . . . . . 100

Power cords . . . . . . . . . . . . . . . . . . . . . . . . . 101

Appendix B. Diagnostics and troubleshooting . . . . . . . . . . . . 103

Obtaining chassis and component status . . . . . . . . . . . . . . . 103

Interpreting POST and boot results . . . . . . . . . . . . . . . . . 104

POST . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Validating the installation of port blades . . . . . . . . . . . . . . . 105

Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . 106

Powering off the SAN768B . . . . . . . . . . . . . . . . . . . . 109

Appendix C. Port numbering templates . . . . . . . . . . . . . . 111

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . .117

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . .119

Electronic emission notices . . . . . . . . . . . . . . . . . . . . 120

Federal Communications Commission (FCC) Class A Statement . . . . . 120

Industry Canada Class A Emission Compliance Statement . . . . . . . 120

Avis de conformité à la réglementation d’Industrie Canada . . . . . . . 120

European Union (EU) Electromagnetic Compatibility Directive . . . . . . 120

Germany Electromagnetic Compatibility Directive . . . . . . . . . . . 121

Japan VCCI Class A ITE Electronic Emission Statement . . . . . . . . 122

People’s Republic of China Class A Electronic Emission Statement . . . . 122

Korea Class A Electronic Emission Statement . . . . . . . . . . . . 122

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

viii SAN768B Installation, Service, and User’s Guide

Page 11

Figures

1. Port side of the SAN768B (sample configuration) . . . . . . . . . . . . . . . . . . .3

2. Blower (non-port) side of the SAN768B (sample configuration) . . . . . . . . . . . . . .4

3. Parts in the 14U rack mount kit . . . . . . . . . . . . . . . . . . . . . . . . .15

4. Left and right shelf brackets installed on the cabinet rails . . . . . . . . . . . . . . . .16

5. Installing clip or retainer nuts on the vertical cabinet rails . . . . . . . . . . . . . . . .17

6. Chassis installation from rear (exhaust aisle side) of the cabinet . . . . . . . . . . . . .19

7. Locking the shelf into position . . . . . . . . . . . . . . . . . . . . . . . . . .20

8. Attaching the port side of the chassis to the cabinet rails . . . . . . . . . . . . . . . .21

9. Aligning the chassis door with the ball studs on the chassis . . . . . . . . . . . . . . .22

10. Details of the chassis and door, showing the ball studs on the chassis and the mounting hardware

| ||

on the door . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

11. Control processor blade (CP8) LEDs . . . . . . . . . . . . . . . . . . . . . . .34

12. Core switch blade (CR8) LEDs . . . . . . . . . . . . . . . . . . . . . . . . . .36

13. FR4-18i router blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

14. FC8-16 port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

15. FC8-32 port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

16. FC8-48 port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

17. FC10-6 port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

18. Power supply LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46

19. Blower assembly LEDs and other parts . . . . . . . . . . . . . . . . . . . . . .47

20. LEDs on the WWN card and bezel . . . . . . . . . . . . . . . . . . . . . . . .49

21. Chassis door . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52

22. Cable management comb . . . . . . . . . . . . . . . . . . . . . . . . . . .53

23. Port blade removal and installation (16-port blade shown as example) . . . . . . . . . . .55

24. Port blade removal and installation (16-port blade shown as example) . . . . . . . . . . .57

25. Removing the control processor (CP) blade . . . . . . . . . . . . . . . . . . . . .61

26. Removing the core switch blade . . . . . . . . . . . . . . . . . . . . . . . . .65

27. Removing and installing a power supply . . . . . . . . . . . . . . . . . . . . . .70

28. Blower assembly LEDs and other parts . . . . . . . . . . . . . . . . . . . . . .72

29. WWN bezel and card removal and installation . . . . . . . . . . . . . . . . . . . .75

30. SFP extraction tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

31. Port side populated with eight FC8-48 port blades, two CR8 blades, and two CP8 blades 112

32. Port side populated with eight FC8-32 port blades, two CR8 blades, and two CP8 blades 113

33. Port side populated with eight FC8-16 port blades, two CR8 blades, and two CP8 blades 114

34. FC10-6 port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115

35. FC4-18IP port blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116

Page 12

x SAN768B Installation, Service, and User’s Guide

Page 13

Tables

1. Brocade and IBM product and model number matrix . . . . . . . . . . . . . . . . . xxvi

2. Blades available for the SAN768B . . . . . . . . . . . . . . . . . . . . . . . . .4

3. Security features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

4. Port blades available on the SAN768B . . . . . . . . . . . . . . . . . . . . . . .8

5. Items supplied with the 14U rack mount kit . . . . . . . . . . . . . . . . . . . . .14

6. Configuration parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

7. Example login without changing the password . . . . . . . . . . . . . . . . . . . .27

8. Example of viewing enabled licenses . . . . . . . . . . . . . . . . . . . . . . .31

9. CP blade LED descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . .34

10. CR blade LED descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . .36

11. FR4-18i router blade LED patterns during normal operation . . . . . . . . . . . . . . .39

12. Port and application blade LED descriptions . . . . . . . . . . . . . . . . . . . . .45

13. Power supply LED descriptions . . . . . . . . . . . . . . . . . . . . . . . . .46

14. Blower assembly LED descriptions . . . . . . . . . . . . . . . . . . . . . . . .47

15. Messages that may indicate WWN card failure . . . . . . . . . . . . . . . . . . . .48

16. WWN bezel LED descriptions . . . . . . . . . . . . . . . . . . . . . . . . . .49

17. WWN LED patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73

18. Commands identifying the WWN card status . . . . . . . . . . . . . . . . . . . .73

19. WWN card related system log messages . . . . . . . . . . . . . . . . . . . . . .74

20. Critical information checklist example . . . . . . . . . . . . . . . . . . . . . . .87

21. Sample cable routing table for SAN768B (Switch Name: swDir) . . . . . . . . . . . . .93

22. Cable routing table for SAN768B . . . . . . . . . . . . . . . . . . . . . . . . .93

23. System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

24. System sizes and weights . . . . . . . . . . . . . . . . . . . . . . . . . . .97

25. System FRU weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97

26. Power specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98

27. Environmental requirements . . . . . . . . . . . . . . . . . . . . . . . . . . .99

28. Data transmission ranges by cable type and port speed . . . . . . . . . . . . . . . .99

29. Environmental status and maintenance commands . . . . . . . . . . . . . . . . . . 104

30. Commands for validating installation of port blades . . . . . . . . . . . . . . . . . . 106

31. Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Page 14

xii SAN768B Installation, Service, and User’s Guide

Page 15

Safety and environmental notices

This section contains information about:

v “Safety notices and labels” v “Rack safety” on page xix v “Safety inspections” on page xxi v “Product recycling and disposal” on page xxii

Safety notices and labels

When using this product, observe the danger, caution, and attention notices contained in this guide. The notices are accompanied by symbols that represent the severity of the safety condition. The danger and caution notices are listed in numerical order based on their IDs, which are displayed in parentheses, for example (D004), at the end of each notice. Use this ID to locate the translation of these danger and caution notices in the IBM Systems Safety Notices (G229–9054) publication, which is on the CD-ROM that accompanies this product.

The following notices and statements are used in IBM documents. They are listed below in order of increasing severity of potential hazards. Follow the links for more detailed descriptions and examples of the danger, caution, and attention notices in the sections that follow.

v Note: These notices provide important tips, guidance, or advice. v “Attention notices” on page xviii: These notices indicate potential damage to

programs, devices, or data.

v “Caution notices” on page xvi: These statements indicate situations that can

be potentially hazardous to you.

v “Danger notices”: These statements indicate situations that can be potentially

lethal or extremely hazardous to you. Safety labels are also attached directly to

products to warn of these situations.

v In addition to these notices, “Safety labels” on page xvii may be attached to the

product to warn of potential hazards.

Danger notices

A danger notice calls attention to a situation that is potentially lethal or extremely hazardous to people. A lightning bolt symbol accompanies a danger notice to represent a dangerous electrical condition. Read and comply with the following danger notices before installing or servicing this device.

DANGER

prevent a possible shock from touching two surfaces with

different protective ground (earth), use one hand, when possible, to connect or disconnect signal cables. (D001)

Page 16

DANGER

Overloading

a branch circuit is potentially a fire hazard and a

shock hazard under certain conditions. To avoid these hazards, ensure that your system electrical requirements do not exceed branch circuit protection requirements. Refer to the information that is provided with your device or the power rating label for electrical specifications. (D002)

DANGER

the receptacle has a metal shell, do not touch the shell until

you have completed the voltage and grounding checks. Improper wiring or grounding could place dangerous voltage on the metal shell. If any of the conditions are not as described, STOP. Ensure the improper voltage or impedance conditions are corrected before proceeding. (D003)

DANGER

electrical outlet that is not correctly wired could place

hazardous voltage on metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (D004)

A general electrical danger notice provides instructions on how to avoid shock hazards when servicing equipment. Unless instructed otherwise, follow the procedures in the following danger notice.

xiv SAN768B Installation, Service, and User’s Guide

Page 17

DANGER

When

working on or around the system, observe the following

precautions:

Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard:

v Connect power to this unit only with the IBM provided power

cord. Do not use the IBM provided power cord for any other product.

v Do not open or service any power supply assembly.

v Do not connect or disconnect any cables or perform

installation, maintenance, or reconfiguration of this product during an electrical storm.

v The product might be equipped with multiple power cords. To

remove all hazardous voltages, disconnect all power cords.

v Connect all power cords to a properly wired and grounded

electrical outlet. Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate.

v Connect any equipment that will be attached to this product to

properly wired outlets.

v When possible, use one hand only to connect or disconnect

signal cables.

v Never turn on any equipment when there is evidence of fire,

water, or structural damage.

v Disconnect the attached power cords, telecommunications

systems, networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures.

v Connect and disconnect cables as described below when

installing, moving, or opening covers on this product or attached devices.

Disconnect:

1. Turn off everything (unless instructed otherwise).

2. Remove the power cords from the outlets.

3. Remove the signal cables from the connectors.

4. Remove all cables from the devices.

Connect:

1. Turn off everything (unless instructed otherwise).

2. Attach all cables to the devices.

3. Attach the signal cables to the connectors.

4. Attach the power cords to the outlets.

5. Turn on the devices.

(D005)

The weight of the SAN cabinet (2109-C36) with two SAN768Bs is greater than 227 kg (500 lb), and has a fully loaded capacity of 816 kg (1795 lb). Under these conditions, the following statement and notice apply.

Safety and environmental notices xv

Page 18

Delivery and subsequent transportation of the equipment

The customer should prepare his environment to accept the new product based on the installation planning information provided, with assistance from an IBM Installation Planning Representative (IPR) or IBM authorized service provider. In anticipation of the equipment delivery, the final installation site should be prepared in advance such that professional movers/riggers can transport the equipment to the final installation site within the computer room. If for some reason, this is not possible at the time of delivery, the customer will need to make arrangements to have professional movers/riggers return to finish the transportation at a later date. Only professional movers/riggers should transport the equipment. The IBM authorized service provider will only perform minimal frame repositioning within the computer room, as needed, to perform required service actions. The customer is also responsible for using professional movers/riggers in the case of equipment relocation or disposal.

>(>)500 lbs. 227 kg.

DANGER

Heavy

equipment—personal injury or equipment damage might

result if mishandled. (D006)

a69i0333

Caution notices

A caution notice calls attention to a situation that is potentially hazardous to people because of some existing condition. A caution notice can be accompanied by different symbols, as in the examples below:

If the symbol

is... It means....

55 kg ( 121.2 lbs)

>55kg (121.2 lb)

P/N 18P5850-B

SJ000752

A hazardous electrical condition with less severity than electrical danger.

A generally hazardous condition not represented by other safety symbols.

A specification of product weight that requires safe lifting practices. The weight range of the product is listed below the graphic, and the wording of the caution varies, depending on the weight of the device.

svc00169

A potential hazard of pinching the hand or other body parts between parts.

A hazardous condition due to moving parts nearby.

A hazardous condition due to the use of a laser in the product. Laser symbols are always accompanied by the classification of the laser as defined by the U. S. Department of Health and Human Services (for example, Class I, Class II, and so forth).

xvi SAN768B Installation, Service, and User’s Guide

Page 19

Read and comply with the following caution notices before installing or servicing this device.

55 kg ( 121.2 lbs)

>55kg (121.2 lb)

CAUTION: Energy hazard present. Shorting may result in system outage and possible physical injury. Remove all metallic jewelry before servicing. (C001)

CAUTION: The weight of this part or unit is more than 55 kg (121.2 lb). It takes specially trained persons, a lifting device, or both to safely lift this part or unit. (C011)

svc00169

CAUTION: This product is equipped with a 3-wire (two conductors and ground) power cable and plug. Use this power cable with a properly grounded electrical outlet to avoid electrical shock. (C018)

CAUTION: Servicing of this product or unit is to be performed by trained service personnel only. (C032)

Safety labels

As an added precaution, safety labels are often installed directly on products or product components to warn of potential hazards. These can be either danger or caution notices, depending upon the level of the hazard.

The actual product safety labels may differ from these sample safety labels:

DANGER

Hazardous

voltage, current, or energy levels are present inside

any component that has this label attached. Do not open any cover or barrier that contains this label. (L001)

DANGER

Rack-mounted

devices are not to be used as a shelf or work

space. (L002)

Safety and environmental notices xvii

Page 20

DANGER

Multiple

power cords. The product might be equipped with

multiple power cords. To remove all hazardous voltages, disconnect all power cords. (L003)

DANGER

Hazardous

voltage present. Voltages present constitute a shock

hazard, which can cause severe injury or death. (L004)

CAUTION: Hazardous energy present. Voltages with hazardous energy might cause heating when shorted with metal, which might result in splattered metal, burns, or both. (L005)

P/N 18P5850-B

Attention notices

An attention notice indicates the possibility of damage to a program, device, or system, or to data. An exclamation point symbol may accompany an attention notice, but is not required. A sample attention notice follows:

CAUTION: Hazardous moving parts nearby (L008)

CAUTION: Pinch hazard. (L012)

SJ000752

Attention: Do not bend a fibre cable to a radius less than 5 cm (2 in.); you can

damage the cable. Tie wraps are not recommended for optical cables because they can be easily overtightened, causing damage to the cable.

xviii SAN768B Installation, Service, and User’s Guide

Page 21

Rack safety

Rack installation

DANGER

Observe

the following precautions when working on or around your IT rack system:

v Heavy equipment—personal injury or equipment damage might result if

mishandled.

v Always lower the leveling pads on the rack cabinet.

v Always install stabilizer brackets on the rack cabinet.

v To avoid hazardous conditions due to uneven mechanical loading, always install

the heaviest devices in the bottom of the rack cabinet. Always install servers and optional devices starting from the bottom of the rack cabinet.

v Rack-mounted devices are not to be used as shelves or work spaces. Do not

place objects on top of rack-mounted devices.

v Each rack cabinet might have more than one power cord. Be sure to disconnect

all power cords in the rack cabinet when directed to disconnect power during servicing.

v Connect all devices installed in a rack cabinet to power devices installed in the

same rack cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power device installed in a different rack cabinet.

v An electrical outlet that is not correctly wired could place hazardous voltage on

the metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock.

(R001 part 1 of 2) CAUTION:

v Do not install a unit in a rack where the internal rack ambient temperatures will

exceed the manufacturer’s recommended ambient temperature for all your rack-mounted devices.

v Do not install a unit in a rack where the air flow is compromised. Ensure that air

flow is not blocked or reduced on any side, front, or back of a unit used for air flow through the unit.

v Consideration should be given to the connection of the equipment to the supply

circuit so that overloading of the circuits does not compromise the supply wiring or overcurrent protection. To provide the correct power connection to a rack, refer to the rating labels located on the equipment in the rack to determine the total power requirement of the supply circuit.

v (For sliding drawers) Do not pull out or install any drawer or feature if the rack

stabilizer brackets are not attached to the rack. Do not pull out more than one drawer at a time. The rack might become unstable if you pull out more than one drawer at a time.

v (For fixed drawers) This drawer is a fixed drawer and must not be moved for

servicing unless specified by the manufacturer. Attempting to move the drawer partially or completely out of the rack might cause the rack to become unstable or cause the drawer to fall out of the rack.

part 2 of 2)

(R001

Safety and environmental notices xix

Page 22

Rack relocation (19″ rack)

CAUTION: Removing components from the upper positions in the rack cabinet improves rack stability during relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a room or building:

v Reduce the weight of the rack cabinet by removing equipment starting at

the top of the rack cabinet. When possible, restore the rack cabinet to the configuration of the rack cabinet as you received it. If this configuration is not known, you must do the following:

– Remove all devices in the 32U position and above. – Ensure that the heaviest devices are installed in the bottom of the rack

cabinet.

– Ensure that there are no empty U-levels between devices installed in the

rack cabinet below the 32U level.

– If the rack cabinet you are relocating is part of a suite of rack cabinets,

detach the rack cabinet from the suite.

– Inspect the route that you plan to take when moving the rack to

eliminate potential hazards.

– Verify that the route that you choose can support the weight of the

loaded rack cabinet. Refer to the documentation that came with your

rack cabinet for the weight of a loaded rack cabinet. – Verify that all door openings are at least 760 x 2030 mm (30 x 80 in.). – Ensure that all devices, shelves, drawers, doors, and cables are secure. – Ensure that the four leveling pads are raised to their highest position. – Ensure that there is no stabilizer bracket installed on the rack cabinet

during movement. – Do not use a ramp inclined at more than 10 degrees. – Once the rack cabinet is in the new location, do the following:

- Lower the four leveling pads.

- Install stabilizer brackets on the rack cabinet.

- If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest position to the highest position.

– If a long distance relocation is required, restore the rack cabinet to the

configuration of the rack cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent. Also, lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the pallet.

(R002)

xx SAN768B Installation, Service, and User’s Guide

Page 23

Safety inspections

Perform the following safety checks to identify unsafe conditions. Be cautious of potential safety hazards that are not covered in the safety checks. If unsafe conditions are present, determine how serious the hazards are and whether you should continue before you correct the problem.

Removing ac power

Perform the following steps to remove the alternating current (ac) power:

1. Perform a controlled system shutdown.

2. Set the power switch on the product to the off position.

3. Disconnect the power cables from the power source.

External machine checks

Perform the following external machine checks:

1. Verify that all external covers are present and are not damaged.

2. Ensure that all latches and hinges are in correct operating condition.

3. Check the power cable for damage.

4. Check the external signal cable for damage.

5. Check the cover for sharp edges, damage, or alterations that expose the internal parts of the device.

6. Check that any unused serial ports are covered for dust and ESD protection. The cover should be kept on the serial port whenever it is not being used.

7. Correct any problems that you find.

DANGER

Multiple power cords. (L003)

Internal machine checks

Perform the following internal machine checks:

1. Check for any non-IBM changes that might have been made to the machine. If any are present, obtain the “Non-IBM Alteration Attachment Survey” form, number R009, from the IBM branch office. Complete the form and return it to the branch office.

2. Check the condition of the inside of the machine for:

v Metal or other contaminants v Indications of water or other fluid v Fire v Smoke damage

3. Check for any obvious mechanical problems, such as loose components.

4. Check any exposed cables and connectors for wear, cracks, or pinching.

Safety and environmental notices xxi

Page 24

Product recycling and disposal

This unit must be recycled or discarded according to applicable local and national regulations. IBM encourages owners of information technology (IT) equipment to responsibly recycle their equipment when it is no longer needed. IBM offers a variety of product return programs and services in several countries to assist equipment owners in recycling their IT products. Information on IBM product recycling offerings can be found on IBM’s Internet site at http://www.ibm.com/ibm/ environment/products/index.shtml

Esta unidad debe reciclarse o desecharse de acuerdo con lo establecido en la normativa nacional o local aplicable. IBM recomienda a los propietarios de equipos de tecnología de la informacion (TI) que reciclen responsablemente sus equipos cuando éstos ya no les sean utiles. IBM dispone de una serie de programas y servicios de devolucion de productos en varios países, a fin de ayudar a los propietarios de equipos a reciclar sus productos de TI. Se puede encontrar informacion sobre las ofertas de reciclado de productos de IBM en el sitio web de IBM http://www.ibm.com/ibm/environment/products/index.shtml.

Note: This mark applies only to countries within the European Union (EU) and

Norway.

Appliances are labeled in accordance with European Directive 2002/96/EC concerning waste electrical and electronic equipment (WEEE). The Directive determines the framework for the return and recycling of used appliances as applicable throughout the European Union. This label is applied to various products to indicate that the product is not to be thrown away, but rather reclaimed upon end of life per this Directive.

In accordance with the European WEEE Directive, electrical and electronic equipment (EEE) is to be collected separately and to be reused, recycled, or recovered at end of life. Users of EEE with the WEEE marking per Annex IV of the WEEE Directive, as shown above, must not dispose of end of life EEE as unsorted municipal waste, but use the collection framework available to customers for the return, recycling and recovery of WEEE. Customer participation is important to minimize any potential effects of EEE on the environment and human health due to

xxii SAN768B Installation, Service, and User’s Guide

Page 25

the potential presence of hazardous substances in EEE. For proper collection and treatment, contact your local IBM representative.

Battery return program

This product may contain sealed lead acid, nickel cadmium, nickel metal hydride, lithium, or lithium ion battery. Consult your user manual or service manual for specific battery information. The battery must be recycled or disposed of properly. Recycling facilities may not be available in your area. For information on disposal of batteries outside the United States, go to http://www.ibm.com/ibm/environment/ products/index.shtml

In the United States, IBM has established a return process for reuse, recycling, or proper disposal of used IBM sealed lead acid, nickel cadmium, nickel metal hydride, and other battery packs from IBM equipment. For information on proper disposal of these batteries, contact IBM at 1-800-426-4333. Please have the IBM part number listed on the battery available prior to your call.

For Taiwan:

or contact your local waste disposal facility.

Please recycle batteries.

For the European Union:

Note: This mark applies only to countries within the European Union (EU).

Batteries or packaging for batteries are labeled in accordance with European Directive 2006/66/EC concerning batteries and accumulators and waste batteries and accumulators. The Directive determines the framework for the return and recycling of used batteries and accumulators as applicable throughout the European Union. This label is applied to various batteries to indicate that the battery is not to be thrown away, but rather reclaimed upon end of life per this Directive.

Les batteries ou emballages pour batteries sont étiquetés conformément aux directives européennes 2006/66/EC, norme relative aux batteries et accumulateurs en usage et aux batteries et accumulateurs usés. Les directives déterminent la marche à suivre en vigueur dans l’Union Européenne pour le retour et le recyclage des batteries et accumulateurs usés. Cette étiquette est appliquée sur diverses batteries pour indiquer que la batterie ne doit pas être mise au rebut mais plutôt récupérée en fin de cycle de vie selon cette norme.

Safety and environmental notices xxiii

Page 26

In accordance with the European Directive 2006/66/EC, batteries and accumulators are labeled to indicate that they are to be collected separately and recycled at end of life. The label on the battery may also include a chemical symbol for the metal concerned in the battery (Pb for lead, Hg for mercury and Cd for cadmium). Users of batteries and accumulators must not dispose of batteries and accumulators as unsorted municipal waste, but use the collection framework available to customers for the return, recycling and treatment of batteries and accumulators. Customer participation is important to minimize any potential effects of batteries and accumulators on the environment and human health due to the potential presence of hazardous substances. For proper collection and treatment, contact your local IBM representative.

This notice is provided in accordance with Royal Decree 106/2008 of Spain: The retail price of batteries, accumulators and power cells includes the cost of the environmental management of their waste.

xxiv SAN768B Installation, Service, and User’s Guide

Page 27

About this document

This document describes how to install and service the IBM System Storage SAN768B fabric backbone. Throughout this document, the product is referred to as the SAN768B.

The sections that follow provide information about:

v “Who should read this document” v “Product documents” v “Brocade documents”

Who should read this document

This document is intended for trained service representatives who are responsible for installing and servicing the SAN768B, and for network administrators responsible for maintaining and using the SAN768B.

Product documents

The following documents contain information related to this product: v IBM System Storage SAN768B Installation, Service, and User’s Guide,

GA32-0574 (this document)

v IBM Systems Safety Notices, G229–9054 v IBM System Storage SAN 2499 Statement of Limited Warranty, GA32-0575

Brocade documents

IBM switches use software licensed from Brocade Communications Systems, Inc. You can find information related to the software that supports the SAN768B in the following documents on the CD-ROM supplied with this product:

Brocade Fabric OS

v Fabric OS Administrator’s Guide

v Fabric OS Command Reference Manual

v Fabric OS MIB Reference Manual

v Fabric OS Message Reference Manual

v Fabric OS Troubleshooting and Diagnostics Guide

v SAN Glossary

Brocade Fabric OS optional features

v Fabric Watch Administrator’s Guide

v Web Tools Administrator’s Guide

IBM and Brocade product matrix

When you use any of the Brocade documents, you will notice that the model numbers reflect the original Brocade products. Table 1 on page xxvi provides a product matrix for you to use to correlate the Brocade model numbers to the IBM product names and machine types and model numbers.

Page 28

Table 1. Brocade and IBM product and model number matrix

IBM machine type and

Brocade product name IBM product name

model number

Brocade DCX SAN768B 2499 Model 384 Brocade 48000 SAN256B Director 2109 Model M48 Brocade 5300 SAN80B-4 2498 Model B80 Brocade 5100 SAN40B-4 2498 Models B40 and 40E Brocade 300 SAN24B-4 2498 Models B24 and 24E Brocade 7500E SAN04B–R 2005 Model R04 Brocade 5000 SAN32B-3 2005 Models B5K and 5KB Silkworm 7500 SAN18B-R 2005 Model R18 Silkworm 200E SAN16B-2 2005 Models B16 and 16B Silkworm 4900 SAN64B-2 2005 Model B64

xxvi SAN768B Installation, Service, and User’s Guide

Page 29

Chapter 1. Introduction

This chapter introduces the features and components of the IBM System Storage SAN768B fabric backbone. Throughout the remainder of this document, the product will be referred to as the SAN768B. This chapter contains the following information:

v “Overview of the SAN768B” v “Hardware components” on page 2 v “High availability” on page 6 v “Reliability” on page 6 v “Serviceability” on page 6 v “Software features” on page 7

Overview of the SAN768B

The SAN768B represents the next generation of advanced Fibre Channel directors used to intelligently interconnect storage devices, hosts, and servers in a Storage Area Network (SAN). It satisfies the most demanding Reliability, Availability, and Serviceability (RAS), performance, and scalability requirements, while delivering investment protection, interoperability, and fabric-based intelligence advantages found only in the IBM SAN product family.

Key features of the SAN768B include:

v Up to 384 ports in a single chassis, providing high port density for a scalable

solution to drive high-port-count SAN configurations.

v Support for high-performance port blades running at 1-, 2-, 4-, 8-, or 10-Gbps,

enabling flexible system configuration.

v Supports 1-, 2-, 4-, and 8-Gbps auto-sensing Fibre Channel ports. Trunking

technology groups up to eight ports to create high performance 64-Gbps ISL trunks between switches. (10 Gbps ports (FC10-6) are 10 Gbps only.)

v Dual-redundant control processor blades (CP8) and core switch blades (CR8)

provide high availability and enable nondisruptive software upgrades.

v Redundant and hot-swappable CP8 and CR8 blades, power supplies, blower

assemblies, and WWN cards enable a high availability platform for mission critical SAN applications.

v Inter-chassis linking (ICL) through CR8 blades. v Universal ports self-configure as E_ports, F_ports, FL_ports, Ex_ports, and

M_ports (mirror ports). – 10 Gbps ports (FC10-6) are E-Ports only. – Ex_ports are supported only on the FR4-18i application blade. – FL_ports are not supported on the FC8-48 port blade.

Page 30

Hardware components

The SAN768B features a modular and scalable mechanical construction that allows a wide range of flexibility in installation, fabric design, and maintenance. The chassis may be mounted with the cables facing the front of the equipment rack or to the rear, and consists of the following:

v Up to eight hot-swappable port blade assemblies can be configured in a single

chassis, delivering up to 384 Fibre Channel ports.

v Two slots for control processor blades (CP8):

– A single active CP8 blade can control all 384 ports in the chassis. – The standby CP8 blade assumes control of the SAN768B if the active CP

Two slots for core switch blades (CR8):

– CR8 blade interconnects all port blades – Two inter-chassis link (ICL) connectors per blade to connect to another

– Both CR8 blades are active

Modular hot-swappable port blades:

– 16-port, 8-Gbps blades (FC8-16) – 32-port, 8-Gbps blades (FC8-32) – 48-port, 8-Gbps blades (FC8-48) – 6-port, 10-Gbps blades (FC10-6)

v Modular hot-swappable application blades:

– FR4-18i: 18-port (16 FC + 2 GbE), up to 4 blades per chassis, supporting

v Modular hot-swappable field replaceable units (FRUs):

– Three blower assemblies – Up to four power supplies – Two WWN cards – Small Form-factor Pluggable (SFP) optical transceivers (4- and 8-Gbps) – Extended Form-factor Pluggable (XFP) optical transceivers (10-Gbps)

– 1 GbE copper SFP transceivers for the IP ports on the routing blade

v All blades are serviced from the port side of the SAN768B. Blowers, power

supplies, and power cables are serviced from the nonport side.

v Improved cable management using a redesigned cable management comb and

chassis door

v Constant intake and FRU temperature monitoring v World Wide Name (WWN) cards on the nonport side, to maintain chassis-specific

information such as WWNs, IP addresses, and summary status information of each port blade and power supply through LEDs

v Redundant AC primary power connections to ensure high availability. Each power

supply has its own connector

fails.

chassis

Fibre Channel Routing Services and FCIP

Airflow in the SAN768B is from the non-port (non-cable) side to the port

Note:

(cable) side and out the exhaust vent.

1 on page 3 shows a sample configuration of the port side of the SAN768B.

Figure

2 SAN768B Installation, Service, and User’s Guide

Page 31

b768ig013

Figure 1. Port side of the SAN768B (sample configuration)

1 Exhaust vent 3 Control processor blade (CP8) 2 Core switch blade (CR8) 4 FC8-48 port blade

5 Cable management comb

Figure 2 on page 4 shows a sample configuration of the non-port side of the SAN768B.

Chapter 1. Introduction 3

Page 32

Figure 2. Blower (non-port) side of the SAN768B (sample configuration)

1 WWN bezel 3 Blower assembly 2 Power supply

SAN768B blades

Table 2 summarizes the port, application, control processor, and core switch blades that are available for the SAN768B.

b768ig014

Table 2. Blades available for the SAN768B

Description Feature Code Name Function

Control processor blade

N/A CP8 The CP8 blade

contains the control plane for the chassis. There are two CP8 blades for redundancy.

4 SAN768B Installation, Service, and User’s Guide

Page 33

Table 2. Blades available for the SAN768B (continued)

Description Feature Code Name Function

Core switch blade N/A CR8 The CR8 blade

contains the ASICs for switching between port blades. Every port blade connects to every core switch blade. There are 384 total ports for port blades. Each core switch blade connects to 192 backplane ports. Core switch blades have additional front port connectivity to connect multiple chassis and backplane connections for the storage server blade.

16-port 8-Gbps port blade

FC3816 FC8-16 A 16-port blade

supporting 1–, 2–, 4–, and 8–Gbps port speeds.

32-port 8-Gbps port blade

FC3832 FC8-32 A 32-port blade

supporting 1-, 2-, 4-, and 8-Gbps port speeds.

48-port 8-Gbps port blade

FC3848 FC8-48 A 48-port blade

supporting 1-, 2-, 4-, and 8-Gbps port speeds.

6-port 10-Gbps port blade

FC3870 FC10-6 A 6-port blade

supporting 10-Gbps port speed. Blade provides 10-Gbps ISLs.

Fibre Channel router blade

FC3850 FR4-18i The FR4-18i blade

has 16 4-Gbps physical Fibre Channel SFP ports supporting Fibre Channel Routing Services and two physical Gigabit Ethernet (GbE) SFP ports supporting Fibre Channel Over IP (FCIP). The two physical GbE ports can support up to 16 virtual E_ports.

Chapter 1. Introduction 5

Page 34

High availability

The following features contribute to the SAN768B’s high-availability design:

v Redundant, hot-swappable blades and FRUs v Enhanced data integrity on all data paths v Fabric Shortest Path First (FSPF) rerouting around failed links v Integration with Simple Network Management Protocol (SNMP) managers v Automatic control processor failover v Nondisruptive ″hot″ software code loads and activation v Easy configuration, save, and restore v Hot-swappable World Wide Name (WWN) cards

The high-availability software architecture of the SAN768B provides a common framework for all applications that reside on the system, allowing global and local status to be maintained through any component failure. High-availability elements consist of the High Availability Manager, the heartbeat, the fault/health framework, the replicated database, initialization, and software upgrade.

The High Availability Manager controls access to the standby control processor, facilitates software upgrades, prevents extraneous switchover activity, closes and flushes streams as needed, provides flow control and message buffering, and supports a centralized active and standby state.

Reliability

Serviceability

The SAN768B uses the following error detection and correction mechanisms to ensure reliability of data:

v Data is protected by the Error Detection and Correction mechanism, which

checks for encoder errors and fault isolation , such as cyclic redundancy checking (CRC), parity checking, checksum, and illegal address checking

v Power-on self test (POST) v Dual control processors that enable hot, nondisruptive fast firmware upgrades v Each control processor contains one serial port and two Ethernet ports, for

management and for service. Offline control processor diagnostics and remote diagnostics simplify troubleshooting. The standby control processor monitors diagnostics to ensure it is operational, should a failover be necessary

v Bus monitoring and control of blades and other field-replaceable units (FRUs).

The SAN768B provides the following features to enhance and ensure serviceability:

v Modular design with hot-swappable components v Flash memory that stores two firmware images per control processor v Nonvolatile random-access memory (NVRAM), containing the OEM serial

number, IBM serial number, revision information, and part number information

v Background health-check daemon v Memory scrubber, self test, and bus ping to determine if a bus is not functioning v RASlog messages v SMI-S compliant v Watchdog timers

6 SAN768B Installation, Service, and User’s Guide

Page 35

Software features

| | | | |

v Status LEDs v Predictive diagnostics analysis through Fabric Watch v SNMP (including version 3) integration with higher-layer managers

The Fabric OS allows any Fibre Channel-compliant device to attach to the switches as long as it conforms to the device login, name service, and related Fibre Channel standards. Each operating environment requires that a Fibre Channel host bus adapter (HBA) be available with a standards-compliant driver for correct interface to the fabric.

Fabric OS consists of a set of embedded applications running on top of an embedded Linux operating system kernel. These applications include:

v Name server v Alias server v Zone server v Simple Network Management Protocol (SNMP) agent v SMI-S compliant API v Syslog auditing v RCS (Reliable Commit Service) v NTP v Tasks to manage address assignment, routing, link initialization, fabric

initialization, link shutdown, shutdown, and the user interface.

v Integrated Routing–This feature allows any port in a SAN768B, SAN80B-4, and

SAN40B-4 to be configured as an EX_port supporting Fibre Channel (FC) routing. This eliminates the need to add a routing blade or use of the SAN18B-R for FC routing (FCR) purposes, and this also provides double the bandwidth for each FC router connection when connected to another 8 Gbps-capable port.

Security

Table 3 highlights some of the key security features available for the SAN768B and for other switches running Fabric OS 5.2.0 or newer.

Table 3. Security features

Security Feature Description

DH-CHAP Login banner SSHv2 (using AES, 3DES, RSA) Monitoring of attempted security breaches

(via audit logging)

HTTPS (using AES) Monitoring of attempted security breaches

(via Fabric Watch Security Class)

SNPMv3 Fibre Channel security policies: DCC and

SCC

FC-SP Trusted Switch (FCS) for central security

management

Secure RPC Management access controls (SNMPv3,

Telnet, FTP, serial port, front panel)

Secure file copy (SCP) Hardware-enforced zoning by WWN and/or

domain/port ID

Telnet disable Default zoning

Chapter 1. Introduction 7

Page 36

Table 3. Security features (continued)

Security Feature Description

Telnet timeout RSCN suppression and aggregation IP filters (block listeners) Configurable RSCN suppression by port Secure passwords (centralized control via

RADIUS/CHAP) Multiple user accounts (MUAs). Up to 255. Event auditing Role-based access controls (RBACs) Change tracking Administrative domains/Virtual fabrics Firmware change alerts in Fabric Manager Boot PROM password reset Persistent port disable Password hardening policies Persistent domain ID Upfront login in Web Tools E_port disable

Network Manageability

The SAN768B has a single domain and is managed as a single element with the Brocade Enterprise Fabric Connectivity Manager (EFCM) or Brocade Fabric Manager (FM). The SAN768B responds to its own IP address and appears as a separate entity to the Telnet protocol and SNMP.

NTPv3 (to synchronize timestamps)

Port numbering

All management interfaces, such as Telnet, Web Tools, standards compliant SMI-S, and Management Server, support a “port N within blade M” naming scheme.

The SAN768B supports SNMPv1 and SNPMv3. When SNMP devices send SNMP messages to a management console running SAN management software, the information is stored in a management information base (MIB). Fabric OS v6.0.0 and later supports the latest Fibre Alliance Fibre Channel Management (FCMGMT) and Storage Management Initiative (SMI) MIBs, which allow common information necessary for management software to provide information to a SAN administrator.

Note: Refer to the Fabric OS MIB Reference for additional MIB information.

Table 4 has a list of available port blades for the SAN768B.

Table 4. Port blades available on the SAN768B

Blade label IBM blade name Feature code

FC8-16 16 port 8Gb Switch Blade 3816 FC8-32 32 port 8Gb Switch Blade 3832 FC8-48 48 port 8Gb Switch Blade 3848 FC10-6 10Gb Fibre Channel Blade 3870 FR4-18i FC Routing Blade 3850

The SAN768B uses the following port numbering scheme:

v FC8-16 port blade - Ports are numbered from 0 through 15 from bottom to top. v FC8-32 port blade - Ports are numbered from 0 through 15 from bottom to top on

the left set of ports and 16 through 31 from bottom to top on the right set of ports.

8 SAN768B Installation, Service, and User’s Guide

Page 37

v FC8-48 port blade - Ports are numbered from 0 through 23 from bottom to top on

v FC10-6 port blade - Ports are numbered from 0 through 5 from bottom to top. v FR4-18i router blade - The 16 physical Fibre Channel ports are numbered from 0

Slots SAN768B. Control processor blades (CP8) can be installed only in slots 6 and 7. Core switch blades (CR8) can be installed only in slots 5 and 8.

See Appendix C, “Port numbering templates,” on page 111 for port numbering templates for different SAN768B configurations and optional blades.

Managing cables

The cable management comb (shown in Figure 1 on page 3) is attached to the chassis under the chassis door and allows for simple cable management. The comb can be installed without service disruption.

the left set of ports and 24 through 47 from bottom to top on the right set of ports.

through 15 from bottom to top. The two GbE ports, numbered GE0 and GE1, are located below the Fibre Channel ports. these ports, when fully configured, enable 16 VE_ports or VEX_ports and appear in the switchShow command as ports 16 through 31.

are numbered 1 through 12, from left to right when facing the port side of the

Route cables down in front of the blades to keep LEDs visible. Leave at least one meter of slack for each fiber optic cable to provide room to remove and replace blades.

Attention: Do not route the cables in front of the air exhaust vent, which is

located at the top of the port side of the chassis.

If ISL Trunking is in use, group the cables by trunking group. The ports are color-coded to indicate which ports can be used in the same ISL Trunking group: eight ports marked with solid black ovals alternate with eight ports marked with oval outlines.

Chapter 1. Introduction 9

Page 38

10 SAN768B Installation, Service, and User’s Guide

Page 39

Chapter 2. Installing and removing a SAN768B

The SAN768B is only available for installation in the IBM TotalStorage 2109 SAN Switch Cabinet C36. This SAN768B is to be installed and serviced only by qualified IBM service representatives. The SAN768B can be ordered pre-installed in the cabinet, or the SAN768B can be added to an existing C36 cabinet. The cabinet is designed to support a total of two SAN256Bs or SAN768Bs.

Attention: The SAN768B is only available for installation in the IBM TotalStorage

2109 SAN Switch Cabinet C36. The SAN768B can be pre-installed in the cabinet,

| | | | | |

or it can be added to an existing IBM C36 cabinet. For installation into an existing C36 cabinet, a special bridge tool ships with the SAN768B which is designed to attach to the IBM C36 cabinet. This bridge tool supports the weight of the director as it is moved into the cabinet. This tool may not attach to non-IBM cabinets. There is a real risk of personal injury or machine damage during installation into any cabinet other than the approved IBM cabinets .

This chapter describes how to install the SAN768B into a C36 cabinet, and how to remove the SAN768B from the cabinet in the event you need to move or replace the SAN768B. One SAN768B is installed in the cabinet at the factory. If the SAN768B is already installed in the cabinet, continue on to Chapter 3, “Starting and configuring the SAN768B,” on page 25.

Note: Some illustrations in this chapter display different models of IBM SAN

directors. Details, such as the cable management comb and blades may differ, but the procedures remain the same for the SAN768B.

Attention: Refer to “Safety and environmental notices” on page xiii for general

safety instructions, and “Rack safety” on page xix before performing any installation or service procedures.

55 kg ( 121.2 lbs)

CAUTION:

CAUTION: The weight of this part or unit is more than 104 kg (228 lb). It takes specially trained persons with a lifting device to safely lift this part or unit. (C011)

svc00169

A fully populated SAN768B weighs approximately 104 kg (228 lb). Before you install it, verify that the additional weight of the chassis does not exceed the cabinet’s weight limit or unbalance the cabinet. When you calculate the additional weight, include the weights of all components that can potentially be added, to avoid overloading in the future.

Before you install a SAN768B into the cabinet, perform the following tasks:

v To ensure adequate cooling, plan to install the chassis with the port side facing

the aisle where exhaust air is released (usually called the service aisle). This prevents the fans from pulling in heated exhaust air.

v Plan for managing the cables before you install the chassis. A fully populated

SAN768B will have a large number of cables that must be carefully routed to minimize problems with installation and replacement of components, and in order

Page 40

to maintain the minimum bend radius required for optical cables. Refer to “Managing cables” on page 9 for more specific information. You can manage the cables in a variety of ways, such as:

– Routing the cables down through the cable management tray – Routing the cables out either side of the chassis – Routing the cables through the cable channels on the sides of the cabinet – Using patch panels

Ensure that the airflow in the vicinity of the SAN768B is a minimum of 595 cubic

meters per hour (350 cubic feet per minute).

v Ensure that the air temperature that is measured at the blower inlet does not

exceed 40°C (104°F ) during operation.

Ordering the lift tool

Important

The lift tool is required only when you install an additional SAN768B or when you install or remove the SAN768B from the cabinet.

The ordering procedures for the lift tool vary depending on your location. You

should direct questions about these procedures to your regional representative.

The bridge tool (PN 18P5855), which is required for installation, is shipped

Note:

with a new or a replacement SAN768B. The bridge tool, when installed, serves as a temporary ramp between the lift tool and the SAN768B rails that are installed in a C36 cabinet. You do not need to return the bridge tool when you are finished with it.

World trade locations

The following ordering procedures are for world trade locations:

v Order the lift tool by using the parts order system, like any other part. v Use the following part numbers when you order:

– Lift tool: PN 09P2481 – 24–inch load plate: PN 11P4369

Yo u do not record parts usage.

v Return the lift tool and the 24–inch load plate to the parts center after you

complete the installation or removal of the SAN768B.

United States locations

In the United States, call UPS Logistics at 800–528–6070 to order the lift tool and the 24–inch load plate.

Note: For the SSR branch and territory, the United States cannot order the lift tool

or 24–inch load plate through the parts order system. UPS Logistics are used to ship and return the lift tool and 24–inch load plate.

Use the following part numbers when you order:

v Lift tool: PN 09P2481 v 24–inch load plate: PN 11P4369

12 SAN768B Installation, Service, and User’s Guide

Page 41

Note: When you order the lift tool, you will receive an 18–inch load plate. Do not

use the 18–inch load plate. You must order and use a 24–inch load plate when you install the SAN768B.

You must provide the following information when you order the lift tool. This information is necessary to ensure that the lift tool is delivered when you need it. Failure to provide this information might delay the completion of the order request and the shipping request. It might also result in a time and date to return the tool that is different from what you need.

v Phone number and customer contact v Account code: 98577 v Time and date of delivery v Accurate destination address with zip code v Time and date of return pickup

You must return the lift tool at the time that was scheduled with UPS Logistics. If you need to change the scheduled return time or date, contact UPS Logistics.

You are responsible for ensuring that all of the paperwork and components are packed and restored in the arbocrate (shipping container) of the lift tool. Ensure that the lift tool is functioning properly before you release the tool to UPS Logistics for return. Yo u are accountable for the lift tool until UPS Logistics picks up the lift tool for return delivery to their parts storage facility.

Contact your branch office tools coordinator or your region specialist if you have any questions or concerns.

Installation guidelines

Follow these general installation guidelines:

1. Provide a space that is 14 rack units (14U) high, 61.29 cm (24.09 in.) deep, and

43.74 cm (17.22 in.). 1U is equal to 4.45 cm (1.75 in.).

2. Plan to install the SAN768B with the non-port side facing the air-intake aisle. The SAN768B can be installed facing either direction, if serviceability and cooling requirements are met.

3. Plan for cable management before installing the chassis. See “Managing cables” on page 9.

Cables can be managed in a variety of ways, such as by routing cables below the chassis, to either side of the chassis, through cable channels on the sides of the cabinet, or by using patch panels.

4. Ensure that dedicated electrical branch circuits with the following characteristics are available:

v 200 – 240 VAC, 50–60 Hz (two branch circuits) v Two cables for 200 - 240 VA C service v Protected by a circuit breaker in accordance with local electrical codes v Supply circuit, line fusing, and wire size adequate to the electrical rating on

the chassis nameplate

v Location close to the chassis and easily accessible v Grounded outlets installed by a licensed electrician, compatible with the

power cords

Attention: To maximize fault tolerance, connect each power cord to a

separate power source.

Chapter 2. Installing and removing a SAN768B 13

Page 42

5. Ensure that the air intake and exhaust vents have a minimum of 2 in. of airspace.

6. Ensure that the air temperature on the air intake side is less than 40 degrees Celsius (104 degrees Fahrenheit) during operation.

CAUTION:

Use safe lifting practices when moving the product. (C015)

A fully populated SAN768B (eight FC8-48 port cards, 384 ports) weighs

Note:

approximately 104 kg (228 lbs) and requires the specified lift tool to install it.

Installing a SAN768B in a cabinet

This section describes how to install the SAN768B in the Model C36 cabinet. These procedures use parts that are included in the 14U rack mount kit.

Time required

Allow approximately 3 hours to complete this procedure. This includes unpacking, assembling, and repacking the lift tool. This is for the physical installation of the SAN768B only. It does not include installing Ethernet cables, SFPs, attaching and routing the fiber optic cables, or completing the initial configuration.

Tools required

Parts list

|||

||||

|||

||||

|||

Note: Assembly instructions for the lift tool are in the shipping container.

You will require the following items to install the SAN768B in the cabinet:

v Torque wrench with #2 Phillips screwdriver tip v Flathead screwdriver v Lift tool

The following parts list refers to items illustrated in Figure 3 on page 15

Table 5. Items supplied with the 14U rack mount kit

Item Description Quantity

For all types of installations A Left rack mount shelf bracket (rail brackets may differ from the

illustration)

B Right rack mount shelf bracket (rail brackets may differ from the

illustration) C 10-32 x 5/8 inch (1.58 cm) panhead Phillips screw, washer 6 For cabinets that have vertical rails with round holes D 10-32 clip nut 6 E 1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with lock

washer For cabinets that have vertical rails with square holes F 10-32 retainer nut 6 G 1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with glue 16 H 0.375 in. (0.953 cm) alignment washer 16

14 SAN768B Installation, Service, and User’s Guide

Page 43

| | |

Item B

Item A

(6x)

10-32 x 5/8"

F (6x) G (16x) H (16x)

10-32 Retainer Nut

Figure 3. Parts in the 14U rack mount kit

| |

Unpacking the SAN768B

| |

To unpack the SAN768B, follow these steps:

1. Cut the bands that encircle the packaging.

2. Remove the lid, the 14U rack mount kit, the accessory kit, and the packing foam from the top of the chassis. Set the kits aside for later use.

3. Lift the cardboard box off the chassis and remove the plastic bag from around the chassis. Save the packing materials if you are returning an old chassis.

4. Leave the chassis on top of the plastic shipping tray if the chassis must be transported to the installation location.

D (6x)

Clip Nut

1/4-20 x 1/2" Alignment Washer

E (16x)

1/4-20 x 1/2"

B768ig027

The standard packaging does not incorporate a wood pallet and pallet

| |

| | |

Note:

brackets. The chassis sits on top of a plastic shipping tray.

5. Use the lift tool to transport the new chassis to the installation area. Doorways must be wider than 91 cm (36 in.) to accommodate the chassis on the shipping tray.

Chapter 2. Installing and removing a SAN768B 15

Page 44

Attaching the shelf brackets

| | |

| | | |

Complete the following steps to install the shelf brackets to the cabinet rails:

1. Find the left and right rack mount shelf brackets (A and B in Figure 4) in the 14U rack mount kit. These brackets are adjustable to allow for different cabinet depths.

2. Loosen the adjusting screws on both brackets enough to allow the brackets to slide easily to adjust the length.

A le

(4x)

Setup for rails with square holes

Rail

Adjustable ends

Shown with round hole rails

Figure 4. Left and right shelf brackets installed on the cabinet rails

| |

| | |

3. Position the shelf brackets (A and B) with the adjustable ends at the intake aisle side of the cabinet (see Figure 4) and attach them to the cabinet rails.

v For rails with round holes: Use eight screws with lock washers (E) for each

bracket, four at each end. Tighten the screws to a torque of 92 cm-kg (80 in.-lb).

v For rails with square holes: Use eight screws (G) eight alignment washers

(H) for each bracket, four at each end. Tighten the screws to a torque of 92 cm-kg (80 in.-lb).

(4x)

B768ig028

Standard EIA rails have holes in sets of three; spaces between the holes

| | | |

| |

Note:

are 1.58 cm, 1.58 cm, 1.27 cm (5/8 in., 5/8 in., and 1/2 in.). If cables are to be routed down through the cable management comb, allow at least 1U space below the brackets for cable management.

4. Tighten the adjusting screws on the shelf brackets to a torque of 37 cm-kg (32 in.-lb).

16 SAN768B Installation, Service, and User’s Guide

Page 45

| | | | | |

| | | | |

5. Attach the clip nuts D (for rails with round holes) or retainer nuts F (for rails with square holes) to the vertical rails on the exhaust aisle side of the cabinet (see Figure 5). These nuts are used for securing the port side of the chassis to the rack rails using 10-32 x 5/8-inch screws. Use three nuts on each rail. Place the nuts in optimum locations for securing the chassis to the rails. See Figure 6 on page 19 for approximate positions.

Do not align the clip or retainer nuts with the top or bottom holes of the

Note:

SAN768B mounting brackets because the door will interfere with the screw heads.

Attached clip nuts

(item D)

Setup for rails with square holes

Retainer nut (item F)

Rail

Top screws in shelf bracket

Hole 1

Bottom screws in shelf bracket

1.58 cm (5/8 in.)

1.27 cm (½ in.)

1.58 cm (5/8 in.)

Shown with clip nuts for round-hole rails

B768ig029

Figure 5. Installing clip or retainer nuts on the vertical cabinet rails

| |

Removing the chassis door

| | | | |

The chassis door must be removed before the chassis can be installed into the cabinet. The door is not hinged. It is held to the chassis by mounting hardware on the door that snaps onto eight ball studs on the chassis. To remove the door, grip both sides of the door, and gently pull the door straight out until it releases from the ball studs on the chassis.

Using the lift tool, the 24–inch load plate, and the bridge tool

Step 1 on page 18 through step 9 on page 19 describe how to use the lift tool (PN 09P2481) and the 24–inch load plate (PN 11P4369) to install a SAN768B in the cabinet. Step 10 on page 19 through step 21 on page 21 describe how to install the bridge tool (PN 18P5855) in the cabinet for the lower SAN768B or the upper

Chapter 2. Installing and removing a SAN768B 17

Page 46

SAN768B. This procedure uses parts from the 14U chassis mount kit. See “Items required” on page 84 for a list of these parts.

CAUTION: A fully populated SAN768B weighs approximately 105 kg (231 lb) and requires a minimum of two people and a lift tool to install it. (C011)

55 kg ( 121.2 lbs)

Attention: Before you install it, verify that the additional weight of the chassis

svc00169

does not exceed the cabinet’s weight limits or unbalance the cabinet. When you calculate the additional weight, include the cards or power supplies that partially extend out of the chassis.

P/N 18P5850-B

CAUTION: A pinch point exists between the load plate and the bridge tool.

SJ000752

Perform the following steps to install the SAN768B:

1. Assemble the lift tool if it is not assembled. The assembly and disassembly instructions are included with the lift tool.

2. Attach the 24–inch load plate to the lift tool.

| | |

3. Although not required, you can remove the power supplies and blower assemblies from the SAN768B to reduce the weight. This can reduce the weight by up to 27 kg (60 lb).

4. Move the lift tool next to the pallet that contains the SAN768B. a. Adjust the load platform to the same height. b. Slide the SAN768B from the pallet onto the load platform. c. Center the switch on the load platform.

5. Attach the retention straps around the drawer.

6. If the load platform is not at its lowest position, lower the load platform to its lowest position.

7. Move the lift tool to a position near the rear of the cabinet. See Figure 6 on page 19.

18 SAN768B Installation, Service, and User’s Guide

Page 47

Rack cabinet (shown without sheet metal)

Air intake side of chassis

Direction of installation

B768ig030

| |

Figure 6. Chassis installation from rear (exhaust aisle side) of the cabinet

Note: If you are passing through a narrow passage or door, it might be

necessary to rotate the drawer on the load platform.

8. Set the wheel brake by pressing down on the wheel brake pedal.

9. Locate the anti-tips bars. If they are not attached, attach them to the lift tool by performing step 9a through step 9d. If the anti-tip bars were previously attached, go to step 10.

a. Pull up on the leg lock pins and insert the anti-tip bars inside the legs. b. Pull on the legs and anti-tip bars to ensure that they are locked into

position. c. Raise the platform to allow enough room to attach the anti-tip bracket. d. Secure the anti-tip bars by using the anti-tip bracket and bolt.

If you are installing the lower SAN768B, go to step 11. If you are installing the

10. upper SAN768B, go to step 12 on page 20.

11. For the lower SAN768B, install the bridge tool by completing the following

steps: a. Remove the bridge tool from the box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 3, and

move the two supports another 45°. Make sure that the following conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

c. Align the two-pin section of the support on the back of the vertical rail with

the bottom and middle holes in EIA unit 3.

Chapter 2. Installing and removing a SAN768B 19

Page 48

d. Align the one-pin section of the support on the front of the vertical rail with

the middle hole in EIA unit 1.

Tilt the bridge tool towards the back of the cabinet approximately

Note:

45° to make it easier to align the two-pin and the one-pin sections with the vertical support.

e. Lock the shelf into position. See Figure 7.

The shelf in the picture below is shown with a different director.

Note:

Figure 7. Locking the shelf into position

12. For the upper SAN768B, install the bridge tool by completing the following steps:

a. Remove the bridge tool from the box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 20,

and move the two supports another 45°. Make sure that the following conditions are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section is on the front of the vertical support.

Tilt the bridge tool towards the back of the cabinet approximately

Note:

45° to make it easier to align the two-pin and one-pin sections with the vertical support.

c. Align the two-pin section of the support on the front of the vertical rail with

the bottom hole and the middle hole in EIA unit 20.

d. Align the one-pin section of the support on the front of the vertical rail with

the middle hole in EIA unit 18.

Turn the winch crank counterclockwise, 1/4 turn, to set the winch brake.

13.

20 SAN768B Installation, Service, and User’s Guide

SJ000680

Page 49

14. Set the wheel brake and place the wheel chocks around the wheels.

15. Remove the retention straps.

16. Slide the SAN768B into the cabinet until it rests securely on the shelf brackets.

17. Remove the wheel chocks, release the wheel brake, and move the lift tool away from the cabinet.

18. Uninstall the bridge tool and return it to its original box.

| | | | |

19. Reinstall the power supplies and blower assemblies if any were removed.

20. Fasten the port side of the chassis to the cabinet rails. Use three 10–32 x 5/8-in. Phillips panhead screws with washers C on each side to attach to the clip nuts (D) or retainer nuts (F) installed on the rails earlier. Tighten the screws to a torque of 37 cm-kg (32 in-lb). See Figure 8 for the approximate location of the screws.

| | | |

Do not use the top or bottom holes of the SAN768B mounting brackets

Note:

because the screw heads will interfere with the chassis door.

B768ig031

| |

Figure 8. Attaching the port side of the chassis to the cabinet rails

| | | |

21. Reinstall the chassis door. Align the door with the chassis and push it into place, as shown in Figure 9 on page 22 and Figure 10 on page 22.

Chapter 2. Installing and removing a SAN768B 21

Page 50

Figure 9. Aligning the chassis door with the ball studs on the chassis

| |

Inside view of door

Ball stud on chassis

Mounting hardware

Ball stud on chassis

B768ig032

Outside view of door

B768ig033

Figure 10. Details of the chassis and door, showing the ball studs on the chassis and the mounting hardware on the

door

Note: For turning on the power and for the configuration instructions, see

Chapter 3, “Starting and configuring the SAN768B,” on page 25.

22 SAN768B Installation, Service, and User’s Guide

Page 51

Attention: Do not connect the SAN768B to the network until the IP addresses are

correctly set.

Removing a SAN768B from the cabinet

Step 1 through step 5 describes how to use the lift tool (PN 09P2481) and the 24–inch load plate (PN 11P4369) to remove a SAN768B from the cabinet. Step 6 through step 17 on page 24 describes how to remove the bridge tool (PN 18P5855) from the cabinet.

Perform the following steps to remove a SAN768B:

1. Assemble the lift tool if it is not assembled. The assembly and disassembly instructions are included with the lift tool.

2. Remove both power cords from the SAN768B.

| |

3. Optionally, remove the power supplies and blower assemblies to reduce the weight.

4. Remove the chassis door from the chassis. See “Removing the chassis door” on page 17.

5. Remove the three 10-32 x 5/8 in. Phillips panhead screws with washers on each side. See Figure 8 on page 21 for the location of the screws.

6. If you are replacing the lower SAN768B, go to step 7. If you are replacing the upper SAN768B, go to step 8.

7. Install the bridge tool by completing the following steps: a. Remove the bridge tool from its box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 3. c. Move the two supports another 45°. Make sure that the following conditions

are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

d. Align the two-pin section of the support on the back of the vertical rail with

the bottom and middle holes in EIA unit 3.

e. Align the one-pin section of the support on the front of the vertical rail with

the middle hole in EIA unit 1.

Note: Tilt the bridge tool towards the back of the cabinet approximately

45° to make it easier to align the two-pin and one-pin sections with the vertical support.

f. Lock the shelf into position.

8. For the upper SAN768B, install the bridge tool by completing the following

steps: a. Remove the bridge tool from its box and adjust the two supports 45°. b. Hold the bridge tool between the two vertical supports near EIA unit 20. c. Move the two supports another 45°. Make sure that the following conditions

are met:

v The two-pin section of the support is on the back of the vertical support. v The one-pin section of the support is on the front of the vertical support.

d. Align the two-pin section of the support on the back of the vertical rail with

the bottom and middle holes in EIA unit 20.

e. Align the one-pin section of the support on the front of the vertical rail with

the middle hole in EIA unit 18.

Chapter 2. Installing and removing a SAN768B 23

Page 52

f. Lock the shelf into position.

9. Locate the anti-tip bars and if they are not attached, attach them to the lift tool.

If anti-tip bars were previously attached, go to step 10. a. Pull up on the leg-lock pins and insert the anti-tip bars inside the legs. b. Pull on the legs and anti-tip bars to ensure that they are locked into

position. c. Raise the platform to allow enough room to attach the anti-tip bracket. d. Secure the anti-tip bars with the anti-tip bracket and bolt.

10. Move the lift tool to a position near the rear of the cabinet.

11. Turn the winch crank clockwise to raise the switch. Move the lift tool into position in front of the cabinet while you align the 24-inch load plate with the line on the bridge tool.

12. Turn the winch crank counterclockwise 1/4 turn to set the winch brake.

13. Set the wheel brake. Place the wheel chocks around the wheels.

55 kg ( 121.2 lbs)

CAUTION: A fully populated SAN768B weighs approximately 104 kg (228 lb) and requires a minimum of two people and a lift tool to install it. (C011)

svc00169

CAUTION: A pinch point exists between the load plate and the bridge tool.

P/N 18P5850-B

SJ000752

14. Slide the SAN768B from the cabinet onto the 24–inch load plate. Center the

SAN768B on the platform.

15. Install the retention straps.

16. Remove the wheel chocks, release the wheel brake, and move the lift tool

away from the cabinet.

17. Lower the load platform to its lowest position.

18. Remove the bridge tool.

24 SAN768B Installation, Service, and User’s Guide

Page 53

Chapter 3. Starting and configuring the SAN768B

Attention: Refer to the latest Fabric OS Release Notes or Interoperability Matrix

(see http://www.ibm.com/servers/storage/support/san) for information regarding Fabric OS compatibility. This is particularly important for fabrics with 1-Gbps switches using earlier Fabric OS versions.

The SAN768B must be configured before it is connected to the fabric, and all of the configuration commands must be entered through the active CP blade. The SAN768B configuration includes the following parameters:

v IP address and subnet mask for the SAN768B v IP addresses (or ″native″ IP address), host names, subnet masks, and gateway

addresses for both CP blades

v Switch name v Domain ID for the SAN768B (optional) v WWN for the SAN768B

Note: The SAN768B WWN is initially set by the factory to match the license ID

(which is based on the chassis serial number). The WWN can be changed, but the license ID cannot be modified.

configuration information is mirrored to the standby CP blade, which allows the

The most current configuration to remain available even if the active CP blade fails. The configuration information for the SAN768B is stored in the WWN blade and the flash memory of the CP blades. The configuration can be backed up to a workstation (uploaded) and then downloaded to the active CP blade if necessary.

The initial physical hardware installation continues with the following steps:

v “Providing power to the SAN768B” on page 26 v “Establishing a serial connection and logging on to the SAN768B” on page 26 v “Configuring the SAN768B IP addresses” on page 27

following additional steps are customer responsibilities required for the initial

The configuration:

1. Establish an Ethernet connection. See “Establish an Ethernet connection” on page 29.

2. Specify a switch name. See “Customize a switch name” on page 30.

3. Specify a domain ID for the SAN768B. See “Set the Domain ID” on page 30.

4. Upgrade the Fabric OS (optional). See “Software features” on page 7.

5. Verify the PID mode and connect the SAN768B to the fabric. See “Verifying the PID mode and connecting to the fabric” on page 30.

6. Enable software licenses. See “Enabling software licenses” on page 31.

7. Back up the configuration. See “Backing up the configuration” on page 32.

Page 54

Providing power to the SAN768B

DANGER

the supplied power cords. Ensure the facility power receptacle is the

Use correct type, supplies the required voltage, and is properly grounded. (D004)

To provide power to the SAN768B:

1. Connect the AC power cords to the power supply assemblies. One to four power cords are required depending on electrical service.

2. Connect the power cords to a power source with voltage of 200 to 240 VAC, 47 to 63 Hz (normally two power cords).

3. Turn the AC power switches on the power supplies to ON (I). The AC power switches light green when switched on and power is supplied.

4. The SAN768B performs a power-on self-test (POST) each time it is powered on. POST takes approximately 10 minutes and is complete when indicator light activity indicates the operational state. For information about LED patterns, see Chapter 4, “Monitoring system components,” on page 33.

You can bypass POST by using the fastBoot command. You can also disable POST for successive reboots on the SAN768B using the diagDisablePost command.

Attention: To prevent a potential IP address conflict, do not connect the

SAN768B to the network until the IP addresses are configured.

To power off the SAN768B, see “Powering off the SAN768B” on page

Note:

109.

Establishing a serial connection and logging on to the SAN768B

Note: The preferred method of establishing a connection to the SAN768B is via the

serial port. However, if a serial connection is not possible (for example, if a serial cable is not available), then an Ethernet connection can be established using an Ethernet crossover cable. To establish an Ethernet connection, you must know the IP address of the control processor.

To establish a serial connection and log on to the SAN768B:

1. Verify that the SAN768B is powered on and that POST is complete by verifying that all power LED indicators on the port, control processor, and core switch blades display a steady green light.

2. Remove the shipping cap from the SERVICE port on the active CP. The active CP blade is indicated by an illuminated (blue) LED. Use the serial cable provided with the SAN768B to connect the SERVICE port on the active CP to a computer workstation.

Attention: The SERVICE port is intended primarily for the initial setting of the

IP address and for service purposes.

3. Access the SAN768B using a terminal emulator application (such as HyperTerminal in a Windows environment or TERM in a UNIX environment).

4. Disable any serial communication programs running on the workstation (such as synchronization programs).

5. Open the terminal emulator application and configure as follows:

26 SAN768B Installation, Service, and User’s Guide

Page 55

v For most Microsoft® Windows® systems, use the configuration parameters

shown in Table 6.

Table 6. Configuration parameters

Parameter Value

Bits per second 9600 Data bits 8 Parity None Stop bits 1 Flow control None

v For most UNIX® systems, type the string at the prompt:

tip /dev/ttyb -9600

the terminal emulator application stops reporting information, press Enter.

When You will receive the following login prompt:

CP0 Console Login:

6. Log in to the SAN768B as admin. The default password is “password.” At the initial login, the user is prompted to enter new admin and user passwords. Make sure to write down the new passwords and keep this information in a secure location.

Table 7. Example login without changing the password

Fabric OS (swDir)

swDir login: admin Password: Please change your passwords now. Use Control-C to exit or press ’Enter’ key to proceed.

Password was not changed. Will prompt again at next login until password is changed. swDir:admin>

7. (Optional) Modify passwords. Passwords can be 8 to 40 characters long. They must begin with an alphabetic character. They can include numeric characters, the dot (.), and the underscore (_). Passwords are case-sensitive, and they are not displayed when you enter them on the command line. To skip modifying the password, press Ctrl-C. For more information on passwords, refer to the Fabric OS Administrator’s Guide.

Configuring the SAN768B IP addresses

The SAN768B requires three IP addresses; one for the single logical switch in the SAN768B chassis (shown as SWITCH in the ipAddrShow command), and one for each of the CP blades (CP0 and CP1). These IP addresses are set using the

ipAddrSet command through the serial or Ethernet connection to the active CP

blade.

Notes:

1. The default IP addresses and host names for the SAN768B are as follows:

v 10.77.77.75 for CP0 (the CP blade in slot 6 at the time of configuration) v 10.77.77.74 for CP1 (the CP blade in slot 7 at the time of configuration)

Chapter 3. Starting and configuring the SAN768B 27

Page 56

2. The default password is "password."

Attention: Resetting an IP address while the SAN768B has active IP traffic such

as Fabric Manager, Fabric Watch, SNMP, or other applications can cause traffic to be interrupted or stopped. Changing the IP address causes a domain address format RSCN to be issued.

Follow these steps to configure the IP addresses for both CP blades (from the active CP blade):

1. Log in to the active CP as admin using the serial or Ethernet connection.

2. Set up the SAN768B IP address by entering the ipaddrset -sw 0 command at the prompt.

swDir:admin> ipAddrSet -sw 0

Enter the requested information at the prompts. Specify the -sw 0 IP address. There is no -sw 1 IP address.

The addresses 10.0.0.0 through 10.0.0.255 are reserved and used

Note:

internally by the SAN768B. External IPs cannot use these addresses either.

3. Set up the CP0 blade IP address by entering the ipaddrset -cp 0 command at the prompt.

swDir:admin> ipAddrSet -cp 0

Enter the requested information at the prompts.

4. Set up the CP1 blade IP address by entering the ipaddrset -cp 1 command at the prompt.

swDir:admin> ipAddrSet -cp 1

Enter the requested information at the prompts. The following is a sample IP configuration for the SAN768B (sw -0) and the two

CP blades (cp0 and cp1).

28 SAN768B Installation, Service, and User’s Guide

Page 57

swDir:admin> ipaddrset -sw 0 Ethernet IP Address [0.0.0.0]: 123.123.123.120 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Fibre Channel IP Address [0.0.0.0]: Fibre Channel Subnetmask [0.0.0.0]: Issuing gratuitous ARP...Done. Committing configuration...Done. swDir:admin> ipaddrset -cp 0 Host Name [cp0]: Ethernet IP Address [10.77.77.75]: 123.123.123.121 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Gateway IP Address [0.0.0.0]: 123.123.123.124 IP address is being changed...Done. Committing configuration...Done. swDir:admin> ipaddrset -cp 1 Host Name [cp1]: Ethernet IP Address [10.77.77.74]: 123.123.123.122 Ethernet Subnetmask [0.0.0.0]: 123.123.123.123 Gateway IP Address [0.0.0.0]: 123.123.123.124 IP address of remote CP is being changed...Done. Committing configuration...Done. swDir:admin> reboot

5. Reboot the SAN768B by typing reboot at the prompt.

The serial port can be used to monitor error messages through a serial connection. It is not recommended as a command interface during normal operations. If this port is not going to be in ongoing use, remove the serial cable and protect the port from dust by replacing the shipping cap.

Establish an Ethernet connection

After using a serial connection to configure the IP addresses for the SAN768B, you can connect the active CP blade to the local area network (LAN) if desired.

Note: Connecting the CP blades to a private network/VLAN is recommended.

By establishing an Ethernet connection, you can complete the SAN768B configuration using either the serial session or a Telnet session, or through the graphical management interfaces: Web Tools and Fabric Manager. However, you must ensure that the SAN768B is not modified from other connections at the same time.

To establish an Ethernet connection to the SAN768B, follow these steps:

1. Remove the shipping plug from the Ethernet port on the active CP blade.

2. Insert one end of an Ethernet cable into the Ethernet port.

3. Connect the other end to an Ethernet 10/100 Base-T LAN. The SAN768B can now be accessed by remote connection using any of the

available management tools, such as Telnet, Web Tools, or Fabric Manager.

4. To complete any additional SAN768B configuration procedures through a Telnet session, log in to the SAN768B by Telnet, using the admin login. The default password is password.

Chapter 3. Starting and configuring the SAN768B 29

Page 58

Customize a switch name

The switch name of the SAN768B can be up to 15 characters long; can include alpha, numeric, and underscore characters; and must begin with an alpha character.

Changing the name causes a domain address format RSCN to be issued.

Note:

To customize the name, follow these steps:

1. Enter the switchName command with the new name in quotes.

2. Record the new name for future reference.

Set the Domain ID

Each switch in the fabric must have a unique Domain ID. The Domain ID can be set using the configure command. You can also allow the Domain ID to be automatically set. The default Domain ID for the SAN768B is ″1″.

To set the Domain ID, follow these steps:

1. Enter the fabricshow command to determine the already assigned Domain IDs.

2. Enter the switchdisable command to disable the SAN768B.

3. Enter the configure command.

4. Enter y at the ″Fabric parameters″ prompt:

5. Enter a unique Domain ID:

6. Complete the remaining prompts or press Ctrl+D to accept the other settings

7. Enter the switchEnable command to reenable the SAN768B.

swDir:admin> switchName "swDirector5" Committing configuration... Done. swDirector5:admin

Fabric parameters (yes, y, no, n): [no] y

Domain: (1..239) [1] 3

and to exit.

Verifying the PID mode and connecting to the fabric

Before connecting the SAN768B to the fabric, verify that the port identifier (PID) mode on the SAN768B matches the other switches in the fabric. This parameter must be set identically on all switches in the fabric. This parameter is set using the

configure command.

For information on PID mode, refer to the Fabric OS Administrator’s Guide. Before starting, also refer to “Managing cables” on page 9 for more information on how to organize and route cables to minimize potential problems with large numbers of fiber optic cables.

To connect the SAN768B to the fabric, follow these steps:

1. Add SFP transceivers and cables to the fibre channel ports, as required. The ports are color-coded to indicate which ones can be used in the same port

group for trunking. Trunking port groups can be up to eight ports. Trunk groups

30 SAN768B Installation, Service, and User’s Guide

Page 59

are displayed by shaded or nonshaded ports. The ports and cables used in trunking groups must meet specific requirements.

For a list of these requirements, refer to the Fabric OS Administrator’s Guide.

2. Position one of the SFP transceivers so that the key is oriented correctly to the port. Insert the transceiver into the port until it is firmly seated and the latching mechanism clicks.

Transceivers are keyed so that they can only be inserted with the correct orientation. If a transceiver does not slide in easily, ensure that it is correctly oriented. For instructions that are specific to the transceiver model, refer to the transceiver manufacturer’s documentation.

3. Position a cable so that the key (the ridge on one side of the cable connector) is aligned with the slot in the transceiver; then, insert the cable into the transceiver until the latching mechanism clicks.

Cables are keyed so that they can only be inserted with the correct orientation. If a cable does not slide in easily, ensure that it is correctly oriented.

4. Repeat step 1 on page 30 through step 3 for the remaining ports.

5. Organize the cables as required. For recommendations regarding cable management, refer to “Managing cables” on page 9.

6. Verify the SAN768B for switch and port status by entering the switchshow command.

The switchshow command provides detailed information about the SAN768B.

7. Verify fabric connectivity by entering the fabricshow command. The fabricshow command provides general information about the fabric.

Enabling software licenses

To determine which licenses are currently enabled, enter the licenseShow command.

Table 8. Example of viewing enabled licenses

swDir:admin> licenseshow AAbbccDDeeFFeeGG:

Web license Zoning license Extended Fabric license Fabric Watch license Performance Monitor license Trunking license Security license

In the example above, the license key is AAbbccDDeeFFeeGG. Make sure you keep a hardcopy of your license key in a secure location for emergency reference.

The 64-bit chassis ID is required to obtain and activate licenses for the SAN768B. The chassis ID is available through the licenseIdShow command.

Both the licenseShow and licenseIdShow commands must be entered through the active CP blade.

For information about activating licenses, refer to the Fabric OS Administrator’s Guide.

Chapter 3. Starting and configuring the SAN768B 31

Page 60

Backing up the configuration

Save all key configuration data for the SAN768B, including license key information, and upload it to a host for emergency reference. Routine backups of the configuration are recommended to ensure the current configuration is available if needed.

After all zoning configurations and other changes are complete, back up the configuration by entering the configUpload command at the prompt.

The configUpload command uploads the SAN768B configuration to the server so that it is available for downloading to a replacement SAN768B, if necessary.

An easy way to backup configuration information is to enable logging on your Telnet session and then running the following commands and saving the output in a file on a secure host. Once you enable logging, run the following commands:

v configShow v ipAddrShow v licenseShow v switchShow

hardcopy versions of key data such as passwords, license keys, and IP

Keep addresses. This is sensitive information and should be kept in a secure location.

Note: Passwords are not saved in the configuration file, and are not uploaded

during a configUpload. Password information should be saved in hardcopy in a secure location for emergency reference, subject to your security and password management policy.

32 SAN768B Installation, Service, and User’s Guide

Page 61

Chapter 4. Monitoring system components

The SAN768B is engineered for reliability and requires no routine operational steps or maintenance. This chapter provides information about determining the status of the following components using LEDs and CLI commands.

v “Control processor blade (CP8)” v “Core switch blade (CR8)” on page 35 v “FR4-18i router blade” on page 37 v “Port or application blades” on page 40 v “Power supply” on page 45 v “Blower assembly” on page 46 v “WWN bezel and WWN card” on page 48

Refer to the Web Tools Administrator’s Guide and the Fabric OS Administrator’s Guide for additional information.

Control processor blade (CP8)

To determine the status of a CP blade:

1. Check the LED indicators on the CP blade (see Figure 11 on page 34). The LED patterns may temporarily change during POST and other diagnostic tests. For information about how to interpret the LED patterns, see Table 9 on page

34.

2. Check port blade status by typing slotShow and haShow.

11 on page 34 identifies the control processor blade (CP8).

Figure

Page 62

Figure 11 . Control processor blade (CP8) LEDs

1 Status LED 5 Console port (10101) 2 Power LED 6 Ethernet port (MGMT) 3 USB LED 7 Ethernet port (SERVICE) 4 USB port 8 Active CP LED

Table 9. CP blade LED descriptions

LED purpose Color Status

Power Steady green CP blade has valid

34 SAN768B Installation, Service, and User’s Guide

power.

No light (LED is off) CP blade does not

have incoming power.

Recommended Action

No action required.

Ensure blade is firmly seated and has power.

Page 63

Table 9. CP blade LED descriptions (continued)

Recommended

LED purpose Color Status

Status No light (LED is off) CP blade is either

healthy or does not

Action

Verify that the power LED is on.

have power.

Steady amber CP blade is faulty or

the switch is still booting.

Ensure blade is firmly seated and switch has completed booting. If LED remains yellow, contact IBM.

Slow-flashing amber (on 2 seconds; then off 2 seconds)

CP blade is not seated correctly or is faulty.

Pull blade out and reseat it. If LED continues to flash, replace blade.

Fast-flashing amber (on 1/2 second; then off 1/2 second)

Environmental range exceeded.

Check for out-of-bounds environmental condition and correct it.

Ethernet Link Status No light (LED is off) Either an Ethernet

link is not detected, or it does not have incoming power.

Ensure the blade has power, Ethernet cable is firmly seated, and connected device is functioning.

Flickering green/amber

Ethernet link is healthy and traffic is

No action required.

flowing through port.

Ethernet Link Speed No light (LED is off) Ethernet link speed is

10 Mb/s or CP blade does not have incoming power.

Ensure CP has power.

Note: To force a

persistent Ethernet link speed, type the

ifModeSet command.

Steady green Ethernet link speed is

No action required.

100/1000 Mb/s.

Active CP Steady blue Active CP blade. No action required.

No light (LED is off) Standby CP blade. No action required.

Core switch blade (CR8)

To determine the status of a CR blade:

1. Check the LED indicators on the core switch blade (Figure 12 on page 36). The LED patterns may temporarily change during POST and other diagnostic tests; for information about how to interpret the LED patterns, see Table 10 on page

36.

2. Check core switch blade status by typing slotShow and haShow.

12 on page 36 identifies the core switch blade.

Figure

Chapter 4. Monitoring system components 35

Page 64

Figure 12. Core switch blade (CR8) LEDs

1 Status LED 4 ATTN LED 2 Power LED 5 ICL Connector 3 LINK LED

Table 10. CR blade LED descriptions

LED purpose Color Status

Power Steady green CP blade has valid

36 SAN768B Installation, Service, and User’s Guide

power.

No light (LED is off) CP blade does not

have incoming power.

Recommended Action

No action required.

Ensure blade is firmly seated and has power.

Page 65

Table 10. CR blade LED descriptions (continued)

Recommended

LED purpose Color Status

Status No light (LED is off) CP blade is either

healthy or does not

Action

Verify that the power LED is on.

have power.

Steady amber CP blade is faulty or

the switch is still booting.

Ensure blade is firmly seated and switch has completed booting. If LED remains yellow, contact IBM..

Slow-flashing amber (on 2 seconds; then off 2 seconds)

CP blade is not seated correctly or is faulty.

Pull blade out and reset it. If LED continues to flash, replace blade.

Fast-flashing amber (on 1/2 second; then off 1/2 second)

Environmental range exceeded.

Check for out-of-bounds environmental condition and correct it.

LINK LED No light (LED is off) No ICL or ICL cable

present, but no

Ensure ICL cable is correctly installed.

connection.

Steady green ICL connected No action required.

ATTN LED No light (LED is off) No ICL No action required.

Blinking amber Attention required Ensure ICL cable is

correctly installed.

FR4-18i router blade

Power on the router blade by setting the slider switch at the top of the blade to the On position (see Figure 13 on page 38). If the router blade is operational, powering off the blade is disruptive to traffic across the blade.

Attention: All ports on the router blades are by default, persistently disabled to

avoid unintentional merging of fabrics. Before you can successfully configure FCIP tunnels, you must enable the ports. When the blade is first powered up, all port status LEDs will continue to flash amber. This is normal. Use the portCfgShow command to view the status of the ports. Use the portCfgPersistentEnable command to enable a port. Refer to the Brocade Fabric OS Administrator’s Guide and the Brocade Fabric OS Command Reference Manual for more information on these commands.

Perform the following to determine the status of a FR4-18i router blade:

1. The FR4-18i router blade must be installed in a SAN768B in order to be powered on.

2. Power is supplied to the blade once it is installed in the SAN768B and the power switch on the blade is moved to the On position. The router blade runs POST by default each time it is powered on. It typically requires from 1 to 3 minutes to boot and complete POST.

3. Log in to the SAN768B as admin and run the slotShow command. The status of the FR4-18i blade is displayed in the output.

Chapter 4. Monitoring system components 37

Page 66

4. Run the errShow command to determine if any errors detected during POST are written to the system log. For information about this command, see the Fabric OS Command Reference Manual. For information about specific error messages, see the Fabric OS System Error Message Reference Manual.

5. Check the LED indicators on the power supply (see Figure 13.) The LED patterns may temporarily change during POST and other diagnostic tests; for information about how to interpret the LED patterns and verify that all components are healthy, refer to Table 11 on page 39.

6. If any of the LEDs appear to indicate a problem, verify that the LEDs are not set to "beacon" (this can be determined through the switchShow command or Web Tools). For information about how to turn beaconing on and off, see the Brocade

Fabric OS Administrator’s Guide or the Brocade Web Tools Administrator’s Guide. Follow the recommended action for the observed LED behavior, as listed

in Table 11 on page 39.

scale:

5/16" = 1"

scale:

15/32" = 1"

scale:

15/32" = 1"

Figure 13. FR4-18i router blade

1 FR4-18i router blade 7 Fiber Channel port status LED

38 SAN768B Installation, Service, and User’s Guide

scale:

5/16" = 1"

FR4

18 i

FR4

18 i

Page 67

2 On/Off switch (in On position) 8 GbE ports (x2) 3 On/Off switch (in Off position) 9 Thumb screw 4 Power LED 10 Upper ejector 5 Status LED 11 Lower ejector 6 Fibre Channel port (16) 12 GbE Port Status LED

Table 11 describes the FR4-18i router blade LED patterns. It lists the name, reference number, and possible colors of each LED. It also lists the status of the component that is associated with each LED color and action that you can take in response to that status.

Table 11. FR4-18i router blade LED patterns during normal operation

LED name, location Color of LED Status of hardware Action

Power LED

4

Status LED

5

No light Blade does not have

incoming power.

Perform the following steps:

v Verify that the SAN768B is operational. v Verify that the router blade is seated

correctly.

v Verify that the power slider switch is in the

up position.

v Verify that all power supplies are operational.

the unit still does not power up, the unit may

be faulty. Contact IBM. Steady green Blade has valid power. No action required No light Blade is either functioning

Verify that the power status LED is green

correctly or does not have power.

Steady amber (for more than five seconds)

Boot failed, the blade is faulty. The blade displays a steady amber during boot or POST, this is normal and does not indicate a fault.

Perform the following steps:

v Reboot the blade.

v Check the error logs for information on the

failure.

v Contact IBM, if necessary.

Slow-flashing amber (on 2 seconds; then off 2 seconds)

Fast-flashing amber (on 1/2 second; then off 1/2 second)

Attention is required. The Blade is not seated correctly or is faulty.

Attention is required. Environmental range exceeded.

Perform the following steps:

v Check the error log for details on the cause

of the status.

v Pull the blade out and reseat it.

the LED continues to flash, contact IBM.

Perform the following steps:

v Check the error log for details on the cause

of the status.

v Check for out-of-bounds environmental

condition and correct it.

the LED continues to flash, contact IBM.

Chapter 4. Monitoring system components 39

Page 68

Table 11. FR4-18i router blade LED patterns during normal operation (continued)

LED name, location Color of LED Status of hardware Action

Fiber Status Channel Port Status 7 GbE Port Status LEDs 12

No light Indicates one of the

following:

v No signal or light carrier

(media or cable) detected.

v Blade may be currently

initializing.

Perform the following steps:

v Verify the power LED is on, and check the

SFP and cable.

v Verify the blade is not currently being

initialized.

v Verify the status of the connected device.

v Connected device is

configured in an offline state.

Steady green Port is online (connected to

No action required. external device) but has no traffic.

Slow-flashing green (on 1 second; then off 1 second)

Fast-flashing green (on 1/4 second; then off

Port is online but segmented because of a loopback cable or incompatible switch connection.

Port is online and an internal loopback diagnostic test is running.

Verify that the correct device is attached to the

blade.

No action required.

1/4 second) Flickering green Port is online and frames are

No action required. flowing through the port.

Steady amber Port is receiving light or

No action required. signal carrier, but it is not online yet.

Slow-flashing amber (on 2 seconds; then off 2 seconds)

Port is disabled because of diagnostics or the

portDisable or portCfgPersistentDisable

Reset the port.

Note: The portCfgPersistentDisable

command is persistent across reboots in the

Fabric OS v6.0.0. commands.

Fast-flashing amber (on 1/2 second; then off 1/2 second)

SFP or port is faulty. Perform the following steps:

v Replace the SFP

v Reset the port.

Port or application blades

To determine the status of a port or application blade:

1. Check the LEDs on the blade.

v Figure 14 on page 41 illustrates the FC8-16 port blade. v Figure 15 on page 42 illustrates the FC8-32 port blade. v Figure 16 on page 43 illustrates the FC8-48 port blade. v Figure 17 on page 44 illustrates the FC10-6 port blade. v Figure 13 on page 38 illustrates the FR4-18i routing blade.

The LED patterns may temporarily change during POST and other diagnostic

2. tests. For information about how to interpret the LED patterns, see Table 12 on page 45.

3. Check the blade status by typing slotShow.

40 SAN768B Installation, Service, and User’s Guide

Page 69

Table 12 on page 45 describes the port and routing blade LED patterns and the recommended actions for those patterns.

1 Power LED 3 Fibre Channel port 2 Status LED 4 Port Status LED

Figure 14. FC8-16 port blade

Chapter 4. Monitoring system components 41

Page 70

1 Power LED 3 Fibre Channel port 2 Status LED 4 Port Status LED

Figure 15. FC8-32 port blade

42 SAN768B Installation, Service, and User’s Guide

Page 71

1 Power LED 3 Fibre Channel port 2 Status LED 4 Port Status LED

Figure 16. FC8-48 port blade

Chapter 4. Monitoring system components 43

Page 72

1 FC10-6 Port Blade 5 Ejector 2 On/Off Switch 6 Power LED 3 Fibre Channel Ports 7 Blade Status LED 4 Thumb Screw 8 Port Status LED

Figure 17. FC10-6 port blade

44 SAN768B Installation, Service, and User’s Guide

Page 73

Table 12. Port and application blade LED descriptions

LED purpose Color Status Recommended Action

Power LED Steady green Blade has been enabled. No action required.

No light (LED is off) Blade has not been enabled. Ensure blade is firmly seated.

Status LED No light (LED is off) Blade is either healthy or does

Steady amber Blade is faulty. Ensure blade is firmly seated

Slow-flashing amber (on 2 seconds, then off 2 seconds)

Fast-flashing amber (on 1/2 second, then off 1/2 second)

Port Status No light (LED is off) Port has no incoming power, or

Steady green Port is online (connected to an

Slow-flashing green (on 1 second, then off 1 second)

Fast-flashing green (on 1/4 second, then off 1/4 second

Flickering green Port is online, with traffic

Steady amber Port is receiving light or signal

Slow-flashing amber (on 2 seconds, then off 2 seconds)

Fast-flashing amber (on 1/2 second, then off 1/2 second)

Alternating green/amber Port is bypassed Reset port from workstation

not have power.

Blade is not seated correctly or is faulty.

Environmental range exceeded. Check for out-of-bounds

there is no light or signal carrier detected.

Polling is in progress. Allow 60 seconds for polling to

Connected device is configured in an offline state.

external device) but has no traffic.

Port is online but segmented, indicating a loopback plug or cable or an incompatible switch.

Port is in internal loopback (diagnostic).

flowing through port.

carrier, but it is not online yet.

Port is disabled due to diagnostic tests or portDisable or portCfgPersistentEnable command.

Transceiver or port is faulty. Change transceiver or reset

Verify that the power LED is on.

and check status with slotShow command. If LED remains amber, contact IBM.

Pull blade out and reseat it. If LED continues to flash, replace blade.

environmental condition and correct it.

Verify that the power LED is on, check the transceiver and cable.

complete. Verify the status of the

connected device. No action required.

Verify that the correct device is attached to the SAN768B.

No action required.

Reset port from workstation using the portEnable or

portCfgPersistentEnable

command. Reset port from workstation

using the portEnable or

portCfgPersistentEnable

command.

switch from workstation.

using the portEnable or

portCfgPersistentEnable

command.

Power supply

To determine the status of a power supply:

1. Check the LED indicator on the power supply (see Figure 18 on page 46). The LED patterns may temporarily change during POST and other diagnostic tests; for information about how to interpret the LED patterns, see Table 13 on page

46.

Chapter 4. Monitoring system components 45

Page 74

2. Check power supply status by typing psShow. The power supply status displays OK, Absent, or Faulty. If a power supply is

absent or faulty, contact IBM.

Figure 18. Power supply LEDs

Table 13. Power supply LED descriptions

LED purpose Color Status

Power No light (LED is off) Power supply does

Blower assembly

To determine the status of a blower assembly:

1. Check the LED indicators on the blower assembly (see Figure 19 on page 47).

2. Check the blower assembly status using the fanShow command.

1 Power LED

Recommended Action

Ensure power supply not have incoming power and is not providing power to the SAN768B.

is firmly seated,

SAN768B has

incoming power, both

power cables are

connected, and AC

power switches are

on.

Steady green Power supply has

No action required. incoming power and is providing power to the SAN768B.

Flashing green Power supply is

about to fail.

Replace power

supply.

The LED patterns may temporarily change during POST and other diagnostic tests; for information about how to interpret the LED patterns, see Table 14 on page 47.

46 SAN768B Installation, Service, and User’s Guide

Page 75

The status for each blower assembly displays OK, Absent, or Faulty. The RPM of each fan in the assembly is also provided. If a blower assembly is absent or faulty, contact IBM.

19 displays the blower assembly.

Figure

b768ig006

Figure 19. Blower assembly LEDs and other parts

Item Description

1 Power LED 2 Fault LED

Table 14. Blower assembly LED descriptions

Recommended

LED purpose Color Status

Power No Light (LED is off) Blower assembly

does not have incoming power.

action

Ensure that the blower assembly is firmly seated and has power.

Steady green Blower assembly has

No action required.

incoming power.

Chapter 4. Monitoring system components 47

Page 76

Table 14. Blower assembly LED descriptions (continued)

Recommended

LED purpose Color Status

Fault No Light (LED is off) Blower assembly is

either healthy or does not have incoming

action

Ensure that the

blower assembly has

incoming power. power.

Steady amber Blower assembly has

a failure (full or

Replace blower

assembly. partial).

Slow-flashing amber (on 2 sec, then off 2 sec)

Blower assembly is not seated correctly or is faulty.

Pull unit out and

reseat. If LED

continues to flash,

replace unit.

Flashing amber (on 1/2 sec, then off 3.5

Fan is disabled. Run fanEnable to

enable the fan.

sec) Fast-flashing amber

(on 1/2 sec, then off 1/2 sec)

Environmental range exceeded.

Check for

out-of-bounds

environmental

condition, resolve any

problems, and reseat

unit. If LED continues

to flash, replace unit.

WWN bezel and WWN card

To determine the status of the WWN card:

The WWN bezel covers the WWN cards. The LEDs on the WWN cards are

Note:

not visible unless the bezel is removed.

1. WWN units correspond to information specific to the WWN card (see Figure 29 on page 75) and are displayed by typing the chassisShow command. Error messages that may indicate problems with a WWN card are summarized in Table 15.

Table 15. Messages that may indicate WWN card failure

Type of Message Sample error message

WWN unit fails its FRU (field replaceable unit) header access.

WWN unit is being faulted. 0x24c (fabos): Switch: switchname,

WWN unit is not present or is not accessible. 0x24c (fabos): Switch: switchname, Error

Writing to the FRU history log (hilSetFruHistory) has failed.

0x24c (fabos): Switch: switchname, error EM-I2C_TIMEOUT, 2, WWN 1 I2C timed out: state 0x4

Critical EM-WWN_UNKNOWN, 1, Unknown WWN #2 is being faulted

EM-WWN_ABSENT, 2, WWN #1 not present

0x24c (fabos): Switch: switchname, Error EM-HIL_FAIL, 2, HIL Error: hilSetFruHistory failed, rc=-3 for SLOT 3

Figure 20 on page 49 displays the WWN bezel.

48 SAN768B Installation, Service, and User’s Guide

Page 77

b768ig004

Figure 20. LEDs on the WWN card and bezel

1 CP blade Status (above) and

Power (below) LEDs

2 CR blade Status (above) and

3 Port blade Status (above) and

Power (below) LEDs

4 Slot numbers

Power (below) LEDs

The WWN card LED patterns and the recommended actions for those patterns are described in Table 16.

Table 16. WWN bezel LED descriptions

Recommended

LED purpose Color Status

Port blade/CP blade/

Steady green Power is OK. No action required.

action

CR blade Power Port blade/CP blade/

CR blade Status

Steady amber Blade is faulty. Check blade. No light (LED is OFF) Blade is OK. No action required.

Note: If a blade slot or power supply bay has a filler panel installed, the corresponding

LEDs on the WWN blade do not light up.

Chapter 4. Monitoring system components 49

Page 78

50 SAN768B Installation, Service, and User’s Guide

Page 79

Chapter 5. Removing and installing components

Note: Read the safety notices before servicing (see “Safety notices and labels” on

page xiii).

The field replaceable units (FRUs) in the SAN768B can be removed and installed without special tools. The SAN768B can continue operating during many of the FRU replacements if the conditions specified in the procedure are followed.

The following sections contain FRU removal and installation procedures.

Removing and installing the chassis door

Note: The chassis door must be installed to ensure the SAN768B meets EMI and

other regulatory certifications.

Time and items required

The replacement procedure for the chassis door takes less than 5 minutes.

Removing the chassis door

To remove a chassis door:

1. Support the door to prevent it from falling.

2. Pull and remove the door.

install a chassis door:

1. Align the door and push it into place. See Figure 21 on page 52.

Page 80

Figure 21. Chassis door

Removing and installing a cable management comb

The SAN768B can continue to operate during the replacement of the cable management comb.

Time and items required

The replacement procedure for the cable management comb takes less than 5 minutes. A #1 Phillips screwdriver is required.

Removing a cable management comb

To remove the cable management comb, do the following:

1. Rearrange the cables around the cable management comb.

2. Unscrew and save the four screws holding the comb to the chassis (see Figure 22 on page 53). Support the comb to prevent it from falling.

3. Remove the cable management comb.

52 SAN768B Installation, Service, and User’s Guide

Page 81

Figure 22. Cable management comb

Installing a cable management comb

To install a cable management comb, do the following:

1. Position and tighten the four screws to secure the cable management comb to the chassis.

2. Arrange the cables along the cable management comb.

Removing and installing port blades

This section describes how to remove and install port blades.

Attention: A filler panel should be removed only when being replaced with a port

blade, or a filler panel. Any slot that is not occupied by a blade should be occupied by a filler panel to ensure correct cooling of the chassis and protection from dust.

Slots are numbered from 1 through 12, from left to right when facing the port side of the SAN768B. Port blades can be installed in slots 1 through 4 and 9 through 12.

Chapter 5. Removing and installing components 53

Page 82

Time and items required

The installation procedure for each port blade takes less than 10 minutes. The following items are required for the blade and filler panel installation:

v ESD (electrostatic discharge) grounding strap v Workstation computer v Installation blade or filler panel v Phillips screwdriver v Small form-factor pluggable (SFP) or extended form-factor pluggable (XFP,

FC10-6 blade only) transceivers (as needed)

v Optical cables (as needed)

Removing a port blade

To remove a port blade, do the following:

Attention: Follow ESD precautions.

1. Remove the chassis door. (See “Removing and installing the chassis door” on page 51.)

2. A failed port or application blade can be identified by inspecting the LEDs on the front panel of each blade. Check the power LED, status LED, and port status LED to identify any possible problems. See Figure 14 on page 41 to Figure 13 on page 38 for LED locations.

3. Establish a Telnet or console session. Before installing a blade, establish a Telnet or console connection to determine

a failure and verify operation after installation. Use the switchShow command to view the status of blades.

4. Check for adequate cable slack. Ensure there is plenty of cable slack to remove a blade without cable obstruction.

5. Ensure that the part number on the unit being installed matches the replacement part number. The chassisShow command displays information about the blades, including part numbers, serial numbers, and additional status.

6. Ensure that traffic is not flowing through the blade (port status LED should be off) prior to disconnecting cables.

Before removing any cables from a blade, note the cable order (identify

Note:

each cable by its physical port). If multiple blades are being installed, install one blade at a time.

7. Ensure that traffic is not flowing through the blade.

8. Disconnect all cables and SFP or XFP transceivers from the blade.

9. For FC8-16, FC8-32, FC8-48, and FC10-6 port blades: Adjust the ejectors to the open position. (See Figure 23 on page 55.) Unscrew the two thumb screws from the top and bottom ejectors on the blade using the Phillips screwdriver. Unscrew the top thumb screw until it pops out. This initiates a hot-swap request.

10. Wait for the power LED to turn off in response to the hot-swap request before removing the blade.

11. Open the ejectors. Pull the blade out of the chassis using the ejectors.

12. If the blade is not being replaced by another blade, install a filler panel. See “Removing and installing port blade filler panels” on page 56.

54 SAN768B Installation, Service, and User’s Guide

Page 83

Figure 23. Port blade removal and installation (16-port blade shown as example)

Installing a port blade

To install a port blade, do the following:

Attention: Follow ESD precautions.

1. Orient the blade so that the ports are at the front of the chassis and the flat side of the blade is on the left.

2. Adjust the ejectors to the open position, align the flat side of the blade inside the upper and lower rail guides in the slot, and slide the blade into the slot until it is firmly seated.

3. Adjust the ejectors to the closed position by pulling them away from the center of the blade.

4. Tighten the two thumb screws near the top and bottom ejectors. The top thumb screw must be tightened for the blade to power on.

5. Verify that the power LED on the blade displays a steady green light (it might require a few seconds to turn on). If it does not turn on, ensure that the blade is firmly seated.

Chapter 5. Removing and installing components 55

Page 84

6. Install SFP or XFP (FC10-6 only) transceivers and cables in the blade.

7. Group and route the cables.

8. Install the chassis door.

Removing and installing port blade filler panels

This section describes how to remove and install port blade filler panels.

Removing a filler panel

Attention: A filler panel should be removed only when being replaced with a port

blade, or new filler panel. Any slot that is not occupied by a blade should be occupied by a filler panel to ensure correct cooling of the chassis and protection from dust.

To remove a filler panel, do the following:

1. Remove the chassis door.

2. Unscrew the thumb screw at the bottom of the panel using the Phillips screwdriver.

3. Pull the filler panel out of the chassis. (See Figure 24 on page 57.)

56 SAN768B Installation, Service, and User’s Guide

Page 85

Figure 24. Port blade removal and installation (16-port blade shown as example)

Installing a filler panel

To install a filler panel, do the following:

1. Orient the filler panel.

2. Slide the filler panel into the slot until it is firmly seated.

3. Tighten the thumb screw at the bottom of the panel.

4. Install the chassis door.

Removing and installing a control processor blade (CP8)

This section describes how to remove and install a control processor (CP8) blade. Each SAN768B has two CP8 blades located in slot 6 and slot 7.

If the new CP blade does not have the same firmware as the active CP

Note:

blade, you must upgrade the new blade to the same firmware version.

Chapter 5. Removing and installing components 57

Page 86

Firmware downloads require that Ethernet connections be made to both CP blades. Procedures for loading firmware are described in the Brocade Fabric OS Administrator’s Guide.

Time and items required

The installation procedure for the CP blade takes approximately 30 minutes. The following items are required for the CP blade installation:

v ESD (electrostatic discharge) grounding strap v Workstation computer v Serial cable v IP address of an FTP server for backing up the SAN768B configuration v Phillips #2 screwdriver v Replacement CP blade. SAN768B requires the CP blade (CP8).

Verifying the necessity of installation

Confirm that you need to install the CP blade. The following events might indicate that a CP blade is faulty:

v The status LED on the CP blade is lit amber, or the power LED is not lit. v The CP blade does not respond to Telnet commands, or the serial console is not

available.

v The slotShow command does not show that the CP blade is enabled. v The haShow command indicates an error. v The clock is inaccurate, or the CP blade does not boot up or shut down normally. v Any of the following messages display in the error log:

– “Slot unknown” message relating to a CP slot – CP blade errors or I2C timeouts – FRU: FRU_FAULTY messages for a CP blade – Configuration loader messages or “Sys PCI config” messages – Generic system driver messages (“FABSYS”) – Platform system driver messages (“Platform”) – EM messages that indicate a problem with a CP blade – Function fail messages for the CP master

more information about error messages, refer to the Fabric OS Message

For Reference.

Recording critical SAN768B information

Back up the SAN768B configuration before you replace a CP blade. Refer to the

Fabric OS Administrator’s Guide for backup information.

To record critical SAN768B information, do the following:

1. Login to the functioning CP blade as admin, using either a Telnet or a serial console connection.

2. Type haShow to determine which CP blade is active:

swDir:admin> haShow Local CP (Slot 7, CP1) : Active Remote CP (Slot 6, CP0) : Standby, Healthy HA Enabled, Heartbeat Up, HA State Synchronized

58 SAN768B Installation, Service, and User’s Guide

Page 87

3. Enter all remaining commands from the serial console for the active CP blade, unless otherwise indicated. For more information about commands, refer to the Fabric OS Command Reference.

4. If the functioning CP blade is performing as the active CP blade, go to step 5. If the faulty CP blade is performing as the active CP blade, failover the blades:

a. Login to the faulty CP blade as admin, using either a Telnet or a serial

console connection. If you connect successfully to the faulty CP, continue to step b. If you cannot connect to the faulty CP, remove the faulty CP blade for repair

as follows:

1) Turn the CP blade off and notify the SAN768B of a hot swap request by sliding the slider switch in the top ejector down to the off position (see Figure 25 on page 61).

2) Wait until the active CP LED on the functioning CP blade is lit. Remove the CP blade.

b. Type haFailover. The functioning CP blade becomes the active CP blade. c. Wait until the status LED on the functioning CP blade is no longer lit. d. Type haShow from the functioning CP blade (the new active CP blade) to

verify the failover. This is an example of failing over a CP blade:

Fabric OS (cp1)

cp1 login: admin Password: swDir:admin> hashow Local CP (Slot 7, CP1): Active Remote CP (Slot 6, CP0): Standby, Healthy HA enabled, Heartbeat Up, HA State synchronized swDir:admin> hafailover Local CP (Slot 7, CP1): Active Remote CP (Slot 6, CP0): Standby, Healthy HA enabled, Heartbeat Up, HA State synchronized

Warning: This command is being run on a redundant control processor(CP) system. If the above status does not indicate ’HA State synchronized’, then the CPs are not synchronized and this operation will cause the active CP to reset. This will cause disruption to devices attached to both switch 0 and switch 1 and will require that existing Telnet sessions be restarted. To reboot a single logical switch on this system, use the switchreboot command while logged in to that logical switch.

Are you sure you want to fail over to the standby CP [y/n]? swDir:admin> haShow Local CP (Slot 7, CP1) : Standby, Healthy Remote CP (Slot 6, CP0) : Active HA Enabled, Heartbeat Up, HA State Synchronized

5. Type firmwareShow to note the firmware version of the active CP blade.

6. Type haDisable from the active CP blade to prevent failover or communication between the CP blades during the replacement.

7. Logged into the active CP, use the configUpload command to upload the SAN768B configuration to a specified FTP server. Enter information at the prompts.

This is a sample of backing up the configuration files:

Chapter 5. Removing and installing components 59

Page 88

swDir:admin> configUpload Protocol (scp or ftp) [ftp]: ftp Server Name or IP Address [host]: 123.456.78.90 User Name [None]: user File Name [config.txt]: config.txt Password: xxxxxxxx upload complete

Removing a control processor blade (CP8)

The SAN768B continues to operate while a CP blade is being installed if the redundant CP blade is active and a failover does not occur. Yo u can prevent failover by entering the haDisable command.

To remove a CP blade, do the following:

1. Remove the chassis door. (See “Removing and installing the chassis door” on page 51.)

2. Login to the active CP as the admin user. Yo u can use a serial cable or Telnet, Web Tools, or Fabric Manger. Determine which CP is active using the haShow command or view the active LED on the front of the CP.

3. If the faulty CP is the active CP, issue the haFailover command. Wait until the failover has completed. Use the haShow command to verify the CPs are synchronized and the failover is complete.

4. Power off the blade by sliding the slider switch in the top ejector down to the off position (see Figure 25 on page 61).

5. Disconnect all cables from the faulty (standby) CP.

6. Unscrew the thumb screw from both ejectors using the Phillips screwdriver.

7. Lever open both ejector handles simultaneously to approximately 45 degrees and pull the CP blade out of the chassis (see Figure 25 on page 61).

60 SAN768B Installation, Service, and User’s Guide

Page 89

Figure 25. Removing the control processor (CP) blade

Installing a control processor blade (CP8)

Attention: Follow ESD precautions.

To install a CP blade (slot 6 or 7):

1. Open the ejector handles to approximately 45 degrees. Orient the CP blade so that the handles are toward you and the flat metal side is on your left.

2. Align the flat metal side of the CP blade inside the lower and upper blade guides in the slot. Slide the CP blade into the slot until it is firmly seated.

3. Tighten the thumb screw inside each handle using the Phillips screwdriver.

4. Turn the CP blade on by sliding the ON/OFF switch in the top handle up, to cover the thumb screw.

5. Verify that the power LED is green. If not, ensure that the CP blade has power and is firmly seated and that the ejectors are in the locked position.

Chapter 5. Removing and installing components 61

Page 90

6. Connect the cables to the new CP blade.

7. Verify the installation. (See “Verifying operation of the new CP blade.”)

8. Install the chassis door. (See “Removing and installing the chassis door” on page 51.)

Verifying operation of the new CP blade

To verify that boot and POST are complete on the new CP blade and that the CP blade has achieved failover redundancy:

1. Log into the active CP as admin.

2. Enable the redundancy feature using the haEnable command.

3. Verify HA is enabled using the haShow command. The command output includes “HA Enabled Heartbeat Up” and “HA-state in sync.”

If not, POST is not complete or the CP blades have not achieved redundancy. Wait one or two minutes and re-enter the command until you can verify that redundancy has been achieved.

4. Type slotShow. The command output shows the new CP blade as “enabled”:

swDir:admin> slotShow

Slot Blade Type ID Status

---------------------------------

1 SW BLADE 16 ENABLED 2 SW BLADE 16 ENABLED 3 SW BLADE 16 ENABLED 4 SW BLADE 16 ENABLED 5 CR BLADE 17 ENABLED 6 CP BLADE 18 ENABLED 7 CP BLADE 18 ENABLED 8 CR BLADE 17 ENABLED 9 SW BLADE 16 ENABLED 10 SW BLADE 16 ENABLED 11 SW BLADE 16 ENABLED 12 SW BLADE 16 ENABLED

swDir:admin>

5. Determine the version by typing firmwareShow.

This example shows sample output for the firmwareShow command:

swDir:admin> firmwareShow Local CP (Slot 7, CP1): Active

Primary partition: v6.0.0 Secondary Partition: v6.0.0

Remote CP (Slot 6, CP0): Standby

Primary partition: v6.0.0 Secondary Partition: v6.0.0

Note: If Local CP and Remote CP have different versions of firmware, please retry firmwaredownload command.

swDir:admin>

6. If the firmware version on the replacement blade does not match that on the

active CP blade, bring the replacement blade to the same firmware level as the active blade.

Firmware downloads require that Ethernet connections be made to both

Note:

CP blades. Procedures for loading firmware are described in the Brocade Fabric OS Administrator’s Guide.

62 SAN768B Installation, Service, and User’s Guide

Page 91

7. Type firmwareDownload -s to download the firmware to one of the CP blades.

To download firmware, go to http://www.ibm.com/servers/storage/support/san and select your product, then click on the download tab. Select the appropriate firmware version and release notes version.Enter all requested information (use default values):

swDir:admin> firmwaredownload -s Server Name or IP Address: 192.168.100.1 User Name: user File Name: /software/v6.0.0/release.plist Password: ******** Do Auto-Commit after Reboot [Y]: Reboot system after download [N]: Firmwaredownload has started. 2007/07/03-14:59:21, [SULB-1001], 923,, WARNING, SAN768B, Firmwaredownload command has started.

Start to install packages......

dir ################################################## ldconfig ################################################## glibc ################################################## bash ################################################## readline ################################################## terminfo ################################################## termcap ################################################## setup ##################################################

<output_truncated=""> tz ################################################## mtracer-tool ################################################## lkcd ################################################## sysstat ################################################## Removing unneeded files, please wait ... Finished removing unneeded files.

All packages have been downloaded successfully. Firmwaredownload has completed successfully. 2007/07/03-15:05:56, [SULB-1002], 924,, INFO, SAN768B, Firmwaredownload command has completed successfully.

8. Type firmwareDownloadStatus to verify that the firmware has been updated.

9. Type haEnable to enable HA again on the active CP blade.

10. Type reboot.

11. Verify that the reboot is complete and the CP blades have achieved failover redundancy:

a. Wait until the status LEDs on both CP blades are not lit. b. Type haShow and verify that the command output includes “HA Enabled

Heartbeat Up”. If not, reenter the command until you have verified that redundancy is achieved.

Type firmwareShow to verify that the firmware version has been updated.

12.

13. Pack the faulty CP blade in the packaging provided with the new CP blade for

return to IBM, following local procedures.

14. Install the chassis door. (See “Removing and installing the chassis door” on

page 51.)

Removing and installing a core switch blade (CR8)

This section describes how to remove and install a core switch blade. Each SAN768B has two core switch blades located in slot 5 and slot 8.

Chapter 5. Removing and installing components 63

Page 92

Time and items required

The installation procedure for the core switch blade takes approximately 30 minutes. The following items are required for the core switch blade installation:

v ESD (electrostatic discharge) grounding strap v Phillips #2 screwdriver v Replacement core switch blade (SAN768B requires the core switch blade (CR8)).

Verifying the necessity of installation

Confirm that you need to replace the core switch blade before continuing. The following events might indicate that a core switch blade is faulty:

v The status LED on the core switch blade is lit amber, or the power LED is not lit. v The slotShow command does not show that the core switch blade is enabled. v The haShow command indicates an error. v Any of the following messages display in the error log:

– “Slot unknown” message relating to a core switch blade slot – core switch blade errors or I2C timeouts – FRU: FRU_FAULTY messages for a core switch blade – Configuration loader messages or “Sys PCI config” messages – Generic system driver messages (“FABSYS”) – Platform system driver messages (“Platform”) – EM messages that indicate a problem with a core switch blade – Function fail messages for the core switch blade master

more information about error messages, refer to the Fabric OS Message

For Reference.

Removing a core switch blade (CR8)

The SAN768B continues to operate while a core switch blade is being replaced.

Attention: Follow ESD precautions.

To remove a CR blade, do the following:

1. Remove the chassis door (see “Removing and installing the chassis door” on page 51).

2. Power off the blade by sliding the slider switch in the top ejector down to the off position (see Figure 26 on page 65).

3. Disconnect cables from the faulty core switch blade.

4. Unscrew the thumb screw from both ejectors using the Phillips screwdriver.

5. Lever open both ejector handles simultaneously to approximately 45 degrees and pull the core switch blade out of the chassis (see Figure 26 on page 65).

64 SAN768B Installation, Service, and User’s Guide

Page 93

Figure 26. Removing the core switch blade

Installing a core switch blade (CR8)

Attention: Follow ESD precautions.

To install a CR blade (slot 5 or 8), do the following:

1. Open the ejector handles to approximately 45 degrees; then, orient the CR blade so that the handles are toward you and the flat metal side is on your left.

2. Align the flat metal side of the core switch blade inside the lower and upper blade guides in the slot; then, slide the core switch blade into the slot until it is firmly seated.

3. Tighten the thumb screw inside each handle using the Phillips screwdriver.

4. Turn the core switch blade on by sliding the ON/OFF switch in the top handle up, to cover the thumb screw.

Chapter 5. Removing and installing components 65

Page 94

5. Verify that the power LED is green (might require a few seconds). If not, ensure that the core switch blade has power and is firmly seated and that the ejectors are in the locked position.

6. Connect to the new core switch blade.

7. Pack the faulty core switch blade in the packaging provided with the new core switch blade for return to IBM, following local procedures.

8. Install the chassis door. (See “Removing and installing the chassis door” on page 51.)

Removing and installing an FR4-18i router blade

This section describes how to remove and install an FR4-18i router blade in the SAN768B. Yo u can install up to two router blades in any of slots 1-4 and 9-12. To ensure proper cooling, install only one router blade in each slot grouping. See Figure 13 on page 38, for blade removal and installation, as an example of how to remove a router blade.

Attention: Follow these precautions for all procedures in this section to avoid

damaging the router blades or chassis:

v Wear a grounded ESD strap when handling the router blade. v To ensure adequate system pressure for cooling purposes, leave the faulty router

blade in the slot until you are ready to perform the installation or replacement procedure.

v Do not force the router blade into the chassis. If the blade does not slide in

easily, ensure that it is correctly aligned inside the rail guides before continuing. Installing a router blade with incorrect alignment can damage both the chassis and the router blade.

v Hold the router blade by the edges of the metal pan. Do not use the ejectors to

hold a router blade.

v Disassembling any part of a router blade voids the part warranty and regulatory

certifications. There are no user-serviceable parts inside the router blade.

Time required

Approximately 10 minutes

Items required

v ESD (electrostatic discharge) grounding strap v Workstation computer v Replacement router blade v Phillips screwdriver

Removing a router blade

A failed router blade can be identified by inspecting its LEDs and by using CLI commands to verify status. Refer to Figure 13 on page 38, for a description of the router blade LEDs. See the Fabric OS Administrator's Guide for information about how to check the status of hardware components using the CLI.

Perform the following steps to remove a router blade:

1. Check the power LED, status LEDs, port speed LEDs, and port status LEDs to identify any possible problems with the router blade.

66 SAN768B Installation, Service, and User’s Guide

Page 95

2. Establish a Telnet or console session before removing an router blade to determine a failure and verify operation before removal. Use the switchShow command to view the status of blades on the SAN768B.

3. Check for adequate cable slack. Make sure there is plenty of cable slack to remove a router blade without

optical, power, or Ethernet cable obstruction. Refer to “Managing cables” on page 9 for cabling guidelines.

4. Ensure that you have the proper spare part (filler panel or replacement router blade).

Make sure that the part numbers match for the unit being replaced. The

chassisShow command displays information about the router blades,

including part numbers, serial numbers, and additional status.

5. Ensure that traffic is not flowing through the router blade (all port status LEDs should be off) prior to disconnecting cables.

Before removing any cables from the faulty router blade, label the

Note:

cables or make a note of the cable order by identifying each cable by the physical port it attaches to. This will reduce confusion when you reattach cables. If two router blades are being replaced, install one blade at a time.

6. Disconnect all cables and remove the SFP transceivers from the Fibre Channel and GbE ports as required.

7. Turn the router blade off by sliding the on/off switch in the top ejector down, to the off position (see 3 in Figure 13 on page 38 as an example). This initiates a hot-swap request.

8. Wait for the power LED to turn off in response to the hot-swap request before removing the router blade.

9. Unscrew the captive screws from the top and bottom ejectors on the blade using a Phillips screwdriver.

10. Lever both ejectors open simultaneously to approximately 45 degrees, and pull the router blade out of the chassis.

11. If the blade is not being replaced by another router blade, install a filler panel

to ensure correct cooling of the chassis and protection from dust. See “Installing a filler panel” on page 57 for instructions.

Installing a router blade

You can install up to two FR4-18i router blade in slots 1–4 or 9-12. If you are installing two router blades, install one in one of the slots 1–4, and the other in one of the slots 9-12 to maintain proper cooling. The control processor (CP) blades occupy slots 6 and 7. The core switch (CR) blades occupy slots 5 and 8. Any slots that are not occupied by a port blade, router blade, or CP blade must have a filler panel installed to ensure proper cooling of the chassis and protection from dust. If you are performing a new installation of a router blade, you must either remove the filler panel or a port blade from the slot where you will install the router blade.

Attention: All ports on the router blades are by default, persistently disabled to

Fabric OS Command Reference Manual for more information on these commands.

Chapter 5. Removing and installing components 67

Page 96

To install an FR4-18i router blade, complete these steps:

1. Enter the chassisConfig command, and verify that it is in mode 5. If the chassisConfig mode is 5, this step is complete. If not, enter the chassisConfig

5 command.

Attention: Changing the chassisConfig mode requires a reboot, and is

disruptive to SAN768B operation.

2. Select an empty slot from slots 1-4 or 9-12 to install the router blade.

3. Make sure the latches on the router blade are unscrewed and open. If you are performing a new installation of a router blade, you must either remove the filler panel or a port blade from the slot where you will install the router blade.

4. Orient the router blade so that the ports are at the front of the chassis and the router blade power switch is at the top.

5. Make sure the latches on the router blade are unscrewed and open.

6. Slide the router blade into the open slot until it is fully seated.

7. Close the latches and tighten the screws on the latches.

8. Power on the blade using the power switch at the top of the blade. Keep the blade in an offline (disabled) state until the ports are configured.

The router blade will begin POST as soon as the power is turned on.

Note:

After POST is complete the firmware version on the blade will autolevel with the firmware version on the active CP.

9. Verify that the blade power and blade status LEDs are green after the blade has completed booting up.

10. Verify the installation. For information about how to check the status of hardware components using the CLI, See the Fabric OS Administrator’s Guide.

11. Log into the SAN768B as the admin user.

Configuring FCIP and Fibre Channel routing services and enabling the ports

Refer to the Fabric OS Administrator’s Guide or the Web Tools Administrator’s Guide for detailed information on configuring the router blade. The following steps

provide only a brief overview of the procedure.

1. Verify that the FR4-18i router blade is in an offline (disabled) state until the ports

are configured.

Attention: The ports on the FR4-18i router blade are initially set to

persistently disabled. They will flash amber until they are enabled. This is normal. If you want to use the FC ports and EX_Ports, you must configure the Fibre Channel Routing Services feature prior to enabling the ports.

2. Use the portCfgPersistentEnable command to enable standard E_ports or

F_ports.

The GbE ports can only be used once you have configured FCIP and

Note:

enabled VE_Ports. See the Fabric OS Administrator’s Guide for detailed instructions on configuring the Fibre Channel Router ports and GbE ports on the FR4-18i router blade.

Cabling the FR4-18i router blade

Complete the router blade configuration by installing the SFP transceivers in the Fibre Channel and GbE ports.

68 SAN768B Installation, Service, and User’s Guide

Page 97

1. Install the SFP transceivers in the Fibre Channel and GbE ports, as required. Refer to “Installing and removing FC transceivers in a port blade” on page 76 for more detailed instructions.

2. Remove the rubber plugs from the ports to be used.

3. Position a transceiver so that it is oriented correctly and insert it into a port until it is firmly seated and the latching mechanism clicks. For instructions specific to the type of transceiver, see the transceiver manufacturer’s documentation.

The transceivers are keyed to ensure correct orientation. If a transceiver

Note:

does not install easily, ensure that it is correctly oriented.

4. Repeat Steps 2 and 3 for the remaining ports, as required.

5. Connect the cables to the transceivers.

Attention: A 50-micron cable should not be bent to a radius less than 2

inches under full tensile load and 1.2 inches with no tensile load. Tie wraps are not recommended for optical cables because they are easily overtightened.

6. Orient a cable connector so that the key (the ridge on one side of connector) aligns with the slot in the transceiver. Then, insert the cable into the transceiver until the latching mechanism clicks. For instructions specific to cable type, see the cable manufacturer’s documentation.

7. Repeat Step 6 for the remaining cables as required.

8. Check the LEDs to verify that all components are functional. For information about LED patterns, see “FR4-18i router blade” on page 37.

9. Log in to the SAN768B as admin.

10. Verify the correct operation of the FR4-18i router blade ports by typing the

switchShow command from the SAN768B command prompt. This command

provides information about switch and port status. Once the FR4-18i router blade is installed and fully configured in a SAN768B,

the switchShow command displays 32 Fibre Channel ports (port numbers 0 through 31) and 2 GbE ports. The first 16 Fibre Channel ports are physical ports on the FR4-18i router blade, Ports 16-23 are virtual ports associated with the GE0 physical GbE link and ports 24-31 are virtual ports associated with GE1 physical GbE links. The GbE ports are displayed as ge0 and ge1 and are not assigned port numbers or area numbers.

Removing and installing a power supply

Use this procedure to remove and install a power supply. The SAN768B can continue operating during the replacement if at least one power supply continues operating.

Time and items required

The installation procedure for each power supply takes less than 5 minutes. A power supply unit or filler panel is required for the power supply installation.

Removing a power supply

To remove a power supply, do the following:

1. If the SAN768B is not operating during the replacement procedure, go to step 2. If the SAN768B is going to continue operating during the replacement, check

the power LEDs to verify that the minimum number of power supplies is functioning. A fully populated SAN768B requires a minimum of one power supply slot at all times to ensure power to the SAN768B.

Chapter 5. Removing and installing components 69

Page 98

2. Pull the handle out and down (see Figure 27).

3. Support the power supply from beneath. Use the handle to remove the power supply out of the chassis.

4. If you are not replacing the power supply, insert a filler panel into the slot.

Figure 27. Removing and installing a power supply

Installing a power supply

To install a power supply, do the following:

1. Remove any filler panels.

2. Insert the power supply into the slot and push the handle up. Verify that the power supply is seated by gently pulling on the handle.

3. Verify that the power LED on the power supply displays a steady green light.

b768ig003

70 SAN768B Installation, Service, and User’s Guide

Page 99

Removing and installing a blower assembly

This procedure provides instructions for removing and installing a blower assembly.

Attention: To ensure continuous adequate cooling, maintain three operating

blower assemblies at all times except for the brief period when replacing a blower assembly.

Time and items required

The installation procedure for each blower assembly takes less than 5 minutes. The following items are required for the blower assembly installation:

v Replacement blower assembly v Phillips #2 screwdriver

Removing a blower assembly

To remove a blower assembly, do the following:

1. Before removing a blower assembly, verify that the other blower assemblies are functioning correctly. The power LEDs should be steady green.

2. Use the screwdriver to loosen the captive screws at the top and bottom of blower assembly (see Figure 28 on page 72).

3. Support the blower assembly from beneath. Use the handle to remove the blower assembly.

Chapter 5. Removing and installing components 71

Page 100

Figure 28. Blower assembly LEDs and other parts

Installing a blower assembly

To install a blower assembly, do the following:

1. Orient the blower assembly and slide it into the chassis, pushing firmly to ensure that it is seated.

2. Verify that the power LED displays a green light.

3. Use the screwdriver or your fingers to tighten the captive screws.

Removing and installing a WWN bezel and WWN card

Note: The World Wide Name (WWN) cards contain fully redundant circuits and

normally do not require installation. Two WWN cards are located beneath the WWN bezel.

Time and items required

Allow approximately 20 minutes to install a WWN card. The following items are needed to install a WWN card:

v Electrostatic discharge (ESD) grounding strap v #2 Phillips screwdriver (required only for some versions of the WWN card)

b768ig001

72 SAN768B Installation, Service, and User’s Guide