Brocade Communications Systems SilkWorm 12000 Design, Deployment And Management Manual
Brocade SilkWorm 12000
Design, Deployment and
Management Guide
Publication Number 53-0000251-02
Copyright ©2002, Brocade Communications Systems, Incorporated.
ALL RIGHTS RESERVED. Publication Number 53-0000251-02
BROCADE, the Brocade B weave logo, Brocade: the Intelligent Platform for Networking Storage, SilkWorm, and
SilkWorm Express, are trademarks or registered trademarks of Brocade Communications Systems, Inc. or its
subsidiaries in the United States and/or in other countries. All other brands, products, or service names are or may be
trademarks or service marks of, and are used to identify, products or services of their respective owners.
Notice: This book was designed and written to provide information about storage area networking architectures. Every
effort has been made to make this book as complete and accurate as possible. However, the information in this book is
provided to you “AS IS,” without warranty of any kind, including, without limitation, any implied warranty of
merchantability, noninfringement or fitness for a particular p urpose. Disclosure of information in this material in no way
grants a recipient any rights under Brocade's patents, copyrights, trade secrets or other intellectual property rights.
Brocade reserves the right to make changes to this docume nt at any time, without notice, and assumes no responsibility
for its use.
IMPORTANT NOTICE
This document is the property of Brocade. This document is confidential to Brocade. It is inte nded solely as an aid for
installing and configuring Storage Area Networks constructed with Brocade switches. This document does not provide a
warranty to any Brocade software, equipment, or service, nor does it imply product availability. Brocade is not
responsible for the use of this document and does not guarantee the results of its use. Brocade does not warrant or
guarantee that anyone will be able to recreate or achieve the results described in this document. The installation and
configuration described in this document made use of third party software and hardware. Brocade does not make any
warranties or guarantees concerning such third party software and hardware.
The authors and Brocade Communications Systems, Inc. shall have no liability or responsibility to any person or entity
with respect to any loss, cost, liability or damages arising from the information contained in this book or the computer
programs that accompany it.
Export of technical data contained in this document may require an export license from the United States Government.
Brocade Communications Systems,
Incorporated
Corporate Headquarters
1745 Technology Drive
San Jose, CA 95110
T: (408) 487-8000
F: (408) 487-8101
European Headquarters
29, route de l-Aeroport
Case Postale 105
1211 Geneva 15,
Switzerland
T: +41 22 799 56 40
F: +41 22 799 56 41
europe-info@brocade.com
Asia-Pacific Headquarters
The Imperial Tower 15th Floor
1-1-1 Uchisaiwaicho
Chiyoda-ku, Tokyo 100-0011
Japan
T: +81 33507 5802
F: +81 33507 5900
apac-info@brocade.com
Contents
Preface
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Audience for This Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Chapter 1 Introducing the SilkWorm 12000
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Fault Monitoring and Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Intelligent Fabric Services Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Advanced Performance Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Advanced Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Extended Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Fabric Watch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
ISL Trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
QuickLoop/Fabric Assist (QLFA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
Chapter 2 SilkWorm 12000 Architecture and What Is New
Fabric OS 4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
Dual Switch Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Dual Control Processors For High Availability . . . . . . . . . . . . . . . . . . . . 2-3
Accessing the SilkWorm 12000 Switches . . . . . . . . . . . . . . . . . . . . . . . . 2-4
How Logical Switch Behavior Differs. . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Port Addressing and Area Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10
Publication Number 53-0000251-02 iii
Software High Availability Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11
Failover Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11
Failover Details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12
Chapter 3 SilkWorm 12000 Based SAN Designs
Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3
Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
ISL Over Subscription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
Device Attachment Strategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5
Attaching Nodes for Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5
Attaching ISLs For Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6
Attaching Nodes for Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7
Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
SilkWorm 12000 Placement In The Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9
SilkWorm 12000 Based Fabric Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10
The Continuum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10
Single Chassis Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14
Core/Edge Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Device Attachment Strategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Core & Edge Switch Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18
Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19
Large Fabric Support Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20
SilkWorm 12000 Reference Topologies. . . . . . . . . . . . . . . . . . . . . . . . . . 3-21
Single chassis topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21
Two Chassis/Four Switch Partial Mesh . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22
Core/edge With Maximal Config
(current support levels). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23
Chapter 4 Deploying the SilkWorm 12000
Deployment Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1
iv Publication Number 53-0000251-02
Unpacking and Installing the SilkWorm 12000 in the Rack. . . . . . . . . . . . . . 4-2
Unpacking the Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Site Planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Installing the Rack Mount Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
Reinstalling the Chassis Door and Cable Management Tray . . . . . . . . . . 4-4
Cable Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5
Chapter 5 Configuring the SilkWorm 12000
Configuring the SilkWorm 12000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1
Basic configuration steps:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1
Logging into the SilkWorm 12000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Using Diagnostic Tests to Verify Hardware (Optional) . . . . . . . . . . . . . . 5-2
Configuring IP Addresses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Configuring the Switch Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Setting the Domain ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Enabling Software Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Return Switches to Default Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6
Chapter 6 SilkWorm 12000 Management Interfaces
Telnet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1
Web Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8
Zoning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-10
Upload/Download. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11
Port Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-12
Configure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-14
Extended Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-15
Fabric Manager 3.0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-16
Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17
Publication Number 53-0000251-02 v
Chapter 7 Maintaining the SilkWorm 12000
Hardware Maintenance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6
Power Supply Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7
Identify A Faulty Power Supply. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
Steps For Installation and Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
Power supply removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
Power supply installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
Blowers Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10
Identify A Faulty Blower Assembly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10
Blower Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-12
Blower Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-12
Control Processor Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-13
Identify A Faulty CP Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14
CP Card Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-15
CP Card Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-16
CP Card Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-17
16-Port Card Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-17
Identifying A Faulty 16-Port Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
16-Port Card Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
To Remove A 16-Port Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
Installing A 16-Port Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-22
16-Port Card Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-22
Software Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-23
Firmware Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-23
License upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-25
License Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-25
vi Publication Number 53-0000251-02
Chapter 8 Troubleshooting/Support
Software Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1
QuickLoop Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
Trunking Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
Web Tools Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Zoning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
Other Possible Software Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
Hardware Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5
Software Command Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-6
Cable-side LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-7
Non-cable Side LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8
Miscellaneous Hardware Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8
Diagnostic Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-9
Publication Number 53-0000251-02 vii
viii Publication Number 53-0000251-02
Preface
Introduction
This document discusses the practical aspects of designing, deploying, and maintaining a SilkWorm
12000 based SAN. The SilkWorm 12000 is a new product because it has: a bladed architecture, two
logical switches in one chassis, a new Fabric OS (version 4.0), and high availability/failover features.
Several SilkWorm 12000 features, while introduced in the SilkWorm 3800, require further discussion
within the context of the SilkWorm 12000, such as Trunking and 1-2 Gbit/sec auto-sensing ports. Other
considerations include designing and deploying SilkWorm 12000 based SANs and the integration of the
SilkWorm 12000 into existing SANs built with SilkWorm 2x00 and 3x00 technology.
This document addresses these new features and capabilities in a clear and concise manner, with liberal
use of examples, and is intended to be used in conjunction with SilkWorm 12000 manuals (see
References later in this section).
While working with the SilkWorm 12000, a multitude of decisions are necessary . This guide is intended
to identify the key decision points expected during the lifecycle of a SilkWorm 12000 deployment and
the advantages and disadvantages of adopting a particular approach. Also included in this guide are tips,
shortcuts, and suggestions for the efficient operation and maintenance of the SilkWorm 12000, which
are gathered from the engineers who developed and tested the SilkWorm 12000.
Audience for This Document
This guide is intended for technically focused personnel directly or indirectly responsible for the design,
deployment, and management of SilkWorm 12000 based SANs. The reader is expected to be familiar
with and have a working knowledge of SAN technology, Brocade SilkWorm switches, and Brocade
Fabric OS.
References
The following Brocade documentation is to be used in reference to this guide.
• Building SANs With Brocade Fabric Switches (Syngress Press) (ISBN: 1-928994-30-x)
• Brocade SAN Design Guide v2.2 (publication number: 53-0000231-03)
• SilkWorm 12000 Hardware Reference Manual –Beta Draft (publication number: 53-0000148-01)
• Brocade Fabric OS Reference Version 3.0/4.0 (publication number: 53-0000182-01)
• Brocade ISL Trunking User’s Guide (publication number: 53-0000189-01)
• Brocade Zoning User’s Guide v 3.0/4.0 (publication number: 53-0000187-01)
Publication Number 53-0000251-02 vii
• Web Tools User’s Guide Version 4.0 (publication number: 53-0000185-01)
• Brocade Fabric Manager User’s Guide Version 3.0 (publication number: 53-0000204-0)
• Brocade MIB Reference Version 3.0/4.0 (publication number: 53-0000184-01)
• Brocade SilkWorm 12000 Co re Migration User’s Guide v1.1 (53-0000477-02)
viii Publication Number 53-0000251-02
Introducing the SilkWorm 12000
This chapter includes the following sections:
• Hardware on page 1-2
• High Availability on page 1-3
• Reliability on page 1-4
• Fault Monitoring and Diagnostics on page 1-4
• Intelligent Fabric Services Architecture on page 1-5
The Brocade SilkWorm 12000 core fabric switch represents the next generation of advanced Fibre
Channel switches used to intelligently interconnect storage devices, hosts, and servers in a Storage Area
Network (SAN). It is a revolutionary product: a dual 64-port Fibre Channel switch that delivers
unprecedented performance, scalability, flexibility, functionality, reliability and availability. Figure1-1
shows two views of the SilkWorm 12000. Several key features of the SilkWorm 12000 and the Fabric
operating system (Fabric OS) are detailed below:
1
• The dual switch capability allows either one or two 64-port switches per chassis. The switches may
be interconnected together to create a single high port count fabric, or they may be used in a highly
available redundant fabric SAN. Dual redundant control processors (CP) provide high availability
within the chassis. The control processors are located on the CP cards.
• The SilkWorm 12000 is based on Brocade’s third generation technology, which supports 1 and 2
Gbit/sec auto-sensing Fibre Channel ports. Trunking technology groups up to four ports together to
create high performance 8 Gbit/sec ISL trunks between switches.
• Universal ports self-configure as E-ports, F-ports, or FL-ports.
• Small Form-Factor Pluggable (SFP) optical transceivers support any combination of Short
Wavelength (SWL) and Long Wavelength (LWL) optical media on a single switch module
(hereafter called 16-port card).
• Fully networkable, the SilkWorm 12000 offers forward and backward compatibility with all
Brocade SilkWorm switches.
• High availability, redundant design, extensive diagnostics, and system monitoring capabilities
integrated with Fabric OS management tools deliver unprecedented Reliability, Availability, and
Serviceability (RAS).
• The SilkWorm 12000 offers a highly available platform for mission-critical SAN-designed
applications.
• Extensible and multi-protocol to support 1 Gbit/sec, 2 Gbit/sec and 10 Gbit/sec Fibre Channel, IP,
and InfiniBand protocols.
Publication Number 53-0000251-02 1-1
1
Introducing the SilkWorm 12000
Cable Side (Front)
Figure 1-1 Cable and non-cable side views of the SilkWorm 12000
Non-Cable Side (Rear)
Hardware
The SilkWorm 12000 features a modular and scalable mechanical construction that allows a wide range
of flexibility in switch installation, Fabric design, and maintenance. Using a 14U (rack unit) mechanical
design, up to three SilkWorm 12000 chassis may be mounted in a standard 42U rack, supporting as many
as 384 Fibre Channel ports in a single rack. As shown in Figure 1-2, the modular multi-card assembly
chassis of the SilkWorm 12000 consists of the following:
• Up to eight hot-swappable 16-port cards, delivering up to two separate 64-port Fibre Channel
switches in a single chassis. Each 64-port switch uses four 16-port cards.
• Two slots for Control Processor cards.
- A single active CP card can control both 64-port switches in the chassis.
- A redundant CP card can assume control of a single or dual switch configuration in the event of
an active CP failure.
• Modular hot-swappable Field Replaceable Units (FRUs):
- 16-Port Card
- Control Processor Card (CP Card)
- Small Form-Factor Pluggable (SFP) optical transceivers
- Blower assembly
- Power supply
Figure 1-2 identifies the components as described above.
1-2 Publication Number 53-0000251-02
Port 15
16-port cards
Slot numbers
1-4
Introducing the SilkWorm 12000
7-10
Power supply #4
Power supply #3
Power supply #2
1
Port 0
Slot 1
Figure 1-2 SilkWorm 12000 With Identified Components
High Availability
High availability is an all-encompassing term, and this term normally includes attributes such as
reliability and availability. If a system is under continuous operation, it is accessible 7 days a week and
24 hours per day by users and the system manager. Availability is designated in terms of 9s. For example,
the architecture of the SilkWorm 12000 is designed for availability in excess of 99.999%.
The following features contribute to the high availability design of the SilkWorm 12000:
• Redundant, hot-swappable components
• Redundant power and cooling subsystems
• No single point of failure
• Enhanced data integrity on all data paths
• Fabric Shortest Path First (FSPF) automatic rerouting around failed links
• Integration with SNMP managers
• Automatic Contro l Pr ocessor fail over
Power supply #1
Slot 10
2 CP cards
Slot numbers 5 and 6
Publication Number 53-0000251-02 1-3
1
Introducing the SilkWorm 12000
The SilkWorm 12000 high availability software architecture provides a common framework for all
applications that reside on the system and allows global and local states to be maintained such that any
component failure is fully manageable. High availability elements consist of the High Availability
Manager, the heartbeat, the fault/health framework, the replicated database, initialization, and software
upgrade. The software high availability model is discussed in more detail later in this section.
The power supplies (four total) support two 64-port switches in a chassis with the ability to tolerate the
failure of as many as two power supplies. The power supplies are hot-swappable, without taking the
switch down and without incurring any outage.
The blower assemblies (three total) are also hot-swappable and the chassis can operate fully with only
two blower assemblies in place allowing for the failed blower assembly to be replaced with no outage.
The recommended systems networks for high availability include the use of redundant fabrics and dual
paths from hosts to storage devices in a SAN. When dual fabrics are used, one switch, one link, or an
entire fabric can go down, but data traffic will be re-routed to the alternate path ensuring that the SAN
remains operational.
Reliability
In addition to being available, the system must be reliable. This means that some, if not all, of its internal
state must be maintained. In a reliable system, a user is not aware of the internal state of the chassis
components and will experience continued system service with zero degradation of service.
The SilkWorm 12000 provides the following features to ensure reliability. All data inside the switch is
protected by the following Error Detection and Correction mechanisms as follows:
• Power-on self test (POST)
• Error detection and fault isolati on (EDFI), such as cyclic redundancy checking (CRC), parity
checking, checksum, and illegal address checking
• Dual control processors, with each control processor containing two serial ports and one Ethernet
port. Offline Control Processor diagnostics and remote diagnostics make troubleshooting
straightforward.
2
• I
C monitoring and control of environmental conditions
Fault Monitoring and Diagnostics
Fault monitoring, diagnostic tests, and system status indicators simplify management and ensure
availability of the SilkWorm 12000.
Diagnostic testing occurs in three areas: Power-On Self Test (POST), switch level testing, and
manufacturing tests. The Power-On Self T est is card oriented and ensures that the switch is ready for use
during power up. Switch level testing is done at the user port level. These tests rely on th e standard
Fabric OS support to provide routing and port setup. Manufacturing support includes long duration
testing.
The WWN card located on the non-cable side of the switch summarizes the system status of each 16-port
card, each Control Processor Card, and each power supply module. LEDs on the blowers show the status
of the blower assemblies.
1-4 Publication Number 53-0000251-02
Introducing the SilkWorm 12000
Brocade’s Fabric Watch exposes enhanced status reporting capabilities of the SilkWorm switches
through all the standard management interfaces, including SNMP, the Fabric Access Layer API, Brocade
Web Tools, Fabric Manager, and the command line interface.
Intelligent Fabric Services Architecture
Fabric OS v4.0 (required for the Silk Worm 12000) provides a wide variety of Advanced Fabric Services
that are designed to improve the investment protection, security, performance, scalability, and efficiency
of Brocade SAN fabrics.
Features of Fabric OS v4.0 include Trunking, Advanced Zoning, and Advanced Performance
Monitoring. These features, some of which are also available in previous versions of Fabric OS, are
discussed in the following pages.
Advanced Performance Monitoring
Advanced Performance Monitoring (available on 2 Gbit/sec switches) can monitor performance
characteristics on any attribute within the first 64 bytes of a Fibre Channel frame. Predefined graphs
measure end-to-end performance; port, switch, and AL-PA bandwidth utilization; SCSI commands
(read, write, and read/write); and protocol comparisons (SCSI versus IP). As a result, performance
monitoring provides the foundation for performance tuning, resource optimization, service level
agreement compliance reporting, and bill-back applications.
1
Advanced Zoning
Advanced Zoning software limits access to data by segmenting a fabric into virtual private SANs. On
1 Gbit/sec and 2 Gbit/sec switches, software-enforced zoning prevents hosts from discovering
unauthorized target devices. Hardware-enforced zoning prevents a host from accessing a device that is
unauthorized. This provides the most secure zoning available. In addition, Advanced Zoning on
2 Gbit/sec switches enables hardware enforcement for devices identified by W orld Wide Name (WWN).
This is new functionality that was not available in the SilkWorm 2000 series switches, which could only
do soft WWN zoning. With WWN zoning, zone enforcement adjusts automatically, even if a device
moves to another port. This new zoning model allows for the continued flexibility that traditional
software-enhanced zoning provides plus garners the security benefits of legacy hardware-enforced
zoning.
Extended Fabrics
Extended Fabrics software enables long distance (100km) ISLs over dark fiber or Dense Wave Division
Multiplexing (DWDM) connections at full bandwidth.
Publication Number 53-0000251-02 1-5
1
Introducing the SilkWorm 12000
Fabric Watch
Fabric Watch software enables organizations to set thresholds and ranges for SAN fabrics, and raise
management alerts when performance or errors vary outside predefined ranges.
ISL Trunking
ISL Trunking increases the performance and availability of links between 2 Gbit/sec switches and
minimizes the SAN management effort. Up to four 2 Gbit/sec ISLs between two switches can be
combined into a single trunk, or logical ISL, at 8 Gbit/sec. Traffic is load balanced across all the links
(i.e. any traffic can go across any available trunk link). This is an improvement over current static routing
and load sharing where servers are allocated individual dedicated links.
QuickLoop/Fabric Assist (QLFA)
QuickLoop/Fabric Assist (QLF A) connects private loop hosts to the SAN fabric for better performance
and fault management, while protecting investments in legacy loop devices. Because many legacy
devices are designed for FC-AL configurations, Fabric OS translative mode protects investments by
supporting private loop target devices. The SilkW orm 12000 running Fabric OS v4.0 supports translative
mode. Therefore switches that do support QuickLoop or Fabric Assist can be connected to a SilkWorm
12000, even though the SilkWorm 12000 does not support QuickLoop or Fabric Assist directly. It is
possible to connect devices that are accessed by QuickLoop/Fabric Assist devices to the SilkWo rm
12000. This means that any type of target device may be attached to a switch running Fabric OS v4.0 and
may be included in a QuickLoop Fabric Assist zone that has its private host attached to a switch running
QuickLoop and Zoning. QuickLoop and Zoning are pre-requisites for QLFA, on Fabric OS v2.3 or later
(SilkWorm 2xxx) or v3.0.1 or later (SilkWorm 3800/3200).
1-6 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
This chapter includes the following sections:
• Fabric OS 4.0 on page 2-1
• Dual Switch Model on page 2-2
• Port Addressing and Area Numbering on page 2-5
• Compatibility on page 2-10
• Software High Availabil ity Model on page 2-11
Note: All topics in this section establish a foundation for further discussions regarding the design,
operation, and management of the SilkWorm 12000 and SilkWorm 12000 based SANs. Where
appropriate, detail is provided in this section to identify key changes between the SilkWorm
12000 and previous SilkWorm switch models or to highlight key architectural features.
2
The SilkWorm 12000 utilizes embedded Linux as its underlying operating system, however all SAN
management is still performed on the Fabric OS level. While the impact to previous users of SilkWorm
switches is nominal, it is important to note what has changed, and what prompted these changes. The
dual switch model introduces several new concepts that are important to understand for the design,
deployment, and maintenance of the SilkWorm 12000 and SilkWorm 12000-based SANs. The 16-port
card design of the SilkWorm 12000 introduces a new “slot” operand to many commands, and a new
model for port identification that should be understood for effective operation of the SilkWorm 12000.
The SilkWorm 12000 is compatible with all Brocade switch models and is interoperable with switches
from vendors such as McData. To enable this compatibility, you must change certain configuration
settings before connecting other switches to the SilkWorm 12000. Finally , the software high availability
architecture is discussed.
Fabric OS 4.0
Fabric OS v4.0 is built upon a real-time version of Linux version 2.4. Linux was chosen due to industry
wide support for hardware and software, portability, and scalability. Figure 2-1 shows the screen output
when booting the switch. Notice the references to Linux.
Publication Number 53-0000251-02 2-1
2
SilkWorm 12000 Architecture and What Is New
The system is coming up, please wait...
Checking system RAM - press any key to stop test
Checking memory address: 08000000
System RAM check complete
Press escape within 4 seconds to enter boot interface.
Entry point at 0x00400000 ...
Loading Initial RAM disk
Uncompressing Linux... done.
read_silkworm_bdinfo: silkworm->board = 10, silkworm->board_rev =2
id mach(): done
Physical Memory: 0x08000000 Used memory = 0x07f00000
Linux version 2.4.2-mvista_010329 (swrel@nermal) (gcc version 2.95.3 20010112
(prerelease)) #1 Tue Dec 11 00:39:11 PST 2001
Figure 2-1 SilkWorm 12000 Boot Up With Fabric OS v4.0
Fabric OS 4.0 is a superset of previous versions of Fabric OS, so most commands used with previous
versions as well as several new commands are available.
Warning: The SilkWorm 12000 is equipped with a Root account intended for diagnostics and
debugging purposes solely by the trained engineers of the equipment vendor. Improper use of the
functionality made available through the Root account, such as treating the SilkWorm 12000 like
a standard Linux system and using the Linux functions and commands, can cause significant
harm and disruption to the operation of the SAN fabric. During normal operation, log in to the
switch as the “admin” user. The “admin” user utilizes a res tricted shell with access to Fabric OS
commands only.
Dual Switch Model
The SilkWorm 12000 houses two separate logical switches within a single chassis. Each switch is
capable of scaling to 64 ports by adding up to four 16-port cards to the respective logical switch. It is
possible to interconnect the two switches inside a chassis to form a fabric. Each logical switch has its
own unique domain ID, WWN, and IP address. Each switch in the chassis is an entity accessible through
telnet and other methods, using the unique IP address of that switch. The switches are numbered zero and
one, as shown in Figure 2-2. You can also access a switch using a serial connection for installation, such
as for setting the IP address of the switch or for diagnostic purposes.
2-2 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
2
Logical
Switch 0
Logical
Switch 1
cp0 cp1
Figure 2-2 SilkWorm 12000 Dual-Switch Model
Dual Control Processors For High Availability
The dual control processors (CP) operate in an active/standby model. The CP cards are named “cp0” (the
CP card on the left, when looking at the chassis) and “cp1” (the CP card on the right when looking at the
chassis), as shown in Figure 2-2. Each CP card is assigned an IP address for maintenance and diagnostic
purposes. Cp0 is located in slot 5 and cp1 is located in slot 6. One CP card is active and manages both
switches. The other CP card is in standby mode and will become active when a failover is required or a
forced failover occurs. Both CP cards should run the same version of Fabric OS. When configured for a
single switch, meaning only one switch is populated with 16-port cards, it is still necessary to have two
CP cards to maintain availability should one CP card fail. Normally, cp0 is given preference to become
the active CP card and cp1 operates as the standby CP card. In the initial release of Fabric OS v4.0, when
a failover occurs, it takes approximately thirty seconds before the standby CP card becomes active. The
use of a dual fabric solution can mitigate or eliminate the impact of this failover period. During the
failover period, I/O is not possible and attached devices must log back into the fabric and re-authenticate.
In properly designed high availability SANs, the real effect on I/O can be limited to as little as four
seconds. The next release of Fabric OS v4.1 is planned to support completely non-disruptive updates to
minimize path interruption.
Publication Number 53-0000251-02 2-3
2
SilkWorm 12000 Architecture and What Is New
The dual CP card model and the concept of logical switches is a change from past SilkWorm switch
implementations. The SilkWorm 2000 and 3000 families of 8-port and 16-port switches all had a static
relationship between the processor and the switch. Now the switch and the processor are de-coupled.
One implication of this model is that instead of downloading firmware to a switch, it is necessary to
download firmware to a CP card.
Note: The time it takes to activate the standby CP, when a failover occurs, will be considerably less
when using Fabric OS 4.1 and greater.
Accessing the SilkWorm 12000 Switches
When accessing SilkWorm 12000 switches, it is possible to access either switch by its respective IP
address or by using a serial connection to the active CP card. It is possible to access a CP card by telnet
using the respective CP card’s IP address or by connecting a serial cable to the CP card. Access to the CP
card should be limited to: installation purposes, setting a switch’s IP address, doing firmware
maintenance (i.e., downloads), or for diagnostic purposes.
When telnetting to an inactive CP card, the user is entered into a limited access environment where no
access to a switch is possible. When a user accesses an active CP card, that user will have access to the
full Fabric OS environment. To determine a CP card state, whether inactive or active, use the command
haShow (see Figure 2-3). When telnetting to a SilkWorm 12000 switch, the user will encounter a login
prompt from the active CP card. This may seem confusing, since the destination is a switch and not the
CP card; however, once logged in, the user is then placed into the tar get switch environment, as shown in
Figure 2-4.
Note: While it is possible to access the switches via the active CP card, either using an Ethernet address
or serial connection, primary access to the switches should be via Ethernet to the switch and not
the CP card.
sun1# telnet sw0_156_22
Trying 192.168.156.22...
Connected to sw0_156_22.
Escape character is '^]'.
Fabric OS (cp1)
cp1 login: admin
Password:
sw0_156_22:admin> hashow
Local CP (Slot 6, CP1): Active
Remote CP (Slot 5, CP0): Standby
HA Enabled, Heartbeat Up
Figure 2-3 Determining a CP Card State Using the haShow Command. CP1 is Active.
sun1# telnet sw1_156_23
Trying 192.168.156.23...
Connected to sw1_156_23.
Escape character is '^]'.
Fabric OS (cp1)
cp1 login: admin
Password:
sw1_156_23:admin>
Figure 2-4 SilkWorm 12000 Log In
2-4 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
2
How Logical Switch Behavior Differs
The behavior of several commands have changed in Fabric OS v4.0 to account for the dual CP card
architecture. For example, the command reboot now will reboot the active CP card and both logical
switches if issued from a switch. This happens since the logical switches run on the active CP card and
the reboot command will cause the active CP card to reboot and a failover to the standby CP card will
occur. There is a new command in Fabric OS v4.0 that should be used to reboot a switch. This command
is called switchReboot and this command will only affect that switch from which the command is issued.
Note: Use the command switchReboot to reboot a switch. Use of the reboot command from a logical
switch will result in the reboot of the active CP card, causing both logical switches to failover to
the standby CP card.
Also, users and their passwords are now associated with a chassis. This means that the user/password
pairs are the same for both logical switches and the CP cards. If the password for user admin is changed
on switch 0, the password will also be changed for switch 1 and the CP cards.
Note: Some commands, such as passwd, are chassis-wide in scope and affect both logical switches.
Port Addressing and Area Numbering
Port addressing is different for the SilkWorm 12000 than with the SilkWorm 2000 and 3000 families of
8-port and 16-port switches. The change in port addressing is driven by several factors, including the
high port density of the SilkWorm 12000, the need to eliminate ambiguity, to enable consistent marking
of port numbers on the 16-port card, and to accommodate future cards that may implement varyi ng port
densities. The physical ports on the 16-port cards are numbered zero through fifteen from bottom to top
and up to four 16-port cards can comprise a logical switch. It is necessary to relate a physical port
number to a card to uniquely identify that port. Port oriented commands, such as portShow, now require
that the slot be specified so that a port can be uniquely identified. The syntax is command slot/
port, as follows in Figure 2-5.
Publication Number 53-0000251-02 2-5
2
SilkWorm 12000 Architecture and What Is New
sw0_156_22:admin> portShow 1/7
portCFlags: 0x0
portFlags: 0x20801 PRESENT DISABLED LED
portType: 1.1
portState: 2 Offline
portPhys: 5 No_Sync
portScn: 2 Offline
portId: 160700
portWwn: 20:07:00:60:69:80:04:a0
Distance: normal
portSpeed: 2Gbps
Interrupts: 107 Link_failure: 1 Frjt: 0
Unknown: 13 Loss_of_sync: 10 Fbsy: 0
Lli: 31 Loss_of_sig: 0
Proc_rqrd: 72 Protocol_err: 0
Timed_out: 0 Invalid_word: 0
Rx_flushed: 0 Invalid_crc: 0
Tx_unavail: 0 Delim_err: 0
Free_buffer: 0 Address_err: 8
Overrun: 0 Lr_in: 0
Suspended: 0 Lr_out: 0
Parity_err: 0 Ols_in: 0
2_parity_err: 0 Ols_out: 0
CMI_bus_err: 0
Figure 2-5 Port Related Commands Require Input Of Slot Number/Port Number
Indirectly affected by the new port-addressing scheme are commands that reference a port ID, which is
the 24-bit fabric address, assigned by the switch. Some examples of commands that are indirectly
affected by the change in port ID are nsShow, nsAllShow, and portLogDump. Knowing the port ID of a
device enables the decoding of the physical location of a particular device. The previous method for
decoding a port ID for the 8-port and 16-port SilkWorm 2000 and 3000 family switches is shown in
Figure 2-6.
This is the Port Addressing Format:
0 x XX 1Y ZZ
where:
• XX is a value between 0x1 to 0xef inclusive and indicates the doma in ID of the switch to
which the device is attached
• The “1” will always be there in 2000 series & 3800 switches
• Y is the port number (0-F hex) that the device is attached
• ZZ is the AL_PA for a FL_Port or 00 for an F_Port
Example: 021500
where:
XX=02 Domain_ID of the switch
Y=5 Port #
ZZ=00 an F_Port
Figure 2-6 Decoding Port ID For Fabric OS v2.x and 3.x
2-6 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
The port-addressing scheme for Fabric OS v4.0 is summarized in Figure 2-7.
This is the Port Addressing Format:
0 x WW XY ZZ
where:
• WW is a value between 0x1 to 0xef inclusive and indicat es the dom a in id of the sw itch to
which the device is attached
• X is the logical port card number
• Y is the port number (0-F hex) that the device is attached
• ZZ is the AL_PA for a FL_Port (Loop) or 00 for an F_Port
Example 1: 170f00
where:
WW =23 Domain_ID of the switch
X= logical port card 0
Y= port number 15 (0xf)
ZZ=00 an F_Port.
2
Example 2: 162ed2
where:
WW =22 Domain_ID of the switch
X= logical port card 2
Y= port number 14 (0xe)
ZZ=d2 ALPA (FL_Port)
Figure 2-7 Decoding Port ID For Fabric OS v4.0
Since the port-addressing scheme has changed for the SilkWorm 12000, so has the decoding for a
particular port ID. The concept of area number is new in the SilkWorm 12000. The area number is used
in the same way a port number is used for the SilkWorm 2000 series and 3800 switches. When
specifying zoning configurations by port number it is necessary to utilize the area number. Also several
commands, such as switchShow or nsShow, specify area number in the output (see Figure 2-8).
Publication Number 53-0000251-02 2-7
2
SilkWorm 12000 Architecture and What Is New
sw1_156_23:admin> switchshow
switchName: sw1_156_23
switchType: 10.1
switchState: Online
switchRole: Subordinate
switchDomain: 23
switchId: fffc17
switchWwn: 10:00:00:60:69:80:04:a1
switchBeacon: OFF
blade7: Beacon: OFF
blade8: Beacon: OFF
Area Slot Port Gbic Speed State
=====================================
0 7 0 id N2 Online E-Port 10:00:00:60:69:50:09:2b "sw1"
(upstream)
1 7 1 id N2 Online E-Port 10:00:00:60:69:50:09:2b "sw1"
2 7 2 -- 2G No_Module
3 7 3 id 2G No_Light
4 7 4 id 2G No_Sync Disabled
5 7 5 -- 2G No_Module
6 7 6 -- 2G No_Module
7 7 7 -- 2G No_Module
8 7 8 -- 2G No_Module
9 7 9 -- 2G No_Module
10 7 10 -- 2G No_Module
11 7 11 -- 2G No_Module
12 7 12 -- 2G No_Module
13 7 13 -- 2G No_Module
14 7 14 -- 2G No_Module
15 7 15 -- 2G No_Module
16 8 0 id N2 Online F-Port 10:00:00:00:c9:27:2c:fe
17 8 1 id N2 Online F-Port 10:00:00:00:c9:28:c8:43
18 8 2 id N1 Online F-Port 20:00:00:60:16:3c:9f:16
19 8 3 id N1 Online F-Port 20:00:00:60:16:3c:9e:e8
20 8 4 -- 2G No_Module
21 8 5 -- 2G No_Module
a. Output trunctuated to fit in diagram
a
Figure 2-8 SwitchShow Command Output Specifies Area Number
Each logical switch consists of four slots and up to four 16-port cards. Both logical switches consist of
logical switch port cards 0 through 3 (see Table 2-2).
To calculate the area number of a port in the SilkWorm 12000, multiply the switch logical port card
number by sixteen, add the port number, and convert the value to hexadecimal. SeeTable 2-1 for a
complete map of physical ports to addresses.
Example:
12000 area number = logical port card number * 16 + port number
convert the value obtained above to hexadecimal
The calculation of a physical switch slot is obtained by adding one to the logical port card number for
switch 0 and seven to the logical port card number for switch 1.
Example:
switch 0 physical slot number = logical port card number + 1
switch 1 physical slot number = logical port card number + 7
2-8 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
In addition to the calculation method, you can determine the area by looking it up in Table 2-1.
Table 2-1 Logical Port Card Numbers
Port 0 1 2 3
15 0F 1F 2F 3F
14 0E 1E 2E 3E
13 0D 1D 2D 3D
12 0C 1C 2C 3C
11 0B 1B 2B 3B
10 0A 1A 2A 3A
909192939
808182838
707172737
606162636
505152535
2
404142434
303132333
202122232
101112131
000102030
Note: If you look at the left-most digit in the Area, that corresponds to the logical port card. The rightmost digit corresponds to the port.
One method to derive the logical switch number (either 0 or 1) is to use the “myid” command:
Example 1:
switch0:root> myid
Current Switch: switch0
Session Detail: Console Port (/dev/ttyS0) Active Redundant
Example 2:
switch0:root> myid
Current Switch: switch0
Session Detail: Console Port (/dev/ttyS0) Standby Redundant
Example 3:
switch1:root> myid
Current Switch: switch0
Session Detail: switch1 (192.168.148.30) Active Redundant
Example 4:
switch0:root> myid
Current Switch: switch1
Session Detail: cp1 (192.168.148.32) Active Non-Redundant
Publication Number 53-0000251-02 2-9
2
SilkWorm 12000 Architecture and What Is New
Additionally, if meaningful switch names are used, you can simply look at the telnet prompt. In the
above example, the switch name is “sw0” which intuitively tells you that you are on switch 0.
All of the above mentioned relationships are summarized in Table 2-2.
Table 2-2 Logical Switch Numbers
Physical Slot Logical Port
Logical Switch 0 1 0
21
32
43
Logical Switch 1 7 0
81
92
10 3
Card
Compatibility
The SilkWorm 12000 is compatible with all previous versions of SilkWorm switches including the
SilkW orm 2000 and 3000 families of switches. The SilkW orm 12000 is interoperable with switches from
other vendors, such as McData. To enable compatibility with 8-port and 16-port SilkWorm 3000 series
switches, it is necessary to run Fabric OS 3.0.2c or later on these switches, and Fabric OS v2.6 or later
for SilkWorm 2000 series switches.
When connecting a switch from the SilkWorm 2000 and 3000 product family to a SilkWorm 12000, you
must enable the Core Switch PID Format parameter. You can do this using the configure command as
shown in Figure 2-9. If you do not enable the Core Switch PID Format parameter (by setting the value to
1), the fabric will segment when connecting these switches to a SilkWorm 12000.
Note: When deploying the SilkWorm 12000 into existing fabrics that also include SilkW orm 2000 and
3000 series switches, it is necessary to change the Core Switch PID format setting on those
switches. Doing so may have an impact on existing applications, as enabling this setting changes
the 24-bit address. A dual fabric architecture can mitigate or eliminate downtime if it is
necessary to change the core PID format.
2-10 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
sw3:admin> switchDisable
sw3:admin> configure
Configure...
Fabric parameters (yes, y, no, n): [no] y
Domain: (1..239) [3]
BB credit: (1..27) [16]
R_A_TOV: (4000..120000) [10000]
E_D_TOV: (1000..5000) [2000]
Data field size: (256..2112) [2112]
Sequence Level Switching: (0..1) [0]
Disable Device Probing: (0..1) [0]
Suppress Class F Traffic: (0..1) [0]
SYNC IO mode: (0..1) [0]
VC Encoded Address Mode: (0..1) [0]
Core Switch PID Format: (0..1) [0] 1
Per-frame Route Priority: (0..1) [0] ^D
Committing configuration...done.
sw3:admin> switchEnable
2
Figure 2-9 Enabling Compatibility Between the SilkWorm 12000 and other SilkWorm
Switches
To enable compatibility between a SilkWorm 12000 switch and another vendor’s switch, it is necessary
to invoke interoperability mode. Operating the Brocade switch in this mode places significant
restrictions on the Brocade SilkWorm 12000 switch. Functionality such as Hard Zoning, Extended
Fabrics, and Virtual Channels will not be available. Refer to the Fabric OS Procedures Guide for
information on procedures and restrictions when implementing interoperability mode.
Software High Availability Model
The high availability software architecture of the SilkWorm 12000 provides a common framework for all
applications that reside on it, enabling global and local states to be maintained such that any component
failure is fully manageable. High availability elements consist of the High Availability Manager, the
heartbeat, the fault/health framework, the replicated database, initialization, and software upgrade.
The High Availability Manager (HAM) uses an Active-Standby model. HAM controls access to the
standby CP card, facilitates software upgrades, prevents extraneous switchover activity, closes and
flushes streams as needed, provides flow control and message buffering, and supports a centralized
active and standby state allowing the Switch of Activity (SWACT) to be controlled from a single point.
Failover Overview
The two methods used to notify each CP card of the health of the other are a network based heartbeats
and hardware handshaking control lines. HAM manages the IP address used to access each logical
switch and the standby CP card. Figure 2-10 is a block diagram of the failover process. In the case of
Figure 2-10, the command haFailover is used to cause a failover. CP0 is the HA master (that is, the
active CP card) and all the switch applications running on it are controlling the fabrics for switch 0 and
switch 1. CP1 is in standby mode and the applications are in wait for active mode. The administrator
Publication Number 53-0000251-02 2-11
2
SilkWorm 12000 Architecture and What Is New
issues the haFailover command and the HAM demon on CP0 reboots the kernel. The HAM demon on
CP1 is notified of the loss of CP0 through the hardware control lines, changes it’s state from standby to
master, configures the switch IP addresses for it’s LAN interface and notifies the switching applications
that they are now running as active on CP1.
Figure 2-10 Block Diagram of the High Availability Fail Over Process
Failover Details
The failover process is essentially invisible to the switch administrator. Whether the active CP card is
CP0 or CP1 really has no bearing on the operation of the switches. All switch information, such as the
switch configuration, zoning, SNMP settings, Fabric Watch settings, etc. is preserved and independent of
the CP card. When a CP card failure occurs, there is some impact to the switches and the associated
fabrics. In the initial release of Fabric OS v4.0, when a failover occurs, it takes approximately twenty
seconds before the standby CP card becomes active. The use of a dual fabric solution can mitigate or
eliminate the impact of this failover period. During the failover period, I/O is not possible and attached
devices must log back into the fabric and re-authenticate. The next release of Fabric OS 4.x is planned to
support completely non-disruptive updates to minimize path interruption. For smaller fabrics, consisting
of a few switches, the fabric becomes operational (meaning I/O can resume) once the CP becomes
2-12 Publication Number 53-0000251-02
SilkWorm 12000 Architecture and What Is New
active. For larger fabrics, it may take longer for the fabric to become operational since the higher number
of switches in the fabric necessitates a longer convergence period before the fabric becomes operational.
At the time a failover occurs, all devices and ISLs connected to switch 0 and switch 1 lose their link until
the failover completes. Also any active telnet sessions to the switches or the CP cards are disconnected.
Note: The time it takes to activate the standby CP, when a failover occurs, will be considerably less
when using Fabric OS 4.1 and greater.
2
Publication Number 53-0000251-02 2-13
2
SilkWorm 12000 Architecture and What Is New
2-14 Publication Number 53-0000251-02
Loading...