HP HyperFabric Administrator's Guide

Download

HyperFabric Administrator’s Guide

HP-UX 11i v2

Edition 14

Manufacturing Part Number : B6257-90043

March 2004

United States

Legal Notices

The information in this document is subject to change without notice.

Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and ﬁtness for a particular purpose. Hewlett-Packard

shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material.

Warranty

A copy of the speciﬁc warranty terms applicable to your Hewlett-Packard product and replacement parts can be obtained from your local Sales and Service Ofﬁce.

U.S. Government License

Proprietary computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Copyright  2004 Hewlett-Packard Development Company L.P. All rights reserved. Reproduction, adaptation, or translation of this document without prior written permission is prohibited, except as allowed under the copyright laws.

Trademark Notices

Oracle  is a registered trademark of Oracle Corporation. UNIX is a registered trademark in the United States and other

countries, licensed exclusively through The Open Group.

About This Document

1. Overview of the HyperFabric Product

About HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

HyperFabric Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Switches and Switch Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Other Product Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

HyperFabric Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2. Planning the Fabric

Preliminary Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and

HMP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

TCP/UDP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Conﬁguration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

TCP/UDP/IP Supported Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Point-to-Point Conﬁgurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Switched Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

High Availability Switched Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Hybrid Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Hyper Messaging Protocol (HMP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

Conﬁguration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

HMP Supported Conﬁgurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Point-to-Point Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Enterprise (Database) Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Technical Computing (Work Stations) Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . 37

Contents

3. Installing HyperFabric

Checking HyperFabric Installation Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Installing HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

iii

Contents

Online Addition and Replacement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Planning and Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Critical Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Card Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Installing the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Loading the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Installing HyperFabric Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Before Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Installing the HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

With the Rail Kit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Without the Rail Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4. Conﬁguring HyperFabric

Conﬁguration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Information You Need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Conﬁguration Information Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Performing the Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Using the clic_init Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Examples of clic_init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Deconﬁguring a HyperFabric Adapter with SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Conﬁguring the HyperFabric EMS Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Conﬁguring HyperFabric with ServiceGuard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

How HyperFabric Handles Adapter Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Conﬁguring HyperFabric with the ServiceGuard Resource Monitor. . . . . . . . . . . 94

Conﬁguring ServiceGuard with HyperFabric Using the ASCII File . . . . . . . . . . . 94

Conﬁguring ServiceGuard with HyperFabric Using SAM . . . . . . . . . . . . . . . . . . . 94

Conﬁguring ServiceGuard for HyperFabric Relocatable IP Addresses . . . . . . . . . 95

Conﬁguring HMP for Transparent Local Failover Support. . . . . . . . . . . . . . . . . . . . . . 96

How Transparent Local Failover Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Conﬁguring HMP for Transparent Local Failover Support - Using SAM. . . . . . . . 102

Deconﬁguring HMP for Local Failover support - Using SAM . . . . . . . . . . . . . . . . . 103

Conﬁguring HMP for Transparent Local Failover Support - Using the clic_init

command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

5. Managing HyperFabric

Starting HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Using the clic_start Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Verifying Communications within the Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

The clic_probe Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Examples of clic_probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Displaying Status and Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

The clic_stat Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Examples of clic_stat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Viewing man Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Stopping HyperFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Using the clic_shutdown Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

6. Troubleshooting HyperFabric

Running Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

The clic_diag Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Example of clic_diag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Using Support Tools Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Useful Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

LED Colors and Their Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

HF2 Switch LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Determining Whether an Adapter or a Cable is Faulty. . . . . . . . . . . . . . . . . . . . . . . . 153

Determining Whether a Switch is Faulty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Replacing a HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Replacing a HyperFabric Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Safety Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Regulatory Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Adapters and Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

FCC Statement (USA only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

DOC Statement (Canada only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Europe RFI Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Australia and New Zealand EMI Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Contents

Radio Frequency Interference (Japan Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Declarations of Conformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Physical Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Environmental . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Figures

Figure 2-1. TCP/UDP/IP Point-To-Point Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . 22

Figure 2-2. TCP/UDP/IP Basic Switched Conﬁguration . . . . . . . . . . . . . . . . . . . . . . 23

Figure 2-3. TCP/UDP/IP High Availability Switched Conﬁguration . . . . . . . . . . . . 24

Figure 2-4. TCP/UDP/IP Hybrid Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 2-5. HMP Point-To-Point Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Figure 2-6. HMP Enterprise (Database) Conﬁguration, Single Connection Between

Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Figure 2-7. Local Failover Supported Enterprise (Database) Conﬁguration, Multiple

Connections between Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Figure 2-8. Technical Computing Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Figure 2-9. Large Technical Computing Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . 39

Figure 3-1. HyperFabric File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Figure 3-2. Front of HF2 Switch (A6388A Switch Module Installed) . . . . . . . . . . . . 59

Figure 3-3. Front of HF2 Switch (A6389A Switch Module Installed) . . . . . . . . . . . . 60

Figure 4-1. Map for Conﬁguration Information Example . . . . . . . . . . . . . . . . . . . . . 74

Figure 4-2. An ServiceGuard Conﬁguration (with Two HyperFabric Switches) . . . 89

Figure 4-3. Node with Two Active HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . 91

Figure 4-4. Node with One Failed HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . 92

Figure 4-5. When All HyperFabric Adapters Fail. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Figure 4-6. A Conﬁguration supporting Local Failover . . . . . . . . . . . . . . . . . . . . . . . 98

Figure 4-7. Adapter, Link or Switch Port Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Figure 4-8. Switch Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Figure 4-9. Cable Failover Between Two Switches . . . . . . . . . . . . . . . . . . . . . . . . . 101

Figure 4-10. Conﬁguring the Transparent Local Failover feature . . . . . . . . . . . . . 104

vii

Figures

viii

Tables

Table 1. HP-UX 11i Releases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Table 2. Publishing History Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Table 3. Organization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Table 2-1. HF2 Throughput and Latency with TCP/UDP/IP Applications. . . . . . . . 19

Table 2-2. Supported HyperFabric Adapter Conﬁgurations . . . . . . . . . . . . . . . . . . . 20

Table 2-3. HF2 Throughput and Latency with HMP Applications . . . . . . . . . . . . . . 31

Table 2-4. Supported HyperFabric Adapter Conﬁgurations . . . . . . . . . . . . . . . . . . . 31

Table 3-1. Important OLAR Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Table 6-1. LED Names (by Adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Table 6-2. HyperFabric Adapter LED Colors and Meanings . . . . . . . . . . . . . . . . . . 145

Table 6-3. HF2 Switch LED Colors and Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . 150

Tables

About This Document

This document describes how to install, conﬁgure, and troubleshoot the HyperFabric product on the HP-UX 11i v2 (HP-UX 11.23) operating system.

Before you install the HyperFabric software, ensure that the operating system software and the appropriate ﬁles, scripts, subsets are installed.

The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The document part number will change when extensive changes are made.

Document updates may be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, you should subscribe to the appropriate product support service. See your HP sales representative for details.

The latest version of this manual can be found online at

http://www.docs.hp.com/hpux/netcom/index.html#HyperFabric.

Intended Audience

This manual is intended for system and network administrators responsible for installing, conﬁguring, and managing the HyperFabric software and hardware. Administrators are expected to have knowledge of operating system concepts, commands, and conﬁguration.

It is helpful to have knowledge of Transmission Control Protocol/Internet Protocol (TCP/IP) networking concepts and network conﬁguration.

This document is not a tutorial.

HP-UX Release Name and Release Identiﬁer

Each HP-UX 11i release has an associated release name and release identiﬁer. The uname (1) command with the -r option returns the release identiﬁer. Table 1 shows the releases available for HP-UX 11i.

Table 1 HP-UX 11i Releases

Supported

Release Identiﬁer Release Name

B.11.11 HP-UX 11i v1 PA-RISC

B.11.20 HP-UX 11i v1.5 Intel Itanium

B.11.22 HP-UX 11i v1.6 Intel Itanium

B.11.23 HP-UX 11i v2 Intel Itanium

Processor

Architecture

Publishing History

Table 2 provides the publication date for the pertinent edition number.

Table 2 Publishing History Details

Edition Number Publication Date

xii

First March 1998

Second June 1998

Third August 1998

Fourth October 1998

Fifth December 1998

Sixth February 1999

Seventh April 1999

Eighth March 2000

Ninth June 2000

Tenth December 2000

Table 2 Publishing History Details (Continued)

Edition Number Publication Date

Eleventh June 2001

Twelfth September 2002

Thirteenth July 2003

Fourteenth March 2004

What Is in This Document

HyperFabric Administrator’s Guide is divided into several chapters, each of which contains information about installing, conﬁguring, or troubleshooting HyperFabric. The appendixes contain supplemental information.

The following list describes the content in more detail.

Table 3 Organization

Chapter Description

Overview of the HyperFabric Product

Planning the Fabric Describes the steps that need to be

Installing HyperFabric Describes the tasks to install the

Conﬁguring HyperFabric Describes the tasks to conﬁgure

Managing HyperFabric Describes the tasks to start, stop and

Troubleshooting HyperFabric

Presents an overview of HyperFabric and lists the components that the HyperFabric product contains

followed while planning the fabric

HyperFabric products on the HP-UX 11i v2 operating system

HyperFabric

manage HyperFabric

Describes how to troubleshoot HyperFabric

xiii

New and Changed Information in This Edition

This edition includes information about the transparent local failover feature of Hyper Messaging Protocol (HMP). This feature is available with the HyperFabric version B.11.23.01.

Typographic Conventions

This document uses the following typographic conventions: Book Title Italic (slanted) type indicates document and book

names.

daemon Courier font type indicates daemons, ﬁles, commands,

manpages, and option names.

[ ] { } In syntax deﬁnitions, square brackets indicate items

that are optional and braces indicate items that are required.

HP Welcomes Your Feedback

HP welcomes any comments and suggestions you have on this manual. You can send your comments in the following ways:

• Internet electronic mail: netinfo_feedback@cup.hp.com

xiv

• Using a feedback form located at the following URL:

http://docs.hp.com/assistance/feedback.html

Please include the following information along with your comments:

• The complete title of the manual and the part number. (The part number appears on the title page of printed and PDF versions of a manual.)

• The section numbers and page numbers of the information on which you are commenting.

• The version of HP-UX that you are using.

Please note that the HP-UX networking communications publications group does not provide technical support for HP products.

1 Overview of the HyperFabric

Product

This chapter contains the following sections that give general information about HyperFabric:

Chapter 1 1

Overview of the HyperFabric Product

• “About HyperFabric” on page 3

• “HyperFabric Products” on page 4

• “HyperFabric Concepts” on page 7

Chapter 12

Overview of the HyperFabric Product

About HyperFabric

HyperFabric is an HP high-speed, packet-based interconnect for node-to-node communications. HyperFabric provides higher speed, lower network latency and uses less CPU than other industry standard protocols (for example, Fibre Channel and Gigabit Ethernet). Instead of using a traditional bus-based technology, HyperFabric is built around switched fabric architecture, providing the bandwidth necessary for high speed data transfer. This clustering solution delivers the performance, scalability, and high availability required by the following:

• Parallel Database Clusters

— Oracle 9i Real Application Clusters (RAC) — Oracle 8i Parallel Servers (OPS)

• Parallel Computing Clusters

• Client/Server Architecture Interconnects (for example, SAP)

• Multi-Server Batch Applications (for example, SAS Systems)

• Enterprise Resource Planning (ERP)

• Technical Computing Clusters

• HP Message Passing Interface (MPI) based applications

• OpenView Data Protector (earlier known as Omniback)

• Network Backup

• Data Center Network Consolidation

• E-services

Chapter 1 3

Overview of the HyperFabric Product

HyperFabric Products

HyperFabric hardware consists of host-based interface adapter cards, interconnect cables, and optional switches. HyperFabric software resides in Application Speciﬁc Integrated Circuits (ASICs) and ﬁrmware on the adapter cards and includes user-space components and HP-UX drivers.

Currently, ﬁber-based HyperFabric hardware are available. In addition, a hybrid switch that has 8-ﬁber ports is available to support HF2 clusters.

This section describes the various HyperFabric products. For more information on HP 9000 systems that support HyperFabric products, see the HyperFabric Release Notes, available at

http://docs.hp.com/hpux/netcom/index.html#HyperFabric.

NOTE This document uses the term HyperFabric (HF) to refer to the

hardware and software that form the HyperFabric cluster interconnect product.

The term HyperFabric2 (HF2) refers to the following ﬁber-based hardware components:

• The A6386A adapter

• The A6384A switch chassis

• The A6388A and A6389A switch modules. (Although the A6389A switch module has 4-copper ports, it is still considered an HF2 component because it can only be used with the A6384A HF2 switch chassis).

• The C7524A, C7525A, C7526A, and C7527A cables

HyperFabric Adapters

The HyperFabric adapters are as follows:

• A6386A HF2 PCI (4X) adapter with a ﬁber interface.

The A6092A HyperFabric adapter is supported beginning with the following HyperFabric software versions:

Chapter 14

Overview of the HyperFabric Product

HyperFabric Products

• HP-UX 11.0: HyperFabric software version B.11.00.09

• HP-UX 11i v1: HyperFabric software version B.11.11.00

• HP-UX 11i v2: HyperFabric software version B.11.23.00

The A6386A HyperFabric2 adapter is supported beginning with the following HyperFabric software versions:

• HP-UX 11.0: HyperFabric software version B.11.00.11

• HP-UX 11i v1: HyperFabric software version B.11.11.01

• HP-UX 11i v2: HyperFabric software version B.11.23.00

Switches and Switch Modules

The HyperFabric2 switches are as follows:

• A6384A HF2 ﬁber switch chassis with one integrated Ethernet management LAN adapter card, one integrated 8-port ﬁber card, and one expansion slot. For the chassis to be a functional switch, install one of the following switch modules in the expansion slot:

— The A6388A HF2 8-port ﬁber switch module. This gives 16-ﬁber

ports to the switch (8 from the integrated ﬁber card and 8 from the A6388A switch module).

— The A6389A HF2 4-port copper switch module. This gives 12

ports to the switch - a mixture of 8-ﬁber ports (from the integrated ﬁber card) and 4-copper ports (from the A6389A module).

The A6384A HF2 switch chassis with either module installed is supported beginning with the following HyperFabric software versions:

• HP-UX 11.0: HyperFabric software version B.11.00.11

• HP-UX 11i v1: HyperFabric software version B.11.11.01

• HP-UX 11i v2: HyperFabric software version B.11.23.00

NOTE In this manual, the terms HyperFabric2 switch or HF2 switch refer

to the functional switch (the A6384A switch chassis with one of the switch modules installed).

Chapter 1 5

Overview of the HyperFabric Product

HyperFabric Products

IMPORTANT HF2 adapters and switches are not supported by software versions

earlier than those listed in “HyperFabric Adapters” on page 4 and “Switches and Switch Modules” on page 5.

To determine the version of HyperFabric, issue the following command:

$ swlist | grep -i hyperfabric

Other Product Elements

The following are the other elements of the HyperFabric product family:

• HF2 ﬁber cables

— C7524A (2m length) — C7525A (16m length) — C7526A (50m length) — C7527A (200m length)

• The HyperFabric software: The software resides in ASICs and ﬁrmware on the adapter cards and includes user-space components and HP-UX drivers.

HyperFabric supports the IP network protocol stack, speciﬁcally TCP/IP and UDP/IP.

HyperFabric software includes HyperMessaging Protocol (HMP). HMP provides higher bandwidth, lower CPU overhead, and lower latency (the time a message takes to get from one point to another). However, these HMP beneﬁts are available only when applications that are developed on top of HMP are running. HMP can only be used on HP 9000 systems running HP-UX 11.0 or 11i v1, provided HyperFabric A6092A or A6386A (PCI 4X) adapter cards are installed on those systems.

Chapter 16

Overview of the HyperFabric Product

HyperFabric Concepts

This section brieﬂy describes some of the basic HyperFabric concepts and terms.

The fabric is the physical conﬁguration that consists of all the HyperFabric adapters, the HyperFabric switches (if any), and the HyperFabric cables connecting them. The network software controls data transfer over the fabric.

The HyperFabric conﬁguration contains two or more HP 9000 systems and optional HyperFabric switches. Each HP 9000 acts as a node in the conﬁguration. Each node has a minimum of one and a maximum of eight HyperFabric adapters installed in it. (For information on the maximum number of adapters that can be installed in each system, see Chapter 2, “Planning the Fabric,” on page 9.) Each HF2 switch can be conﬁgured with 12 or 16 ports. HyperFabric supports a maximum of 4 HyperFabric switches. You can mesh HyperFabric switches and conﬁgurations with up to four levels of meshed switches are supported.

You can plan a HyperFabric cluster as a High Availability (HA) conﬁguration, when it is necessary to ensure that each node can always participate in the fabric. This is done by using ServiceGuard (earlier known as MC/ServiceGuard), ServiceGuard OPS Edition (earlier known as MC/LockManager), and the Event Monitoring Service (EMS). Conﬁgurations of up to 8 nodes are supported under ServiceGuard.

Beginning with HyperFabric software versions B.11.00.05 and B.11.11.00,you can use relocatable IP addresses as part of an HA conﬁguration. Relocatable IP addresses permit a client application to reroute through an adapter on a remote node, allowing that application to continue processing without interruption. The rerouting is transparent. This function is associated with ServiceGuard (see “Conﬁguring ServiceGuard for HyperFabric Relocatable IP Addresses” on page 95). When the monitor for HyperFabric detects a failure and the backup adapter takes over, the relocatable IP address is transparently migrated to the backup adapter. Throughout this migration process, the client application continues to execute normally.

When you start HyperFabric (with the clic_start command, through SAM, or by booting the HP 9000 system), you start the management process. This process must be active for HyperFabric to run. If the HyperFabric management process on a node stops running for some

Chapter 1 7

Overview of the HyperFabric Product

HyperFabric Concepts

reason (for example, if it is killed), all HyperFabric-related communications on that node are stopped immediately. This makes the node unreachable by other components in the fabric.

When you start HyperFabric, the fabric is veriﬁed automatically. This is because each node performs a self diagnosis and veriﬁcation over each adapter installed in the node. In addition, the management process performs automatic routing and conﬁguring for each switch (if switches are part of the fabric). You can, if needed, run the clic_stat command to get a textual map of the fabric, which can be used as another method of quick veriﬁcation.

Notice that the commands to administer HyperFabric have a preﬁx of clic_, and some of the other components have CLIC as part of their name (for example, the CLIC ﬁrmware and the CLIC software). CLIC stands for CLuster InterConnect, and it is used to differentiate those HyperFabric commands or components from other commands or components. For example, the HyperFabric command clic_init is different from the HP-UX init command.

Chapter 18

2 Planning the Fabric

This chapter contains the following sections that include general guidelines and protocol-speciﬁc considerations for planning HyperFabric clusters that run TCP/UDP/IP or HMP applications.

Chapter 2 9

Planning the Fabric

• “Preliminary Considerations” on page 11

• “HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and HMP Applications” on page 13

• “TCP/UDP/IP” on page 14

• “Hyper Messaging Protocol (HMP)” on page 26

Chapter 210

Planning the Fabric

Preliminary Considerations

Before assembling a fabric physically, do the following to address all of the appropriate issues:

Step 1. Read Chapter 1, “Overview of the HyperFabric Product,” on page 1, to

get a basic understanding of HyperFabric and its components.

Step 2. Read this chapter, Planning the Fabric, to gain an understanding of

protocol speciﬁc conﬁguration guidelines for TCP/UDP/IP and HMP applications.

Step 3. Read “Conﬁguration Overview” on page 69, “Information You Need” on

page 71, and “Conﬁguration Information Example” on page 74, to understand how to conﬁgure the fabric.

Step 4. Decide the number of nodes that will be interconnected in the fabric.

Step 5. Decide the type of HP 9000 system for each node (for a list of supported

HP 9000 systems, see the HyperFabric Release Notes available at

http://docs.hp.com/hpux/netcom/index.html#HyperFabric.

Step 6. Determine the network bandwidth requirements for each node.

Step 7. Determine the number of adapters needed for each node.

Step 8. Determine if a High Availability (ServiceGuard) conﬁguration will be

needed. If ServiceGuard is used, each node should have at least two adapters.

Step 9. Decide the topology of the fabric.

Step 10. Determine how many switches will be used based on the number of

nodes in the fabric. The only conﬁguration that can be supported without a switch is the node-to-node conﬁguration (HA or non-HA). HyperFabric supports meshed switches up to a depth of four switches, starting with the following versions of the HyperFabric software:

• For HF2 switches: software versions B.11.00.11, B.11.11.01, and B.11.23.00.

Chapter 2 11

Planning the Fabric

Preliminary Considerations

Step 11. Draw the cable connections from each node to switches (if the fabric will

contain switches). If you use an HA conﬁguration with switches, it requires more than one switch for complete redundancy and to avoid a single point of failure. For example, each adapter can be connected to its own switch, or two switches can be connected to four adapters.

Chapter 212

Planning the Fabric

HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and HMP Applications

The following sections in this chapter deﬁne HyperFabric functionality for TCP/UDP/IP applications and Hyper Messaging Protocol (HMP) applications. There are distinct differences in supported hardware, available features and performance, depending on which protocol is used by applications running on the HyperFabric.

Chapter 2 13

Planning the Fabric

TCP/UDP/IP

TCP/UDP/IP is supported on all HF2 hardware. Although some of the HyperFabric adapter cards support both HMP and TCP/UDP/IP applications, in this section, the focus is on TCP/UDP/IP HyperFabric applications.

Application Availability

All applications, including Oracle 9i and HP-MPI, that use the TCP/UDP/IP stack are supported.

Features

This section discusses the following HyperFabric features on TCP/UDP/IP:

• OnLine Addition and Replacement (OLAR): Supported The OLAR feature allows the replacement or addition of

HyperFabric adapter cards while the system (node) is running. HyperFabric supports this functionality on the SD64A, rx8620, rx4640, rp54xx (L-class), rp74xx (N-class), rp8400 and Superdome systems, running on the HP-UX 11i v2 platform.

For more information on OLAR, including instructions for implementing this feature, see “Online Addition and Replacement” on page 44 and Conﬁguring HP-UX for Peripherals Part Number B2355-90698 November 2000 Edition.

• Event Monitoring Service (EMS): Supported In the HyperFabric version B.11.23.01, the HyperFabric EMS

monitor enables the system administrator to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator deﬁnes the condition to trigger a notiﬁcation (usually a change in interface status). Notiﬁcation can be accomplished with one of the following:

— A Simple Network Management Protocol (SNMP) trap

Chapter 214

Planning the Fabric

— Logging into a user speciﬁed log ﬁle with a choice of severity — Email to a user deﬁned email address. For more information on EMS, including instructions for

implementing this feature, see “Conﬁguring the HyperFabric EMS Monitor” on page 85 and the EMS Hardware Monitors User’s Guide Part Number B6191-90028 September 2001 Edition.

• ServiceGuard: Supported Within a cluster, ServiceGuard groups application services

(individual HP-UX processes) into packages. In the event of a single service failure (node, network, or other resource), EMS provides notiﬁcation and ServiceGuard transfers control of the package to another node in the cluster, allowing services to remain available with minimal interruption.

ServiceGuard via EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). ServiceGuard uses a heartbeat LAN to monitor the nodes in a cluster. ServiceGuard cannot use the HyperFabric interconnect as a heartbeat LAN. Instead, use a separate LAN for the heartbeat.

TCP/UDP/IP

For more information on conﬁguring ServiceGuard, see “Conﬁguring HyperFabric with ServiceGuard” on page 87, and Managing MC/ServiceGuard Part Number B3936-90065 March 2002 Edition.

• High Availability (HA): Supported To create a highly available HyperFabric cluster, there should not be

any single point of failure. Once the HP 9000 nodes and the HyperFabric hardware have been conﬁgured with no single point of failure, ServiceGuard and EMS can be conﬁgured to monitor and fail over nodes and services using ServiceGuard packages.

If any HyperFabric resource in a cluster fails (adapter card, cable or switch port), the HyperFabric driver transparently routes trafﬁc over other available HyperFabric resources with no disruption of service.

The ability of the HyperFabric driver to transparently fail over trafﬁc reduces the complexity of conﬁguring highly available clusters with ServiceGuard, because ServiceGuard has to take care of node and service failover only.

Chapter 2 15

Planning the Fabric

TCP/UDP/IP

ServiceGuard uses a “heartbeat” to monitor the cluster. The HyperFabric links cannot be used for the heartbeat. Instead, an alternate LAN connection such as 100BaseT, Ethernet, Token Ring, or FDDI must be made between the nodes for use as a heartbeat link.

End-To-End HA: HyperFabric provides end-to-end HA on the entire cluster fabric at the link level. If any of the available routes in the fabric fails, HyperFabric transparently redirects all the trafﬁc to a functional route and, if conﬁgured, notiﬁes ServiceGuard or other enterprise management tools.

Active-Active HA: In conﬁgurations where there are multiple routes between nodes, the HyperFabric software uses a hashing function to determine an adapter or a route through which it sends messages. This is done on a message-by-message basis. All of the available HyperFabric resources in the fabric are used for communication.

In contrast to Active-Passive HA, where one set of resources is not utilized until another set fails, Active-Active HA provides the best return on investment because all of the resources are utilized simultaneously. ServiceGuard is not required for Active-Active HA operation.

For more information on setting up HA HyperFabric clusters, see Figure 2-3 “TCP/UDP/IP High Availability Switched Conﬁguration” on page 24.

• Dynamic Resource Utilization (DRU): Supported If you add a new resource (node, adapter, cable or switch) to a cluster,

the HyperFabric subsystem dynamically identiﬁes the added resource and starts using it. The same process takes place when a resource is removed from a cluster. The difference between DRU and OLAR is that OLAR applies only to the addition or replacement of adapter cards from nodes.

• Load Balancing: Supported When an HP 9000 HyperFabric cluster is running TCP/UDP/IP

applications, the HyperFabric driver balances the load across all available resources in the cluster, including nodes, adapter cards, links, and multiple links between switches.

• Switch Management: Not Supported

Chapter 216

Planning the Fabric

TCP/UDP/IP

Switch Management is not supported. Switch management will not operate properly if you enable it on a HyperFabric cluster.

• Diagnostics: Supported Diagnostics can be run to obtain information on many of the

HyperFabric components using the clic_diag, clic_probe and clic_stat commands, as well as the Support Tools Manager (STM).

For more information on HyperFabric diagnostics, see “Running Diagnostics” on page 133.

Conﬁguration Parameters

This section describes the maximum limits for TCP/UDP/IP HyperFabric conﬁgurations. There are numerous variables that can impact the performance of any HyperFabric conﬁguration. For guidance on speciﬁc HyperFabric conﬁgurations for TCP/UDP/IP applications, see the section, “TCP/UDP/IP Supported Conﬁgurations” on page 21.

• HyperFabric is supported only on the HP 9000 series servers and workstations.

• TCP/UDP/IP is supported for all HyperFabric hardware and software.

• Maximum Supported Nodes and Adapter Cards In point-to-point conﬁgurations, the complexity and performance

limitations of having a large number of nodes in a cluster make it necessary to include switching in the fabric. Typically, point-to-point conﬁgurations consist of only 2 or 3 nodes.

In switched conﬁgurations, HyperFabric supports a maximum of 64 interconnected adapter cards.

A maximum of 8 HyperFabric adapter cards are supported per instance of the HP-UX operating system. The actual number of adapter cards a particular node is able to accommodate also depends on slot availability and system resources. See node speciﬁc documentation for details.

HyperFabric subsystem supports a maximum of 8 conﬁgured IP addresses per instance of the HP-UX operating system.

• Maximum Number of Switches

Chapter 2 17

Planning the Fabric

TCP/UDP/IP

You can interconnect (mesh) up to 4 switches (16-port fiber, or Mixed 8 fiber ports) in a single HyperFabric cluster.

• Trunking Between Switches (multiple connections) You can use trunking between switches to increase bandwidth and

cluster throughput. Trunking is also a way to eliminate a possible single point of failure. The number of trunked cables between nodes is limited only by port availability. To assess the effects of trunking on the performance of any particular HyperFabric conﬁguration, contact your HP representative.

• Maximum Cable Lengths HF2 (fiber): The maximum distance is 200m (Four standard cable

lengths are sold and supported: 2m, 16m, 50m and 200m). TCP/UDP/IP supports up to four HF2 switches connected in series

with a maximum cable length of 200m between the switches and 200m between switches and nodes.

TCP/UDP/IP supports up to four hybrid HF2 switches connected in series with a maximum cable length of 200m between ﬁber ports.

Chapter 218

Planning the Fabric

TCP/UDP/IP

• Throughput and Latency

Table 2-1 HF2 Throughput and Latency with TCP/UDP/IP Applications

Server Class Maximum Throughput Latency

rp7400 2 + 2 Gbps full duplex per link < 42 microsec

Chapter 2 19

Planning the Fabric

TCP/UDP/IP

Table 2-2 Supported HyperFabric Adapter Conﬁgurations

Adapter

A6386A PCI (4X) rx2600 servers 11i v2 No 1

A6386A PCI (4X) rx56XX

A6386A PCI (4X) zx6000

A6386A PCI (4X) SD64A servers 11i v2 Yes 8 (maximum

A6386A PCI (4X) rx7620 servers 11i v2 No 8 (maximum

A6386A PCI (4X) rx8620 servers 11i v2 Yes 8 (maximum

A6386A PCI (4X) rx4640 servers 11i v2 Yes 6

Bus

Type

Supported

HP Systems

servers

workstations

HP-UX

Version

11i v2 No 4

11i v2 No 1

OLAR

Support?

Maximum

Adapters

per System

4 per PCI card cage)

Chapter 220

Planning the Fabric

TCP/UDP/IP

TCP/UDP/IP Supported Conﬁgurations

Multiple TCP/UDP/IP HyperFabric configurations are supported to match the cost, scaling, and performance requirements of each installation.

In the previous section, “Conﬁguration Parameters” on page 17, the maximum limits for TCP/UDP/IP enabled HyperFabric hardware conﬁgurations were outlined. In this section the TCP/UDP/IP enabled HyperFabric conﬁgurations that HP supports are explained. These recommended conﬁgurations offer an optimal mix of performance and availability for a variety of operating environments.

There are many variables that can impact HyperFabric performance. If you are considering a conﬁguration that is beyond the scope of the following HP supported conﬁgurations, contact your HP representative.

Point-to-Point Conﬁgurations

You can interconnect large servers like the HP Superdome to run Oracle RAC 9i and enterprise resource planning applications. These applications are typically consolidated on large servers.

Point-to-point connections between servers support the performance beneﬁts of HMP without investing in HyperFabric switches. This is a good solution in small conﬁgurations where the beneﬁts of a switched HyperFabric cluster might not be required (see conﬁguration A and conﬁguration C in Figure 2-1).

If there are multiple point-to-point connections between two nodes, trafﬁc load is balanced over those links. If one link fails, the load fails over to the remaining links (see conﬁguration B in Figure 2-1).

Running applications using TCP/UDP/IP on a HyperFabric cluster provides major performance beneﬁts compared to other technologies such as Ethernet. If a HyperFabric cluster is originally set up to run enterprise applications using TCP/UDP/IP and the computing environment stabilizes with a requirement for higher performance, migration to HMP is always an option.

Chapter 2 21

Planning the Fabric

TCP/UDP/IP

Figure 2-1 TCP/UDP/IP Point-To-Point Conﬁgurations

Chapter 222

Switched Conﬁguration

This conﬁguration offers the same beneﬁts as the point-to-point conﬁgurations illustrated in Figure 2-1, but it has the added advantage of greater connectivity (see Figure 2-2).

Figure 2-2 TCP/UDP/IP Basic Switched Conﬁguration

Planning the Fabric

TCP/UDP/IP

Chapter 2 23

Planning the Fabric

TCP/UDP/IP

High Availability Switched Conﬁguration

This conﬁguration has no single point of failure. The HyperFabric driver provides end-to-end HA. If any HyperFabric resource in the cluster fails, trafﬁc is transparently rerouted through other available resources. This conﬁguration provides high performance and high availability (see Figure 2-3).

Figure 2-3 TCP/UDP/IP High Availability Switched Conﬁguration

Chapter 224

Hybrid Conﬁguration

You can interconnect servers and workstations in a single heterogeneous HyperFabric cluster.

In this conﬁguration, the servers are highly available. In addition, the workstations and the servers can run the same application or different applications (see Figure 2-4).

Figure 2-4 TCP/UDP/IP Hybrid Conﬁguration

Planning the Fabric

TCP/UDP/IP

Chapter 2 25

Planning the Fabric

Hyper Messaging Protocol (HMP)

Hyper Messaging Protocol (HMP) is an HP patented, high performance cluster interconnect protocol. HMP provides reliable, high speed, low latency, low CPU overhead, datagram service to applications running on the HP-UX operating system.

HMP was jointly developed with Oracle Corp. The resulting feature set was tuned to enhance the scalability of the Oracle Cache Fusion clustering technology. It is implemented using Remote DMA (RDMA) paradigms.

HMP is integral to the HP-UX HyperFabric driver. It can be enabled or disabled at HyperFabric initialization using the clic_init command or SAM. The HMP functionality is used by the applications listed in the following “Application Availability” section.

HMP signiﬁcantly enhances the performance of parallel and technical computing applications.

HMP ﬁrmware on HyperFabric adapter cards provides a shortcut that bypasses several layers in the protocol stack, boosting link performance and lowering latency. By avoiding interruptions and buffer copying in the protocol stack, communication task processing is optimized.

Application Availability

The following are the two families of applications that can use HMP over the HyperFabric interface:

• Oracle 9i Database, Release 1 (9.0.1) and Release 2 (9.2.0.1.0). HMP has been certiﬁed on Oracle 9i Database Release 1 with HP-UX

11.0, 11i v1, and 11i v2. HMP has been certiﬁed on Oracle 9i Database Release 2 with HP-UX

11.0, 11i v1, and 11i v2.

• Technical Computing Applications that use the HP Message Passing Interface (HP-MPI).

Chapter 226

Planning the Fabric

Hyper Messaging Protocol (HMP)

HP MPI is a native implementation of version 1.2 of the Message-Passing Interface Standard. It has become the industry standard for distributed technical applications and is supported on most technical computing platforms.

Features

The following are the HyperFabric features on HMP:

• OnLine Addition and Replacement (OLAR) The OLAR feature, which allows the replacement or addition of

HyperFabric adapter cards while the system (node) is running, is supported when applications use HMP to communicate.

• Event Monitoring Service (EMS): Supported In the HyperFabric version B.11.23.01, the HyperFabric EMS

For more information on EMS, including instructions for implementing this feature, see “Conﬁguring the HyperFabric EMS Monitor” on page 85 in this manual, and the EMS Hardware Monitors User’s Guide Part Number B6191-90028 September 2001 Edition.

• ServiceGuard: Supported Within a cluster, ServiceGuard groups application services

(individual HP-UX processes) into packages. In the event of a single service failure (node or network), EMS provides notification and ServiceGuard transfers control of the package to another node in the cluster, allowing services to remain available with minimal interruption. ServiceGuard using EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). ServiceGuard uses a heartbeat LAN to monitor the

Chapter 2 27

Planning the Fabric

Hyper Messaging Protocol (HMP)

nodes in a cluster. ServiceGuard cannot use the HyperFabric interconnect as a heartbeat link. Instead, a separate LAN must be used for the heartbeat.

For more information on configuring ServiceGuard, see “Conﬁguring HyperFabric with ServiceGuard” on page 87, as well as Managing MC/ServiceGuard Part Number B3936-90065 March 2002 Edition.

• High Availability (HA): Supported When applications use HMP to communicate between HP 9000

nodes in a HyperFabric cluster, you can conﬁgure ServiceGuard and the EMS monitor to identify node failure and automatically failover to a functioning HP 9000 node.

For more information on HA when running HMP applications, contact your HP representative.

• Transparent Local Failover: Supported HMP supports Transparent Local Failover in the HyperFabric

version B.11.23.01. When a HyperFabric resource (adapter, cable, switch or switch port)

fails in a cluster, HMP transparently fails over trafﬁc using other available resources. This is accomplished using card pairs, each of which is a logical entity that comprises a pair of HF2 adapters on a HP 9000 node. Only Oracle applications can make use of the Local Failover feature. HMP trafﬁc can only fail over between adapters that belong to the same card pair. Trafﬁc does not fail over if both the adapters in a card pair fail. However, administrators do not need to conﬁgure HF2 adapters as card pairs if TCP/UDP/IP is run over HF2 or MPI uses HMP.

When HMP is conﬁgured in the local failover mode, all the resources in the cluster are utilized. If a resource fails in the cluster and is restored, HMP does not utilize that resource until another resource fails.

For more information on Transparent Local Failover while running HMP applications, see “Conﬁguring HMP for Transparent Local Failover Support” on page 96.

• Dynamic Resource Utilization (DRU): Partially Supported If you add a new HyperFabric resource (node, cable or switch) to a

cluster running an HMP application, the HyperFabric subsystem will dynamically identify the added resource and start using it. The

Chapter 228

Planning the Fabric

Hyper Messaging Protocol (HMP)

same process takes place when a resource is removed from a cluster. However, DRU is not supported if you add or remove an adapter from a node that is running an HMP application. This is consistent with the fact that OLAR is not supported when an HMP application is running on HyperFabric.

• Load Balancing: Supported When an HP 9000 HyperFabric cluster is running HMP applications,

the HyperFabric driver balances the load across all available resources in the cluster, including nodes, adapter cards, links, and multiple links between switches.

• Switch Management: Not Supported Switch Management is not supported. Switch management will not

operate properly if it is enabled on a HyperFabric cluster.

• Diagnostics: Supported You can run diagnostics to obtain information on many of the

HyperFabric components using the clic_diag, clic_probe and clic_stat commands, as well as the Support Tools Manager (STM).

For more information on HyperFabric diagnostics, see “Running Diagnostics” on page 149.

Conﬁguration Parameters

This section discusses the maximum limits for HMP HyperFabric conﬁgurations. There are numerous variables that can impact the performance of any particular HyperFabric conﬁguration. For more information on speciﬁc HyperFabric conﬁgurations for HMP applications, see “HMP Supported Conﬁgurations” on page 33.

• HyperFabric is supported on the HP 9000 series servers and workstations only.

• HMP is supported on the HF2 adapter, A6386A, only.

• The performance advantages that HMP offers are not completely realized unless HMP is used with A6386A HF2 (ﬁber) adapters and related ﬁber hardware. See Table 2-2 on page 20 for details. The local failover conﬁguration of HMP is supported only on the A6386A HF2 adapters.

• Maximum Supported Nodes and Adapter Cards

Chapter 2 29

Planning the Fabric

Hyper Messaging Protocol (HMP)

HyperFabric clusters running HMP applications are limited to supporting a maximum of 64 adapter cards. However, in local failover conﬁgurations, a maximum of only 52 adapters are supported.

In point-to-point conﬁgurations running HMP applications, the complexity and performance limitations of having a large number of nodes in a cluster make it necessary to include switches in the fabric. Typically, point-to-point conﬁgurations consist of only 2 or 3 nodes.

In switched conﬁgurations running HMP applications, HyperFabric supports a maximum of 64 interconnected adapter cards.

A maximum of 8 conﬁgured IP addresses are supported by the HyperFabric subsystem per instance of the HP-UX operating system.

• Maximum Number of Switches You can interconnect (mesh) up to 4 switches (16-port fiber or Mixed

8 fiber ports) in a single HyperFabric cluster.

• Trunking Between Switches (multiple connections) Trunking between switches can be used to increase bandwidth and

cluster throughput. Trunking is also a way to eliminate a possible single point of failure. The number of trunked cables between nodes is only limited by port availability. To assess the effects of trunking on the performance of any particular HyperFabric conﬁguration, contact your HP representative.

• Maximum Cable Lengths HF2 (fiber): The maximum distance is 200m (4 standard cable

lengths are sold and supported: 2m, 16m, 50m and 200m). HMP supports up to 4 HF2 switches connected in series with a

maximum cable length of 200m between the switches and 200m between switches and nodes.

HMP supports up to 4 hybrid HF2 switches connected in series with a maximum cable length of 200m between ﬁber ports.

Chapter 230

Planning the Fabric

Hyper Messaging Protocol (HMP)

• HMP is supported on A400, A500, rp2400, rp2450, rp54xx (N-class), rp74xx (L-class), rp8400, and Superdome servers running 64-bit HP-UX.

• HMP is supported on HyperFabric starting HyperFabric versions B.11.00.11, B.11.11.01, and B.11.23.00.

• HMP is not supported on the A180 or A180C server.

• HMP is not supported on 32-bit versions of HP-UX.

• Throughput and Latency

Table 2-3 HF2 Throughput and Latency with HMP Applications

Server Class Maximum Throughput Latency

rp 7400 2 + 2 Gbps full duplex per link < 22 microsec

Table 2-4 Supported HyperFabric Adapter Conﬁgurations

Adapter

Bus

Type

Supported

HP Systems

HP-UX

Version

OLAR

Support?

Maximum

Adapters

per System

A6386A PCI (4X) rx2600 servers 11i v2 No 1

A6386A PCI (4X) rx56XX

11i v2 No 4

servers

A6386A PCI (4X) zx6000

11i v2 No 1

workstations

A6386A PCI (4X) SD64A servers 11i v2 Yes 8 (maximum

4 per PCI card cage)

A6386A PCI (4X) rx7620 servers 11i v2 No 8 (maximum

4 per PCI card cage)

A6386A PCI (4X) rx8620 servers 11i v2 Yes 8 (maximum

4 per PCI card cage)

A6386A PCI (4X) rx4640 servers 11i v2 Yes 6

Chapter 2 31

Planning the Fabric

Hyper Messaging Protocol (HMP)

NOTE The local failover conﬁguration on HMP is supported only on the A6386A

HF2 adapters.

Chapter 232

Planning the Fabric

Hyper Messaging Protocol (HMP)

HMP Supported Conﬁgurations

Multiple HMP HyperFabric configurations are supported to match the performance, cost and scaling requirements of each installation.

In the section, “Conﬁguration Parameters” on page 29, the maximum limits for HMP enabled HyperFabric hardware conﬁgurations were outlined. This section discusses the HMP enabled HyperFabric conﬁgurations that HP supports. These recommended conﬁgurations offer an optimal mix of performance and availability for a variety of operating environments.

Point-to-Point Conﬁguration

You can interconnect large servers like the HP Superdome to run Oracle RAC 9i and enterprise resource planning applications. These applications are typically consolidated on large servers.

If an HMP application is running over HyperFabric and another node or adapter is added to either of the nodes, then it is necessary to also add a HyperFabric switch to the cluster (see conﬁguration B in Figure 2-5).

Chapter 2 33

Planning the Fabric

Hyper Messaging Protocol (HMP)

Figure 2-5 HMP Point-To-Point Conﬁgurations

Chapter 234

Planning the Fabric

Hyper Messaging Protocol (HMP)

Enterprise (Database) Conﬁguration

The HMP enterprise conﬁguration illustrated in Figure 2-6 is very popular for running Oracle RAC 9i.

Superdomes or other large servers make up the Database Tier. Database Tier nodes communicate with each other using HMP.

Application Tier nodes communicate with each other and to the Database Tier using TCP/UDP/IP.

Figure 2-6 HMP Enterprise (Database) Conﬁguration, Single Connection

Between Nodes

Chapter 2 35

Planning the Fabric

Hyper Messaging Protocol (HMP)

Enterprise (Database) - Local Failover Supported Conﬁguration

The HMP enterprise conﬁguration is a scalable solution. For high availability and performance, you can easily scale the HMP enterprise conﬁguration with multiple connections between the HyperFabric resources. Any single point of failure in the database tier of the fabric is eliminated in Figure 2-7.

Figure 2-7 Local Failover Supported Enterprise (Database) Conﬁguration,

Multiple Connections between Nodes

In this conﬁguration, if a HyperFabric resource (adapter, cable, switch or switch port) fails in a cluster, HMP transparently fails over trafﬁc using another available resource. For more information, see “Conﬁguring HMP for Transparent Local Failover Support” on page 96.

Chapter 236

Planning the Fabric

Hyper Messaging Protocol (HMP)

Technical Computing (Work Stations) Conﬁguration

This conﬁguration is typically used to run technical computing applications with HP-MPI. A large number of small nodes are interconnected to achieve high throughput (see Figure 2-8). High availability is not usually a requirement in technical computing environments.

HMP provides the high performance, low latency path necessary for these technical computing applications. You can interconnect up to 56 nodes using HP 16-port switches. You cannot link more than four 16-port switches in a single cluster (see Figure 2-9).

The HP “J”, “B”, and “C” class workstations provide excellent performance and return on investment in technical computing conﬁgurations.

Chapter 2 37

Planning the Fabric

Hyper Messaging Protocol (HMP)

Figure 2-8 Technical Computing Conﬁguration

Chapter 238

Hyper Messaging Protocol (HMP)

Figure 2-9 Large Technical Computing Conﬁguration

Planning the Fabric

Chapter 2 39

Planning the Fabric

Hyper Messaging Protocol (HMP)

Chapter 240

3 Installing HyperFabric

This chapter contains the following sections that describe the HyperFabric installation:

• “Checking HyperFabric Installation Prerequisites” on page 43.

Chapter 3 41

Installing HyperFabric

• “Installing HyperFabric Adapters” on page 44.

• “Installing the Software” on page 51.

• “Installing HyperFabric Switches” on page 57.

Chapter 342

Installing HyperFabric

Checking HyperFabric Installation Prerequisites

Before installing HyperFabric, ensure that the following hardware and software prerequisites are met:

✓ Check the HyperFabric Release Notes for known problems, required

patches, or other information needed for installation.

✓ Conﬁrm that the /usr/bin, /usr/sbin, and /sbin directories are in

your PATH by logging in as root and using the echo $PATH command.

✓ Conﬁrm that the HP-UX operating system is the correct version. Use

the uname -a command to determine the HP-UX version. For more information about the required operating system versions,

see HyperFabric Release Notes.

✓ If you are installing an HF2 switch, conﬁrm that you have four

screws with over-sized heads.

✓ Conﬁrm that there are cables of proper length and type (ﬁber) to

make connections in the fabric (adapter to adapter, adapter to switch, or switch to switch).

✓ Conﬁrm that there is at least one loopback plug for testing the

adapters and switches (a ﬁber loopback plug [HP part number A6384-67004] is shipped with each HF2 switch).

✓ Conﬁrm that necessary tools are available to install the HyperFabric

switch mounting hardware. In addition, check the HP 9000 system’s documentation to determine if additional tools may be required for component installation.

✓ Conﬁrm that the software media is correct. ✓ Create a map of the fabric (optional). ✓ Conﬁrm that HP-UX super-user privileges are available; they are

necessary to complete the HyperFabric installation.

The ﬁrst HyperFabric installation step is installing HyperFabric adapter cards in the nodes.

Chapter 3 43

Installing HyperFabric

Installing HyperFabric Adapters

This section contains information about installing HyperFabric adapters in HP 9000 systems. Online Addition and Replacement (OLAR) information is provided in the section, “Online Addition and Replacement” on page 44.

CAUTION HyperFabric adapters contain electronic components that can easily be

damaged by small amount of electricity. To avoid damage, follow these guidelines:

• Store adapters in their antistatic plastic bags until installation.

• Work in a static-free area, if possible.

• Handle adapters by the edges only. Do not touch electronic components or electrical traces.

• Use the disposable grounding wrist strap provided with each adapter. Follow the instructions included with the grounding strap.

• Use a suitable ground—any exposed metal surface on the computer chassis.

For speciﬁc instructions, see system-speciﬁc documentation on “installing networking adapters” for each type of HP 9000 system that HyperFabric adapters will be installed into.

When the HyperFabric adapters have been installed, go to “Installing the Software” on page 51.

Online Addition and Replacement

Online Addition and Replacement (OLAR) allows PCI I/O cards,

adapters or controllers to be replaced or added to HP 9000 systems, without the need for completely shutting down and rebooting the system, or adversely affecting other system components. This feature is only

Chapter 344

Installing HyperFabric

Installing HyperFabric Adapters

available on HP 9000 systems that are designed to support OLAR. The system hardware uses the per-slot power control combined with OS support to enable this feature.

NOTE OLAR is supported only on TCP/UDP/IP over HF2 adapters.

Not all add-in cards have this capability, but over time many cards will be gaining this capability.

The HyperFabric Release Notes contains information about which HP 9000 systems and HyperFabric adapters OLAR is supported for.

IMPORTANT At this time, Superdome systems are not intended for access by users.

HP recommends that these systems only be opened by a qualiﬁed HP engineer. Failure to observe this requirement can invalidate any support agreement or warranty to which the owner might otherwise be entitled.

There are two methods to add or replace OLAR-compatible cards:

• Using the SAM utility.

• Issuing command-line commands, through olrad, that refer to the HyperFabric OLAR script (/usr/sbin/olard.d/clicd).

HP recommends that SAM be used for OLAR procedures, instead of the rad command. This is primarily because SAM prevents the user from doing things that might have adverse effects. This is not true when the rad command is used.

For detailed information about using either of these two procedures, see Conﬁguring HP-UX For Peripherals. You can order that document from HP, or you can view, download, and print it from the following URL: http://www.docs.hp.com.

Chapter 3 45

Installing HyperFabric

Installing HyperFabric Adapters

Table 3-1 below explains some important OLAR-related terms.

Table 3-1 Important OLAR Terms

Term Meaning

OLAR All aspects of the OLAR feature

Power Domain A grouping of 1 or more interface

target card / target card slot The interface card which will be

affected card / affected card slot Interface cards and the card slots

including Online Addition (OLA) and Online Replacement (OLR).

card slots that are powered on or off as a unit. (Note: Multi-slot power domains are not currently supported.)

added or replaced using OLAR, and the card slot in which it resides.

they reside in, which are in the same power domain as the target slot.

IMPORTANT In many cases, other interface cards and slots within the system are

dependent on the target card. For example, if the target card is a multiple-port card, suspending or deleting drivers for the target card slot also suspends individual drivers for the multiple hardware paths on that card.

During a card replacement operation, SAM performs a Critical Resource Analysis (CRA), which checks all ports on the target card for critical resources that would be temporarily unavailable while the card is shut down.

Chapter 346

Installing HyperFabric

Installing HyperFabric Adapters

Planning and Preparation

As mentioned previously, for the most part, SAM prevents the user from performing OLAR procedures that would adversely affect other areas of the HP 9000 system. See Conﬁguring HP-UX For Peripherals for detailed information.

Critical Resources

The effects of shutting down a card’s functions must be considered. Replacing a card that is still operating can have extensive consequences. Power to a slot must be turned off when a card is removed and a new card is inserted.

This is particularly important if there is no online failover or backup card to pick up those functions. For example:

• Which mass storage devices will be temporarily disconnected when a card is shut down?

• Will a critical networking connection be lost?

A critical resource is one that would cause a system crash or prevent an operation from successfully completing if the resource were temporarily suspended or disconnected. For example, if the SCSI controller is connected to the unmirrored root disk or swap space, the system will crash when the SCSI controller is shut down.

During an OLAR procedure, it is essential to check the targeted card for critical resources, as well as the effects of existing disk mirrors and other situations where a card’s functions can be taken over by another card that will not be affected.

As mentioned earlier, SAM performs a thorough CRA automatically, and presents options based on its ﬁndings. If it is determined that critical resources will be affected by the OLAR procedure, the card could be replaced when the system is ofﬂine. If action must be taken immediately, an online addition of a backup card and deletion of the target card could be attempted using the rad command.

Card Compatibility

This section explains card compatibility considerations for doing OLAR.

Chapter 3 47

Installing HyperFabric

Installing HyperFabric Adapters

Online Addition (OLA) Multiple cards can be added at the same time. When adding a card online, the ﬁrst issue to resolve is whether the new card is compatible with the system. Each OLAR-capable PCI slot provides a set amount of power. The replacement card cannot require more power than there is available.

The card must also operate at the slot’s bus frequency. A PCI card must run at any frequency lower than its maximum capability, but a card that could operate at only 33 MHz would not work on a bus running at 66 MHz. rad provides information about the bus frequency and power available at a slot, as well as other slot-related data.

If an HP 9000 system has one or more slots that support OLAR and OLA will be used to install a HyperFabric adapter in one of those slots—install the adapter in the HP 9000 system according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the “Conﬁguring HP-UX Peripherals” manual.

After adding a new HyperFabric adapter, SAM tries to locate the HyperFabric software. If SAM cannot locate the HyperFabric software, the new adapter cannot be used until the software is installed (software installation requires a system reboot). If SAM locates the HyperFabric software, SAM determines whether the new adapter is functional. If it is not functional, SAM displays an error message.

If the new adapter is functional, SAM displays a message telling the user to conﬁgure the adapter and start HyperFabric. If only one adapter is being added, issue the clic_init -c command or use SAM to conﬁgure the adapter, and then issue the clic_start command or use SAM to start HyperFabric. If multiple adapters are being added, add all of the adapters ﬁrst, and then run clic_init -c and clic_start or use SAM. See “Performing the Conﬁguration” on page 78 and “Starting HyperFabric” on page 109 for more information about conﬁguring and starting HyperFabric.

CAUTION Do not change any conﬁguration information for an existing HyperFabric

adapter or switch while you are using clic_init -c to conﬁgure a new adapter.

When you have completed the adapter installation, go to “Installing the Software” on page 51.

Chapter 348

Installing HyperFabric

Installing HyperFabric Adapters

Online Replacement (OLR) When replacing an interface card online, the replacement card must be identical to the card being replaced (or at least be able to operate using the same driver as the replaced card). This is referred to as like-for-like replacement and should be adhered to, because using a similar but not identical card can cause unpredictable results. For example, a newer version of the target card that is identical to the older card in terms of hardware might contain an updated ﬁrmware version that could potentially conﬂict with the current driver. An A6386A adapter must be replaced with another A6386A adapter, etc. Also, the old adapter and new adapter must have the same revision levels.

When a replacement card is added to an HP 9000 system, the appropriate driver for that card must be conﬁgured in the kernel before beginning the replacement operation. SAM ensures the correct driver is present. (In most cases, the replacement card will be the same type as a card already in the system, and this requirement will be automatically met.) Keep the following things in mind:

• If the necessary driver is not present and the driver is a dynamically loadable kernel module (DLKM), it can be loaded manually. See the “Dynamically Loadable Kernel Modules” section in “Conﬁguring HP-UX For Peripherals” for more information.

• If the driver is static and not conﬁgured in the kernel, then the card cannot be added online. The card could be physically inserted online, but no driver would claim it.

If there is any question about the driver’s presence, or if it is uncertain that the replacement card is identical to the existing card, ioscan can be used together with rad to investigate.

If more than one operational HyperFabric adapter is present when SAM requests the suspend operation for all ports on the target adapter, HyperFabric will redirect the target adapter’s trafﬁc to a local backup adapter using local failover. Client applications using the replaced adapter will not be interrupted in any way.

If the adapter being replacing is active and it is the only operational HyperFabric adapter on the HP 9000 system, SAM displays the following warning message:

WARNING: You have 1 operational HyperFabric card. If you go ahead with this operation you will lose network access via HyperFabric until the on-line replaced HyperFabric card becomes operational.

Chapter 3 49

Installing HyperFabric

Installing HyperFabric Adapters

You are asked if you want to continue. If you reply Yes, client applications are suspended. Replace the adapter according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the Conﬁguring HP-UX Peripherals manual.

When an adapter has been replaced, client application activity resumes unless the TCP timers or the application timers have popped.

CAUTION Do not use the clic_start command or the clic_shutdown command,

while an installed adapter is suspended. Do not use SAM to start or stop HyperFabric while an installed adapter is suspended. The operation will fail and an error message will be displayed.

After a HyperFabric adapter has been replaced, SAM checks the replacement adapter to ensure that it is permitted according to the like-for-like rules. If the adapter is permitted, SAM automatically activates it. If it is not permitted, SAM displays an error message.

Chapter 350

Installing the Software

This section describes the HyperFabric ﬁle structure and the steps necessary to load the software. The software must be installed on each instance of the HP-UX operating system in the fabric.

File Structure

The HyperFabric ﬁle structure is shown in Figure 3-1 below. The structure is shown for informational purposes only. The user cannot modify any of the ﬁles or move them to a different directory.

Figure 3-1 HyperFabric File Structure

Installing HyperFabric

Installing the Software

/resmon

/dictionary

/clic_01

/usr

/conf

/lib

/libclic_dlpi_drv.a /libha_drv.a

/opt

/master.d

/etc

/clic

/libclic_mgmt.a

/rc.config.d

/clic_global_conf

/lib

/clic_diag /clic_dump /clic_init /clic_mgmtd /clic_mond /clic_ping

/clic_probe /clic_shutdown /clic_start /clic_stat

/opt

/clic

/bin

/sbin

/init.d

/clic

/firmware

/clic_fw /clic_fw_1x32c

/clic_fw_4x8c /clic_fw_4x32c /clic_fw_hf28c

/clic_fw_hf232c /clic_fw_db

/var/adm

/clic_ip_drv.trc /clic_ip_drv.trc0 /clic_ip_drv.trc1

/clic_log

/clic_log.old /OLDclic_log

/share

/man

/man1m.Z

Chapter 3 51

Installing HyperFabric

Installing the Software

The commands and ﬁles used to administer HyperFabric typically have a preﬁx of clic_. CLIC stands for CLuster InterConnect, and it is used to differentiate those HyperFabric commands/ﬁles from other commands/ﬁles. For example, the HyperFabric command clic_init is different from the HP-UX init command.

Each of the ﬁles shown in Figure 3-1 above is brieﬂy described below:

• /etc/opt/resmon/dictionary/clic_01

• /etc/rc.config.d/clic_global_conf

• /sbin/init.d/clic

• /var/adm/clic_ip_drv.trc

The HyperFabric dictionary ﬁle for the Event Monitoring Service (EMS).

The global conﬁguration ﬁle, which contains the IP addresses for each adapter and each HyperFabric switch (if any) in the fabric.

The system boot startup script for the HyperFabric management process.

One of the software’s trace ﬁles. This ﬁle is created when the clic_diag -D TCP_IP command is run.

• /var/adm/clic_ip_drv.trc0 One of the HyperFabric software’s trace ﬁles. This is the primary ﬁle

that is created when the clic_diag -C TCP_IP command is run.

• /var/adm/clic_ip_drv.trc1 One of the HyperFabric software’s trace ﬁles. This ﬁle is created

when the clic_diag -C TCP_IP command is run, and the primary trace ﬁle (clic_ip_drv.trc0) becomes full.

• /var/adm/clic_log The global log ﬁle that is updated by the HyperFabric management

process.

• /var/adm/clic_log.old The backup copy of the log ﬁle that is created when the log ﬁle grows

larger than 100 Kbytes.

Chapter 352

Installing HyperFabric

Installing the Software

• /var/adm/OLDclic_log The log ﬁle from the previous time the clic_start command was

executed.

• /usr/conf/lib/libclic_dlpi_drv.a The kernel library that contains the HyperFabric software.

• /usr/conf/lib/libha_drv.a The kernel library that contains the High Availability (HA) software.

• /usr/conf/master.d/clic This ﬁle is described along with the other master ﬁles in the master

man page (type man master at the HP-UX prompt).

• /opt/clic/lib/libclic_mgmt.a The HyperFabric management API library.

• /opt/clic/bin The directory containing the HyperFabric management commands:

clic_diag, clic_init, clic_probe, clic_shutdown, clic_start, clic_stat, and clic_dump. (Note that clic_dump is for HP internal

use only.) Also, clic_ping is replaced by clic_probe.This directory also contains the HyperFabric management process (clic_mgmtd) and the HyperFabric EMS monitor process (clic_mond).

• /opt/clic/firmware/clic_fw The 1X HSC HyperFabric 8-bit CRC ﬁrmware. This ﬁle must not be

modiﬁed.

• /opt/clic/firmware/clic_fw_1x32c The 1X HSC HyperFabric 32-bit CRC ﬁrmware. This ﬁle must not be

modiﬁed.

Chapter 3 53

Installing HyperFabric

Installing the Software

• /opt/clic/firmware/clic_fw_4x8c

• /opt/clic/firmware/clic_fw_4x32c

• /opt/clic/firmware/clic_fw_hf28c

• /opt/clic/firmware/clic_fw_hf232c

• /opt/clic/firmware/clic_fw_db

• /opt/clic/share/man/man1m.Z

The 4X PCI HyperFabric 8-bit CRC ﬁrmware. This ﬁle must not be modiﬁed.

The 4X HyperFabric PCI 32-bit CRC ﬁrmware. This ﬁle must not be modiﬁed.

The HyperFabric2 8-bit ﬁrmware. This ﬁle must not be modiﬁed.

The HyperFabric2 32-bit ﬁrmware. This ﬁle must not be modiﬁed.

A binary ﬁle where adapter-speciﬁc conﬁguration information is stored. The management process creates this ﬁle using default values.

The man pages for the HyperFabric commands.

Chapter 354

Installing HyperFabric

Installing the Software

Loading the Software

Listed below are the steps you must follow to load the HyperFabric software, using the HP-UX swinstall program.

Step 1. Log on to the system as root.

Step 2. Insert the software media into the appropriate drive. If the software is

being loaded from a CD-ROM, go to step 3; otherwise, go to step 4.

Step 3. Mount the CD-ROM drive by using the following command at the

command prompt:

$ mount

where

Step 4. Run the swinstall program using the following command:

$ /usr/sbin/swinstall

This opens the “Software Selection” window.

Step 5. Change the Source Host Name, if necessary, and then enter the mount

point of the drive in the Source Depot Path ﬁeld. Select the OK button to return to the “Software Selection” window.

The “Software Selection” window now contains a list of available software to install.

Step 6. Highlight the HyperFabric software:

• HP-UX 11i v2: HyprFabrc-00

Step 7. Choose Mark for Install from the “Actions” menu; this chooses the

highlighted software.

Step 8. From the “Actions” menu, select the “Install...” menu, and then choose

Install. This begins product installation and opens the “Install Analysis” window.

device_name

is the name assigned to the CD-ROM drive.

Step 9. Select the OK button in the “Install Analysis” window when the Status

ﬁeld displays a “Ready” message.

Step 10. Select the YES button in the “Conﬁrmation” window to start software

installation.

Chapter 3 55

Installing HyperFabric

Installing the Software

swinstall loads the ﬁleset, runs the control script for the ﬁlesets, and builds the kernel. When the processing is ﬁnished, the “Status” ﬁeld displays a “Ready” message. Select “Done” and then the “Note” window opens.

Step 11. Select the OK button in the “Note” window to reboot. The user interface

disappears and the system reboots.

Step 12. When the system comes back up, log on to the system as root and view

the /var/adm/sw/swagent.log and /var/adm/sw/swinstall.log ﬁles to view error or warning messages that may have occurred during the installation.

Step 13. While still logged in as root, view the /etc/services ﬁle to ensure that

these two HyperFabric-related lines are present:

• hp-clic 3384/tcp #clic management daemon

• hp-clic 3384/udp #clic switch management

These lines are used by the HyperFabric software, and are not comments; therefore, do not remove them from the ﬁle.

Step 14. Verify that all installed HyperFabric adapters have a software state of

“CLAIMED,” by running the ioscan -nf -C clic command.

A check is also done to make sure all of the HyperFabric adapters have been claimed when clic_init is activated or when SAM is used to conﬁgure HyperFabric.

Step 15. If one or more HyperFabric switches are included in the conﬁguration, go

to the next section of this chapter, “Installing HyperFabric Switches”; otherwise, go to Chapter 4, “Conﬁguring HyperFabric,” on page 67.

Chapter 356

Installing HyperFabric

Installing HyperFabric Switches

This section contains the information you need to install HyperFabric switches. As stated earlier, the term HyperFabric2 (HF2) switch refers to the functional switch (the A6384A switch chassis with one of the switch modules installed).

Before Installation

Before you install the HyperFabric switch, you should be aware of the following:

❏ The A6384A HF2 switch is supported beginning with the following

HyperFabric software versions:

— HP-UX 11.0: version B.11.00.11 — HP-UX 11i v1: version B.11.11.01 — HP-UX 11i v2: version B.11.23.00. HyperFabric switches are not supported by software versions earlier

than those mentioned above, respectively. To determine the version of HyperFabric you have, issue the

following command at the command prompt:

$ swlist | grep -i hyperfabric

❏ For the HF2 switch, HP recommends that you use the rails shipped

with the switch when you mount it in a standard 19-inch rack, even though the switch can be mounted in the rack by itself (without the rails).

WARNING To prevent overheating, you must leave one rack unit (1 EIA)

of empty space above the HyperFabric switch.

❏ After the HyperFabric switch is mounted in the rack, attach the

various cables to the switch. To avoid damage to any of the cables, follow these guidelines:

Chapter 3 57

Installing HyperFabric

Installing HyperFabric Switches

— If your cables have dust caps over the connectors, keep them in

— Be careful not to stretch, puncture, or crush the cable.

To install an HF2 switch, see “Installing the HF2 Switch” on page 59.

place until you are ready to connect them. This prevents dirt and oils from soiling any important surfaces.

Chapter 358

Installing HyperFabric

Installing HyperFabric Switches

Installing the HF2 Switch

This section contains information on installing an HF2 switch. The front of the HF2 switch has a ﬂange—or “wing”—on each side, with

two holes for attaching the switch to the rack. The following ﬁgures do not show the ﬂanges.

Figure 3-2 shows the front of the HF2 switch with an A6388A HF2 8-port ﬁber switch module installed in the switch’s expansion slot.

Figure 3-2 Front of HF2 Switch (A6388A Switch Module Installed)

Integrated Ethernet management

LAN card

Status

Powe r

Por t

Ethernet

Por t Main

Por t

A6388A HF2 8-port ﬁber switch module in expansion slot

Ethernet

Por t

Aux

Label showing Ethernet MAC address

Por t

Port LED colors and meanings

legend

Por t

Integrated 8-port ﬁber card

Por t

Chapter 3 59

Installing HyperFabric

Installing HyperFabric Switches

Figure 3-3 shows the front of the HF2 switch with an A6389A HF2 4-port copper switch module installed in the switch’s expansion slot.

Figure 3-3 Front of HF2 Switch (A6389A Switch Module Installed)

Integrated Ethernet management

LAN card

Status

Powe r

Por t

Ethernet

Por t

Main

Por t

A6389A HF2 4-port copper switch module in expansion slot

You can install the HF2 switch in one of the following ways:

• Using the rail kit that is shipped with the switch (see the next section, “With the Rail Kit”). HP strongly recommends installing the HF2 switch this way.

Ethernet

Por t

Aux

Label showing Ethernet MAC address

Por t

Port LED colors and meanings

legend

Por t

Integrated 8-port

ﬁber card

Por t

• Attaching the switch directly to the rack (see “Without the Rail Kit” on page 65).

Chapter 360

Installing HyperFabric

Installing HyperFabric Switches

With the Rail Kit

As mentioned earlier, HP strongly recommends installing the HF2 switch using the rail kit.

When you install the HF2 switch, you will be putting the front of the switch—the end with the ﬂanges (“wings”)—at the back of the rack. The steps for installing the HF2 switch using the rail kit are as follows:

Step 1. Prepare the rack for rail and switch installation.

Step 2. Install and secure the rails in the rack, using two screws per rail. The

following ﬁgure shows the rack with the rails installed.

rack back

screws

rack

front

Step 3. From the front of the rack, slide the switch—with the front of the switch

facing the back of the rack—into the rack, on the rails. Move it until it is snug against the back of the rack. You might not have enough clearance between the switch and the rail screws; this may prevent you from easily

Chapter 3 61

Installing HyperFabric

Installing HyperFabric Switches

sliding the switch. If so, try lifting the switch over the rail screws. If you cannot do this, remove the rail screws, slide the switch into position, and put the rail screws back in.

Step 4. Align the two holes in each ﬂange (“wing”) on the switch’s front with the

holes in the rack frame. Fasten each ﬂange of the switch to the rack by putting a screw in each of the four holes in the ﬂanges. Be sure to use screws with over-sized heads. The following ﬁgure shows the rack with the switch in this position.

rack back

front of switch

(with flanges)

rack

front

Step 5. From the front of the rack, install a bracket on the outside of each rail,

using two screws per bracket. Be sure to use the upper screw holes on each bracket. Put the screws in the two square holes closest to the back of the switch, so that the brackets—referred to as “bracket 1” and “bracket 2” in these steps—are snug against the switch. Be sure that the brackets make contact with the beveled (sloped) part of the switch (not

Chapter 362

Installing HyperFabric

Installing HyperFabric Switches

the ﬂat part). Otherwise, the brackets will not secure the back of the switch properly. The following ﬁgure shows the rack with these two brackets installed.

rack back

bracket 2

front of switch

(with flanges)

rack

front

bracket 1

Step 6. When you are sure the brackets are snug against the beveled part of the

switch, tighten all four screws in brackets 1 and 2.

Step 7. For each port that will be connected to an HyperFabric adapter in an

HP 9000 system, attach the cable from the corresponding adapter. Your connections must be copper-to-copper and ﬁber-to-ﬁber (including cables).

Step 8. Connect the switch to the Ethernet network.

Chapter 3 63

Installing HyperFabric

Installing HyperFabric Switches

Step 9. Plug the switch’s power cord into the rack’s PDU, if it has one.

Alternatively, you can plug a power cord that is compatible with your country’s requirements into a power strip or outlet that you want to use for the switch. (In this case, you are responsible for obtaining a compatible power cord.)

Step 10. Power on the HF2 switch by plugging the power cord into the AC inlet on

the back of the switch. (There is no power switch.)

Step 11. Once the power is on, check these LEDs on the integrated Ethernet

management LAN adapter card (in the top slot of the switch):

✓ The “Operating/Fault” LED shows solid green.

✓ The “Power A” and “Power B” LEDs show solid green.

✓ The “Ethernet Port Main” and “Ethernet Port Aux” LEDs are

showing solid green (connected) or ﬂashing green (Ethernet trafﬁc is ﬂowing to the switch). See Figure 3-2 on page 59 or Figure 3-3 on page 60 for the locations of the LEDs.

Step 12. On the integrated 8-port ﬁber card (in the middle slot of the switch),

check that for each switch port that is connected to an HF2 adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.

Step 13. On the switch module in the expansion slot (the bottom slot of the

switch), check that for each switch port that is connected to an HyperFabric adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.

For more information about the switch’s LEDs, see “HF2 Switch LEDs” on page 148.

Step 14. If you want to install another HF2 switch using the rail kit, go back to

step 1.

If you want to install another HF2 switch without using the rail kit, go to the next section, “Without the Rail Kit”.

Otherwise, go to Chapter 4, “Conﬁguring HyperFabric,” on page 67.

Chapter 364

Installing HyperFabric

Installing HyperFabric Switches

Without the Rail Kit

As mentioned earlier, HP strongly recommends installing the HF2 switch using the rail kit (described in the previous section, “With the Rail Kit” on page 61).

When you install the HF2 switch, you will be putting the front of the switch—the end with the ﬂanges (“wings”) — at the back of the rack. The steps for installing the HF2 switch without using the rail kit are as follows:

Step 1. Prepare the rack for switch installation.

Step 2. Insert the HF2 switch into the rack, with the front of the switch snug

against the back of the rack.

Step 3. Align the two holes in each ﬂange on the switch’s front with the holes in

the rack frame.

Step 4. Fasten each ﬂange of the switch to the rack by putting a screw in each of

the four holes in the ﬂanges. Be sure to use screws with over-sized heads.

Step 5. Tighten all of the screws so that the HF2 switch is ﬁrmly mounted in the

rack.

Step 6. For each port that will be connected to an HyperFabric adapter in an

HP 9000 system, attach the cable from the corresponding adapter. Your connections must be copper-to-copper and ﬁber-to-ﬁber (including cables).

Step 7. Connect the switch to the Ethernet network.

Step 8. Plug the switch’s power cord into the rack’s PDU, if it has one.

Step 9. Power on the HF2 switch by plugging the power cord into the AC inlet on

the back of the switch. (There is no power switch.)

Chapter 3 65

Installing HyperFabric

Installing HyperFabric Switches

Step 10. Once the power is on, check these LEDs on the integrated Ethernet

management LAN adapter card (in the top slot of the switch):

✓ The “Operating/Fault” LED shows solid green.

✓ The “Power A” and “Power B” LEDs show solid green.

✓ The “Ethernet Port Main” and “Ethernet Port Aux” LEDs are

showing solid green (connected) or ﬂashing green (Ethernet trafﬁc is ﬂowing to the switch). See Figure 3-2 or Figure 3-3 below for the locations of the LEDs.

Step 11. On the integrated 8-port ﬁber card (in the middle slot of the switch),

Step 12. On the switch module in the expansion slot (the bottom slot of the

switch), check that for each switch port that is connected to a HyperFabric adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.

For more information about the switch’s LEDs, see “HF2 Switch LEDs” on page 148.

Step 13. If you want to install another HF2 switch without using the rail kit, go to

step 1.

If you want to install another HF2 switch using the rail kit, go to “With the Rail Kit” on page 61.

Otherwise, go to Chapter 4, “Conﬁguring HyperFabric,” on page 67.

Chapter 366

4 Conﬁguring HyperFabric

This chapter contains the following sections that describe conﬁguring HyperFabric:

• “Conﬁguration Overview” on page 69.

Chapter 4 67

Conﬁguring HyperFabric

• “Information You Need” on page 71.

• “Performing the Conﬁguration” on page 78.

• “Deconﬁguring a HyperFabric Adapter with SAM” on page 83.

• “Conﬁguring the HyperFabric EMS Monitor” on page 85.

• “Conﬁguring HyperFabric with ServiceGuard” on page 87.

• “Conﬁguring HMP for Transparent Local Failover Support” on page 96.

Chapter 468

Conﬁguring HyperFabric

Conﬁguration Overview

You do not need to conﬁgure the HyperFabric switch because the HyperFabric management process performs automatic routing and conﬁguring for the switch. So, conﬁguring HyperFabric consists only of creating the HyperFabric /etc/rc.config.d/clic_global_conf global conﬁguration ﬁle on each node in the fabric. The conﬁguration ﬁle contains the following information:

• The IP addresses and subnet mask of the HyperFabric adapters installed in the node.

• For each HyperFabric switch in the fabric — the switch’s IP address, and the MAC address of the switch’s Ethernet port. This applies only if you enable switch management. In addition, you cannot enable switch management through SAM — you must use the clic_init command.

• The IP multicast address that all the switches and nodes in the fabric will register to (if you are going to enable switch management).

• The IP address of the local node’s Ethernet LAN interface. This LAN interface must be on the same subnet as Ethernet ports of the HyperFabric switches (if you are going to enable switch management). (A node might have multiple LAN interfaces.)

NOTE HP recommends that you do not enable switch management.

You can create the global conﬁguration ﬁle by either running the clic_init command or using SAM to conﬁgure each HyperFabric adapter.

The clic_init command and SAM also place the necessary entries into the following three ﬁles:

• The system /etc/rc.config.d/netconf ﬁle.

Chapter 4 69

Conﬁguring HyperFabric

Conﬁguration Overview

IMPORTANT In this ﬁle, clic_init and SAM add some HyperFabric-related lines

that end with the characters #clic. These lines are used by the HyperFabric software — and are not comments — so do not remove them from the ﬁle.

• The system /etc/rc.config.d/clic_global_conf ﬁle.

• The /etc/rarpd.conf (Reverse Address Resolution Protocol [RARP]) support ﬁle. This ﬁle is used in the management of the HyperFabric switches (if you are going to enable switch management).

The clic_init command is described in “Using the clic_init Command” on page 79. Using SAM to conﬁgure an adapter is described in “Using SAM” on page 81.

After you have used the clic_init command or SAM, you can conﬁgure HyperFabric with ServiceGuard, if necessary. For more information, see “Conﬁguring HyperFabric with ServiceGuard” on page 87.

You can conﬁgure card pairs for the Transparent Local Failover Feature of HMP available with the B.11.23.01 release of HyperFabric. For more information, see “Conﬁguring HMP for Transparent Local Failover Support” on page 96.

Chapter 470

Conﬁguring HyperFabric

Information You Need

When you run the clic_init command or use SAM for conﬁguration, you have to provide certain conﬁguration information. So, before you run clic_init or use SAM, you should have the following information:

❏ For each node in the fabric, determine if that node will need to

interoperate with other nodes that are using; any HP-UX 11.0 HyperFabric versions earlier than B.11.00.11 or any HP-UX 11i v1 HyperFabric versions earlier than B.11.11.01.

❏ For each HyperFabric adapter installed in the local node:

✓ The adapter’s IP address.

IMPORTANT The last 10 bits of each adapter’s IP address must be unique

throughout the entire fabric. And, remember that the last part of the address cannot be 0 (that is, the IP address cannot be

n.n.n

.0). Also, note that HyperFabric converts these 10 bits to a decimal value called the Virtual Route IDentiﬁer (VRID), which is used in some HyperFabric command input and output.

✓ The subnet mask. When you run clic_init or use SAM, if you

do not specify a value for this, a default subnet mask is chosen based on the adapter’s IP address.

When clic_init begins to prompt you for the information for each adapter, it assigns an ID (for example, clic0) to that adapter and displays it as part of the ﬁrst prompt. If you use SAM, it assigns the adapter an ID and displays it in the “Adapter Name” column of the “Conﬁgure HyperFabric Adapter” screen. You can also determine an adapter’s ID by running the clic_stat command (see “The clic_stat Command” on page 117). You should note each adapter’s ID, because it is used as input to other HyperFabric commands.

❏ For using the Transparent Local Failover feature of HMP available

in the version B.11.23.01 of HyperFabric, you need to deﬁne the card pairs.

❏ For each HyperFabric switch in the fabric (if you are going to enable

switch management):

Chapter 4 71

Conﬁguring HyperFabric

Information You Need

✓ The IP address of the switch. ✓ The MAC address of the switch’s Ethernet port. If you do not

already know the switch’s MAC address, it is printed on a label on the back of the HF switch and on the front of the HF2 switch. See Figure 3-2 on page 66 for the location of the label on the HF switch, and Figure 3-2 on page 59 and Figure 3-3 on page 60 for the location of the label on the HF2 switch.

IMPORTANT You cannot enable switch management through SAM — you must

use the clic_init command.

When clic_init begins to prompt you for the information for each switch, it assigns an ID (for example, sw_clic0) to that switch and displays it as part of the ﬁrst prompt. Note that you can also determine a switch’s ID by running the clic_stat command (see “The clic_stat Command” on page 117). You should note each switch’s ID, because it is used as input to other HyperFabric commands.

❏ For the entire fabric, you need the IP multicast address that all the

switches and nodes in the fabric will register to. The address must be a class D address. Note that if you do not have switch management enabled, you do not need this information (clic_init will not prompt you for it).

❏ For each node in the fabric, you need the IP address of the node’s

Ethernet LAN interface that is on the same subnet as the switches. (As mentioned earlier, a node might have multiple LAN interfaces.) Note that if you do not have switch management enabled, you do not need this information (clic_init will not prompt you for it).

As stated earlier, HP recommends that you do not enable switch management.

IMPORTANT You should also check your /etc/hosts ﬁle — when you are using ﬁles

for host name look up — to ensure that the entries for all of the systems are in the correct format: the ofﬁcial host name, which is the full domain extended host name, and any alias names. For example:

Chapter 472

Conﬁguring HyperFabric

Information You Need

IP_address IP_address IP_address

bently6.corp3.com bently6 bently4.corp7.com test1 bently2.corp4.com test3

Chapter 4 73

Conﬁguring HyperFabric

Information You Need

Conﬁguration Information Example

This example uses some “dummy” (that is, not valid) addresses to the components in Figure 4-1. The dummy addresses are used only to show the ﬂow of the information provided as input to the clic_init command and SAM. Do not try to use these addresses in your conﬁguration.

To use clic_init to conﬁgure the Transparent Local Failover feature on HMP, see the section, “Conﬁguring HMP for Transparent Local Failover Support - Using the clic_init command” on page 104.

Figure 4-1 Map for Conﬁguration Information Example

Ethernet LAN

Switch ID: sw_clic0 IP address: 193.0.0.20

Ethernet MAC address:

00:60:b0:d0:02:57

IP multicast address:

226.10.1.1

HF switch 0

switch 1

Switch ID: sw_clic1

IP address: 193.0.0.21 Ethernet MAC address:

00:60:b0:d0:02:56

IP multicast address:

226.10.1.1

adapter 0

Adapter ID:

clic0

IP address:

192.0.0.1

subnet mask:

255.255.255.0

node A

IP address: 193.0.0.10 IP address: 193.0.0.11

adapter 1

Adapter ID:

clic1

IP address:

192.0.8.3

subnet mask:

255.255.255.0

lan0 lan0

Adapter ID:

clic0

IP address:

192.0.0.2

subnet mask:

255.255.255.0

adapter 0

node A

node B

adapter 1

Adapter ID:

clic1

IP address:

192.0.8.4

subnet mask:

255.255.255.0

Chapter 474

Conﬁguring HyperFabric

Information You Need

Using the conﬁguration information in Figure 4-1, the information you would specify when you run clic_init or SAM on each of the nodes is listed below. This example is not an exact depiction of the prompts produced by clic_init nor the ﬁelds in SAM, but merely an example of the ﬂow of information input. In addition, you should not try to use the dummy addresses in your actual conﬁguration.

On node A:

1. How many HyperFabric adapters are installed on the node?

2. Do you want this node to interoperate with nodes running any HyperFabric versions earlier than B.11.00.11 or B.11.11.01?

3. What is the IP address of the ﬁrst adapter (clic0)? (192.0.0.1)

4. What is the subnet mask of the ﬁrst adapter? (255.255.255.0) If you do not specify a value for this, a default mask is chosen. You

will most likely just accept the default. However, in this example, we are showing a value for the subnet mask just to illustrate the correlation between the “dummy” information in Figure 4-1 and where that information is speciﬁed or generated during clic_init and SAM.

5. What is the IP address of the second adapter (clic1)? (192.0.8.3)

6. What is the subnet mask of the second adapter? (255.255.225.0)

7. Do you want to enable switch management? You cannot enable switch management through SAM (you must use the clic_init command).

As stated earlier, HP recommends that you do not enable switch management. However, if you do enable it, you must provide the information in items 8 through 14.

8. If switch management has been enabled, how many switches will be conﬁgured? As stated earlier, HP recommends that you do not enable switch management.

9. What is the IP address of the ﬁrst switch (sw_clic0)? (193.0.0.20)

10. What is the Ethernet hardware address of the ﬁrst switch? (0060b0d00257)

11. What is the IP address of the second switch (sw_clic1)? (193.0.0.21)

Chapter 4 75

Conﬁguring HyperFabric

Information You Need

12. What is the Ethernet hardware address of the second switch? (0060b0d00256)

13. What is the Multicast address for the switches to use? (226.10.1.1)

14. What is the IP address for the LAN card on the same subnet as the switches? (193.0.0.10)

(In Figure 4-1, this is the IP address for lan0 on node A.)

On node B:

1. How many HyperFabric adapters are installed on the node?

2. Do you want this node to interoperate with nodes running any HyperFabric versions earlier than B.11.00.11 or B.11.11.01?

3. What is the IP address of the ﬁrst adapter (clic0)? (192.0.0.2)

4. What is the subnet mask of the ﬁrst adapter? (255.255.255.0) If you do not specify a value for this, a default mask is chosen. You

will most likely just accept the default. However, in this example, we are showing a value for the subnet mask just to illustrate the correlation between the dummy information in Figure 4-1 and where that information is speciﬁed or generated during clic_init and SAM.

5. What is the IP address of the second adapter (clic1)? (192.0.8.4)

6. What is the subnet mask of the second adapter? (255.255.225.0)

7. Do you want to enable switch management? You cannot enable switch management through SAM (you must use the clic_init command).

As stated earlier, HP recommends that you do not enable switch management. However, if you do enable it, you must provide the information in items 8 through 14.

8. If switch management has been enabled, how many switches will be conﬁgured? As stated earlier, HP recommends that you do not enable switch management.

9. What is the IP address of the ﬁrst switch (sw_clic0)? (193.0.0.20)

10. What is the Ethernet hardware address of the ﬁrst switch? (0060b0d00257)

11. What is the IP address of the second switch (sw_clic1)? (193.0.0.21)

Chapter 476

Conﬁguring HyperFabric

Information You Need

12. What is the Ethernet hardware address of the second switch? (0060b0d00256)

13. What is the Multicast address for the switches to use? (226.10.1.1)

14. What is the IP address for the LAN card on the same subnet as the switches? (193.0.0.11)

(In Figure 4-1, this is the IP address for lan0 on node B.)

Chapter 4 77

Conﬁguring HyperFabric

Performing the Conﬁguration

As explained in “Conﬁguration Overview” on page 69, you must create the global conﬁguration ﬁle (/etc/rc.config.d/clic_global_conf)on each node in the fabric. This consists mostly of specifying HyperFabric adapter-related information. (If you are also going to enable switch management — which HP does not recommend doing — you need to specify additional conﬁguration information.)

NOTE Specifying conﬁguration information adds or changes only the addresses

and other information in the global conﬁguration ﬁle, based on the information you supply. It does not perform any operations to check the relationships between that information and any physical connections within the fabric.

You need to create the global conﬁguration ﬁle in the following situations:

• You have just installed the HyperFabric hardware and software on the system.

• You want to change the information in the HyperFabric global conﬁguration ﬁle (see the preceding note).

IMPORTANT Creating the global conﬁguration ﬁle also modiﬁes the

/etc/rc.config.d/netconf ﬁle, adding some HyperFabric-related lines that end with the characters #clic. These lines are used by the HyperFabric software — and are not comments — so do not remove them from the ﬁle.

You can create the global conﬁguration ﬁle by using the clic_init command (described in “Using the clic_init Command” on page 79) or SAM (described in “Using SAM” on page 81). You cannot enable switch management through SAM (you must use the clic_init command).

Chapter 478

Conﬁguring HyperFabric

Performing the Conﬁguration

Using the clic_init Command

Run the clic_init command to create the global conﬁguration ﬁle. To view the man page for clic_init see “Viewing man Pages” on

page 127 of this manual. To use clic_init to conﬁgure the Transparent Local Failover feature on

HMP, see the section, “Conﬁguring HMP for Transparent Local Failover Support - Using the clic_init command” on page 104.

IMPORTANT If the global conﬁguration ﬁle already exists and you are running

clic_init again (to change the ﬁle), you have the option of retaining or modifying the existing conﬁguration information, in addition to adding new information pertaining to new hardware.

Also, once you have completed your changes and clic_init ends its processing, you must stop HyperFabric (by running the clic_shutdown command or using SAM) and then start HyperFabric (by running the clic_start command or using SAM). Otherwise, your conﬁguration information changes will not take effect. See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.

If you include /opt/clic/bin in your PATH statement, you can run the command as it is shown below. Otherwise, you must include

/opt/clic/bin as part of the command name (that is, /opt/clic/bin/clic_init).

You must be logged in as root to run this command. The syntax is as follows:

clic_init [-c] [-?]

where

• -c speciﬁes that you want to create the global conﬁguration ﬁle. You are prompted for the information described in “Information You Need” on page 71. Note that if the global conﬁguration ﬁle already exists (for example, when you are adding an adapter to an existing fabric), clic_init prompts you with the existing conﬁguration information. As you are prompted with each piece of information, you can then conﬁrm that you want to keep it or you can change it.

• -? displays the online help for clic_init.

Chapter 4 79

Conﬁguring HyperFabric

Performing the Conﬁguration

If you do not specify any of the above parameters, the online help for clic_init is displayed.

After you have entered the information for all the adapters in the node and all of the switches (if any) in the fabric, a summary of the conﬁguration information is displayed.

Once clic_init has ﬁnished, you do one of the following things:

• If you want to conﬁgure HyperFabric with ServiceGuard, complete the conﬁguration described in “Conﬁguring HyperFabric with ServiceGuard” on page 87, then run clic_start or use SAM to start HyperFabric.

• If you have just created the global conﬁguration ﬁle on the local node for the ﬁrst time (and you are not conﬁguring ServiceGuard), run clic_start or use SAM to start HyperFabric.

• If you have just changed an existing conﬁguration ﬁle on the node, run clic_shutdown or use SAM to stop HyperFabric, and then run clic_start or use SAM to start HyperFabric. Until you do those two things, your conﬁguration changes will not take effect.

See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.

Examples of clic_init

Some examples of using the clic_init command are shown below.

• Example 1 To create the global conﬁguration ﬁle on the local node, issue this

command:

$ clic_init -c

• Example 2 To display the online help for the clic_init command, issue this

command:

$ clic_init -?

or this command:

$ clic_init

Chapter 480

Conﬁguring HyperFabric

Performing the Conﬁguration

Using SAM

This section describes how to use SAM to conﬁgure HyperFabric. For information on how to use SAM to conﬁgure and deconﬁgure local failover feature on HMP, see “Conﬁguring HMP for Transparent Local Failover Support - Using SAM” on page 102 and “Deconﬁguring HMP for Local Failover support - Using SAM” on page 103.

IMPORTANT If the global conﬁguration ﬁle already exists, and you are running SAM

again (to change the ﬁle), you can keep or modify the existing conﬁguration information, in addition to adding new information pertaining to new hardware.

Also, once you’ve completed your changes and SAM ends its processing, you must stop HyperFabric (by running the clic_shutdown command or using SAM) and then start HyperFabric (by running the clic_start command or using SAM). Otherwise, your conﬁguration information changes will not take effect. See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.

To use SAM to create the global conﬁguration ﬁle on an HP 9000 system running HP-UX 11i v2, follow these steps:

Step 1. Start SAM.

Step 2. Select the “Networking and Communications” area.

Step 3. Select “HyperFabric.”

All HyperFabric adapters installed in the system are listed; installed adapters that are not yet conﬁgured show Not Configured in the “Status” ﬁeld.

Step 4. Highlight the adapter you want to conﬁgure.

Step 5. Pull down the “Actions” menu and select Configure Adapter.

Step 6. In the “Conﬁgure HyperFabric Adapter” screen, specify information for

the following ﬁelds:

• Internet Address—Required. The IP address of the adapter.

Chapter 4 81

Conﬁguring HyperFabric

Performing the Conﬁguration

• Subnet Mask—Optional. The adapter’s subnet mask. If you do not specify this, a default mask is chosen based on the adapter’s IP address.

• Interoperability Enabled—Required. Whether you want the adapter to be able to interoperate with adapters that are using; any HP-UX 11.0 HyperFabric versions earlier than B.11.00.11 or any HP-UX 11i v1 HyperFabric versions earlier than B.11.11.01. Note that if you select No, the HyperFabric software on the system will not be backwards compatible with previous releases. This means you must update all of the other systems in the fabric to the version that is running on the system.

Default: No.

Step 7. Select OK (remember, you cannot enable switch management within

SAM).

Step 8. Exit SAM.

Once SAM has ﬁnished, you do one of the following things:

• If you have just created the global conﬁguration ﬁle on the local node for the ﬁrst time (and you are not conﬁguring ServiceGuard), run clic_start or use SAM to start HyperFabric.

See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.

Chapter 482

Deconﬁguring a HyperFabric Adapter with SAM

To use SAM to deconﬁgure a HyperFabric adapter on an HP 9000 system running HP-UX 11i v2, complete the following steps:

Step 1. Start SAM.

Step 2. Select the “Networking and Communications” area.

Step 3. Select “HyperFabric.”

All HyperFabric adapters installed in the system are listed. Installed adapters that are conﬁgured show Configured in the “Status” ﬁeld, and installed adapters that are not yet conﬁgured show Not Configured in the “Status” ﬁeld. You can deconﬁgure only an adapter with a status of Configured.

Step 4. Highlight the adapter you want to deconﬁgure.

Conﬁguring HyperFabric

Step 5. Pull down the “Actions” menu and select Deconfigure Adapter.

Step 6. In the pop-up window, if you want to deconﬁgure the adapter, select OK to

conﬁrm it.

If you do not want to deconﬁgure the adapter, select Cancel.

Step 7. If you selected OK, the entry for the adapter is deleted from the

HyperFabric conﬁguration ﬁles (/etc/rc.config.d/clic_global_conf and /etc/rc.config.d/netconf).

If you selected Cancel, you remain in the main “HyperFabric Conﬁguration” screen.

Step 8. Exit SAM.

NOTE If you have conﬁgured HMP for Transparent Local Failover support and

if you select Deconfigure Adapter, HyperFabric will verify if the selected adapter is conﬁgured to be part of any card pair. If yes, the user

Chapter 4 83

Conﬁguring HyperFabric

Deconﬁguring a HyperFabric Adapter with SAM

is informed and the card pair entry is removed from the

/etc/rc.config.d/netconf and /etc/rc.config.d/clic_global_conf ﬁles.

Chapter 484

Conﬁguring HyperFabric

Conﬁguring the HyperFabric EMS Monitor

In the HyperFabric version B.11.23.01, the HyperFabric Event Monitoring Service (EMS) monitor allows system administrators to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem.

The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator deﬁnes the condition to trigger a notiﬁcation (usually a change in interface status). Notiﬁcation can be accomplished with a SNMP trap or by logging into the syslog ﬁle with a choice of severity, or by email to a user deﬁned email address.

To conﬁgure the HyperFabric EMS monitor, it is necessary to have the EMS HA monitor product installed (Product Number B7609BA). This product is available on the applications CD media.

Use SAM to initiate monitoring of any particular HyperFabric resource. following the procedure outlined below:

1. Start SAM (Use the online help at any time for details)

2. Select “Resource Management”

3. Select “Event Monitoring Service”

4. Select “Action” and “Add Monitoring Request”

5. Select the location /net/interfaces/clic (class for HyperFabric resources)

6. Select a resource instance (either all instances or a speciﬁc instance from the list)

7. Validate your choice by clicking on OK at the bottom of the screen

8. A Monitoring Request Parameters window opens, showing the resource and its status (if All instances have been selected, then no value is displayed)

9. Deﬁne a condition that will trigger a notiﬁcation (for instance, “When Value is”, “equal to”, “UP”)

10. Deﬁne a polling interval (default is 300 seconds)

11. Deﬁne a way of notiﬁcation: SNMP trap, log in syslog with a choice of severity, or email to a user deﬁned email address

Chapter 4 85

Conﬁguring HyperFabric

Conﬁguring the HyperFabric EMS Monitor

12. Validate by pressing OK

NOTE Although EMS is able to monitor each HyperFabric adapter on every

node in the fabric, as well as the entire HyperFabric subsystem, EMS is not able to monitor HyperFabric switches.

For more detailed information on EMS, including instructions for implementing this feature, see the EMS Hardware Monitors Users Guide Part Number B6191-90028 September 2001 Edition.

Chapter 486

HP HyperFabric Administrator's Guide

Specifications and Main Features

Frequently Asked Questions

User Manual

About HyperFabric

HyperFabric Products

HyperFabric Adapters

Switches and Switch Modules

Other Product Elements

HyperFabric Concepts

2 Planning the Fabric

Preliminary Considerations

TCP/UDP/IP

Application Availability

Features

Hyper Messaging Protocol (HMP)

Application Availability

Features

3 Installing HyperFabric

Checking HyperFabric Installation Prerequisites

Installing HyperFabric Adapters

Online Addition and Replacement

Planning and Preparation

Critical Resources

Card Compatibility

Installing the Software

File Structure

Loading the Software

Installing HyperFabric Switches

Before Installation

Installing the HF2 Switch

With the Rail Kit

Without the Rail Kit

Information You Need

Using the clic_init Command

Examples of clic_init

Using SAM