HP HyperFabric Administrator's Guide

HyperFabric Administrator’s Guide

HP-UX 11i v2

Edition 14

Manufacturing Part Number : B6257-90043

March 2004

United States

Legal Notices

The information in this document is subject to change without notice.

Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and ﬁtness for a particular purpose. Hewlett-Packard

shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material.

Warranty

A copy of the speciﬁc warranty terms applicable to your Hewlett-Packard product and replacement parts can be obtained from your local Sales and Service Ofﬁce.

U.S. Government License

Proprietary computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Copyright  2004 Hewlett-Packard Development Company L.P. All rights reserved. Reproduction, adaptation, or translation of this document without prior written permission is prohibited, except as allowed under the copyright laws.

Trademark Notices

Oracle  is a registered trademark of Oracle Corporation. UNIX is a registered trademark in the United States and other

countries, licensed exclusively through The Open Group.

About This Document

1. Overview of the HyperFabric Product

About HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

HyperFabric Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Switches and Switch Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Other Product Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

HyperFabric Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2. Planning the Fabric

Preliminary Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and

HMP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

TCP/UDP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Conﬁguration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

TCP/UDP/IP Supported Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Point-to-Point Conﬁgurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Switched Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

High Availability Switched Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Hybrid Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Hyper Messaging Protocol (HMP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

Conﬁguration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

HMP Supported Conﬁgurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Point-to-Point Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Enterprise (Database) Conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Technical Computing (Work Stations) Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . 37

Contents

3. Installing HyperFabric

Checking HyperFabric Installation Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Installing HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

iii

Contents

Online Addition and Replacement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Planning and Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Critical Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Card Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Installing the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Loading the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Installing HyperFabric Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Before Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Installing the HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

With the Rail Kit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Without the Rail Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4. Conﬁguring HyperFabric

Conﬁguration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Information You Need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Conﬁguration Information Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Performing the Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Using the clic_init Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Examples of clic_init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Deconﬁguring a HyperFabric Adapter with SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Conﬁguring the HyperFabric EMS Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Conﬁguring HyperFabric with ServiceGuard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

How HyperFabric Handles Adapter Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Conﬁguring HyperFabric with the ServiceGuard Resource Monitor. . . . . . . . . . . 94

Conﬁguring ServiceGuard with HyperFabric Using the ASCII File . . . . . . . . . . . 94

Conﬁguring ServiceGuard with HyperFabric Using SAM . . . . . . . . . . . . . . . . . . . 94

Conﬁguring ServiceGuard for HyperFabric Relocatable IP Addresses . . . . . . . . . 95

Conﬁguring HMP for Transparent Local Failover Support. . . . . . . . . . . . . . . . . . . . . . 96

How Transparent Local Failover Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Conﬁguring HMP for Transparent Local Failover Support - Using SAM. . . . . . . . 102

Deconﬁguring HMP for Local Failover support - Using SAM . . . . . . . . . . . . . . . . . 103

Conﬁguring HMP for Transparent Local Failover Support - Using the clic_init

command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

5. Managing HyperFabric

Starting HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Using the clic_start Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Verifying Communications within the Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

The clic_probe Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Examples of clic_probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Displaying Status and Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

The clic_stat Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Examples of clic_stat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Viewing man Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Stopping HyperFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Using the clic_shutdown Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

6. Troubleshooting HyperFabric

Running Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

The clic_diag Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Example of clic_diag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Using Support Tools Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Useful Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

LED Colors and Their Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

HF2 Switch LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Determining Whether an Adapter or a Cable is Faulty. . . . . . . . . . . . . . . . . . . . . . . . 153

Determining Whether a Switch is Faulty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Replacing a HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Replacing a HyperFabric Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Safety Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Regulatory Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Adapters and Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

FCC Statement (USA only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

DOC Statement (Canada only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Europe RFI Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Australia and New Zealand EMI Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Contents

Radio Frequency Interference (Japan Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Declarations of Conformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Physical Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Environmental . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Figures

Figure 2-1. TCP/UDP/IP Point-To-Point Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . 22

Figure 2-2. TCP/UDP/IP Basic Switched Conﬁguration . . . . . . . . . . . . . . . . . . . . . . 23

Figure 2-3. TCP/UDP/IP High Availability Switched Conﬁguration . . . . . . . . . . . . 24

Figure 2-4. TCP/UDP/IP Hybrid Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 2-5. HMP Point-To-Point Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Figure 2-6. HMP Enterprise (Database) Conﬁguration, Single Connection Between

Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Figure 2-7. Local Failover Supported Enterprise (Database) Conﬁguration, Multiple

Connections between Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Figure 2-8. Technical Computing Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Figure 2-9. Large Technical Computing Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . 39

Figure 3-1. HyperFabric File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Figure 3-2. Front of HF2 Switch (A6388A Switch Module Installed) . . . . . . . . . . . . 59

Figure 3-3. Front of HF2 Switch (A6389A Switch Module Installed) . . . . . . . . . . . . 60

Figure 4-1. Map for Conﬁguration Information Example . . . . . . . . . . . . . . . . . . . . . 74

Figure 4-2. An ServiceGuard Conﬁguration (with Two HyperFabric Switches) . . . 89

Figure 4-3. Node with Two Active HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . 91

Figure 4-4. Node with One Failed HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . 92

Figure 4-5. When All HyperFabric Adapters Fail. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Figure 4-6. A Conﬁguration supporting Local Failover . . . . . . . . . . . . . . . . . . . . . . . 98

Figure 4-7. Adapter, Link or Switch Port Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Figure 4-8. Switch Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Figure 4-9. Cable Failover Between Two Switches . . . . . . . . . . . . . . . . . . . . . . . . . 101

Figure 4-10. Conﬁguring the Transparent Local Failover feature . . . . . . . . . . . . . 104

vii

Figures

viii

Tables

Table 1. HP-UX 11i Releases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Table 2. Publishing History Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Table 3. Organization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Table 2-1. HF2 Throughput and Latency with TCP/UDP/IP Applications. . . . . . . . 19

Table 2-2. Supported HyperFabric Adapter Conﬁgurations . . . . . . . . . . . . . . . . . . . 20

Table 2-3. HF2 Throughput and Latency with HMP Applications . . . . . . . . . . . . . . 31

Table 2-4. Supported HyperFabric Adapter Conﬁgurations . . . . . . . . . . . . . . . . . . . 31

Table 3-1. Important OLAR Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Table 6-1. LED Names (by Adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Table 6-2. HyperFabric Adapter LED Colors and Meanings . . . . . . . . . . . . . . . . . . 145

Table 6-3. HF2 Switch LED Colors and Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . 150

Tables

About This Document

This document describes how to install, conﬁgure, and troubleshoot the HyperFabric product on the HP-UX 11i v2 (HP-UX 11.23) operating system.

Before you install the HyperFabric software, ensure that the operating system software and the appropriate ﬁles, scripts, subsets are installed.

The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The document part number will change when extensive changes are made.

Document updates may be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, you should subscribe to the appropriate product support service. See your HP sales representative for details.

The latest version of this manual can be found online at

http://www.docs.hp.com/hpux/netcom/index.html#HyperFabric.

Intended Audience

This manual is intended for system and network administrators responsible for installing, conﬁguring, and managing the HyperFabric software and hardware. Administrators are expected to have knowledge of operating system concepts, commands, and conﬁguration.

It is helpful to have knowledge of Transmission Control Protocol/Internet Protocol (TCP/IP) networking concepts and network conﬁguration.

This document is not a tutorial.

HP-UX Release Name and Release Identiﬁer

Each HP-UX 11i release has an associated release name and release identiﬁer. The uname (1) command with the -r option returns the release identiﬁer. Table 1 shows the releases available for HP-UX 11i.

Table 1 HP-UX 11i Releases

Supported

Release Identiﬁer Release Name

B.11.11 HP-UX 11i v1 PA-RISC

B.11.20 HP-UX 11i v1.5 Intel Itanium

B.11.22 HP-UX 11i v1.6 Intel Itanium

B.11.23 HP-UX 11i v2 Intel Itanium

Processor

Architecture

Publishing History

Table 2 provides the publication date for the pertinent edition number.

Table 2 Publishing History Details

Edition Number Publication Date

xii

First March 1998

Second June 1998

Third August 1998

Fourth October 1998

Fifth December 1998

Sixth February 1999

Seventh April 1999

Eighth March 2000

Ninth June 2000

Tenth December 2000

Table 2 Publishing History Details (Continued)

Edition Number Publication Date

Eleventh June 2001

Twelfth September 2002

Thirteenth July 2003

Fourteenth March 2004

What Is in This Document

HyperFabric Administrator’s Guide is divided into several chapters, each of which contains information about installing, conﬁguring, or troubleshooting HyperFabric. The appendixes contain supplemental information.

The following list describes the content in more detail.

Table 3 Organization

Chapter Description

Overview of the HyperFabric Product

Planning the Fabric Describes the steps that need to be

Installing HyperFabric Describes the tasks to install the

Conﬁguring HyperFabric Describes the tasks to conﬁgure

Managing HyperFabric Describes the tasks to start, stop and

Troubleshooting HyperFabric

Presents an overview of HyperFabric and lists the components that the HyperFabric product contains

followed while planning the fabric

HyperFabric products on the HP-UX 11i v2 operating system

HyperFabric

manage HyperFabric

Describes how to troubleshoot HyperFabric

xiii

New and Changed Information in This Edition

This edition includes information about the transparent local failover feature of Hyper Messaging Protocol (HMP). This feature is available with the HyperFabric version B.11.23.01.

Typographic Conventions

This document uses the following typographic conventions: Book Title Italic (slanted) type indicates document and book

names.

daemon Courier font type indicates daemons, ﬁles, commands,

manpages, and option names.

[ ] { } In syntax deﬁnitions, square brackets indicate items

that are optional and braces indicate items that are required.

HP Welcomes Your Feedback

HP welcomes any comments and suggestions you have on this manual. You can send your comments in the following ways:

• Internet electronic mail: netinfo_feedback@cup.hp.com

xiv

• Using a feedback form located at the following URL:

http://docs.hp.com/assistance/feedback.html

Please include the following information along with your comments:

• The complete title of the manual and the part number. (The part number appears on the title page of printed and PDF versions of a manual.)

• The section numbers and page numbers of the information on which you are commenting.

• The version of HP-UX that you are using.

Please note that the HP-UX networking communications publications group does not provide technical support for HP products.

1 Overview of the HyperFabric

Product

This chapter contains the following sections that give general information about HyperFabric:

Chapter 1 1

Overview of the HyperFabric Product

• “About HyperFabric” on page 3

• “HyperFabric Products” on page 4

• “HyperFabric Concepts” on page 7

Chapter 12

Overview of the HyperFabric Product

About HyperFabric

HyperFabric is an HP high-speed, packet-based interconnect for node-to-node communications. HyperFabric provides higher speed, lower network latency and uses less CPU than other industry standard protocols (for example, Fibre Channel and Gigabit Ethernet). Instead of using a traditional bus-based technology, HyperFabric is built around switched fabric architecture, providing the bandwidth necessary for high speed data transfer. This clustering solution delivers the performance, scalability, and high availability required by the following:

• Parallel Database Clusters

— Oracle 9i Real Application Clusters (RAC) — Oracle 8i Parallel Servers (OPS)

• Parallel Computing Clusters

• Client/Server Architecture Interconnects (for example, SAP)

• Multi-Server Batch Applications (for example, SAS Systems)

• Enterprise Resource Planning (ERP)

• Technical Computing Clusters

• HP Message Passing Interface (MPI) based applications

• OpenView Data Protector (earlier known as Omniback)

• Network Backup

• Data Center Network Consolidation

• E-services

Chapter 1 3

Overview of the HyperFabric Product

HyperFabric Products

HyperFabric hardware consists of host-based interface adapter cards, interconnect cables, and optional switches. HyperFabric software resides in Application Speciﬁc Integrated Circuits (ASICs) and ﬁrmware on the adapter cards and includes user-space components and HP-UX drivers.

Currently, ﬁber-based HyperFabric hardware are available. In addition, a hybrid switch that has 8-ﬁber ports is available to support HF2 clusters.

This section describes the various HyperFabric products. For more information on HP 9000 systems that support HyperFabric products, see the HyperFabric Release Notes, available at

http://docs.hp.com/hpux/netcom/index.html#HyperFabric.

NOTE This document uses the term HyperFabric (HF) to refer to the

hardware and software that form the HyperFabric cluster interconnect product.

The term HyperFabric2 (HF2) refers to the following ﬁber-based hardware components:

• The A6386A adapter

• The A6384A switch chassis

• The A6388A and A6389A switch modules. (Although the A6389A switch module has 4-copper ports, it is still considered an HF2 component because it can only be used with the A6384A HF2 switch chassis).

• The C7524A, C7525A, C7526A, and C7527A cables

HyperFabric Adapters

The HyperFabric adapters are as follows:

• A6386A HF2 PCI (4X) adapter with a ﬁber interface.

The A6092A HyperFabric adapter is supported beginning with the following HyperFabric software versions:

Chapter 14

Overview of the HyperFabric Product

HyperFabric Products

• HP-UX 11.0: HyperFabric software version B.11.00.09

• HP-UX 11i v1: HyperFabric software version B.11.11.00

• HP-UX 11i v2: HyperFabric software version B.11.23.00

The A6386A HyperFabric2 adapter is supported beginning with the following HyperFabric software versions:

• HP-UX 11.0: HyperFabric software version B.11.00.11

• HP-UX 11i v1: HyperFabric software version B.11.11.01

• HP-UX 11i v2: HyperFabric software version B.11.23.00

Switches and Switch Modules

The HyperFabric2 switches are as follows:

• A6384A HF2 ﬁber switch chassis with one integrated Ethernet management LAN adapter card, one integrated 8-port ﬁber card, and one expansion slot. For the chassis to be a functional switch, install one of the following switch modules in the expansion slot:

— The A6388A HF2 8-port ﬁber switch module. This gives 16-ﬁber

ports to the switch (8 from the integrated ﬁber card and 8 from the A6388A switch module).

— The A6389A HF2 4-port copper switch module. This gives 12

ports to the switch - a mixture of 8-ﬁber ports (from the integrated ﬁber card) and 4-copper ports (from the A6389A module).

The A6384A HF2 switch chassis with either module installed is supported beginning with the following HyperFabric software versions:

• HP-UX 11.0: HyperFabric software version B.11.00.11

• HP-UX 11i v1: HyperFabric software version B.11.11.01

• HP-UX 11i v2: HyperFabric software version B.11.23.00

NOTE In this manual, the terms HyperFabric2 switch or HF2 switch refer

to the functional switch (the A6384A switch chassis with one of the switch modules installed).

Chapter 1 5

Overview of the HyperFabric Product

HyperFabric Products

IMPORTANT HF2 adapters and switches are not supported by software versions

earlier than those listed in “HyperFabric Adapters” on page 4 and “Switches and Switch Modules” on page 5.

To determine the version of HyperFabric, issue the following command:

$ swlist | grep -i hyperfabric

Other Product Elements

The following are the other elements of the HyperFabric product family:

• HF2 ﬁber cables

— C7524A (2m length) — C7525A (16m length) — C7526A (50m length) — C7527A (200m length)

• The HyperFabric software: The software resides in ASICs and ﬁrmware on the adapter cards and includes user-space components and HP-UX drivers.

HyperFabric supports the IP network protocol stack, speciﬁcally TCP/IP and UDP/IP.

HyperFabric software includes HyperMessaging Protocol (HMP). HMP provides higher bandwidth, lower CPU overhead, and lower latency (the time a message takes to get from one point to another). However, these HMP beneﬁts are available only when applications that are developed on top of HMP are running. HMP can only be used on HP 9000 systems running HP-UX 11.0 or 11i v1, provided HyperFabric A6092A or A6386A (PCI 4X) adapter cards are installed on those systems.

Chapter 16

Overview of the HyperFabric Product

HyperFabric Concepts

This section brieﬂy describes some of the basic HyperFabric concepts and terms.

The fabric is the physical conﬁguration that consists of all the HyperFabric adapters, the HyperFabric switches (if any), and the HyperFabric cables connecting them. The network software controls data transfer over the fabric.

The HyperFabric conﬁguration contains two or more HP 9000 systems and optional HyperFabric switches. Each HP 9000 acts as a node in the conﬁguration. Each node has a minimum of one and a maximum of eight HyperFabric adapters installed in it. (For information on the maximum number of adapters that can be installed in each system, see Chapter 2, “Planning the Fabric,” on page 9.) Each HF2 switch can be conﬁgured with 12 or 16 ports. HyperFabric supports a maximum of 4 HyperFabric switches. You can mesh HyperFabric switches and conﬁgurations with up to four levels of meshed switches are supported.

You can plan a HyperFabric cluster as a High Availability (HA) conﬁguration, when it is necessary to ensure that each node can always participate in the fabric. This is done by using ServiceGuard (earlier known as MC/ServiceGuard), ServiceGuard OPS Edition (earlier known as MC/LockManager), and the Event Monitoring Service (EMS). Conﬁgurations of up to 8 nodes are supported under ServiceGuard.

Beginning with HyperFabric software versions B.11.00.05 and B.11.11.00,you can use relocatable IP addresses as part of an HA conﬁguration. Relocatable IP addresses permit a client application to reroute through an adapter on a remote node, allowing that application to continue processing without interruption. The rerouting is transparent. This function is associated with ServiceGuard (see “Conﬁguring ServiceGuard for HyperFabric Relocatable IP Addresses” on page 95). When the monitor for HyperFabric detects a failure and the backup adapter takes over, the relocatable IP address is transparently migrated to the backup adapter. Throughout this migration process, the client application continues to execute normally.

When you start HyperFabric (with the clic_start command, through SAM, or by booting the HP 9000 system), you start the management process. This process must be active for HyperFabric to run. If the HyperFabric management process on a node stops running for some

Chapter 1 7

Overview of the HyperFabric Product

HyperFabric Concepts

reason (for example, if it is killed), all HyperFabric-related communications on that node are stopped immediately. This makes the node unreachable by other components in the fabric.

When you start HyperFabric, the fabric is veriﬁed automatically. This is because each node performs a self diagnosis and veriﬁcation over each adapter installed in the node. In addition, the management process performs automatic routing and conﬁguring for each switch (if switches are part of the fabric). You can, if needed, run the clic_stat command to get a textual map of the fabric, which can be used as another method of quick veriﬁcation.

Notice that the commands to administer HyperFabric have a preﬁx of clic_, and some of the other components have CLIC as part of their name (for example, the CLIC ﬁrmware and the CLIC software). CLIC stands for CLuster InterConnect, and it is used to differentiate those HyperFabric commands or components from other commands or components. For example, the HyperFabric command clic_init is different from the HP-UX init command.

Chapter 18

2 Planning the Fabric

This chapter contains the following sections that include general guidelines and protocol-speciﬁc considerations for planning HyperFabric clusters that run TCP/UDP/IP or HMP applications.

Chapter 2 9

Planning the Fabric

• “Preliminary Considerations” on page 11

• “HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and HMP Applications” on page 13

• “TCP/UDP/IP” on page 14

• “Hyper Messaging Protocol (HMP)” on page 26

Chapter 210

Planning the Fabric

Preliminary Considerations

Before assembling a fabric physically, do the following to address all of the appropriate issues:

Step 1. Read Chapter 1, “Overview of the HyperFabric Product,” on page 1, to

get a basic understanding of HyperFabric and its components.

Step 2. Read this chapter, Planning the Fabric, to gain an understanding of

protocol speciﬁc conﬁguration guidelines for TCP/UDP/IP and HMP applications.

Step 3. Read “Conﬁguration Overview” on page 69, “Information You Need” on

page 71, and “Conﬁguration Information Example” on page 74, to understand how to conﬁgure the fabric.

Step 4. Decide the number of nodes that will be interconnected in the fabric.

Step 5. Decide the type of HP 9000 system for each node (for a list of supported

HP 9000 systems, see the HyperFabric Release Notes available at

http://docs.hp.com/hpux/netcom/index.html#HyperFabric.

Step 6. Determine the network bandwidth requirements for each node.

Step 7. Determine the number of adapters needed for each node.

Step 8. Determine if a High Availability (ServiceGuard) conﬁguration will be

needed. If ServiceGuard is used, each node should have at least two adapters.

Step 9. Decide the topology of the fabric.

Step 10. Determine how many switches will be used based on the number of

nodes in the fabric. The only conﬁguration that can be supported without a switch is the node-to-node conﬁguration (HA or non-HA). HyperFabric supports meshed switches up to a depth of four switches, starting with the following versions of the HyperFabric software:

• For HF2 switches: software versions B.11.00.11, B.11.11.01, and B.11.23.00.

Chapter 2 11

Planning the Fabric

Preliminary Considerations

Step 11. Draw the cable connections from each node to switches (if the fabric will

contain switches). If you use an HA conﬁguration with switches, it requires more than one switch for complete redundancy and to avoid a single point of failure. For example, each adapter can be connected to its own switch, or two switches can be connected to four adapters.

Chapter 212

Planning the Fabric

HyperFabric Features, Parameters and Supported Conﬁgurations for TCP/UDP/IP and HMP Applications

The following sections in this chapter deﬁne HyperFabric functionality for TCP/UDP/IP applications and Hyper Messaging Protocol (HMP) applications. There are distinct differences in supported hardware, available features and performance, depending on which protocol is used by applications running on the HyperFabric.

Chapter 2 13

Planning the Fabric

TCP/UDP/IP

TCP/UDP/IP is supported on all HF2 hardware. Although some of the HyperFabric adapter cards support both HMP and TCP/UDP/IP applications, in this section, the focus is on TCP/UDP/IP HyperFabric applications.

Application Availability

All applications, including Oracle 9i and HP-MPI, that use the TCP/UDP/IP stack are supported.

Features

This section discusses the following HyperFabric features on TCP/UDP/IP:

• OnLine Addition and Replacement (OLAR): Supported The OLAR feature allows the replacement or addition of

HyperFabric adapter cards while the system (node) is running. HyperFabric supports this functionality on the SD64A, rx8620, rx4640, rp54xx (L-class), rp74xx (N-class), rp8400 and Superdome systems, running on the HP-UX 11i v2 platform.

For more information on OLAR, including instructions for implementing this feature, see “Online Addition and Replacement” on page 44 and Conﬁguring HP-UX for Peripherals Part Number B2355-90698 November 2000 Edition.

• Event Monitoring Service (EMS): Supported In the HyperFabric version B.11.23.01, the HyperFabric EMS

monitor enables the system administrator to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator deﬁnes the condition to trigger a notiﬁcation (usually a change in interface status). Notiﬁcation can be accomplished with one of the following:

— A Simple Network Management Protocol (SNMP) trap

Chapter 214

Planning the Fabric

— Logging into a user speciﬁed log ﬁle with a choice of severity — Email to a user deﬁned email address. For more information on EMS, including instructions for

implementing this feature, see “Conﬁguring the HyperFabric EMS Monitor” on page 85 and the EMS Hardware Monitors User’s Guide Part Number B6191-90028 September 2001 Edition.

• ServiceGuard: Supported Within a cluster, ServiceGuard groups application services

(individual HP-UX processes) into packages. In the event of a single service failure (node, network, or other resource), EMS provides notiﬁcation and ServiceGuard transfers control of the package to another node in the cluster, allowing services to remain available with minimal interruption.

ServiceGuard via EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). ServiceGuard uses a heartbeat LAN to monitor the nodes in a cluster. ServiceGuard cannot use the HyperFabric interconnect as a heartbeat LAN. Instead, use a separate LAN for the heartbeat.

TCP/UDP/IP

For more information on conﬁguring ServiceGuard, see “Conﬁguring HyperFabric with ServiceGuard” on page 87, and Managing MC/ServiceGuard Part Number B3936-90065 March 2002 Edition.

• High Availability (HA): Supported To create a highly available HyperFabric cluster, there should not be

any single point of failure. Once the HP 9000 nodes and the HyperFabric hardware have been conﬁgured with no single point of failure, ServiceGuard and EMS can be conﬁgured to monitor and fail over nodes and services using ServiceGuard packages.

If any HyperFabric resource in a cluster fails (adapter card, cable or switch port), the HyperFabric driver transparently routes trafﬁc over other available HyperFabric resources with no disruption of service.

The ability of the HyperFabric driver to transparently fail over trafﬁc reduces the complexity of conﬁguring highly available clusters with ServiceGuard, because ServiceGuard has to take care of node and service failover only.

Chapter 2 15

Planning the Fabric

TCP/UDP/IP

ServiceGuard uses a “heartbeat” to monitor the cluster. The HyperFabric links cannot be used for the heartbeat. Instead, an alternate LAN connection such as 100BaseT, Ethernet, Token Ring, or FDDI must be made between the nodes for use as a heartbeat link.

End-To-End HA: HyperFabric provides end-to-end HA on the entire cluster fabric at the link level. If any of the available routes in the fabric fails, HyperFabric transparently redirects all the trafﬁc to a functional route and, if conﬁgured, notiﬁes ServiceGuard or other enterprise management tools.

Active-Active HA: In conﬁgurations where there are multiple routes between nodes, the HyperFabric software uses a hashing function to determine an adapter or a route through which it sends messages. This is done on a message-by-message basis. All of the available HyperFabric resources in the fabric are used for communication.

In contrast to Active-Passive HA, where one set of resources is not utilized until another set fails, Active-Active HA provides the best return on investment because all of the resources are utilized simultaneously. ServiceGuard is not required for Active-Active HA operation.

For more information on setting up HA HyperFabric clusters, see Figure 2-3 “TCP/UDP/IP High Availability Switched Conﬁguration” on page 24.

• Dynamic Resource Utilization (DRU): Supported If you add a new resource (node, adapter, cable or switch) to a cluster,

the HyperFabric subsystem dynamically identiﬁes the added resource and starts using it. The same process takes place when a resource is removed from a cluster. The difference between DRU and OLAR is that OLAR applies only to the addition or replacement of adapter cards from nodes.

• Load Balancing: Supported When an HP 9000 HyperFabric cluster is running TCP/UDP/IP

applications, the HyperFabric driver balances the load across all available resources in the cluster, including nodes, adapter cards, links, and multiple links between switches.

• Switch Management: Not Supported

Chapter 216

+ 160 hidden pages