HP HyperFabric Administrator's Guide

HyperFabric Administrator’s Guide
HP-UX 11i v2
Edition 14
Manufacturing Part Number : B6257-90043
March 2004
United States
© Copyright 2004 Hewlett-Packard Development Company L.P. All rights reserved.
The information in this document is subject to change without notice.
Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard
shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material.
Warranty
A copy of the specific warranty terms applicable to your Hewlett-Packard product and replacement parts can be obtained from your local Sales and Service Office.
U.S. Government License
Proprietary computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.
Copyright Notice
Copyright 2004 Hewlett-Packard Development Company L.P. All rights reserved. Reproduction, adaptation, or translation of this document without prior written permission is prohibited, except as allowed under the copyright laws.
Trademark Notices
Oracle is a registered trademark of Oracle Corporation. UNIX is a registered trademark in the United States and other
countries, licensed exclusively through The Open Group.
ii
About This Document
1. Overview of the HyperFabric Product
About HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
HyperFabric Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Switches and Switch Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Other Product Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
HyperFabric Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2. Planning the Fabric
Preliminary Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
HyperFabric Features, Parameters and Supported Configurations for TCP/UDP/IP and
HMP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
TCP/UDP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
TCP/UDP/IP Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Point-to-Point Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Switched Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
High Availability Switched Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Hybrid Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Hyper Messaging Protocol (HMP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Application Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
HMP Supported Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Point-to-Point Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Enterprise (Database) Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Technical Computing (Work Stations) Configuration . . . . . . . . . . . . . . . . . . . . . . . 37
Contents
3. Installing HyperFabric
Checking HyperFabric Installation Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Installing HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
iii
Contents
Online Addition and Replacement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Planning and Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Critical Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Card Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Installing the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Loading the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Installing HyperFabric Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Before Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Installing the HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
With the Rail Kit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Without the Rail Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4. Configuring HyperFabric
Configuration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Information You Need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Configuration Information Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Performing the Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Using the clic_init Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Examples of clic_init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Deconfiguring a HyperFabric Adapter with SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Configuring the HyperFabric EMS Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Configuring HyperFabric with ServiceGuard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
How HyperFabric Handles Adapter Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Configuring HyperFabric with the ServiceGuard Resource Monitor. . . . . . . . . . . 94
Configuring ServiceGuard with HyperFabric Using the ASCII File . . . . . . . . . . . 94
Configuring ServiceGuard with HyperFabric Using SAM . . . . . . . . . . . . . . . . . . . 94
Configuring ServiceGuard for HyperFabric Relocatable IP Addresses . . . . . . . . . 95
Configuring HMP for Transparent Local Failover Support. . . . . . . . . . . . . . . . . . . . . . 96
How Transparent Local Failover Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Configuring HMP for Transparent Local Failover Support - Using SAM. . . . . . . . 102
Deconfiguring HMP for Local Failover support - Using SAM . . . . . . . . . . . . . . . . . 103
Configuring HMP for Transparent Local Failover Support - Using the clic_init
command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
iv
5. Managing HyperFabric
Starting HyperFabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Using the clic_start Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Verifying Communications within the Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
The clic_probe Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Examples of clic_probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Displaying Status and Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
The clic_stat Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Examples of clic_stat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Viewing man Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Stopping HyperFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Using the clic_shutdown Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6. Troubleshooting HyperFabric
Running Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
The clic_diag Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Example of clic_diag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Using Support Tools Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Useful Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
LED Colors and Their Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
HF2 Switch LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Determining Whether an Adapter or a Cable is Faulty. . . . . . . . . . . . . . . . . . . . . . . . 153
Determining Whether a Switch is Faulty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Replacing a HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Replacing a HyperFabric Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Safety Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Regulatory Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Adapters and Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
FCC Statement (USA only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
DOC Statement (Canada only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Europe RFI Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Australia and New Zealand EMI Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Contents
v
Contents
Radio Frequency Interference (Japan Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Declarations of Conformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Physical Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Environmental . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
vi
Figures
Figure 2-1. TCP/UDP/IP Point-To-Point Configurations . . . . . . . . . . . . . . . . . . . . . . 22
Figure 2-2. TCP/UDP/IP Basic Switched Configuration . . . . . . . . . . . . . . . . . . . . . . 23
Figure 2-3. TCP/UDP/IP High Availability Switched Configuration . . . . . . . . . . . . 24
Figure 2-4. TCP/UDP/IP Hybrid Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Figure 2-5. HMP Point-To-Point Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Figure 2-6. HMP Enterprise (Database) Configuration, Single Connection Between
Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Figure 2-7. Local Failover Supported Enterprise (Database) Configuration, Multiple
Connections between Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Figure 2-8. Technical Computing Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Figure 2-9. Large Technical Computing Configuration . . . . . . . . . . . . . . . . . . . . . . . 39
Figure 3-1. HyperFabric File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Figure 3-2. Front of HF2 Switch (A6388A Switch Module Installed) . . . . . . . . . . . . 59
Figure 3-3. Front of HF2 Switch (A6389A Switch Module Installed) . . . . . . . . . . . . 60
Figure 4-1. Map for Configuration Information Example . . . . . . . . . . . . . . . . . . . . . 74
Figure 4-2. An ServiceGuard Configuration (with Two HyperFabric Switches) . . . 89
Figure 4-3. Node with Two Active HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . 91
Figure 4-4. Node with One Failed HyperFabric Adapter. . . . . . . . . . . . . . . . . . . . . . 92
Figure 4-5. When All HyperFabric Adapters Fail. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Figure 4-6. A Configuration supporting Local Failover . . . . . . . . . . . . . . . . . . . . . . . 98
Figure 4-7. Adapter, Link or Switch Port Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Figure 4-8. Switch Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Figure 4-9. Cable Failover Between Two Switches . . . . . . . . . . . . . . . . . . . . . . . . . 101
Figure 4-10. Configuring the Transparent Local Failover feature . . . . . . . . . . . . . 104
vii
Figures
viii
Tables
Table 1. HP-UX 11i Releases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Table 2. Publishing History Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Table 3. Organization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Table 2-1. HF2 Throughput and Latency with TCP/UDP/IP Applications. . . . . . . . 19
Table 2-2. Supported HyperFabric Adapter Configurations . . . . . . . . . . . . . . . . . . . 20
Table 2-3. HF2 Throughput and Latency with HMP Applications . . . . . . . . . . . . . . 31
Table 2-4. Supported HyperFabric Adapter Configurations . . . . . . . . . . . . . . . . . . . 31
Table 3-1. Important OLAR Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Table 6-1. LED Names (by Adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Table 6-2. HyperFabric Adapter LED Colors and Meanings . . . . . . . . . . . . . . . . . . 145
Table 6-3. HF2 Switch LED Colors and Meanings. . . . . . . . . . . . . . . . . . . . . . . . . . 150
ix
Tables
x
About This Document
This document describes how to install, configure, and troubleshoot the HyperFabric product on the HP-UX 11i v2 (HP-UX 11.23) operating system.
Before you install the HyperFabric software, ensure that the operating system software and the appropriate files, scripts, subsets are installed.
The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The document part number will change when extensive changes are made.
Document updates may be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, you should subscribe to the appropriate product support service. See your HP sales representative for details.
The latest version of this manual can be found online at
http://www.docs.hp.com/hpux/netcom/index.html#HyperFabric.
Intended Audience
This manual is intended for system and network administrators responsible for installing, configuring, and managing the HyperFabric software and hardware. Administrators are expected to have knowledge of operating system concepts, commands, and configuration.
It is helpful to have knowledge of Transmission Control Protocol/Internet Protocol (TCP/IP) networking concepts and network configuration.
This document is not a tutorial.
xi
HP-UX Release Name and Release Identifier
Each HP-UX 11i release has an associated release name and release identifier. The uname (1) command with the -r option returns the release identifier. Table 1 shows the releases available for HP-UX 11i.
Table 1 HP-UX 11i Releases
Supported
Release Identifier Release Name
B.11.11 HP-UX 11i v1 PA-RISC
B.11.20 HP-UX 11i v1.5 Intel Itanium
B.11.22 HP-UX 11i v1.6 Intel Itanium
B.11.23 HP-UX 11i v2 Intel Itanium
Processor
Architecture
Publishing History
Table 2 provides the publication date for the pertinent edition number.
Table 2 Publishing History Details
Edition Number Publication Date
xii
First March 1998
Second June 1998
Third August 1998
Fourth October 1998
Fifth December 1998
Sixth February 1999
Seventh April 1999
Eighth March 2000
Ninth June 2000
Tenth December 2000
Table 2 Publishing History Details (Continued)
Edition Number Publication Date
Eleventh June 2001
Twelfth September 2002
Thirteenth July 2003
Fourteenth March 2004
What Is in This Document
HyperFabric Administrator’s Guide is divided into several chapters, each of which contains information about installing, configuring, or troubleshooting HyperFabric. The appendixes contain supplemental information.
The following list describes the content in more detail.
Table 3 Organization
Chapter Description
Overview of the HyperFabric Product
Planning the Fabric Describes the steps that need to be
Installing HyperFabric Describes the tasks to install the
Configuring HyperFabric Describes the tasks to configure
Managing HyperFabric Describes the tasks to start, stop and
Troubleshooting HyperFabric
Presents an overview of HyperFabric and lists the components that the HyperFabric product contains
followed while planning the fabric
HyperFabric products on the HP-UX 11i v2 operating system
HyperFabric
manage HyperFabric
Describes how to troubleshoot HyperFabric
xiii
New and Changed Information in This Edition
This edition includes information about the transparent local failover feature of Hyper Messaging Protocol (HMP). This feature is available with the HyperFabric version B.11.23.01.
Typographic Conventions
This document uses the following typographic conventions: Book Title Italic (slanted) type indicates document and book
names.
daemon Courier font type indicates daemons, files, commands,
manpages, and option names.
[ ] { } In syntax definitions, square brackets indicate items
that are optional and braces indicate items that are required.
HP Welcomes Your Feedback
HP welcomes any comments and suggestions you have on this manual. You can send your comments in the following ways:
Internet electronic mail: netinfo_feedback@cup.hp.com
xiv
Using a feedback form located at the following URL:
http://docs.hp.com/assistance/feedback.html
Please include the following information along with your comments:
The complete title of the manual and the part number. (The part number appears on the title page of printed and PDF versions of a manual.)
The section numbers and page numbers of the information on which you are commenting.
The version of HP-UX that you are using.
Please note that the HP-UX networking communications publications group does not provide technical support for HP products.
1 Overview of the HyperFabric
Product
This chapter contains the following sections that give general information about HyperFabric:
Chapter 1 1
Overview of the HyperFabric Product
“About HyperFabric” on page 3
“HyperFabric Products” on page 4
“HyperFabric Concepts” on page 7
Chapter 12
Overview of the HyperFabric Product

About HyperFabric

About HyperFabric
HyperFabric is an HP high-speed, packet-based interconnect for node-to-node communications. HyperFabric provides higher speed, lower network latency and uses less CPU than other industry standard protocols (for example, Fibre Channel and Gigabit Ethernet). Instead of using a traditional bus-based technology, HyperFabric is built around switched fabric architecture, providing the bandwidth necessary for high speed data transfer. This clustering solution delivers the performance, scalability, and high availability required by the following:
Parallel Database Clusters
— Oracle 9i Real Application Clusters (RAC) — Oracle 8i Parallel Servers (OPS)
Parallel Computing Clusters
Client/Server Architecture Interconnects (for example, SAP)
Multi-Server Batch Applications (for example, SAS Systems)
Enterprise Resource Planning (ERP)
Technical Computing Clusters
HP Message Passing Interface (MPI) based applications
OpenView Data Protector (earlier known as Omniback)
Network Backup
Data Center Network Consolidation
E-services
Chapter 1 3
Overview of the HyperFabric Product

HyperFabric Products

HyperFabric Products
HyperFabric hardware consists of host-based interface adapter cards, interconnect cables, and optional switches. HyperFabric software resides in Application Specific Integrated Circuits (ASICs) and firmware on the adapter cards and includes user-space components and HP-UX drivers.
Currently, fiber-based HyperFabric hardware are available. In addition, a hybrid switch that has 8-fiber ports is available to support HF2 clusters.
This section describes the various HyperFabric products. For more information on HP 9000 systems that support HyperFabric products, see the HyperFabric Release Notes, available at
http://docs.hp.com/hpux/netcom/index.html#HyperFabric.
NOTE This document uses the term HyperFabric (HF) to refer to the
hardware and software that form the HyperFabric cluster interconnect product.
The term HyperFabric2 (HF2) refers to the following fiber-based hardware components:
The A6386A adapter
The A6384A switch chassis
The A6388A and A6389A switch modules. (Although the A6389A switch module has 4-copper ports, it is still considered an HF2 component because it can only be used with the A6384A HF2 switch chassis).
The C7524A, C7525A, C7526A, and C7527A cables

HyperFabric Adapters

The HyperFabric adapters are as follows:
A6386A HF2 PCI (4X) adapter with a fiber interface.
The A6092A HyperFabric adapter is supported beginning with the following HyperFabric software versions:
Chapter 14
Overview of the HyperFabric Product
HyperFabric Products
HP-UX 11.0: HyperFabric software version B.11.00.09
HP-UX 11i v1: HyperFabric software version B.11.11.00
HP-UX 11i v2: HyperFabric software version B.11.23.00
The A6386A HyperFabric2 adapter is supported beginning with the following HyperFabric software versions:
HP-UX 11.0: HyperFabric software version B.11.00.11
HP-UX 11i v1: HyperFabric software version B.11.11.01
HP-UX 11i v2: HyperFabric software version B.11.23.00

Switches and Switch Modules

The HyperFabric2 switches are as follows:
A6384A HF2 fiber switch chassis with one integrated Ethernet management LAN adapter card, one integrated 8-port fiber card, and one expansion slot. For the chassis to be a functional switch, install one of the following switch modules in the expansion slot:
— The A6388A HF2 8-port fiber switch module. This gives 16-fiber
ports to the switch (8 from the integrated fiber card and 8 from the A6388A switch module).
— The A6389A HF2 4-port copper switch module. This gives 12
ports to the switch - a mixture of 8-fiber ports (from the integrated fiber card) and 4-copper ports (from the A6389A module).
The A6384A HF2 switch chassis with either module installed is supported beginning with the following HyperFabric software versions:
HP-UX 11.0: HyperFabric software version B.11.00.11
HP-UX 11i v1: HyperFabric software version B.11.11.01
HP-UX 11i v2: HyperFabric software version B.11.23.00
NOTE In this manual, the terms HyperFabric2 switch or HF2 switch refer
to the functional switch (the A6384A switch chassis with one of the switch modules installed).
Chapter 1 5
Overview of the HyperFabric Product
HyperFabric Products
IMPORTANT HF2 adapters and switches are not supported by software versions
earlier than those listed in “HyperFabric Adapters” on page 4 and “Switches and Switch Modules” on page 5.
To determine the version of HyperFabric, issue the following command:
$ swlist | grep -i hyperfabric

Other Product Elements

The following are the other elements of the HyperFabric product family:
HF2 fiber cables
— C7524A (2m length) — C7525A (16m length) — C7526A (50m length) — C7527A (200m length)
The HyperFabric software: The software resides in ASICs and firmware on the adapter cards and includes user-space components and HP-UX drivers.
HyperFabric supports the IP network protocol stack, specifically TCP/IP and UDP/IP.
HyperFabric software includes HyperMessaging Protocol (HMP). HMP provides higher bandwidth, lower CPU overhead, and lower latency (the time a message takes to get from one point to another). However, these HMP benefits are available only when applications that are developed on top of HMP are running. HMP can only be used on HP 9000 systems running HP-UX 11.0 or 11i v1, provided HyperFabric A6092A or A6386A (PCI 4X) adapter cards are installed on those systems.
Chapter 16
Overview of the HyperFabric Product

HyperFabric Concepts

HyperFabric Concepts
This section briefly describes some of the basic HyperFabric concepts and terms.
The fabric is the physical configuration that consists of all the HyperFabric adapters, the HyperFabric switches (if any), and the HyperFabric cables connecting them. The network software controls data transfer over the fabric.
The HyperFabric configuration contains two or more HP 9000 systems and optional HyperFabric switches. Each HP 9000 acts as a node in the configuration. Each node has a minimum of one and a maximum of eight HyperFabric adapters installed in it. (For information on the maximum number of adapters that can be installed in each system, see Chapter 2, “Planning the Fabric,” on page 9.) Each HF2 switch can be configured with 12 or 16 ports. HyperFabric supports a maximum of 4 HyperFabric switches. You can mesh HyperFabric switches and configurations with up to four levels of meshed switches are supported.
You can plan a HyperFabric cluster as a High Availability (HA) configuration, when it is necessary to ensure that each node can always participate in the fabric. This is done by using ServiceGuard (earlier known as MC/ServiceGuard), ServiceGuard OPS Edition (earlier known as MC/LockManager), and the Event Monitoring Service (EMS). Configurations of up to 8 nodes are supported under ServiceGuard.
Beginning with HyperFabric software versions B.11.00.05 and B.11.11.00,you can use relocatable IP addresses as part of an HA configuration. Relocatable IP addresses permit a client application to reroute through an adapter on a remote node, allowing that application to continue processing without interruption. The rerouting is transparent. This function is associated with ServiceGuard (see “Configuring ServiceGuard for HyperFabric Relocatable IP Addresses” on page 95). When the monitor for HyperFabric detects a failure and the backup adapter takes over, the relocatable IP address is transparently migrated to the backup adapter. Throughout this migration process, the client application continues to execute normally.
When you start HyperFabric (with the clic_start command, through SAM, or by booting the HP 9000 system), you start the management process. This process must be active for HyperFabric to run. If the HyperFabric management process on a node stops running for some
Chapter 1 7
Overview of the HyperFabric Product
HyperFabric Concepts
reason (for example, if it is killed), all HyperFabric-related communications on that node are stopped immediately. This makes the node unreachable by other components in the fabric.
When you start HyperFabric, the fabric is verified automatically. This is because each node performs a self diagnosis and verification over each adapter installed in the node. In addition, the management process performs automatic routing and configuring for each switch (if switches are part of the fabric). You can, if needed, run the clic_stat command to get a textual map of the fabric, which can be used as another method of quick verification.
Notice that the commands to administer HyperFabric have a prefix of clic_, and some of the other components have CLIC as part of their name (for example, the CLIC firmware and the CLIC software). CLIC stands for CLuster InterConnect, and it is used to differentiate those HyperFabric commands or components from other commands or components. For example, the HyperFabric command clic_init is different from the HP-UX init command.
Chapter 18

2 Planning the Fabric

This chapter contains the following sections that include general guidelines and protocol-specific considerations for planning HyperFabric clusters that run TCP/UDP/IP or HMP applications.
Chapter 2 9
Planning the Fabric
“Preliminary Considerations” on page 11
“HyperFabric Features, Parameters and Supported Configurations for TCP/UDP/IP and HMP Applications” on page 13
“TCP/UDP/IP” on page 14
“Hyper Messaging Protocol (HMP)” on page 26
Chapter 210
Planning the Fabric

Preliminary Considerations

Preliminary Considerations
Before assembling a fabric physically, do the following to address all of the appropriate issues:
Step 1. Read Chapter 1, “Overview of the HyperFabric Product,” on page 1, to
get a basic understanding of HyperFabric and its components.
Step 2. Read this chapter, Planning the Fabric, to gain an understanding of
protocol specific configuration guidelines for TCP/UDP/IP and HMP applications.
Step 3. Read “Configuration Overview” on page 69, “Information You Need” on
page 71, and “Configuration Information Example” on page 74, to understand how to configure the fabric.
Step 4. Decide the number of nodes that will be interconnected in the fabric.
Step 5. Decide the type of HP 9000 system for each node (for a list of supported
HP 9000 systems, see the HyperFabric Release Notes available at
http://docs.hp.com/hpux/netcom/index.html#HyperFabric.
Step 6. Determine the network bandwidth requirements for each node.
Step 7. Determine the number of adapters needed for each node.
Step 8. Determine if a High Availability (ServiceGuard) configuration will be
needed. If ServiceGuard is used, each node should have at least two adapters.
Step 9. Decide the topology of the fabric.
Step 10. Determine how many switches will be used based on the number of
nodes in the fabric. The only configuration that can be supported without a switch is the node-to-node configuration (HA or non-HA). HyperFabric supports meshed switches up to a depth of four switches, starting with the following versions of the HyperFabric software:
For HF2 switches: software versions B.11.00.11, B.11.11.01, and B.11.23.00.
Chapter 2 11
Planning the Fabric
Preliminary Considerations
Step 11. Draw the cable connections from each node to switches (if the fabric will
contain switches). If you use an HA configuration with switches, it requires more than one switch for complete redundancy and to avoid a single point of failure. For example, each adapter can be connected to its own switch, or two switches can be connected to four adapters.
Chapter 212
Planning the Fabric
HyperFabric Features, Parameters and Supported Configurations for TCP/UDP/IP and HMP Applications
HyperFabric Features, Parameters and Supported Configurations for TCP/UDP/IP and HMP Applications
The following sections in this chapter define HyperFabric functionality for TCP/UDP/IP applications and Hyper Messaging Protocol (HMP) applications. There are distinct differences in supported hardware, available features and performance, depending on which protocol is used by applications running on the HyperFabric.
Chapter 2 13
Planning the Fabric

TCP/UDP/IP

TCP/UDP/IP
TCP/UDP/IP is supported on all HF2 hardware. Although some of the HyperFabric adapter cards support both HMP and TCP/UDP/IP applications, in this section, the focus is on TCP/UDP/IP HyperFabric applications.

Application Availability

All applications, including Oracle 9i and HP-MPI, that use the TCP/UDP/IP stack are supported.

Features

This section discusses the following HyperFabric features on TCP/UDP/IP:
OnLine Addition and Replacement (OLAR): Supported The OLAR feature allows the replacement or addition of
HyperFabric adapter cards while the system (node) is running. HyperFabric supports this functionality on the SD64A, rx8620, rx4640, rp54xx (L-class), rp74xx (N-class), rp8400 and Superdome systems, running on the HP-UX 11i v2 platform.
For more information on OLAR, including instructions for implementing this feature, see “Online Addition and Replacement” on page 44 and Configuring HP-UX for Peripherals Part Number B2355-90698 November 2000 Edition.
Event Monitoring Service (EMS): Supported In the HyperFabric version B.11.23.01, the HyperFabric EMS
monitor enables the system administrator to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator defines the condition to trigger a notification (usually a change in interface status). Notification can be accomplished with one of the following:
— A Simple Network Management Protocol (SNMP) trap
Chapter 214
Planning the Fabric
— Logging into a user specified log file with a choice of severity — Email to a user defined email address. For more information on EMS, including instructions for
implementing this feature, see “Configuring the HyperFabric EMS Monitor” on page 85 and the EMS Hardware Monitors User’s Guide Part Number B6191-90028 September 2001 Edition.
ServiceGuard: Supported Within a cluster, ServiceGuard groups application services
(individual HP-UX processes) into packages. In the event of a single service failure (node, network, or other resource), EMS provides notification and ServiceGuard transfers control of the package to another node in the cluster, allowing services to remain available with minimal interruption.
ServiceGuard via EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). ServiceGuard uses a heartbeat LAN to monitor the nodes in a cluster. ServiceGuard cannot use the HyperFabric interconnect as a heartbeat LAN. Instead, use a separate LAN for the heartbeat.
TCP/UDP/IP
For more information on configuring ServiceGuard, see “Configuring HyperFabric with ServiceGuard” on page 87, and Managing MC/ServiceGuard Part Number B3936-90065 March 2002 Edition.
High Availability (HA): Supported To create a highly available HyperFabric cluster, there should not be
any single point of failure. Once the HP 9000 nodes and the HyperFabric hardware have been configured with no single point of failure, ServiceGuard and EMS can be configured to monitor and fail over nodes and services using ServiceGuard packages.
If any HyperFabric resource in a cluster fails (adapter card, cable or switch port), the HyperFabric driver transparently routes traffic over other available HyperFabric resources with no disruption of service.
The ability of the HyperFabric driver to transparently fail over traffic reduces the complexity of configuring highly available clusters with ServiceGuard, because ServiceGuard has to take care of node and service failover only.
Chapter 2 15
Planning the Fabric
TCP/UDP/IP
ServiceGuard uses a “heartbeat” to monitor the cluster. The HyperFabric links cannot be used for the heartbeat. Instead, an alternate LAN connection such as 100BaseT, Ethernet, Token Ring, or FDDI must be made between the nodes for use as a heartbeat link.
End-To-End HA: HyperFabric provides end-to-end HA on the entire cluster fabric at the link level. If any of the available routes in the fabric fails, HyperFabric transparently redirects all the traffic to a functional route and, if configured, notifies ServiceGuard or other enterprise management tools.
Active-Active HA: In configurations where there are multiple routes between nodes, the HyperFabric software uses a hashing function to determine an adapter or a route through which it sends messages. This is done on a message-by-message basis. All of the available HyperFabric resources in the fabric are used for communication.
In contrast to Active-Passive HA, where one set of resources is not utilized until another set fails, Active-Active HA provides the best return on investment because all of the resources are utilized simultaneously. ServiceGuard is not required for Active-Active HA operation.
For more information on setting up HA HyperFabric clusters, see Figure 2-3 “TCP/UDP/IP High Availability Switched Configuration” on page 24.
Dynamic Resource Utilization (DRU): Supported If you add a new resource (node, adapter, cable or switch) to a cluster,
the HyperFabric subsystem dynamically identifies the added resource and starts using it. The same process takes place when a resource is removed from a cluster. The difference between DRU and OLAR is that OLAR applies only to the addition or replacement of adapter cards from nodes.
Load Balancing: Supported When an HP 9000 HyperFabric cluster is running TCP/UDP/IP
applications, the HyperFabric driver balances the load across all available resources in the cluster, including nodes, adapter cards, links, and multiple links between switches.
Switch Management: Not Supported
Chapter 216
Planning the Fabric
TCP/UDP/IP
Switch Management is not supported. Switch management will not operate properly if you enable it on a HyperFabric cluster.
Diagnostics: Supported Diagnostics can be run to obtain information on many of the
HyperFabric components using the clic_diag, clic_probe and clic_stat commands, as well as the Support Tools Manager (STM).
For more information on HyperFabric diagnostics, see “Running Diagnostics” on page 133.
Configuration Parameters
This section describes the maximum limits for TCP/UDP/IP HyperFabric configurations. There are numerous variables that can impact the performance of any HyperFabric configuration. For guidance on specific HyperFabric configurations for TCP/UDP/IP applications, see the section, “TCP/UDP/IP Supported Configurations” on page 21.
HyperFabric is supported only on the HP 9000 series servers and workstations.
TCP/UDP/IP is supported for all HyperFabric hardware and software.
Maximum Supported Nodes and Adapter Cards In point-to-point configurations, the complexity and performance
limitations of having a large number of nodes in a cluster make it necessary to include switching in the fabric. Typically, point-to-point configurations consist of only 2 or 3 nodes.
In switched configurations, HyperFabric supports a maximum of 64 interconnected adapter cards.
A maximum of 8 HyperFabric adapter cards are supported per instance of the HP-UX operating system. The actual number of adapter cards a particular node is able to accommodate also depends on slot availability and system resources. See node specific documentation for details.
HyperFabric subsystem supports a maximum of 8 configured IP addresses per instance of the HP-UX operating system.
Maximum Number of Switches
Chapter 2 17
Planning the Fabric
TCP/UDP/IP
You can interconnect (mesh) up to 4 switches (16-port fiber, or Mixed 8 fiber ports) in a single HyperFabric cluster.
Trunking Between Switches (multiple connections) You can use trunking between switches to increase bandwidth and
cluster throughput. Trunking is also a way to eliminate a possible single point of failure. The number of trunked cables between nodes is limited only by port availability. To assess the effects of trunking on the performance of any particular HyperFabric configuration, contact your HP representative.
Maximum Cable Lengths HF2 (fiber): The maximum distance is 200m (Four standard cable
lengths are sold and supported: 2m, 16m, 50m and 200m). TCP/UDP/IP supports up to four HF2 switches connected in series
with a maximum cable length of 200m between the switches and 200m between switches and nodes.
TCP/UDP/IP supports up to four hybrid HF2 switches connected in series with a maximum cable length of 200m between fiber ports.
Chapter 218
Planning the Fabric
TCP/UDP/IP
Throughput and Latency
Table 2-1 HF2 Throughput and Latency with TCP/UDP/IP Applications
Server Class Maximum Throughput Latency
rp7400 2 + 2 Gbps full duplex per link < 42 microsec
Chapter 2 19
Planning the Fabric
TCP/UDP/IP
Table 2-2 Supported HyperFabric Adapter Configurations
HF
Adapter
A6386A PCI (4X) rx2600 servers 11i v2 No 1
A6386A PCI (4X) rx56XX
A6386A PCI (4X) zx6000
A6386A PCI (4X) SD64A servers 11i v2 Yes 8 (maximum
A6386A PCI (4X) rx7620 servers 11i v2 No 8 (maximum
A6386A PCI (4X) rx8620 servers 11i v2 Yes 8 (maximum
A6386A PCI (4X) rx4640 servers 11i v2 Yes 6
Bus
Type
Supported
HP Systems
servers
workstations
HP-UX
Version
11i v2 No 4
11i v2 No 1
OLAR
Support?
Maximum
Adapters
per System
4 per PCI card cage)
4 per PCI card cage)
4 per PCI card cage)
Chapter 220
Planning the Fabric
TCP/UDP/IP
TCP/UDP/IP Supported Configurations
Multiple TCP/UDP/IP HyperFabric configurations are supported to match the cost, scaling, and performance requirements of each installation.
In the previous section, “Configuration Parameters” on page 17, the maximum limits for TCP/UDP/IP enabled HyperFabric hardware configurations were outlined. In this section the TCP/UDP/IP enabled HyperFabric configurations that HP supports are explained. These recommended configurations offer an optimal mix of performance and availability for a variety of operating environments.
There are many variables that can impact HyperFabric performance. If you are considering a configuration that is beyond the scope of the following HP supported configurations, contact your HP representative.
Point-to-Point Configurations
You can interconnect large servers like the HP Superdome to run Oracle RAC 9i and enterprise resource planning applications. These applications are typically consolidated on large servers.
Point-to-point connections between servers support the performance benefits of HMP without investing in HyperFabric switches. This is a good solution in small configurations where the benefits of a switched HyperFabric cluster might not be required (see configuration A and configuration C in Figure 2-1).
If there are multiple point-to-point connections between two nodes, traffic load is balanced over those links. If one link fails, the load fails over to the remaining links (see configuration B in Figure 2-1).
Running applications using TCP/UDP/IP on a HyperFabric cluster provides major performance benefits compared to other technologies such as Ethernet. If a HyperFabric cluster is originally set up to run enterprise applications using TCP/UDP/IP and the computing environment stabilizes with a requirement for higher performance, migration to HMP is always an option.
Chapter 2 21
Planning the Fabric
TCP/UDP/IP
Figure 2-1 TCP/UDP/IP Point-To-Point Configurations
Chapter 222
Switched Configuration
This configuration offers the same benefits as the point-to-point configurations illustrated in Figure 2-1, but it has the added advantage of greater connectivity (see Figure 2-2).
Figure 2-2 TCP/UDP/IP Basic Switched Configuration
Planning the Fabric
TCP/UDP/IP
Chapter 2 23
Planning the Fabric
TCP/UDP/IP
High Availability Switched Configuration
This configuration has no single point of failure. The HyperFabric driver provides end-to-end HA. If any HyperFabric resource in the cluster fails, traffic is transparently rerouted through other available resources. This configuration provides high performance and high availability (see Figure 2-3).
Figure 2-3 TCP/UDP/IP High Availability Switched Configuration
Chapter 224
Hybrid Configuration
You can interconnect servers and workstations in a single heterogeneous HyperFabric cluster.
In this configuration, the servers are highly available. In addition, the workstations and the servers can run the same application or different applications (see Figure 2-4).
Figure 2-4 TCP/UDP/IP Hybrid Configuration
Planning the Fabric
TCP/UDP/IP
Chapter 2 25
Planning the Fabric

Hyper Messaging Protocol (HMP)

Hyper Messaging Protocol (HMP)
Hyper Messaging Protocol (HMP) is an HP patented, high performance cluster interconnect protocol. HMP provides reliable, high speed, low latency, low CPU overhead, datagram service to applications running on the HP-UX operating system.
HMP was jointly developed with Oracle Corp. The resulting feature set was tuned to enhance the scalability of the Oracle Cache Fusion clustering technology. It is implemented using Remote DMA (RDMA) paradigms.
HMP is integral to the HP-UX HyperFabric driver. It can be enabled or disabled at HyperFabric initialization using the clic_init command or SAM. The HMP functionality is used by the applications listed in the following “Application Availability” section.
HMP significantly enhances the performance of parallel and technical computing applications.
HMP firmware on HyperFabric adapter cards provides a shortcut that bypasses several layers in the protocol stack, boosting link performance and lowering latency. By avoiding interruptions and buffer copying in the protocol stack, communication task processing is optimized.

Application Availability

The following are the two families of applications that can use HMP over the HyperFabric interface:
Oracle 9i Database, Release 1 (9.0.1) and Release 2 (9.2.0.1.0). HMP has been certified on Oracle 9i Database Release 1 with HP-UX
11.0, 11i v1, and 11i v2. HMP has been certified on Oracle 9i Database Release 2 with HP-UX
11.0, 11i v1, and 11i v2.
Technical Computing Applications that use the HP Message Passing Interface (HP-MPI).
Chapter 226
Planning the Fabric
Hyper Messaging Protocol (HMP)
HP MPI is a native implementation of version 1.2 of the Message-Passing Interface Standard. It has become the industry standard for distributed technical applications and is supported on most technical computing platforms.

Features

The following are the HyperFabric features on HMP:
OnLine Addition and Replacement (OLAR) The OLAR feature, which allows the replacement or addition of
HyperFabric adapter cards while the system (node) is running, is supported when applications use HMP to communicate.
Event Monitoring Service (EMS): Supported In the HyperFabric version B.11.23.01, the HyperFabric EMS
monitor enables the system administrator to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator defines the condition to trigger a notification (usually a change in interface status). Notification can be accomplished with a SNMP trap, or by logging into a user specified log file with a choice of severity, or by email to a user defined email address.
For more information on EMS, including instructions for implementing this feature, see “Configuring the HyperFabric EMS Monitor” on page 85 in this manual, and the EMS Hardware Monitors User’s Guide Part Number B6191-90028 September 2001 Edition.
ServiceGuard: Supported Within a cluster, ServiceGuard groups application services
(individual HP-UX processes) into packages. In the event of a single service failure (node or network), EMS provides notification and ServiceGuard transfers control of the package to another node in the cluster, allowing services to remain available with minimal interruption. ServiceGuard using EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). ServiceGuard uses a heartbeat LAN to monitor the
Chapter 2 27
Planning the Fabric
Hyper Messaging Protocol (HMP)
nodes in a cluster. ServiceGuard cannot use the HyperFabric interconnect as a heartbeat link. Instead, a separate LAN must be used for the heartbeat.
For more information on configuring ServiceGuard, see “Configuring HyperFabric with ServiceGuard” on page 87, as well as Managing MC/ServiceGuard Part Number B3936-90065 March 2002 Edition.
High Availability (HA): Supported When applications use HMP to communicate between HP 9000
nodes in a HyperFabric cluster, you can configure ServiceGuard and the EMS monitor to identify node failure and automatically failover to a functioning HP 9000 node.
For more information on HA when running HMP applications, contact your HP representative.
Transparent Local Failover: Supported HMP supports Transparent Local Failover in the HyperFabric
version B.11.23.01. When a HyperFabric resource (adapter, cable, switch or switch port)
fails in a cluster, HMP transparently fails over traffic using other available resources. This is accomplished using card pairs, each of which is a logical entity that comprises a pair of HF2 adapters on a HP 9000 node. Only Oracle applications can make use of the Local Failover feature. HMP traffic can only fail over between adapters that belong to the same card pair. Traffic does not fail over if both the adapters in a card pair fail. However, administrators do not need to configure HF2 adapters as card pairs if TCP/UDP/IP is run over HF2 or MPI uses HMP.
When HMP is configured in the local failover mode, all the resources in the cluster are utilized. If a resource fails in the cluster and is restored, HMP does not utilize that resource until another resource fails.
For more information on Transparent Local Failover while running HMP applications, see “Configuring HMP for Transparent Local Failover Support” on page 96.
Dynamic Resource Utilization (DRU): Partially Supported If you add a new HyperFabric resource (node, cable or switch) to a
cluster running an HMP application, the HyperFabric subsystem will dynamically identify the added resource and start using it. The
Chapter 228
Planning the Fabric
Hyper Messaging Protocol (HMP)
same process takes place when a resource is removed from a cluster. However, DRU is not supported if you add or remove an adapter from a node that is running an HMP application. This is consistent with the fact that OLAR is not supported when an HMP application is running on HyperFabric.
Load Balancing: Supported When an HP 9000 HyperFabric cluster is running HMP applications,
the HyperFabric driver balances the load across all available resources in the cluster, including nodes, adapter cards, links, and multiple links between switches.
Switch Management: Not Supported Switch Management is not supported. Switch management will not
operate properly if it is enabled on a HyperFabric cluster.
Diagnostics: Supported You can run diagnostics to obtain information on many of the
HyperFabric components using the clic_diag, clic_probe and clic_stat commands, as well as the Support Tools Manager (STM).
For more information on HyperFabric diagnostics, see “Running Diagnostics” on page 149.
Configuration Parameters
This section discusses the maximum limits for HMP HyperFabric configurations. There are numerous variables that can impact the performance of any particular HyperFabric configuration. For more information on specific HyperFabric configurations for HMP applications, see “HMP Supported Configurations” on page 33.
HyperFabric is supported on the HP 9000 series servers and workstations only.
HMP is supported on the HF2 adapter, A6386A, only.
The performance advantages that HMP offers are not completely realized unless HMP is used with A6386A HF2 (fiber) adapters and related fiber hardware. See Table 2-2 on page 20 for details. The local failover configuration of HMP is supported only on the A6386A HF2 adapters.
Maximum Supported Nodes and Adapter Cards
Chapter 2 29
Planning the Fabric
Hyper Messaging Protocol (HMP)
HyperFabric clusters running HMP applications are limited to supporting a maximum of 64 adapter cards. However, in local failover configurations, a maximum of only 52 adapters are supported.
In point-to-point configurations running HMP applications, the complexity and performance limitations of having a large number of nodes in a cluster make it necessary to include switches in the fabric. Typically, point-to-point configurations consist of only 2 or 3 nodes.
In switched configurations running HMP applications, HyperFabric supports a maximum of 64 interconnected adapter cards.
A maximum of 8 HyperFabric adapter cards are supported per instance of the HP-UX operating system. The actual number of adapter cards a particular node is able to accommodate also depends on slot availability and system resources. See node specific documentation for details.
A maximum of 8 configured IP addresses are supported by the HyperFabric subsystem per instance of the HP-UX operating system.
Maximum Number of Switches You can interconnect (mesh) up to 4 switches (16-port fiber or Mixed
8 fiber ports) in a single HyperFabric cluster.
Trunking Between Switches (multiple connections) Trunking between switches can be used to increase bandwidth and
cluster throughput. Trunking is also a way to eliminate a possible single point of failure. The number of trunked cables between nodes is only limited by port availability. To assess the effects of trunking on the performance of any particular HyperFabric configuration, contact your HP representative.
Maximum Cable Lengths HF2 (fiber): The maximum distance is 200m (4 standard cable
lengths are sold and supported: 2m, 16m, 50m and 200m). HMP supports up to 4 HF2 switches connected in series with a
maximum cable length of 200m between the switches and 200m between switches and nodes.
HMP supports up to 4 hybrid HF2 switches connected in series with a maximum cable length of 200m between fiber ports.
Chapter 230
Planning the Fabric
Hyper Messaging Protocol (HMP)
HMP is supported on A400, A500, rp2400, rp2450, rp54xx (N-class), rp74xx (L-class), rp8400, and Superdome servers running 64-bit HP-UX.
HMP is supported on HyperFabric starting HyperFabric versions B.11.00.11, B.11.11.01, and B.11.23.00.
HMP is not supported on the A180 or A180C server.
HMP is not supported on 32-bit versions of HP-UX.
Throughput and Latency
Table 2-3 HF2 Throughput and Latency with HMP Applications
Server Class Maximum Throughput Latency
rp 7400 2 + 2 Gbps full duplex per link < 22 microsec
Table 2-4 Supported HyperFabric Adapter Configurations
HF
Adapter
Bus
Type
Supported
HP Systems
HP-UX
Version
OLAR
Support?
Maximum
Adapters
per System
A6386A PCI (4X) rx2600 servers 11i v2 No 1
A6386A PCI (4X) rx56XX
11i v2 No 4
servers
A6386A PCI (4X) zx6000
11i v2 No 1
workstations
A6386A PCI (4X) SD64A servers 11i v2 Yes 8 (maximum
4 per PCI card cage)
A6386A PCI (4X) rx7620 servers 11i v2 No 8 (maximum
4 per PCI card cage)
A6386A PCI (4X) rx8620 servers 11i v2 Yes 8 (maximum
4 per PCI card cage)
A6386A PCI (4X) rx4640 servers 11i v2 Yes 6
Chapter 2 31
Planning the Fabric
Hyper Messaging Protocol (HMP)
NOTE The local failover configuration on HMP is supported only on the A6386A
HF2 adapters.
Chapter 232
Planning the Fabric
Hyper Messaging Protocol (HMP)
HMP Supported Configurations
Multiple HMP HyperFabric configurations are supported to match the performance, cost and scaling requirements of each installation.
In the section, “Configuration Parameters” on page 29, the maximum limits for HMP enabled HyperFabric hardware configurations were outlined. This section discusses the HMP enabled HyperFabric configurations that HP supports. These recommended configurations offer an optimal mix of performance and availability for a variety of operating environments.
There are many variables that can impact HyperFabric performance. If you are considering a configuration that is beyond the scope of the following HP supported configurations, contact your HP representative.
Point-to-Point Configuration
You can interconnect large servers like the HP Superdome to run Oracle RAC 9i and enterprise resource planning applications. These applications are typically consolidated on large servers.
Point-to-point connections between servers support the performance benefits of HMP without investing in HyperFabric switches. This is a good solution in small configurations where the benefits of a switched HyperFabric cluster might not be required (see configuration A in Figure 2-5).
If an HMP application is running over HyperFabric and another node or adapter is added to either of the nodes, then it is necessary to also add a HyperFabric switch to the cluster (see configuration B in Figure 2-5).
Chapter 2 33
Planning the Fabric
Hyper Messaging Protocol (HMP)
Figure 2-5 HMP Point-To-Point Configurations
Chapter 234
Planning the Fabric
Hyper Messaging Protocol (HMP)
Enterprise (Database) Configuration
The HMP enterprise configuration illustrated in Figure 2-6 is very popular for running Oracle RAC 9i.
Superdomes or other large servers make up the Database Tier. Database Tier nodes communicate with each other using HMP.
Application Tier nodes communicate with each other and to the Database Tier using TCP/UDP/IP.
Figure 2-6 HMP Enterprise (Database) Configuration, Single Connection
Between Nodes
Chapter 2 35
Planning the Fabric
Hyper Messaging Protocol (HMP)
Enterprise (Database) - Local Failover Supported Configuration
The HMP enterprise configuration is a scalable solution. For high availability and performance, you can easily scale the HMP enterprise configuration with multiple connections between the HyperFabric resources. Any single point of failure in the database tier of the fabric is eliminated in Figure 2-7.
Figure 2-7 Local Failover Supported Enterprise (Database) Configuration,
Multiple Connections between Nodes
In this configuration, if a HyperFabric resource (adapter, cable, switch or switch port) fails in a cluster, HMP transparently fails over traffic using another available resource. For more information, see “Configuring HMP for Transparent Local Failover Support” on page 96.
Chapter 236
Planning the Fabric
Hyper Messaging Protocol (HMP)
Technical Computing (Work Stations) Configuration
This configuration is typically used to run technical computing applications with HP-MPI. A large number of small nodes are interconnected to achieve high throughput (see Figure 2-8). High availability is not usually a requirement in technical computing environments.
HMP provides the high performance, low latency path necessary for these technical computing applications. You can interconnect up to 56 nodes using HP 16-port switches. You cannot link more than four 16-port switches in a single cluster (see Figure 2-9).
The HP “J”, “B”, and “C” class workstations provide excellent performance and return on investment in technical computing configurations.
Chapter 2 37
Planning the Fabric
Hyper Messaging Protocol (HMP)
Figure 2-8 Technical Computing Configuration
Chapter 238
Hyper Messaging Protocol (HMP)
Figure 2-9 Large Technical Computing Configuration
Planning the Fabric
Chapter 2 39
Planning the Fabric
Hyper Messaging Protocol (HMP)
Chapter 240

3 Installing HyperFabric

This chapter contains the following sections that describe the HyperFabric installation:
“Checking HyperFabric Installation Prerequisites” on page 43.
Chapter 3 41
Installing HyperFabric
“Installing HyperFabric Adapters” on page 44.
“Installing the Software” on page 51.
“Installing HyperFabric Switches” on page 57.
Chapter 342
Installing HyperFabric

Checking HyperFabric Installation Prerequisites

Checking HyperFabric Installation Prerequisites
Before installing HyperFabric, ensure that the following hardware and software prerequisites are met:
Check the HyperFabric Release Notes for known problems, required
patches, or other information needed for installation.
Confirm that the /usr/bin, /usr/sbin, and /sbin directories are in
your PATH by logging in as root and using the echo $PATH command.
Confirm that the HP-UX operating system is the correct version. Use
the uname -a command to determine the HP-UX version. For more information about the required operating system versions,
see HyperFabric Release Notes.
If you are installing an HF2 switch, confirm that you have four
screws with over-sized heads.
Confirm that there are cables of proper length and type (fiber) to
make connections in the fabric (adapter to adapter, adapter to switch, or switch to switch).
Confirm that there is at least one loopback plug for testing the
adapters and switches (a fiber loopback plug [HP part number A6384-67004] is shipped with each HF2 switch).
Confirm that necessary tools are available to install the HyperFabric
switch mounting hardware. In addition, check the HP 9000 system’s documentation to determine if additional tools may be required for component installation.
Confirm that the software media is correct. Create a map of the fabric (optional). Confirm that HP-UX super-user privileges are available; they are
necessary to complete the HyperFabric installation.
The first HyperFabric installation step is installing HyperFabric adapter cards in the nodes.
Chapter 3 43
Installing HyperFabric

Installing HyperFabric Adapters

Installing HyperFabric Adapters
This section contains information about installing HyperFabric adapters in HP 9000 systems. Online Addition and Replacement (OLAR) information is provided in the section, “Online Addition and Replacement” on page 44.
CAUTION HyperFabric adapters contain electronic components that can easily be
damaged by small amount of electricity. To avoid damage, follow these guidelines:
Store adapters in their antistatic plastic bags until installation.
Work in a static-free area, if possible.
Handle adapters by the edges only. Do not touch electronic components or electrical traces.
Use the disposable grounding wrist strap provided with each adapter. Follow the instructions included with the grounding strap.
Use a suitable ground—any exposed metal surface on the computer chassis.
For specific instructions, see system-specific documentation on “installing networking adapters” for each type of HP 9000 system that HyperFabric adapters will be installed into.
When the HyperFabric adapters have been installed, go to “Installing the Software” on page 51.

Online Addition and Replacement

Online Addition and Replacement (OLAR) allows PCI I/O cards,
adapters or controllers to be replaced or added to HP 9000 systems, without the need for completely shutting down and rebooting the system, or adversely affecting other system components. This feature is only
Chapter 344
Installing HyperFabric
Installing HyperFabric Adapters
available on HP 9000 systems that are designed to support OLAR. The system hardware uses the per-slot power control combined with OS support to enable this feature.
NOTE OLAR is supported only on TCP/UDP/IP over HF2 adapters.
Not all add-in cards have this capability, but over time many cards will be gaining this capability.
The HyperFabric Release Notes contains information about which HP 9000 systems and HyperFabric adapters OLAR is supported for.
IMPORTANT At this time, Superdome systems are not intended for access by users.
HP recommends that these systems only be opened by a qualified HP engineer. Failure to observe this requirement can invalidate any support agreement or warranty to which the owner might otherwise be entitled.
There are two methods to add or replace OLAR-compatible cards:
Using the SAM utility.
Issuing command-line commands, through olrad, that refer to the HyperFabric OLAR script (/usr/sbin/olard.d/clicd).
HP recommends that SAM be used for OLAR procedures, instead of the rad command. This is primarily because SAM prevents the user from doing things that might have adverse effects. This is not true when the rad command is used.
For detailed information about using either of these two procedures, see Configuring HP-UX For Peripherals. You can order that document from HP, or you can view, download, and print it from the following URL: http://www.docs.hp.com.
Chapter 3 45
Installing HyperFabric
Installing HyperFabric Adapters
Table 3-1 below explains some important OLAR-related terms.
Table 3-1 Important OLAR Terms
Term Meaning
OLAR All aspects of the OLAR feature
Power Domain A grouping of 1 or more interface
target card / target card slot The interface card which will be
affected card / affected card slot Interface cards and the card slots
including Online Addition (OLA) and Online Replacement (OLR).
card slots that are powered on or off as a unit. (Note: Multi-slot power domains are not currently supported.)
added or replaced using OLAR, and the card slot in which it resides.
they reside in, which are in the same power domain as the target slot.
IMPORTANT In many cases, other interface cards and slots within the system are
dependent on the target card. For example, if the target card is a multiple-port card, suspending or deleting drivers for the target card slot also suspends individual drivers for the multiple hardware paths on that card.
During a card replacement operation, SAM performs a Critical Resource Analysis (CRA), which checks all ports on the target card for critical resources that would be temporarily unavailable while the card is shut down.
Chapter 346
Installing HyperFabric
Installing HyperFabric Adapters
Planning and Preparation
As mentioned previously, for the most part, SAM prevents the user from performing OLAR procedures that would adversely affect other areas of the HP 9000 system. See Configuring HP-UX For Peripherals for detailed information.
Critical Resources
The effects of shutting down a card’s functions must be considered. Replacing a card that is still operating can have extensive consequences. Power to a slot must be turned off when a card is removed and a new card is inserted.
This is particularly important if there is no online failover or backup card to pick up those functions. For example:
Which mass storage devices will be temporarily disconnected when a card is shut down?
Will a critical networking connection be lost?
A critical resource is one that would cause a system crash or prevent an operation from successfully completing if the resource were temporarily suspended or disconnected. For example, if the SCSI controller is connected to the unmirrored root disk or swap space, the system will crash when the SCSI controller is shut down.
During an OLAR procedure, it is essential to check the targeted card for critical resources, as well as the effects of existing disk mirrors and other situations where a card’s functions can be taken over by another card that will not be affected.
As mentioned earlier, SAM performs a thorough CRA automatically, and presents options based on its findings. If it is determined that critical resources will be affected by the OLAR procedure, the card could be replaced when the system is offline. If action must be taken immediately, an online addition of a backup card and deletion of the target card could be attempted using the rad command.
Card Compatibility
This section explains card compatibility considerations for doing OLAR.
Chapter 3 47
Installing HyperFabric
Installing HyperFabric Adapters
Online Addition (OLA) Multiple cards can be added at the same time. When adding a card online, the first issue to resolve is whether the new card is compatible with the system. Each OLAR-capable PCI slot provides a set amount of power. The replacement card cannot require more power than there is available.
The card must also operate at the slot’s bus frequency. A PCI card must run at any frequency lower than its maximum capability, but a card that could operate at only 33 MHz would not work on a bus running at 66 MHz. rad provides information about the bus frequency and power available at a slot, as well as other slot-related data.
If an HP 9000 system has one or more slots that support OLAR and OLA will be used to install a HyperFabric adapter in one of those slots—install the adapter in the HP 9000 system according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the “Configuring HP-UX Peripherals” manual.
After adding a new HyperFabric adapter, SAM tries to locate the HyperFabric software. If SAM cannot locate the HyperFabric software, the new adapter cannot be used until the software is installed (software installation requires a system reboot). If SAM locates the HyperFabric software, SAM determines whether the new adapter is functional. If it is not functional, SAM displays an error message.
If the new adapter is functional, SAM displays a message telling the user to configure the adapter and start HyperFabric. If only one adapter is being added, issue the clic_init -c command or use SAM to configure the adapter, and then issue the clic_start command or use SAM to start HyperFabric. If multiple adapters are being added, add all of the adapters first, and then run clic_init -c and clic_start or use SAM. See “Performing the Configuration” on page 78 and “Starting HyperFabric” on page 109 for more information about configuring and starting HyperFabric.
CAUTION Do not change any configuration information for an existing HyperFabric
adapter or switch while you are using clic_init -c to configure a new adapter.
When you have completed the adapter installation, go to “Installing the Software” on page 51.
Chapter 348
Installing HyperFabric
Installing HyperFabric Adapters
Online Replacement (OLR) When replacing an interface card online, the replacement card must be identical to the card being replaced (or at least be able to operate using the same driver as the replaced card). This is referred to as like-for-like replacement and should be adhered to, because using a similar but not identical card can cause unpredictable results. For example, a newer version of the target card that is identical to the older card in terms of hardware might contain an updated firmware version that could potentially conflict with the current driver. An A6386A adapter must be replaced with another A6386A adapter, etc. Also, the old adapter and new adapter must have the same revision levels.
When a replacement card is added to an HP 9000 system, the appropriate driver for that card must be configured in the kernel before beginning the replacement operation. SAM ensures the correct driver is present. (In most cases, the replacement card will be the same type as a card already in the system, and this requirement will be automatically met.) Keep the following things in mind:
If the necessary driver is not present and the driver is a dynamically loadable kernel module (DLKM), it can be loaded manually. See the “Dynamically Loadable Kernel Modules” section in “Configuring HP-UX For Peripherals” for more information.
If the driver is static and not configured in the kernel, then the card cannot be added online. The card could be physically inserted online, but no driver would claim it.
If there is any question about the driver’s presence, or if it is uncertain that the replacement card is identical to the existing card, ioscan can be used together with rad to investigate.
If more than one operational HyperFabric adapter is present when SAM requests the suspend operation for all ports on the target adapter, HyperFabric will redirect the target adapter’s traffic to a local backup adapter using local failover. Client applications using the replaced adapter will not be interrupted in any way.
If the adapter being replacing is active and it is the only operational HyperFabric adapter on the HP 9000 system, SAM displays the following warning message:
WARNING: You have 1 operational HyperFabric card. If you go ahead with this operation you will lose network access via HyperFabric until the on-line replaced HyperFabric card becomes operational.
Chapter 3 49
Installing HyperFabric
Installing HyperFabric Adapters
You are asked if you want to continue. If you reply Yes, client applications are suspended. Replace the adapter according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the Configuring HP-UX Peripherals manual.
When an adapter has been replaced, client application activity resumes unless the TCP timers or the application timers have popped.
CAUTION Do not use the clic_start command or the clic_shutdown command,
while an installed adapter is suspended. Do not use SAM to start or stop HyperFabric while an installed adapter is suspended. The operation will fail and an error message will be displayed.
After a HyperFabric adapter has been replaced, SAM checks the replacement adapter to ensure that it is permitted according to the like-for-like rules. If the adapter is permitted, SAM automatically activates it. If it is not permitted, SAM displays an error message.
Chapter 350

Installing the Software

This section describes the HyperFabric file structure and the steps necessary to load the software. The software must be installed on each instance of the HP-UX operating system in the fabric.

File Structure

The HyperFabric file structure is shown in Figure 3-1 below. The structure is shown for informational purposes only. The user cannot modify any of the files or move them to a different directory.
Figure 3-1 HyperFabric File Structure
Installing HyperFabric
Installing the Software
/
/resmon
/dictionary
/clic_01
/usr
/conf
/lib
/libclic_dlpi_drv.a /libha_drv.a
/opt
/master.d
/etc
/clic
/libclic_mgmt.a
/rc.config.d
/clic_global_conf
/lib
/clic_diag /clic_dump /clic_init /clic_mgmtd /clic_mond /clic_ping
/clic_probe /clic_shutdown /clic_start /clic_stat
/opt
/clic
/bin
/sbin
/init.d
/clic
/firmware
/clic_fw /clic_fw_1x32c
/clic_fw_4x8c /clic_fw_4x32c /clic_fw_hf28c
/clic_fw_hf232c /clic_fw_db
/var/adm
/clic_ip_drv.trc /clic_ip_drv.trc0 /clic_ip_drv.trc1
/clic_log
/clic_log.old /OLDclic_log
/share
/man
/man1m.Z
Chapter 3 51
Installing HyperFabric
Installing the Software
The commands and files used to administer HyperFabric typically have a prefix of clic_. CLIC stands for CLuster InterConnect, and it is used to differentiate those HyperFabric commands/files from other commands/files. For example, the HyperFabric command clic_init is different from the HP-UX init command.
Each of the files shown in Figure 3-1 above is briefly described below:
/etc/opt/resmon/dictionary/clic_01
/etc/rc.config.d/clic_global_conf
/sbin/init.d/clic
/var/adm/clic_ip_drv.trc
The HyperFabric dictionary file for the Event Monitoring Service (EMS).
The global configuration file, which contains the IP addresses for each adapter and each HyperFabric switch (if any) in the fabric.
The system boot startup script for the HyperFabric management process.
One of the software’s trace files. This file is created when the clic_diag -D TCP_IP command is run.
/var/adm/clic_ip_drv.trc0 One of the HyperFabric software’s trace files. This is the primary file
that is created when the clic_diag -C TCP_IP command is run.
/var/adm/clic_ip_drv.trc1 One of the HyperFabric software’s trace files. This file is created
when the clic_diag -C TCP_IP command is run, and the primary trace file (clic_ip_drv.trc0) becomes full.
/var/adm/clic_log The global log file that is updated by the HyperFabric management
process.
/var/adm/clic_log.old The backup copy of the log file that is created when the log file grows
larger than 100 Kbytes.
Chapter 352
Installing HyperFabric
Installing the Software
/var/adm/OLDclic_log The log file from the previous time the clic_start command was
executed.
/usr/conf/lib/libclic_dlpi_drv.a The kernel library that contains the HyperFabric software.
/usr/conf/lib/libha_drv.a The kernel library that contains the High Availability (HA) software.
/usr/conf/master.d/clic This file is described along with the other master files in the master
man page (type man master at the HP-UX prompt).
/opt/clic/lib/libclic_mgmt.a The HyperFabric management API library.
/opt/clic/bin The directory containing the HyperFabric management commands:
clic_diag, clic_init, clic_probe, clic_shutdown, clic_start, clic_stat, and clic_dump. (Note that clic_dump is for HP internal
use only.) Also, clic_ping is replaced by clic_probe.This directory also contains the HyperFabric management process (clic_mgmtd) and the HyperFabric EMS monitor process (clic_mond).
/opt/clic/firmware/clic_fw The 1X HSC HyperFabric 8-bit CRC firmware. This file must not be
modified.
/opt/clic/firmware/clic_fw_1x32c The 1X HSC HyperFabric 32-bit CRC firmware. This file must not be
modified.
Chapter 3 53
Installing HyperFabric
Installing the Software
/opt/clic/firmware/clic_fw_4x8c
/opt/clic/firmware/clic_fw_4x32c
/opt/clic/firmware/clic_fw_hf28c
/opt/clic/firmware/clic_fw_hf232c
/opt/clic/firmware/clic_fw_db
/opt/clic/share/man/man1m.Z
The 4X PCI HyperFabric 8-bit CRC firmware. This file must not be modified.
The 4X HyperFabric PCI 32-bit CRC firmware. This file must not be modified.
The HyperFabric2 8-bit firmware. This file must not be modified.
The HyperFabric2 32-bit firmware. This file must not be modified.
A binary file where adapter-specific configuration information is stored. The management process creates this file using default values.
The man pages for the HyperFabric commands.
Chapter 354
Installing HyperFabric
Installing the Software

Loading the Software

Listed below are the steps you must follow to load the HyperFabric software, using the HP-UX swinstall program.
Step 1. Log on to the system as root.
Step 2. Insert the software media into the appropriate drive. If the software is
being loaded from a CD-ROM, go to step 3; otherwise, go to step 4.
Step 3. Mount the CD-ROM drive by using the following command at the
command prompt:
$ mount
where
Step 4. Run the swinstall program using the following command:
$ /usr/sbin/swinstall
This opens the “Software Selection” window.
Step 5. Change the Source Host Name, if necessary, and then enter the mount
point of the drive in the Source Depot Path field. Select the OK button to return to the “Software Selection” window.
The “Software Selection” window now contains a list of available software to install.
Step 6. Highlight the HyperFabric software:
HP-UX 11i v2: HyprFabrc-00
Step 7. Choose Mark for Install from the “Actions” menu; this chooses the
highlighted software.
Step 8. From the “Actions” menu, select the “Install...” menu, and then choose
Install. This begins product installation and opens the “Install Analysis” window.
device_name
device_name
is the name assigned to the CD-ROM drive.
Step 9. Select the OK button in the “Install Analysis” window when the Status
field displays a “Ready” message.
Step 10. Select the YES button in the “Confirmation” window to start software
installation.
Chapter 3 55
Installing HyperFabric
Installing the Software
swinstall loads the fileset, runs the control script for the filesets, and builds the kernel. When the processing is finished, the “Status” field displays a “Ready” message. Select “Done” and then the “Note” window opens.
Step 11. Select the OK button in the “Note” window to reboot. The user interface
disappears and the system reboots.
Step 12. When the system comes back up, log on to the system as root and view
the /var/adm/sw/swagent.log and /var/adm/sw/swinstall.log files to view error or warning messages that may have occurred during the installation.
Step 13. While still logged in as root, view the /etc/services file to ensure that
these two HyperFabric-related lines are present:
hp-clic 3384/tcp #clic management daemon
hp-clic 3384/udp #clic switch management
These lines are used by the HyperFabric software, and are not comments; therefore, do not remove them from the file.
Step 14. Verify that all installed HyperFabric adapters have a software state of
“CLAIMED,” by running the ioscan -nf -C clic command.
A check is also done to make sure all of the HyperFabric adapters have been claimed when clic_init is activated or when SAM is used to configure HyperFabric.
Step 15. If one or more HyperFabric switches are included in the configuration, go
to the next section of this chapter, “Installing HyperFabric Switches”; otherwise, go to Chapter 4, “Configuring HyperFabric,” on page 67.
Chapter 356
Installing HyperFabric

Installing HyperFabric Switches

Installing HyperFabric Switches
This section contains the information you need to install HyperFabric switches. As stated earlier, the term HyperFabric2 (HF2) switch refers to the functional switch (the A6384A switch chassis with one of the switch modules installed).

Before Installation

Before you install the HyperFabric switch, you should be aware of the following:
The A6384A HF2 switch is supported beginning with the following
HyperFabric software versions:
— HP-UX 11.0: version B.11.00.11 — HP-UX 11i v1: version B.11.11.01 — HP-UX 11i v2: version B.11.23.00. HyperFabric switches are not supported by software versions earlier
than those mentioned above, respectively. To determine the version of HyperFabric you have, issue the
following command at the command prompt:
$ swlist | grep -i hyperfabric
For the HF2 switch, HP recommends that you use the rails shipped
with the switch when you mount it in a standard 19-inch rack, even though the switch can be mounted in the rack by itself (without the rails).
WARNING To prevent overheating, you must leave one rack unit (1 EIA)
of empty space above the HyperFabric switch.
After the HyperFabric switch is mounted in the rack, attach the
various cables to the switch. To avoid damage to any of the cables, follow these guidelines:
Chapter 3 57
Installing HyperFabric
Installing HyperFabric Switches
— If your cables have dust caps over the connectors, keep them in
— Be careful not to stretch, puncture, or crush the cable.
To install an HF2 switch, see “Installing the HF2 Switch” on page 59.
place until you are ready to connect them. This prevents dirt and oils from soiling any important surfaces.
Chapter 358
Installing HyperFabric
Installing HyperFabric Switches

Installing the HF2 Switch

This section contains information on installing an HF2 switch. The front of the HF2 switch has a flange—or “wing”—on each side, with
two holes for attaching the switch to the rack. The following figures do not show the flanges.
Figure 3-2 shows the front of the HF2 switch with an A6388A HF2 8-port fiber switch module installed in the switch’s expansion slot.
Figure 3-2 Front of HF2 Switch (A6388A Switch Module Installed)
Integrated Ethernet management
LAN card
Status
Status
Status
Powe r
A
B
Por t
7
Por t
15
Ethernet
Por t Main
Por t
6
Por t
14
A6388A HF2 8-port fiber switch module in expansion slot
Ethernet
Por t
Por t
5
Por t
13
Aux
Label showing Ethernet MAC address
Por t
4
Por t
12
Por t
3
Por t
11
Port LED colors and meanings
legend
Por t
2
Por t
10
Integrated 8-port fiber card
Por t
Por t
1
9
Por t
0
Por t
8
Chapter 3 59
Installing HyperFabric
Installing HyperFabric Switches
Figure 3-3 shows the front of the HF2 switch with an A6389A HF2 4-port copper switch module installed in the switch’s expansion slot.
Figure 3-3 Front of HF2 Switch (A6389A Switch Module Installed)
Integrated Ethernet management
LAN card
Status
Status
Status
Powe r
A
B
Por t
7
Ethernet
Por t
Main
Por t
6
Por t
11
A6389A HF2 4-port copper switch module in expansion slot
You can install the HF2 switch in one of the following ways:
Using the rail kit that is shipped with the switch (see the next section, “With the Rail Kit”). HP strongly recommends installing the HF2 switch this way.
Ethernet
Por t
5
Por t
Aux
Label showing Ethernet MAC address
Por t
4
Por t
10
Por t
3
Port LED colors and meanings
legend
Por t
2
Por t
9
Integrated 8-port
fiber card
Por t
1
Por t
0
Por t
8
Attaching the switch directly to the rack (see “Without the Rail Kit” on page 65).
Chapter 360
Installing HyperFabric
Installing HyperFabric Switches
With the Rail Kit
As mentioned earlier, HP strongly recommends installing the HF2 switch using the rail kit.
When you install the HF2 switch, you will be putting the front of the switch—the end with the flanges (“wings”)—at the back of the rack. The steps for installing the HF2 switch using the rail kit are as follows:
Step 1. Prepare the rack for rail and switch installation.
Step 2. Install and secure the rails in the rack, using two screws per rail. The
following figure shows the rack with the rails installed.
rack back
screws
screws
rack
front
Step 3. From the front of the rack, slide the switch—with the front of the switch
facing the back of the rack—into the rack, on the rails. Move it until it is snug against the back of the rack. You might not have enough clearance between the switch and the rail screws; this may prevent you from easily
Chapter 3 61
Installing HyperFabric
Installing HyperFabric Switches
sliding the switch. If so, try lifting the switch over the rail screws. If you cannot do this, remove the rail screws, slide the switch into position, and put the rail screws back in.
Step 4. Align the two holes in each flange (“wing”) on the switch’s front with the
holes in the rack frame. Fasten each flange of the switch to the rack by putting a screw in each of the four holes in the flanges. Be sure to use screws with over-sized heads. The following figure shows the rack with the switch in this position.
rack back
front of switch
(with flanges)
rack
front
Step 5. From the front of the rack, install a bracket on the outside of each rail,
using two screws per bracket. Be sure to use the upper screw holes on each bracket. Put the screws in the two square holes closest to the back of the switch, so that the brackets—referred to as “bracket 1” and “bracket 2” in these steps—are snug against the switch. Be sure that the brackets make contact with the beveled (sloped) part of the switch (not
Chapter 362
Installing HyperFabric
Installing HyperFabric Switches
the flat part). Otherwise, the brackets will not secure the back of the switch properly. The following figure shows the rack with these two brackets installed.
rack back
bracket 2
front of switch
(with flanges)
rack
front
bracket 1
Step 6. When you are sure the brackets are snug against the beveled part of the
switch, tighten all four screws in brackets 1 and 2.
Step 7. For each port that will be connected to an HyperFabric adapter in an
HP 9000 system, attach the cable from the corresponding adapter. Your connections must be copper-to-copper and fiber-to-fiber (including cables).
Step 8. Connect the switch to the Ethernet network.
Chapter 3 63
Installing HyperFabric
Installing HyperFabric Switches
Step 9. Plug the switch’s power cord into the rack’s PDU, if it has one.
Alternatively, you can plug a power cord that is compatible with your country’s requirements into a power strip or outlet that you want to use for the switch. (In this case, you are responsible for obtaining a compatible power cord.)
Step 10. Power on the HF2 switch by plugging the power cord into the AC inlet on
the back of the switch. (There is no power switch.)
Step 11. Once the power is on, check these LEDs on the integrated Ethernet
management LAN adapter card (in the top slot of the switch):
The “Operating/Fault” LED shows solid green.
The “Power A” and “Power B” LEDs show solid green.
The “Ethernet Port Main” and “Ethernet Port Aux” LEDs are
showing solid green (connected) or flashing green (Ethernet traffic is flowing to the switch). See Figure 3-2 on page 59 or Figure 3-3 on page 60 for the locations of the LEDs.
Step 12. On the integrated 8-port fiber card (in the middle slot of the switch),
check that for each switch port that is connected to an HF2 adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.
Step 13. On the switch module in the expansion slot (the bottom slot of the
switch), check that for each switch port that is connected to an HyperFabric adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.
For more information about the switch’s LEDs, see “HF2 Switch LEDs” on page 148.
Step 14. If you want to install another HF2 switch using the rail kit, go back to
step 1.
If you want to install another HF2 switch without using the rail kit, go to the next section, “Without the Rail Kit”.
Otherwise, go to Chapter 4, “Configuring HyperFabric,” on page 67.
Chapter 364
Installing HyperFabric
Installing HyperFabric Switches
Without the Rail Kit
As mentioned earlier, HP strongly recommends installing the HF2 switch using the rail kit (described in the previous section, “With the Rail Kit” on page 61).
When you install the HF2 switch, you will be putting the front of the switch—the end with the flanges (“wings”) — at the back of the rack. The steps for installing the HF2 switch without using the rail kit are as follows:
Step 1. Prepare the rack for switch installation.
Step 2. Insert the HF2 switch into the rack, with the front of the switch snug
against the back of the rack.
Step 3. Align the two holes in each flange on the switch’s front with the holes in
the rack frame.
Step 4. Fasten each flange of the switch to the rack by putting a screw in each of
the four holes in the flanges. Be sure to use screws with over-sized heads.
Step 5. Tighten all of the screws so that the HF2 switch is firmly mounted in the
rack.
Step 6. For each port that will be connected to an HyperFabric adapter in an
HP 9000 system, attach the cable from the corresponding adapter. Your connections must be copper-to-copper and fiber-to-fiber (including cables).
Step 7. Connect the switch to the Ethernet network.
Step 8. Plug the switch’s power cord into the rack’s PDU, if it has one.
Alternatively, you can plug a power cord that is compatible with your country’s requirements into a power strip or outlet that you want to use for the switch. (In this case, you are responsible for obtaining a compatible power cord.)
Step 9. Power on the HF2 switch by plugging the power cord into the AC inlet on
the back of the switch. (There is no power switch.)
Chapter 3 65
Installing HyperFabric
Installing HyperFabric Switches
Step 10. Once the power is on, check these LEDs on the integrated Ethernet
management LAN adapter card (in the top slot of the switch):
The “Operating/Fault” LED shows solid green.
The “Power A” and “Power B” LEDs show solid green.
The “Ethernet Port Main” and “Ethernet Port Aux” LEDs are
showing solid green (connected) or flashing green (Ethernet traffic is flowing to the switch). See Figure 3-2 or Figure 3-3 below for the locations of the LEDs.
Step 11. On the integrated 8-port fiber card (in the middle slot of the switch),
check that for each switch port that is connected to an HF2 adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.
Step 12. On the switch module in the expansion slot (the bottom slot of the
switch), check that for each switch port that is connected to a HyperFabric adapter, the LED on the port shows as solid green (see Figure 3-2 on page 59 or Figure 3-3 on page 60). This means the connection is operational.
For more information about the switch’s LEDs, see “HF2 Switch LEDs” on page 148.
Step 13. If you want to install another HF2 switch without using the rail kit, go to
step 1.
If you want to install another HF2 switch using the rail kit, go to “With the Rail Kit” on page 61.
Otherwise, go to Chapter 4, “Configuring HyperFabric,” on page 67.
Chapter 366
4 Configuring HyperFabric
This chapter contains the following sections that describe configuring HyperFabric:
“Configuration Overview” on page 69.
Chapter 4 67
Configuring HyperFabric
“Information You Need” on page 71.
“Performing the Configuration” on page 78.
“Deconfiguring a HyperFabric Adapter with SAM” on page 83.
“Configuring the HyperFabric EMS Monitor” on page 85.
“Configuring HyperFabric with ServiceGuard” on page 87.
“Configuring HMP for Transparent Local Failover Support” on page 96.
Chapter 468
Configuring HyperFabric
Configuration Overview
Configuration Overview
You do not need to configure the HyperFabric switch because the HyperFabric management process performs automatic routing and configuring for the switch. So, configuring HyperFabric consists only of creating the HyperFabric /etc/rc.config.d/clic_global_conf global configuration file on each node in the fabric. The configuration file contains the following information:
The IP addresses and subnet mask of the HyperFabric adapters installed in the node.
For each HyperFabric switch in the fabric — the switch’s IP address, and the MAC address of the switch’s Ethernet port. This applies only if you enable switch management. In addition, you cannot enable switch management through SAM — you must use the clic_init command.
The IP multicast address that all the switches and nodes in the fabric will register to (if you are going to enable switch management).
The IP address of the local node’s Ethernet LAN interface. This LAN interface must be on the same subnet as Ethernet ports of the HyperFabric switches (if you are going to enable switch management). (A node might have multiple LAN interfaces.)
NOTE HP recommends that you do not enable switch management.
You can create the global configuration file by either running the clic_init command or using SAM to configure each HyperFabric adapter.
The clic_init command and SAM also place the necessary entries into the following three files:
The system /etc/rc.config.d/netconf file.
Chapter 4 69
Configuring HyperFabric
Configuration Overview
IMPORTANT In this file, clic_init and SAM add some HyperFabric-related lines
that end with the characters #clic. These lines are used by the HyperFabric software — and are not comments — so do not remove them from the file.
The system /etc/rc.config.d/clic_global_conf file.
The /etc/rarpd.conf (Reverse Address Resolution Protocol [RARP]) support file. This file is used in the management of the HyperFabric switches (if you are going to enable switch management).
The clic_init command is described in “Using the clic_init Command” on page 79. Using SAM to configure an adapter is described in “Using SAM” on page 81.
After you have used the clic_init command or SAM, you can configure HyperFabric with ServiceGuard, if necessary. For more information, see “Configuring HyperFabric with ServiceGuard” on page 87.
You can configure card pairs for the Transparent Local Failover Feature of HMP available with the B.11.23.01 release of HyperFabric. For more information, see “Configuring HMP for Transparent Local Failover Support” on page 96.
Chapter 470
Configuring HyperFabric

Information You Need

Information You Need
When you run the clic_init command or use SAM for configuration, you have to provide certain configuration information. So, before you run clic_init or use SAM, you should have the following information:
For each node in the fabric, determine if that node will need to
interoperate with other nodes that are using; any HP-UX 11.0 HyperFabric versions earlier than B.11.00.11 or any HP-UX 11i v1 HyperFabric versions earlier than B.11.11.01.
For each HyperFabric adapter installed in the local node:
The adapter’s IP address.
IMPORTANT The last 10 bits of each adapter’s IP address must be unique
throughout the entire fabric. And, remember that the last part of the address cannot be 0 (that is, the IP address cannot be
n.n.n
.0). Also, note that HyperFabric converts these 10 bits to a decimal value called the Virtual Route IDentifier (VRID), which is used in some HyperFabric command input and output.
The subnet mask. When you run clic_init or use SAM, if you
do not specify a value for this, a default subnet mask is chosen based on the adapter’s IP address.
When clic_init begins to prompt you for the information for each adapter, it assigns an ID (for example, clic0) to that adapter and displays it as part of the first prompt. If you use SAM, it assigns the adapter an ID and displays it in the “Adapter Name” column of the “Configure HyperFabric Adapter” screen. You can also determine an adapter’s ID by running the clic_stat command (see “The clic_stat Command” on page 117). You should note each adapter’s ID, because it is used as input to other HyperFabric commands.
For using the Transparent Local Failover feature of HMP available
in the version B.11.23.01 of HyperFabric, you need to define the card pairs.
For each HyperFabric switch in the fabric (if you are going to enable
switch management):
Chapter 4 71
Configuring HyperFabric
Information You Need
The IP address of the switch. The MAC address of the switch’s Ethernet port. If you do not
already know the switch’s MAC address, it is printed on a label on the back of the HF switch and on the front of the HF2 switch. See Figure 3-2 on page 66 for the location of the label on the HF switch, and Figure 3-2 on page 59 and Figure 3-3 on page 60 for the location of the label on the HF2 switch.
IMPORTANT You cannot enable switch management through SAM — you must
use the clic_init command.
When clic_init begins to prompt you for the information for each switch, it assigns an ID (for example, sw_clic0) to that switch and displays it as part of the first prompt. Note that you can also determine a switch’s ID by running the clic_stat command (see “The clic_stat Command” on page 117). You should note each switch’s ID, because it is used as input to other HyperFabric commands.
For the entire fabric, you need the IP multicast address that all the
switches and nodes in the fabric will register to. The address must be a class D address. Note that if you do not have switch management enabled, you do not need this information (clic_init will not prompt you for it).
For each node in the fabric, you need the IP address of the node’s
Ethernet LAN interface that is on the same subnet as the switches. (As mentioned earlier, a node might have multiple LAN interfaces.) Note that if you do not have switch management enabled, you do not need this information (clic_init will not prompt you for it).
As stated earlier, HP recommends that you do not enable switch management.
IMPORTANT You should also check your /etc/hosts file — when you are using files
for host name look up — to ensure that the entries for all of the systems are in the correct format: the official host name, which is the full domain extended host name, and any alias names. For example:
Chapter 472
Configuring HyperFabric
Information You Need
IP_address IP_address IP_address
bently6.corp3.com bently6 bently4.corp7.com test1 bently2.corp4.com test3
Chapter 4 73
Configuring HyperFabric
Information You Need
Configuration Information Example
This example uses some “dummy” (that is, not valid) addresses to the components in Figure 4-1. The dummy addresses are used only to show the flow of the information provided as input to the clic_init command and SAM. Do not try to use these addresses in your configuration.
To use clic_init to configure the Transparent Local Failover feature on HMP, see the section, “Configuring HMP for Transparent Local Failover Support - Using the clic_init command” on page 104.
Figure 4-1 Map for Configuration Information Example
Ethernet LAN
S
Switch ID: sw_clic0 IP address: 193.0.0.20
Ethernet MAC address:
00:60:b0:d0:02:57
IP multicast address:
226.10.1.1
HF switch 0
HF
switch 1
Switch ID: sw_clic1
IP address: 193.0.0.21 Ethernet MAC address:
00:60:b0:d0:02:56
IP multicast address:
226.10.1.1
HF
adapter 0
Adapter ID:
clic0
IP address:
192.0.0.1
subnet mask:
255.255.255.0
node A
node A
IP address: 193.0.0.10 IP address: 193.0.0.11
HF
adapter 1
Adapter ID:
clic1
IP address:
192.0.8.3
subnet mask:
255.255.255.0
lan0 lan0
Adapter ID:
clic0
IP address:
192.0.0.2
subnet mask:
255.255.255.0
HF
adapter 0
node A
node B
HF
adapter 1
Adapter ID:
clic1
IP address:
192.0.8.4
subnet mask:
255.255.255.0
S
Chapter 474
Configuring HyperFabric
Information You Need
Using the configuration information in Figure 4-1, the information you would specify when you run clic_init or SAM on each of the nodes is listed below. This example is not an exact depiction of the prompts produced by clic_init nor the fields in SAM, but merely an example of the flow of information input. In addition, you should not try to use the dummy addresses in your actual configuration.
On node A:
1. How many HyperFabric adapters are installed on the node?
2. Do you want this node to interoperate with nodes running any HyperFabric versions earlier than B.11.00.11 or B.11.11.01?
3. What is the IP address of the first adapter (clic0)? (192.0.0.1)
4. What is the subnet mask of the first adapter? (255.255.255.0) If you do not specify a value for this, a default mask is chosen. You
will most likely just accept the default. However, in this example, we are showing a value for the subnet mask just to illustrate the correlation between the “dummy” information in Figure 4-1 and where that information is specified or generated during clic_init and SAM.
5. What is the IP address of the second adapter (clic1)? (192.0.8.3)
6. What is the subnet mask of the second adapter? (255.255.225.0)
7. Do you want to enable switch management? You cannot enable switch management through SAM (you must use the clic_init command).
As stated earlier, HP recommends that you do not enable switch management. However, if you do enable it, you must provide the information in items 8 through 14.
8. If switch management has been enabled, how many switches will be configured? As stated earlier, HP recommends that you do not enable switch management.
9. What is the IP address of the first switch (sw_clic0)? (193.0.0.20)
10. What is the Ethernet hardware address of the first switch? (0060b0d00257)
11. What is the IP address of the second switch (sw_clic1)? (193.0.0.21)
Chapter 4 75
Configuring HyperFabric
Information You Need
12. What is the Ethernet hardware address of the second switch? (0060b0d00256)
13. What is the Multicast address for the switches to use? (226.10.1.1)
14. What is the IP address for the LAN card on the same subnet as the switches? (193.0.0.10)
(In Figure 4-1, this is the IP address for lan0 on node A.)
On node B:
1. How many HyperFabric adapters are installed on the node?
2. Do you want this node to interoperate with nodes running any HyperFabric versions earlier than B.11.00.11 or B.11.11.01?
3. What is the IP address of the first adapter (clic0)? (192.0.0.2)
4. What is the subnet mask of the first adapter? (255.255.255.0) If you do not specify a value for this, a default mask is chosen. You
will most likely just accept the default. However, in this example, we are showing a value for the subnet mask just to illustrate the correlation between the dummy information in Figure 4-1 and where that information is specified or generated during clic_init and SAM.
5. What is the IP address of the second adapter (clic1)? (192.0.8.4)
6. What is the subnet mask of the second adapter? (255.255.225.0)
7. Do you want to enable switch management? You cannot enable switch management through SAM (you must use the clic_init command).
As stated earlier, HP recommends that you do not enable switch management. However, if you do enable it, you must provide the information in items 8 through 14.
8. If switch management has been enabled, how many switches will be configured? As stated earlier, HP recommends that you do not enable switch management.
9. What is the IP address of the first switch (sw_clic0)? (193.0.0.20)
10. What is the Ethernet hardware address of the first switch? (0060b0d00257)
11. What is the IP address of the second switch (sw_clic1)? (193.0.0.21)
Chapter 476
Configuring HyperFabric
Information You Need
12. What is the Ethernet hardware address of the second switch? (0060b0d00256)
13. What is the Multicast address for the switches to use? (226.10.1.1)
14. What is the IP address for the LAN card on the same subnet as the switches? (193.0.0.11)
(In Figure 4-1, this is the IP address for lan0 on node B.)
Chapter 4 77
Configuring HyperFabric
Performing the Configuration
Performing the Configuration
As explained in “Configuration Overview” on page 69, you must create the global configuration file (/etc/rc.config.d/clic_global_conf)on each node in the fabric. This consists mostly of specifying HyperFabric adapter-related information. (If you are also going to enable switch management — which HP does not recommend doing — you need to specify additional configuration information.)
NOTE Specifying configuration information adds or changes only the addresses
and other information in the global configuration file, based on the information you supply. It does not perform any operations to check the relationships between that information and any physical connections within the fabric.
You need to create the global configuration file in the following situations:
You have just installed the HyperFabric hardware and software on the system.
You want to change the information in the HyperFabric global configuration file (see the preceding note).
IMPORTANT Creating the global configuration file also modifies the
/etc/rc.config.d/netconf file, adding some HyperFabric-related lines that end with the characters #clic. These lines are used by the HyperFabric software — and are not comments — so do not remove them from the file.
You can create the global configuration file by using the clic_init command (described in “Using the clic_init Command” on page 79) or SAM (described in “Using SAM” on page 81). You cannot enable switch management through SAM (you must use the clic_init command).
Chapter 478
Configuring HyperFabric
Performing the Configuration

Using the clic_init Command

Run the clic_init command to create the global configuration file. To view the man page for clic_init see “Viewing man Pages” on
page 127 of this manual. To use clic_init to configure the Transparent Local Failover feature on
HMP, see the section, “Configuring HMP for Transparent Local Failover Support - Using the clic_init command” on page 104.
IMPORTANT If the global configuration file already exists and you are running
clic_init again (to change the file), you have the option of retaining or modifying the existing configuration information, in addition to adding new information pertaining to new hardware.
Also, once you have completed your changes and clic_init ends its processing, you must stop HyperFabric (by running the clic_shutdown command or using SAM) and then start HyperFabric (by running the clic_start command or using SAM). Otherwise, your configuration information changes will not take effect. See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.
If you include /opt/clic/bin in your PATH statement, you can run the command as it is shown below. Otherwise, you must include
/opt/clic/bin as part of the command name (that is, /opt/clic/bin/clic_init).
You must be logged in as root to run this command. The syntax is as follows:
clic_init [-c] [-?]
where
-c specifies that you want to create the global configuration file. You are prompted for the information described in “Information You Need” on page 71. Note that if the global configuration file already exists (for example, when you are adding an adapter to an existing fabric), clic_init prompts you with the existing configuration information. As you are prompted with each piece of information, you can then confirm that you want to keep it or you can change it.
-? displays the online help for clic_init.
Chapter 4 79
Configuring HyperFabric
Performing the Configuration
If you do not specify any of the above parameters, the online help for clic_init is displayed.
After you have entered the information for all the adapters in the node and all of the switches (if any) in the fabric, a summary of the configuration information is displayed.
Once clic_init has finished, you do one of the following things:
If you want to configure HyperFabric with ServiceGuard, complete the configuration described in “Configuring HyperFabric with ServiceGuard” on page 87, then run clic_start or use SAM to start HyperFabric.
If you have just created the global configuration file on the local node for the first time (and you are not configuring ServiceGuard), run clic_start or use SAM to start HyperFabric.
If you have just changed an existing configuration file on the node, run clic_shutdown or use SAM to stop HyperFabric, and then run clic_start or use SAM to start HyperFabric. Until you do those two things, your configuration changes will not take effect.
See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.
Examples of clic_init
Some examples of using the clic_init command are shown below.
Example 1 To create the global configuration file on the local node, issue this
command:
$ clic_init -c
Example 2 To display the online help for the clic_init command, issue this
command:
$ clic_init -?
or this command:
$ clic_init
Chapter 480
Configuring HyperFabric
Performing the Configuration

Using SAM

This section describes how to use SAM to configure HyperFabric. For information on how to use SAM to configure and deconfigure local failover feature on HMP, see “Configuring HMP for Transparent Local Failover Support - Using SAM” on page 102 and “Deconfiguring HMP for Local Failover support - Using SAM” on page 103.
IMPORTANT If the global configuration file already exists, and you are running SAM
again (to change the file), you can keep or modify the existing configuration information, in addition to adding new information pertaining to new hardware.
Also, once you’ve completed your changes and SAM ends its processing, you must stop HyperFabric (by running the clic_shutdown command or using SAM) and then start HyperFabric (by running the clic_start command or using SAM). Otherwise, your configuration information changes will not take effect. See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.
To use SAM to create the global configuration file on an HP 9000 system running HP-UX 11i v2, follow these steps:
Step 1. Start SAM.
Step 2. Select the “Networking and Communications” area.
Step 3. Select “HyperFabric.”
All HyperFabric adapters installed in the system are listed; installed adapters that are not yet configured show Not Configured in the “Status” field.
Step 4. Highlight the adapter you want to configure.
Step 5. Pull down the “Actions” menu and select Configure Adapter.
Step 6. In the “Configure HyperFabric Adapter” screen, specify information for
the following fields:
Internet Address—Required. The IP address of the adapter.
Chapter 4 81
Configuring HyperFabric
Performing the Configuration
Subnet Mask—Optional. The adapter’s subnet mask. If you do not specify this, a default mask is chosen based on the adapter’s IP address.
Interoperability Enabled—Required. Whether you want the adapter to be able to interoperate with adapters that are using; any HP-UX 11.0 HyperFabric versions earlier than B.11.00.11 or any HP-UX 11i v1 HyperFabric versions earlier than B.11.11.01. Note that if you select No, the HyperFabric software on the system will not be backwards compatible with previous releases. This means you must update all of the other systems in the fabric to the version that is running on the system.
Default: No.
Step 7. Select OK (remember, you cannot enable switch management within
SAM).
Step 8. Exit SAM.
Once SAM has finished, you do one of the following things:
If you want to configure HyperFabric with ServiceGuard, complete the configuration described in “Configuring HyperFabric with ServiceGuard” on page 87, then run clic_start or use SAM to start HyperFabric.
If you have just created the global configuration file on the local node for the first time (and you are not configuring ServiceGuard), run clic_start or use SAM to start HyperFabric.
If you have just changed an existing configuration file on the node, run clic_shutdown or use SAM to stop HyperFabric, and then run clic_start or use SAM to start HyperFabric. Until you do those two things, your configuration changes will not take effect.
See “Stopping HyperFabric” on page 128 and “Starting HyperFabric” on page 109 for more information.
Chapter 482
Deconfiguring a HyperFabric Adapter with SAM
Deconfiguring a HyperFabric Adapter with SAM
To use SAM to deconfigure a HyperFabric adapter on an HP 9000 system running HP-UX 11i v2, complete the following steps:
Step 1. Start SAM.
Step 2. Select the “Networking and Communications” area.
Step 3. Select “HyperFabric.”
All HyperFabric adapters installed in the system are listed. Installed adapters that are configured show Configured in the “Status” field, and installed adapters that are not yet configured show Not Configured in the “Status” field. You can deconfigure only an adapter with a status of Configured.
Step 4. Highlight the adapter you want to deconfigure.
Configuring HyperFabric
Step 5. Pull down the “Actions” menu and select Deconfigure Adapter.
Step 6. In the pop-up window, if you want to deconfigure the adapter, select OK to
confirm it.
If you do not want to deconfigure the adapter, select Cancel.
Step 7. If you selected OK, the entry for the adapter is deleted from the
HyperFabric configuration files (/etc/rc.config.d/clic_global_conf and /etc/rc.config.d/netconf).
If you selected Cancel, you remain in the main “HyperFabric Configuration” screen.
Step 8. Exit SAM.
NOTE If you have configured HMP for Transparent Local Failover support and
if you select Deconfigure Adapter, HyperFabric will verify if the selected adapter is configured to be part of any card pair. If yes, the user
Chapter 4 83
Configuring HyperFabric
Deconfiguring a HyperFabric Adapter with SAM
is informed and the card pair entry is removed from the
/etc/rc.config.d/netconf and /etc/rc.config.d/clic_global_conf files.
Chapter 484
Configuring HyperFabric
Configuring the HyperFabric EMS Monitor
Configuring the HyperFabric EMS Monitor
In the HyperFabric version B.11.23.01, the HyperFabric Event Monitoring Service (EMS) monitor allows system administrators to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem.
The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator defines the condition to trigger a notification (usually a change in interface status). Notification can be accomplished with a SNMP trap or by logging into the syslog file with a choice of severity, or by email to a user defined email address.
To configure the HyperFabric EMS monitor, it is necessary to have the EMS HA monitor product installed (Product Number B7609BA). This product is available on the applications CD media.
Use SAM to initiate monitoring of any particular HyperFabric resource. following the procedure outlined below:
1. Start SAM (Use the online help at any time for details)
2. Select “Resource Management”
3. Select “Event Monitoring Service”
4. Select “Action” and “Add Monitoring Request”
5. Select the location /net/interfaces/clic (class for HyperFabric resources)
6. Select a resource instance (either all instances or a specific instance from the list)
7. Validate your choice by clicking on OK at the bottom of the screen
8. A Monitoring Request Parameters window opens, showing the resource and its status (if All instances have been selected, then no value is displayed)
9. Define a condition that will trigger a notification (for instance, “When Value is”, “equal to”, “UP”)
10. Define a polling interval (default is 300 seconds)
11. Define a way of notification: SNMP trap, log in syslog with a choice of severity, or email to a user defined email address
Chapter 4 85
Configuring HyperFabric
Configuring the HyperFabric EMS Monitor
12. Validate by pressing OK
NOTE Although EMS is able to monitor each HyperFabric adapter on every
node in the fabric, as well as the entire HyperFabric subsystem, EMS is not able to monitor HyperFabric switches.
For more detailed information on EMS, including instructions for implementing this feature, see the EMS Hardware Monitors Users Guide Part Number B6191-90028 September 2001 Edition.
Chapter 486
Loading...