This reference guide lists Fabric OS diagnostic and system error messages. The probable cause
and recommended course of action is provided for each message.
the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be liable for
errors contained herein or for incidental or consequential damages in connection with the furnishing, performance,
or use of this material.
This document contains proprietary information, which is protected by copyright. No part of this document may be
photocopied, reproduced, or translated into another language without the prior written consent of Hewlett-Packard.
The information contained in this document is subject to change without notice.
UNIX® is a registered trademark of The Open Group.
Hewlett-Packard Company shall not be liable for technical or editorial errors or omissions contained herein. The
information is provided “as is” without warranty of any kind and is subject to change without notice. The warranties
for Hewlett-Packard Company products are set forth in the express limited warranty statements for such products.
Nothing herein should be construed as constituting an additional warranty.
Printed in the U.S.A.
Diagnostic and System Error Messages Version 3.1.x Reference Guide
First Edition (July 2003)
Part Number: AA-RUPZA-TE
8Diagnostic and System Error Messages Version 3.1.x Reference Guide
about this
guide
This reference guide provides comprehensive information to help you administer,
operate, maintain, and troubleshoot your Storage Area Network (SAN) switch and
your SAN.
The major topics discussed in this chapter are:
■Audience on page 9
■Related Documentation on page 9
■Conventions on page 10
■Getting Help on page 11
Audience
This reference guide is intended for use by systems administrators and technicians
experienced with networking, Fibre Channel, and SAN technologies.
Related Documentation
For a list of related documents included with this product, see the “Related
Documents” section of the Release Notes that came with this product.
About this Guide
About this Guide
For the latest information, documentation, and firmware releases, visit the HP
StorageWorks website:
To access the technical documentation:
— Locate the networked storage section of the Web page.
—Under networked storage, go to the by type subsection.
— Click SAN infrastructure. The SAN infrastructure page displays.
— Locate the fibre channel switches section.
9Diagnostic and System Error Messages Version 3.1.x Reference Guide
About this Guide
— Click the appropriate product name. The product overview page
displays. Go to the product information section.
— Click technical documents.
For information about Fibre Channel standards, visit the Fibre Channel Industry
Association website, located at
Conventions
Conventions consist of Document Conventions and Text Symbols.
Document Conventions
The text conventions used in this document are specified in Tab le 1.
Table 1: Text Conventions
Cross-reference linksBlue text: Figure 1
http://www.fibrechannel.org
ElementConvention
.
Text Symbols
Key and field names, menu items,
buttons, and dialog box titles
File names, application names, and text
emphasis
User input, command and directory
names, and system responses (output
and messages)
Variables<monospace, italic font>
Website addressesBlue, underlined sans serif font text:
Bold
Italics
Monospace font
COMMAND NAMES are uppercase
monospace font unless they are case
sensitive
http://www.hp.com
The following symbols may be found in the text of this guide. They have the
following meanings.
WARNING: Text set off in this manner indicates that failure to follow
directions in the warning could result in bodily harm or death.
10Diagnostic and System Error Messages Version 3.1.x Reference Guide
Caution: Text set off in this manner indicates that failure to follow directions
could result in damage to equipment or data.
Note: Text set off in this manner presents commentary, sidelights, or interesting points
of information.
Getting Help
For answers to questions not covered in this document, contact an HP authorized
service provider or access our website:
HP Technical Support
Telephone numbers for worldwide technical support are listed on the HP website:
http://www .hp.com/support/
About this Guide
http://www.hp.com
.
. From this website, select the country of origin.
Note: For continuous quality improvement, calls may be recorded or monitored.
Be sure to have the following information available before calling:
■Technical support registration number (if applicable)
■Product serial numbers
■Product model names and numbers
■Applicable error messages
■Operating system type and revision level
■Detailed, specific questions
HP Storage Website
The HP website has the latest information on this product, as well as the latest
drivers. Access storage at:
storage .html
http://www .hp.com/country/us/eng/prodserv/
. From this website, select the appropriate product or solution.
Diagnostic and System Error Messages Version 3.1.x Reference Guide
11
About this Guide
HP Authorized Reseller
For the name of your nearest HP authorized reseller:
■In the United States, call 1-800-345-1518
■In Canada, call 1-800-263-5868
■Elsewhere, see the HP website for locations and telephone numbers:
http://www .hp .com
.
12Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error
Messages
This book supports Fabric OS version 3.1.x and contains diagnostic and system
error messages, with recommended responses to the messages. Error messages are
arranged alphabetically, first by module and then by individual message. A typical
module contains multiple error messages. Each error message description consists
of the following:
■The message text
■An explanation of the message or its probable cause
■A recommended user response
■The message’s severity level.
There may be more than one cause and more than one recommended course of
action for any given message. This document provides the most probable cause
and recommends the typically most beneficial response.
This chapter provides an introduction to the error log system and consists of the
following major topics:
1
■Error Message Severity Levels on page 13
■Overview of System Logs on page 15
■View or Configure System Logs on page 17
■Reading a System Error Message on page 19
■Responding to a System Error Message on page 22
Error Message Severity Levels
There are six levels of severity, ranging from 0 (panic) to 5 (debug). The
definitions of the severity levels are quite general and should be used as guidelines
for troubleshooting. In each case, you should thoroughly assess each error log
description before taking a recovery action.
13Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
If you have any questions, collect the applicable data and then contact technical
support for further clarification.
Telephone numbers for worldwide technical support are listed on the HP website:
http://www .hp.com/support/
The severity levels are specified in Tab le 2 .
. From this website, select the country of origin.
14Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
Table 2: Error Message Severity Levels
Severity LevelDescription
0 = PanicPanic-level messages indicate that a specific software subsystem has detected a
fatal or irrecoverable error condition—for example, memory allocation failure,
system call failure, and software detection of misbehaving ASIC or hardware
subsystems. Such errors indicate either partial or complete failure of a subsystem.
A panic frequently results in a reboot of a single-processor switch or a failover of a
StorageWorks Core switch operating in a fully redundant state.
1 = CriticalCritical-level messages indicate serious problems detected by the software that will
eventually cause a partial or complete failure of a subsystem—for example, a
power supply or sensor failure can generate a critical level error message. Some
critical errors may overlap in severity with panic-level errors.
2 = ErrorError-level messages indicate conditions that may not be fatal. These messages are
considered less severe than panic- or critical-level error messages. For example,
error-level messages may indicate timeouts seen on certain operations, failures of
certain operations after retries, invalid parameters, or failure to perform a
requested operation.
3 = WarningWarning-level messages are less severe than error-level messages. These messages
may indicate temporary failures detected by a software module or the detection of
a parameter under monitoring that exceeded a specific threshold value.
4 = InformationalInformational-level messages require no corrective or recovery action. They record
events in the system—for example, the disabling of a port or the clearing of a
switch error log—and provide a description of those events to the user.
5 = DebugDebug-level messages are produced by code inserted to inform HP that a
suspected problem has occurred.
Overview of System Logs
This section describes the System Logs and the types of messages saved in the
System Logs, and explains how to view the information in the log files.
Note: The contents of the Port Logs and instructions for setting up syslogd are discussed
HP StorageWorks HP StorageWorks Fabric OS Procedures Guide
in the
the Panic Trace Logs are intended for support use only.
l
. The contents of
15Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
System Error Log
The Fabric OS maintains an internal System Error Log of all diagnostic and
system error messages. The internal log buffers are limited in size; when the
internal buffers are full, new messages overwrite old messages.
The following are features of the System Error Log:
■Each switch has a System Error Log. Messages are lost over power cycles and
reboots.
■The System Error Log can save a maximum of 1536 messages in RAM. That
is a total of 256 messages for each error message level (panic, error, warning,
informational, and debug).
■The System Error Log is implemented as a circular buffer. When more than
the maximum number of entries are added to the log file, old entries are
overwritten by new ones.
■The errdump and errorshow commands display all the system error
messages.
Syslogd Daemon
Syslogd is a process that runs on UNIX or Linux systems that reads and logs
messages to the system console, log files, other machines, and users specified by
its configuration file. Refer to the manual pages and related documentation for
your particular UNIX host system for more information on the syslogd process
and its capabilities.
The Fabric OS can be configured to use a UNIX-style syslog daemon (syslogd)
process to read system events and error messages and forward these messages to
users and/or write the events to log files on a remote UNIX host system.
The SAN switch can be configured to send error log messages to a UNIX host
system that supports syslogd. This host system can be configured to receive error
and event messages from the switch and store them in files on the computer hard
drive. This enables the storage of switch error log messages on a host system and
overcomes the size limitations of the internal log buffers on the SAN switch.
The host system can be running UNIX, Linux, or any other operating system, as
long as it supports standard syslogd functionality. The Core switch or 2 Gb SAN
switch do not assume any particular operating system to be running on the host
system. The only requirement is that the host system must support standard
syslogd to receive error log messages from the StorageWorks Core switch or
StorageWorks 2 Gb SAN switch.
16Diagnostic and System Error Messages Version 3.1.x Reference Guide
Port Logs
Introduction to System Error Messages
For information on configuring the syslogd functionality, refer to the HP
StorageWorks Fabric OS Procedures Guide.
The Fabric OS maintains an internal log of all port activity. Each switch or logical
switch maintains a log file for each port. Port Logs are circular log files that can
save up to 8000 entries per logical switch. When the log is full, new log entries
overwrite the oldest entries. Port Logs capture switch-to-device, device-to-switch,
some deviceA-to-deviceB, and control information. Port Logs are not persistent
and are lost over power cycles and reboots.
Use the portlogshow command to display the Port Logs for a particular port.
Refer to the HP StorageWorks Fabric OS Procedures Guide for information on
interpreting the output from the portlogdump command.
Note: The Port Log functionality is completely separate from the System Error Log
functionality. Typically, Port Logs are used to troubleshoot connection of devices.
s
System Console
The system console displays messages through the serial port. If you telnet into
the Ethernet port, you will not receive console messages. The system console
displays both system error messages and panic trace messages. These messages
are mirrored only to the system console and are always saved in one of the system
logs.
View or Configure System Logs
Tabl e 3 lists commands that are used to view or configure the error logs.
Table 3: Commands to View or Configure System Logs
CommandDescription
agtcfgdefaultReset the SNMP recipients to default values.
agtcfgsetConfigure the SNMP recipients.
agtcfgshowDisplay the current configuration of the SNMP recipients.
errclearClear the error log.
17Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
Table 3: Commands to View or Configure System Logs (Continued)
CommandDescription
errdumpDisplay the entire error log with no page breaks.
errshowDisplay the entire error log with page breaks.
memshowDisplay the current memory usage of the switch.
poterrshowDisplay the port error summary.
portflagsshowDisplay the port status bitmaps for all ports in a switch.
portlogclearClear the Port Log. If the Port Log is disabled, this
portlogdisableDisable the Port Log facility.
portlogdumpDisplay the Port Log without page breaks.
portlogdumpportDisplay the Port Log of specified port without page breaks.
portloginshowDisplay port logins.
portlogpdiscSet or clear the debug pdisc_flag.
command enables it.
portlogresetEnable the Port Log facility.
portlogresizeResize the Port Log to the specified number of entries.
portlogshowDisplay the Port Log with page breaks.
portlogshowportDisplay the Port Log of the specified port with page
breaks.
setdbgSet the level of debug messages reported by a particular
module.
seterrlvlSet the level of errors reported by a particular module.
setverboseSet the verbose level of a particular module within the
Fabric OS.
supportshowExecute a list of diagnostic and error display commands.
The output is used by technical support to diagnose and
correct problems with the switch. The output from this
command can be very long.
syslogdipaddAdd an IP address as a recipient of event or error
messages.
syslogdipremoveRemove an IP address as a recipient of event or error
messages.
syslogdipshowView the currently configured IP addresses that are
recipients of event or error messages.
18Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
Reading a System Error Message
This section explains how to read system error messages. Typically, system error
messages are generated by the various modules in the Fabric OS. They are
dumped in the System Error Log and, depending on severity, may be saved to
memory or to flash memory.
Viewing System Error Messages from Advanced Web Tools
To view the System Error Log for a switch from Advanced Web Tools:
1.Launch Advanced Web Tools.
2.Select the desired switch from the Fabric Tree.
The Switch View appears.
3.Select the Switch Events button from the Switch View.
A Switch Events Report appears.
4.View the switch events and messages.
Displaying the Error Log without Page Breaks
To display the Switch Error Log without page breaks:
1.Log in to the switch as the admin user.
2.Enter the errdump command at the command line.
An error log, similar to that shown in Figure 1, displays.
19Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
switch:admin> errdump
Error 04
-------0x76 (fabos): Mar 25 08:26:44 (1)
Switch: 1, Info TRACK-LOGIN, 4, Successful login
Error 03
-------0x576 (fabos): Mar 24 16:01L44 (12)
Switch: 1, info TRACK-CONFIG_CHANGE, 4, Config file change from
task:ZNIPC
Error 02
-------0x2f0 (fabos): Mar 24 15:07:01
Switch: 1, Warning FW-STATUS_SWITCH status changed from
HEALTHY/OK to Marginal/Warning
Error 01
-------0x271 (fabos): mar 24 15:04:06
Switch: 1, Info EM-BOOT, 4, restart reason: Failover
switch:admin>
Figure 1: Error log without page breaks
Displaying the Error Log with Page Breaks
To display the error log with page breaks:
1.Log in to the switch as the admin user.
2.Enter the errclear command to clear the System Error Log.
An error log, similar to that shown in Figure 2, displays.
20Diagnostic and System Error Messages Version 3.1.x Reference Guide
switch:admin> errshow
Error 497
-------0x4a5 (fabos): Oct 03 04:40:14
Switch: 0, Info TRACK-LOGIN, 4, Successful login
.
.
.
Type <CR> to continue, Q<CR> to stop: q
Figure 2: Error log with page breaks
Clearing the Switch Error Log
To clear the error log for a particular switch:
1.Log in to the switch as the admin user.
Introduction to System Error Messages
2.Enter the errclear command to clear the System Error Log.
Sample Error Log Message
Figure 3 shows a typical message from the error log.
Error 1001
-------0x253 (fabos): Nov 03 14:11:53
Switch: 1, Error EM-CP_ERR, 2, CP in slot 5 set to faulty because
of CP ERROR
Figure 3: Sample error log message
The fields in this error message are explained in Table 4.
21Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
Table 4: Error Message Field Description
ExampleVariable NameDescription
Error 1001Error Log buffer
number
Nov 03 14:11:53Date and timeThe date and time the error message occurred.
Switch: 1Switch: <number>The logical switch that was affected (
ErrorSeverity levelThe severity of the message as Panic, Critical,
EM-CP_ERRError module - error
code
2Severity levelThe severity of the error as a number:
CP in slot 5 set
to faulty
because of CP
ERROR
Error descriptionError-specific data, such as the reason for the
Displays a rotating number that describes the
position of this error message in the buffer. This
number is not permanently affiliated with the
error itself and should not be used when
contacting Technical Support.
number
values are 0 and 1).
Error, Warning, Informational, or Debug.
The name of the module that generated the error
This section shows how to respond to system error messages.
Looking Up an Error Message
Chapter 2 lists error messages alphabetically by module. To find a specific
message, either scroll through the list or activate the link directly to the message
from one of the lists in Appendix A.
22Diagnostic and System Error Messages Version 3.1.x Reference Guide
Chapter 2 provides the following information for each message:
■Message text
■Firmware module that generated the error
■Module and code name for the error
■Probable cause
■Recommended action
■Severity level
Gathering Information About the Problem
The following are recommended actions for troubleshooting a system error
message. Note that all of the information collected here should be provided to
Technical Support if you require troubleshooting assistance.
■Run supportshow and pdshow and save the output.
■Document the sequence of events leading up to the problem:
Introduction to System Error Messages
— What actions did you take before the problem occurred?
— Is the problem repeatable?
— What configuration is required to produce the problem?
■Answer the following questions:
— Did a failover occur?
— Was security enabled?
— Was POST enabled?
— Are Serial Port (Console) Logs available?
— Which CP was master?
— What was the last change made?
23Diagnostic and System Error Messages Version 3.1.x Reference Guide
Introduction to System Error Messages
Common Responses
The following are common responses to system error messages:
■Run supportshow and pdshow and provide the output to Technical
Support.
■Gather logs.
■Watch for reoccurrence.
■Reinstall firmware.
■Reboot the machine.
■Revert to previous firmware version.
■Call support.
24Diagnostic and System Error Messages Version 3.1.x Reference Guide
System Error Messages
This chapter contains all of the System Error Messages in alphabetical order. To
find a specific message, you can either scroll through the list or activate the link
directly to the message from one of the lists in Appendix A.
This chapter provides information on the following message types:
AS_ System Error Messages on page 26
Bloom_System Error Messages on page 27
ERRLOG_System Error Messages on page 31
FABRIC_System Error Messages on page 34
FCIU_System Error Messages on page 38
FCP_System Error Messages on page 40
FCPH_System Error Messages on page 41
FLOOD_System Error Messages on page 44
2
FSPF_System Error Messages on page 46
HLO_System Error Messages on page 50
LSDB_System Error Messages on page 53
MCAST_System Error Messages on page 55
MPATH_System Error Messages on page 57
MQ_System Error Messages on page 59
MS_System Error Messages on page 65
NBFSM_System Error Messages on page 73
PS_System Error Messages on page 75
RA PID_System Error Messages on page 77
RCS_System Error Messages on page 78
RTWR_System Error Messages on page 81
25Diagnostic and System Error Messages Version 3.1.x Reference Guide
System Error Messages
SEC_System Error Messages on page 85
SECLIB_System Error Messages on page 91
SLAP_System Error Messages on page 100
SWITCH_System Error Messages on page 103
TRACK_System Error Messages on page 104
TS_System Error Messages on page 109
UCAST_System Error Messages on page 112
UPATH_System Error Messages on page 113
ZONE_System Error Messages on page 114
AS_ System Error Messages
Alias Server provides a multi-casting capability; a single frame can be delivered to
multiple ports. The user defines a group of ports identified by the Alias ID and
delivers a frame to that group using the Alias ID. The Alias Server daemon tracks
the Alias ID.
AS-CTMALLOC
Message
<switch number> Error AS-CTMALLOC, 2, <variable> : ctMalloc for <number of
bytes> bytes failed <variable>
Explanation
Memory allocation failure. Fabric OS error.
Recommended Action
Copy the name of the error (AS-CTMALLOC) and then call Technical Support.
Severity
Error
26Diagnostic and System Error Messages Version 3.1.x Reference Guide
Bloom_System Error Messages
Bloom_System Error Messages
BLOOM is the name of the current ASIC chip. BLOOM error messages come
from the BLOOM ASIC driver for this chip.
BLOOM-BAD_ID
Message
<switch number> Warning BLOOM-BAD_ID, 3, <port number> IU in <message
string> has bad ID (S_ID = <SID number>, D_ID = <DID number>)
Explanation
A bad source ID or destination ID was reported on the specified slot and port
number.
Recommended Action
Run portlogdisable and supportshow (in order) to capture debug
information and then contact Technical Support. Technical Support may also ask
for additional debug information from POST and systemtest.
Severity
Warn ing
BLOOM-BUF_RECLAIMED
Message
<switch number> Info BLOOM-BUF_RECLAIMED, 4, <port number>
Explanation
If the specified port was previously disabled because no buffer was available, the
port is now enabled because a buffer has been made available in the same quad.
27Diagnostic and System Error Messages Version 3.1.x Reference Guide
Two or more bad hardware buffers are reported from the specified quad.
Recommended Action
Run portlogdisable and supportshow (in order) to capture debug
information and then contact Technical Support. Technical Support may also ask
for additional debug information from POST and systemtest.
The specified port was disabled due to lack of available buffers. This usually
happens when one or more ports in the same quad are configured as long distance.
28Diagnostic and System Error Messages Version 3.1.x Reference Guide
Bloom_System Error Messages
Recommended Action
Disable one or more other ports in the same quad to enable the specified slot and
port
Severity
Warn ing
BLOOM-RAMINIT_TO
Message
<switch number> Critical BLOOM-RAMINIT_TO, 1, <port number><port index>
failed to init RAM @ <offset>, busy status = <busy index>
Explanation
RAM initialization cannot be completed within the expected time for the specified
port number.
Recommended Action
Copy the error message information, run portlogdisable and
supportshow (in order) to capture debug information and then contact
Technical Support. Technical Support may also ask for additional debug
information from POST and systemtest.