Brocade Communications Systems CNA 1010, HBA 415, CNA 1020, HBA 425, HBA 804 Troubleshooting Manual

...

Download

53-1001582-01

22 June 2010

Brocade Adapters

Troubleshooting Guide

Supporting CNA models 1010 and 1020 Supporting HBA models 415, 425, 804, 815, and 825

Brocade, the B-wing symbol, BigIron, DCX, Fabric OS, FastIron, IronPoint, IronShield, IronView, IronWare, JetCore, NetIron, SecureIron, ServerIron, StorageX, and TurboIron are registered trademarks, and DCFM, Extraordinary Networks, and SAN Health are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. All other brands, products, or service names are or may be trademarks or service marks of, and are used to identify, products or services of their respective owners.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government.

The authors and Brocade Communications Systems, Inc. shall have no liability or responsibility to any person or entity with respect to any loss, cost, liability, or damages arising from the information contained in this book or the computer programs that accompany it.

The product described by this document may contain “open source” software covered by the GNU General Public License or other open source license agreements. To find-out which open source software is included in Brocade products, view the licensing terms applicable to the open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.

Brocade Communications Systems, Incorporated

Corporate and Latin American Headquarters Brocade Communications Systems, Inc. 1745 Technology Drive San Jose, CA 95110 Tel: 1-408-333-8000 Fax: 1-408-333-8101 E-mail: info@brocade.com

European Headquarters Brocade Communications Switzerland Sàrl Centre Swissair Tour B - 4ème étage 29, Route de l'Aéroport Case Postale 105 CH-1215 Genève 15 Switzerland Tel: +41 22 799 5640 Fax: +41 22 799 5641 E-mail: emea-info@brocade.com

Asia-Pacific Headquarters Brocade Communications Systems China HK, Ltd. No. 1 Guanghua Road Chao Yang District Units 2718 and 2818 Beijing 100020, China Tel: +8610 6588 8888 Fax: +8610 6588 9999 E-mail: china-info@brocade.com

Asia-Pacific Headquarters Brocade Communications Systems Co., Ltd. (Shenzhen WFOE) Citic Plaza No. 233 Tian He Road North Unit 1308 – 13th Floor Guangzhou, China Tel: +8620 3891 2000 Fax: +8620 3891 2111 E-mail: china-info@brocade.com

Document History

Title Publication number Summary of changes Date

Brocade Adapters Troubleshooting Guide 53-1001253-01 New document June 2009

Brocade Adapters Troubleshooting Guide 53-1001253-02 New document September 2009

Brocade Adapters Troubleshooting Guide 53-1001582-01 Updates to support the 804

adapter.

June 2010

Contents

About this Document

In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Supported CNA hardware and software . . . . . . . . . . . . . . . . . . . . . . . x

CNA support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Fabric OS and switch support . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Host operating system support. . . . . . . . . . . . . . . . . . . . . . . . . . . x

Supported HBA hardware and software . . . . . . . . . . . . . . . . . . . . . . xii

HBA support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Fabric OS and switch support . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Host operating system support. . . . . . . . . . . . . . . . . . . . . . . . . . xii

What’s new in this document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Document conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Text formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Command syntax conventions . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Command examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Notes, cautions, and warnings . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Key terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Notice to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Additional information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Brocade resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Other industry resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Providing details for support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xvii

Document feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xx

Chapter 1 Introduction to Troubleshooting

In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

How to use this manual for troubleshooting . . . . . . . . . . . . . . . . . . . . 1

Gathering problem information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Chapter 2 Isolating Problems

In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

How to use this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Brocade Adapters Troubleshooting Guide iii 53-1001582-01

General HBA and CNA problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Adapter not reported under server’s PCI subsystem. . . . . . . . . . 8

No adapters reported though BCU adapter --list command . . . . 9

Port link is not active . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Installer program does not autorun (Windows only) . . . . . . . . .10

Host system freezes or crashes . . . . . . . . . . . . . . . . . . . . . . . . .10

Operating system errors (blue screen). . . . . . . . . . . . . . . . . . . . 11

Driver event messages appearing in host system log files. . . . 12

BCU version mismatch warning . . . . . . . . . . . . . . . . . . . . . . . . .12

Errors or problems when entering BCU commands . . . . . . . . .12

Brocade BCU desktop shortcut missing. . . . . . . . . . . . . . . . . . .13

Installing driver update disk (DUD) results in wrong driver versions 13

I/O data traffic issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Driver installation fails and system cannot be booted . . . . . . . 14

Cannot remove Linux driver with uninstaller application or scripts14

Files needed for bfad.sys message appears . . . . . . . . . . . . . . .15

Cannot roll back driver on all adapter instances using Device

Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Errors when installing brocade_driver_linux_<versions>.tar.gz

package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Operating system crashes after invalid HCM operand . . . . . . . 16

UEFI boot problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

BIOS boot problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

HBA problems only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

Quality of Service performance issues. . . . . . . . . . . . . . . . . . . . 22

Quality of Service not functioning. . . . . . . . . . . . . . . . . . . . . . . . 23

Unable to create more than 126 Virtual (NPIV) ports for HBA .23

Ethernet network interface problems (CNAs only) . . . . . . . . . . . . . . 24

Ethernet loopback test problems . . . . . . . . . . . . . . . . . . . . . . . . 24

Ethernet link ports or LOM not coming up on reboot in Linux .25

Loss of adapter hardware address in Linux. . . . . . . . . . . . . . . .25

Loss of adapter IP address in Linux . . . . . . . . . . . . . . . . . . . . . .25

Network stack runs out of heap . . . . . . . . . . . . . . . . . . . . . . . . . 26

NIC numbering unexpected on VMware systems . . . . . . . . . . .26

Ping to remote host is failing. . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Teaming errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

VLAN creation and operation problems . . . . . . . . . . . . . . . . . . .28

Teaming or VLAN operations through HCM fail . . . . . . . . . . . . .29

Poor network performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

iv Brocade Adapters Troubleshooting Guide

53-1001582-01

FCoE and Fibre Channel problems . . . . . . . . . . . . . . . . . . . . . . . . . .31

Loss of sync and loss of signal errors in port statistics . . . . . . 31

Fabric authentication failures. . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Adapter is not showing in the fabric . . . . . . . . . . . . . . . . . . . . . . 31

Virtual devices not listed in name server. . . . . . . . . . . . . . . . . .32

Adapter not registering with the name server or cannot access

storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

FCoE link is down (CNAs only). . . . . . . . . . . . . . . . . . . . . . . . . . .32

I/O problem on connected FCoE device (CNAs only). . . . . . . . .34

I/Os are not failing over immediately on path failure in MPIO setup 34 Disk I/O requests cause low throughput and high latency on Linux34 Disk I/O requests cause low throughput and high latency on VMware 34

CEE network problems (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . .35

CEE is not enabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

HCM and HCM Agent problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Failed to connect to agent on host... error when using HCM . .35

Unable to completely uninstall HCM. . . . . . . . . . . . . . . . . . . . . .38

Time on HCM screens does not match system time . . . . . . . . .39

Verifying Fibre Channel and CEE links(stand-up adapters) . . . . . . .39

Adapter driver installation verification. . . . . . . . . . . . . . . . . . . . . . . .40

Confirming driver package installation with HCM . . . . . . . . . . . 41

Confirming driver package installation in Windows systems . . 41

Confirming driver package installation in Linux systems . . . . . 42

Confirming driver package installation in Solaris systems . . . .43

Confirming driver package installation in VMware systems . . .43

Troubleshooting mezzanine card problems. . . . . . . . . . . . . . . . . . . .44

Additional references for isolating problems. . . . . . . . . . . . . . . . . . . 44

Chapter 3 Tools for Collecting Data

In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

For detailed information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Data to provide support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48

Data collection using host system commands . . . . . . . . . . . . . . . . . 48

Data collection using BCU commands and HCM . . . . . . . . . . . . . . . 50

Support Save . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50

Collecting adapter data using HCM . . . . . . . . . . . . . . . . . . . . . .54

Collecting adapter data using BCU commands . . . . . . . . . . . . . 54

Data collection using Fabric OS commands (Brocade switches only)55

Adapter event messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58

Host system logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58

HCM logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Logging levels adjustment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60

Brocade Adapters Troubleshooting Guide v 53-1001582-01

Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62

Authentication statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

CEE statistics (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

CEE query (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

Ethernet statistics (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . .65

Ethernet IOC statistics (CNAs only). . . . . . . . . . . . . . . . . . . . . . . 66

FCoE statistics (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67

Fabric statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68

IOC statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69

FCP initiator mode statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Logical port statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Port statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Remote port statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72

Quality of Service statistics (HBAs only) . . . . . . . . . . . . . . . . . . .73

Virtual port statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

VLAN Statistics (CNAs only). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Beaconing(stand-up adapters) . . . . . . . . . . . . . . . . . . . . . . . . . .75

Internal and external loopback tests . . . . . . . . . . . . . . . . . . . . . 76

Ethernet port loopback test (CNAs only) . . . . . . . . . . . . . . . . . .77

PCI loopback test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Memory test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79

Pinging Fibre Channel end points. . . . . . . . . . . . . . . . . . . . . . . . 80

Adapter temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81

Queue test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81

Trace route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82

Echo test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82

SCSI test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83

Test Logs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Collecting BIOS data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84

Displaying BIOS data through BCU . . . . . . . . . . . . . . . . . . . . . . .84

Display BIOS data through HCM . . . . . . . . . . . . . . . . . . . . . . . . .84

Collecting LLDP data (CNAs only). . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Collecting SFP data (stand-up adapters). . . . . . . . . . . . . . . . . . . . . .85

SFP properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85

Predictive optical monitoring (POM) . . . . . . . . . . . . . . . . . . . . . .85

Collecting port data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86

Displaying port properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Displaying CEE port properties (CNAs only) . . . . . . . . . . . . . . . .86

Displaying Ethernet port properties (CNAs only) . . . . . . . . . . . . 87

Displaying FCoE port properties (CNAs only) . . . . . . . . . . . . . . . 87

Displaying FCoE port properties . . . . . . . . . . . . . . . . . . . . . . . . . 88

Displaying remote port properties . . . . . . . . . . . . . . . . . . . . . . .88

Displaying logical port properties . . . . . . . . . . . . . . . . . . . . . . . .88

Displaying virtual port properties . . . . . . . . . . . . . . . . . . . . . . . .88

Displaying the port log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89

Displaying the port list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Performing a port query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89

Displaying port speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89

vi Brocade Adapters Troubleshooting Guide

53-1001582-01

Collecting teaming information (CNAs only) . . . . . . . . . . . . . . . . . . . 90

Displaying team data and statistics through HCM . . . . . . . . . .90

Displaying configured team data through BCU . . . . . . . . . . . . .90

Authentication settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91

Displaying authentication settings through HCM . . . . . . . . . . . 91

Displaying authentication settings through BCU . . . . . . . . . . . .91

QoS settings (HBAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Determining QoS settings through HCM. . . . . . . . . . . . . . . . . . .92

Target rate limiting settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Determining target rate limiting settings through BCU . . . . . . .92

Determining settings through HCM . . . . . . . . . . . . . . . . . . . . . .93

Persistent binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Displaying Persistent Binding settings through BCU . . . . . . . . .93

Displaying Persistent Binding settings through HCM . . . . . . . .94

Chapter 4 Performance Optimization

In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

Tuning storage drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

Linux tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

Solaris tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96

Windows tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96

VMware tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Tuning network drivers (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Windows tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98

Linux tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

VMware tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

Appendix A Adapter BIOS and Event Message Reference

Adapter BIOS messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

Adapter driver event messages . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

Appendix B HCM and Installer Message Reference

Index

Brocade Adapters Troubleshooting Guide vii 53-1001582-01

viii Brocade Adapters Troubleshooting Guide

53-1001582-01

About this Document

In this chapter

•How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

•Supported CNA hardware and software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

•Supported HBA hardware and software . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

•What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

•Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

•Notice to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

•Additional information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

•Providing details for support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

•Document feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

How this document is organized

This manual provides troubleshooting information on Brocade host bus adapters (HBAs) and converged network adapters (CNAs). It is organized to help you find the information that you want as quickly and easily as possible.

The document contains the following components:

• Chapter 1, “Introduction to Troubleshooting” provides an introduction and approach to

troubleshooting adapter problems, as well as tips for gathering problem information. A checklist is also provided to verify that required procedures have been followed during installation.

• Chapter 2, “Isolating Problems” provides information on common adapter problems and

procedures to diagnose and recover from these problems.

• Chapter 3, “Tools for Collecting Data” provides a summary of diagnostic and monitoring tools

available through the Host Connectivity Manager (HCM), Brocade Command Line Utility (BCU), Fabric OS commands, and host system to help you isolate and resolve adapter-related problems.

• Chapter 4, “Performance Optimization” contains guidelines for optimizing adapter

performance on your host system.

• Appendix A, “Adapter BIOS and Event Message Reference” contains details on all event

messages generated by adapter drivers.

Brocade Adapters Troubleshooting Guide ix 53-1001582-01

NOTE

This publication is a companion guide to be used with the Brocade Adapters Administrator’s Guide.

NOTE

That publication provides detailed information on adapter monitoring and diagnostic tools in the HCM and the BCU.

Supported CNA hardware and software

This section describes CNA hardware and software support.

CNA support

The following Fibre Channel over Ethernet (FCoE) CNAs are supported in this release:

Install only Brocade-branded small form factor pluggables (SFPs) in these stand-up CNAs.

• Brocade 1010. Single-por t CNA with a per-port maximum of 10 Gbps.

• Brocade 1020. Dual-port CNA with a per-port maximum of 10 Gbps.

Fabric OS and switch support

Brocade CNAs must connect to Fibre Channel SANs and Ethernet data networks through a compatible FCoE switch. For a current list of compatible switches, refer to the latest compatibility matrices on the Brocade website at www.brocade.com. Select Products & Solutions, and then select the Compatibility link.

Host operating system support

This section describes operating systems support for Brocade Host Connectivity Manager (HCM), Brocade Command Line Utility (BCU), and CNA drivers.

Specific operating system release levels, service pack levels, and other patch requirements are detailed in the current HBA and CNA release notes.

Also refer to the latest Brocade compatibility matrices on the Brocade website at www.brocade.com for a list of supported host systems and operating systems. Select Products & Solutions, and then select the Compatibility link.

x Brocade Adapters Troubleshooting Guide

53-1001582-01

HCM support

NOTE

The following operating systems support HCM:

• Windows 2003 R2 SP2 x86, x64, IA64

• Windows Server 2008 x86, x64, IA64

• Windows Server Core for Windows 2008 x86, x64, IA64

HCM is supported only on the guest operating system on Windows Server Core.

• Linux RHEL4, RHEL5, SLES10, and SLES11

• Solaris 10 (x86, x64, and SPARC)

• VMware ESX Server 3.5 and 4.0x86, x64

HCM is supported only on the guest operating system on VMware.

• Windows Vista (HCM only)

• Windows XP (HCM only)

Specific operating system service pack levels and other patch requirements are detailed in the current HBA and CNA release notes for your adapter software version.

FCoE support

The following lists operating systems that support FCoE for the CNAs:

• Windows 2003 x86, x64, IA64

• Windows Server 2008 x86, x64, IA64

• Microsoft Hyper V for Windows 2008 x86, x64, IA64

• Windows Server Core 2008 x86, x64, IA64

• Microsoft WinPE 2.x for Windows 2008 x86, x64, IA64

• Linux RHEL4, RHEL5, SLES10, and SLES11 x86, x64, IA64

• Solaris 10 (x86, x64, and SPARC)

• VMware ESX Server 3.5 U5 and 4.0 x86, x64

Drivers, BCU, and HCM Agent are supported on the VMware “console” Operating System. HCM is supported only on the guest operating system on VMware.

Specific operating system service pack levels, update levels, and patch requirements are detailed in the current HBA and CNA release notes for your adapter software version.

Brocade Adapters Troubleshooting Guide xi 53-1001582-01

Ethernet support

NOTE

The following lists operating systems that support Ethernet for the CNAs:

• Windows 2003 R2 SP2 x86, x64

• Windows Server 2008 x86, x64

• Microsoft WinPE 2.x for Windows 2008 x86, x64

• Windows Server Core x86, x64

• Linux RHEL4, RHEL5, SLES10, and SLES11 x86, x64

• VMware ESX Server 3.5 and 4.0x86, x64

Network drivers are not supported on IA-64 systems.

Specific operating system service pack levels, update levels, and patch requirements are detailed in the current HBA and CNA release notes for your adapter software version.

Supported HBA hardware and software

This section describes HBA hardware and software support.

HBA support

The following Fibre Channel host bus adapters (HBAs) are supported in this release:

• Brocade 804. Dual-port mezzanine HBA with a per-port maximum of 8 Gbps. This HBA installs

in supported server blades that install in supported blade system enclosures.

• Brocade 815. Single-port stand-up HBA with a per-port maximum of 8 Gbps using an 8 Gbps

SFP+.

• Brocade 825. Dual-port stand-up HBA with a per-port maximum of 8 Gbps using an 8 Gbps

SFP+.

• Brocade 415. Single-port stand-up HBA with a per-port maximum of 4 Gbps using a 4 Gbps

SFP.

• Brocade 425 Dual-port stand-up HBA with a per-port maximum of 4 Gbps using a 4 Gbps SFP.

Install only Brocade-branded small form factor pluggable (SFPs) in stand-up HBAs. Mezzanine HBAs do not have SFPs and external port connectors, but utilize internal ports and connections to switch and I/O modules installed in the blade system enclosure.

Note the following about HBA support

• This publication only supports the HBA models listed above and does not provide information

about the Brocade 410 and 420 Fibre Channel HBAs, also known as the Brocade 400 Fibre Channel HBAs.

• Although you can install an 8 Gbps SFP+ into a Brocade 415 or 425 HBA, only 4 Gbps

maximum port speed is possible.

xii Brocade Adapters Troubleshooting Guide

53-1001582-01

• Install only Brocade-branded SFPs in the HBAs.

Fabric OS and switch support

For a current list of servers, switches, and applications compatible with Brocade adapters, refer to the latest compatibility matrixes on the Brocade website at www.brocade.com. Select Products & Solutions, and then select the Compatibility link.

What’s new in this document

This document adds details on the Brocade 804 adapter. For further information about new features not covered in this document and documentation updates, refer to the release notes for your adapter software version.

Document conventions

This section describes text formatting conventions and important notice formats used in this document.

Text formatting

The narrative-text formatting conventions that are used are as follows:

bold text Identifies command names

Identifies the names of user-manipulated GUI elements Identifies keywords and operands Identifies text to enter at the GUI or CLI

italic text Provides emphasis

Identifies variables Identifies paths and Internet addresses Identifies document titles

code text Identifies CLI output

Identifies command syntax examples

For readability, command names in the narrative portions of this guide are presented in mixed lettercase: for example, switchShow. In actual examples, command lettercase is often all lowercase.

Command syntax conventions

Command syntax in this manual follows these conventions:

command Commands are printed in bold.

--option, option Command options are printed in bold.

-argument, arg Arguments.

Brocade Adapters Troubleshooting Guide xiii 53-1001582-01

NOTE

ATTENTION

CAUTION

DANGER

[ ] Optional element.

variable Variables are printed in italics. In the help pages, values are underlined

enclosed in angled brackets < >.

... Repeat the previous element, for example “member[;member...]”

value Fixed values following arguments are printed in plain font. For example,

--show WWN

| Boolean. Elements are exclusive. Example:

--show -mode egress | ingress

Command examples

This book describes how to perform configuration tasks using the Fabric OS command line interface and the BCU interface, but does not describe the commands in detail. For complete descriptions of all commands, including syntax, operand description, and sample output, see the Fabric OS Command Reference and Brocade Adapters Administrator’s Guide.

Notes, cautions, and warnings

The following notices and statements are used in this manual. They are listed below in order of increasing severity of potential hazards.

A note provides a tip, guidance, or advice, emphasizes important information, or provides a reference to related information.

An Attention statement indicates potential damage to hardware or data.

A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware, firmware, software, or data.

A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you. Safety labels are also attached directly to products to warn of these conditions or situations.

Key terms

For definitions specific to Brocade and Fibre Channel, see the technical glossaries on MyBrocade. See “Brocade resources” on page xv for instructions on accessing MyBrocade.

xiv Brocade Adapters Troubleshooting Guide

53-1001582-01

For definitions of SAN-specific terms, visit the Storage Networking Industry Association online dictionary at:

http://www.snia.org/education/dictionary

Notice to the reader

This document may contain references to the trademarks of the following corporations. These trademarks are the properties of their respective companies and corporations.

These references are made for informational purposes only.

Corporation Referenced Trademarks and Products

Microsoft Corporation Windows, Windows Server 2003, Windows Server 2008, Vista,

Oracle Corporation Solaris

Red Hat Inc. Red Hat Enterprise Linux (RHEL)

Novell, Inc. SUSE Linux Enterprise Server (SLES)

VMware, Inc. ESX Server

SPARC International, Inc. SPARC

Hewlett Packard Corp. BladeSystem

XP, PE for Windows, Hyper V for Windows, Windows Automated Installation Kit (WAIK)

Additional information

This section lists additional Brocade and industry-specific documentation that you might find helpful.

Brocade resources

To get up-to-the-minute information, go to http://my.brocade.com to register at no cost for a user ID and password. A variety of resources for Brocade products is available.

Adapters

For adapter resources, such as product information, software, firmware, and documentation, visit the following websites:

• HBA website at www.brocade.com/hba

• CNA website at www.brocade.com/cna

For additional information on Brocade adapters, refer to the following publications:

• Brocade FCoE CNA Quick Installation Guide (provided with your CNA model).

• Brocade Fibre Channel HBA Quick Installation Guide (provided with your HBA model).

• Brocade Adapters Installation and Reference Manual

Brocade Adapters Troubleshooting Guide xv 53-1001582-01

• Brocade Adapters Administrator’s Guide

• CIM Provider for Brocade Adapters Installation Guide

FCoE Switch

For information on the Brocade FCoE Switch for connecting stand-up CNAs, refer to the following publications:

• Brocade 8000 Hardware Reference Manual

• WebTools Administrator’s Guide

• EZSwitchSetup Administrator’s Guide

• Fabric OS Command Reference Manual

Blade servers and blade system enclosure components

Brocade mezzanine are compatible with blade servers, switch modules, interconnect modules, I/O modules, and other components that install in supported blade system enclosures. For compatibility information, visit the blade server and blade system enclosure manufacturer’s website. Also refer to the Hardware Compatibility section in Chapter 1 of the Brocade Adapters Installation and Reference Manual.

SAN information

White papers, online demonstrations, and data sheets are available through the Brocade website at:

http://www.brocade.com/products-solutions/products/index.page

For additional Brocade documentation, visit the Brocade website:

http://www.brocade.com

Other industry resources

For additional resource information, visit the Technical Committee T11 website. This website provides interface standards for high-performance and mass storage applications for Fibre Channel, storage management, and other applications:

http://www.t11.org

For information about the Fibre Channel industry, visit the Fibre Channel Industry Association website:

http://www.fibrechannel.org

Providing details for support

Contact your Brocade FCoE CNA support supplier for hardware, firmware, and software support, including product repairs and part ordering. Provide the following information:

xvi Brocade Adapters Troubleshooting Guide

53-1001582-01

1. General information:

• Brocade adapter model number

• Host operating system version

• Software name and software version, if applicable

• syslog message logs

• bfa_supportsave output.

To expedite your support call, use the bfa_supportsave feature to collect debug information from the driver, internal libraries, and firmware. You can save valuable information to your local file system and send it to support personnel for further investigation. For details on using this feature, refer to “Support Save” on page 48.

• Detailed description of the problem, including the switch or fabric behavior immediately

following the problem, and specific questions.

• Description of any troubleshooting steps already performed and the results.

2. Adapter serial number:

The adapter serial number and corresponding bar code are provided on the serial number label illustrated below. This label is affixed to the adapter card.

*FT00X0054E9*

FT00X0054E9

You can also display the serial number through the following HCM dialog boxes and BCU commands:

• Adapter Properties tab in HCM.

Select an adapter in the device tree, then click the Properties tab in the right pane.

• BCU adapter --list command.

This command lists all adapters in the system and information such as model and serial numbers.

3. Port World-Wide Name (PWWN).

Determine this through the following resources:

• Label affixed on adapter card provides the WWPN for each port.

• Brocade BIOS Configuration Utility.

Select the appropriate adapter port from the initial configuration utility screen, then select Adapter Settings to display the WWNN and PWWN for the port. For details, refer to the Boot Code chapter in the Brocade Adapters Installation and Reference Manual.

• Port Properties tab in HCM.

Select a port for a specific adapter in the device tree, then click the Properties tab in the right pane.

• The following BCU commands:

Brocade Adapters Troubleshooting Guide xvii 53-1001582-01

NOTE

Command Function

port --query <port_id> Displays port information, including the PWWN

for the FCoE port. The port_id parameter is the port number.

port --list Lists all the physical ports on the adapter along

with their basic attributes, such as the PWWN.

4. Media access control (MAC) addresses (CNAs only)

The CNA card MAC address can be found in HCM by selecting the CNA in the device tree and clicking the Properties tab in the right pane. This displays the CNA Properties panel. Look for the MAC Address field.

The CNA MAC address can be found in HCM by selecting the CNA in the device tree and clicking the Properties tab in the right pane to display the CNA Properties panel. Look for the MAC Address field.

Each port has a “burned-in” local port MAC address. This is the source MAC for LLDP communications between the CNA and FCoE switch. To find this MAC address, select a CEE port in the HCM device tree, then click the Properties tab in the right pane to display the port Properties panel. Look for the Local port MAC field.

The Ethernet MAC address is used for normal Ethernet operations. To find this MAC address using HCM, select an Ethernet port in the HCM device tree, then click the Properties tab in the right pane to display the port Properties panel. Look for the Current MAC address and Factory MAC address fields.

Each enode logging in to the fabric through a local CNA port is assigned a MAC address during FCoE Initialization Protocol (FIP) operations. This MAC is assigned for the current FCoE communication only. To find this MAC address, perform one of the following tasks:

• Select an FCoE port in the HCM device tree, then click the Properties tab in the right

pane to display the port Properties panel. Look for the FCoE MAC field.

• Enter the port --query port_id BCU command. Look for the FCoE MAC.

MAC addresses assigned during FCoE initialization operations cannot be changed using device management applications.

The FCoE Forwarder (FCF) MAC address is the address of the attached FCoE switch. Select an FCoE port in the HCM device tree, then click the Properties tab in the right pane to display the port Properties panel. Look for the FCF MAC field.

You can also determine port MAC addresses using the following BCU commands:

Command Function

port ---query port_id Displays port information, including the MAC

addresses. The <port_id> parameter is the port number.

port ---list Lists all the physical ports on the adapter along

with their Ethernet and FCoE MAC addresses.

xviii Brocade Adapters Troubleshooting Guide

53-1001582-01

NOTE

For details on using HCM and BCU commands, refer to the Brocade Adapters Administrator’s Guide.

Document feedback

Quality is our first concern at Brocade and we have made every effort to ensure the accuracy and completeness of this document. However, if you find an error or an omission, or you think that a topic needs further development, we want to hear from you. Forward your feedback to:

documentation@brocade.com

Provide the title and version number of the document and as much detail as possible about your comment, including the topic heading and page number and your suggestions for improvement.

Brocade Adapters Troubleshooting Guide xix 53-1001582-01

xx Brocade Adapters Troubleshooting Guide

53-1001582-01

Chapter

Introduction to Troubleshooting

In this chapter

•How to use this manual for troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . 1

•Gathering problem information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

How to use this manual for troubleshooting

An adapter, such as an HBA or CNA, is one component of a larger network consisting of switches, storage devices, host systems and the cabling and connections to these components. Although there may be a problem in the adapter or an adapter component, the problem could also originate in another network component or connections between these components. Before removing and replacing adapters, launching adapter diagnostics, or even gathering statistics on adapter operation, it is important that you perform the following tasks:

1. Fully describe the problem and gather complete information about the symptoms that suggest a problem exists. Refer to “Gathering problem information” on page 2.

2. Isolate or resolve the problem by first using information in Chapter 2, “Isolating Problems”.

Adapter problems are organized under the following categories:

• “General HBA and CNA problems”

• “HBA problems only”

• “Ethernet network interface problems (CNAs only)”

• “FCoE and Fibre Channel problems”

• “CEE network problems (CNAs only)”

Search through the list of problems in Tabl e 2 on page 5. Problems are organized in the table by problem title, category, and whether the problem is operating-system-specific. Click on a problem to go to the chapter section containing details of possible causes and actions for resolution.

Each problem section in Chapter 2 provides a complete description of the problem, possible causes, and actions for resolution. Fixes and actions may reference BCU commands, HCM features, and host operating system commands. These are described in Chapter 3, “Tools for

Collecting Data” which you can use to further isolate or resolve the problem.

Other helpful sections in Chapter 2 include the following:

• “Verifying Fibre Channel and CEE links (stand-up adapters)” on page 38.

• “Adapter driver installation verification” on page 39.

• “Troubleshooting mezzanine card problems” on page 43.

• “Additional references for isolating problems” on page 43

Brocade Adapters Troubleshooting Guide 1 53-1001582-01

Gathering problem information

NOTE

3. Use the BCU commands, HCM features, and host operating system commands described in

4. Consider these factors when isolating and resolving the problem:

Chapter 3, “Tools for Collecting Data” to gather data for resolving problems.These tools include

event logs, operating statistics, and diagnostics.

• Can the issue be resolved using the latest supported combination of host system BIOS,

operating system, operating system updates, or adapter drivers?

Refer to “Software installation and driver packages” and “Operating system support” in Chapter 1 of the Adapters Installation and Reference Manual for details on driver packages and operating system support. Also refer to the latest release notes on

www.brocade.com/hba or www.brocade.com/cna.

• Does the issue persist when the adapter is installed in a different platform or blade server

or connected to a different switch port?

• Does the problem persist if using a different SFP or cable?

• Can this problem be reproduced on one or more adapters, ports, or host system? Can you

identify specific steps that consistently reproduce this problem on one or more hosts?

• Is the problem documented in release notes for the adapter, operating system, or host

system BIOS?

• Is the problem documented in release notes for the switch and target storage system?

• Is unexpected behavior intermittent or always present?

If the problem is in a Fibre Channel or FCoE switch, storage device, or in connectivity between these components, refer to the documentation, help systems, or service providers of that equipment.

5. If you cannot resolve the problem, gather and provide problem information to your adapter support provider for resolution.

If troubleshooting information in this manual does not resolve problems, check the installed version of the adapter (chip revision) and driver (fw version) using the BCU adapter --query command. To use this command, refer to “Collecting adapter data using BCU commands” on page 52. Also refer to release notes posted on the Brocade adapter website for known problems relating to the adapter and driver versions. The HBA website is www.brocade.com/hba. The CNA website is

www.brocade.com/cna.

Gathering problem information

Perform the following tasks to obtain as much information as possible before contacting technical support. Be sure to take careful notes for use as a record and reference.

• Describe the symptoms that you are observing. Be specific. Here are some examples:

- User experiences, such as slow performance or file access.

- Expected storage devices not visible from the HCM or host system’s storage management

application.

- Adapter not recognized by host system BIOS.

- Adapter not recognized as PCI device by host system operating system.

2 Brocade Adapters Troubleshooting Guide

53-1001582-01

Gathering problem information

- LEDs not functioning on an adapter port that is connected to the fabric (stand-up

adapters).

- All LEDs on adapter port flashing amber.

• What happened prior to the observed symptoms?

• Describe all observed behavior that is unexpected and compare against expected behavior.

• Gather information for support:

- Use appropriate tools on storage targets to gather information such as disk, tape, and

controller model and firmware levels.

- Run the bfa_supportsave BCU command on the host system and save output to a file on

your system.

This command captures all driver, internal libraries, firmware, and other information needed to diagnose suspected system issues. You can save captured information to the local file system and send it to support personnel for further investigation.

- Run the Fabric OS supportSave command on any Brocade switch and save output. This

command collects RASLOG, TRACE, supportShow, core file, FFDC data and other support information.

For details on using the Support Save feature, refer to “Support Save” on page 48.

• Draw a topology map of the SAN from the adapters to the storage targets. Include the

components described in Table 1.

TABLE 1 Topology map details

Component How to identify

adapter Model, World-Wide Name (WWN),

and driver release level.

Fibre Channel switches Model, WWN, and Fabric OS

version.

Fiber optic links between adapter, switches, and storage ports

Host hardware Model and hardware revision.

The bfa_supportsave and FOS supportsave commands can provide current information for the topology map. Also, consider using the Brocade SAN Health products to provide information on your SAN environment, including an inventory of devices, switches, firmware versions, and SAN fabrics, historical performance data, zoning and switch configurations, and other data. Click the Services & Support tab on www.brocade.com for more information on these products.

Port WWNs connected to all links.

• Run appropriate diagnostic tools for storage targets.

• Determine what has changed in the SAN. For example, if the SAN functioned without problems

before installing the adapter, then the problem is most likely in the adapter installation or configuration, adapter hardware, or adapter driver package. Other examples of things to investigate might be changes in the connected switch or storage system firmware, or an offline switch.

For stand-up adapters, investigate disconnected or faulty cables between the adapter, switch, or storage controller fiber optic ports. Check if target storage devices are correctly connected to the switch and are turned on.

Brocade Adapters Troubleshooting Guide 3 53-1001582-01

Gathering problem information

• Record the time and frequency of symptoms and the period of time symptoms have been

• Determine if unexpected behavior is intermittent or always present.

• List steps that have been taken to troubleshoot the problem, including changes attempted to

For mezzanine adapters, make sure that the adapter, the blade server where the adapter is installed, and the modules in the blade system enclosure that support adapter operation are compatible. Verify that the blade server and modules that support adapter operation are installed in the appropriate enclosure bays. Also check whether target storage devices are connected to the appropriate switch, interconnect module, or I/O module in the blade system enclosure and are turned on.

observed.

isolate the problem.

4 Brocade Adapters Troubleshooting Guide

53-1001582-01

Chapter

Isolating Problems

In this chapter

•How to use this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

•General HBA and CNA problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

•HBA problems only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

•Ethernet network interface problems (CNAs only) . . . . . . . . . . . . . . . . . . . . 23

•FCoE and Fibre Channel problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

•CEE network problems (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

•HCM and HCM Agent problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

•Verifying Fibre Channel and CEE links (stand-up adapters) . . . . . . . . . . . . . 38

•Adapter driver installation verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

•Troubleshooting mezzanine card problems. . . . . . . . . . . . . . . . . . . . . . . . . . 43

•Additional references for isolating problems . . . . . . . . . . . . . . . . . . . . . . . . . 43

How to use this chapter

Operation problems are arranged in this chapter in these categories:

• “General HBA and CNA problems”

• “HBA problems only”

• “Ethernet network interface problems (CNAs only)”

• “FCoE and Fibre Channel problems”

• “CEE network problems (CNAs only)”

Use Tabl e 2 to quickly navigate to sections in this chapter that cover specific adapter problems. Each problem section in this chapter contains a description of the problem, possible causes, and actions for resolution. Click on the problem to link to the appropriate problem section.

TABLE 2 Isolate adapter problems

Problem Category OS Specific

“Adapter not reported under server’s PCI subsystem” “General HBA and CNA problems” All

“No adapters reported though BCU adapter --list command”

“Port link is not active” “General HBA and CNA problems” All

“Host system freezes or crashes” “General HBA and CNA problems” All

“Operating system errors (blue screen)” “General HBA and CNA problems” All

“General HBA and CNA problems” All

Brocade Adapters Troubleshooting Guide 5 53-1001582-01

How to use this chapter

TABLE 2 Isolate adapter problems (continued)

Problem Category OS Specific

“Driver event messages appearing in host system log files”

“BCU version mismatch warning” “General HBA and CNA problems” All

“I/O data traffic issues” “General HBA and CNA problems” All

“Driver installation fails and system cannot be booted” “General HBA and CNA problems” Windows

“Installer program does not autorun (Windows only)” “General HBA and CNA problems” Windows

“Files needed for bfad.sys message appears” “General HBA and CNA problems” Windows

“Cannot roll back driver on all adapter instances using Device Manager”

“Errors when installing brocade_driver_linux_<versions>.tar.gz package”

“Operating system crashes after invalid HCM operand” “General HBA and CNA problems” Linux

“System will not boot over SAN or on local disk in Legacy BIOS mode”

“Host not booting from remote LUN” “General HBA and CNA problems”

“Boot devices not available in host’s Boot Manager menu” “General HBA and CNA problems”

“Driver and operating system installation failure on boot LUN”

“Target not visible from host” “General HBA and CNA problems”

“Ctlr-B option does not display when booting host” “General HBA and CNA problems”

“No target devices found or link down message displays in Brocade BIOS Configuration menu”

“Unable to boot from the stored boot device settings in the adapter”

“Remote LUNs are not visible to the host” “General HBA and CNA problems”

“Cannot boot over SAN from installed CNAs” “General HBA and CNA problems”

“Boot from SAN may stop on some Hewlett Packard hosts” “General HBA and CNA problems”

“Adapter <port id>: BIOS not installed displays during boot process”

“Quality of Service performance issues” “HBA problems only” All

“Quality of Service not functioning” “HBA problems only” All

“General HBA and CNA problems” All

2008

“General HBA and CNA problems” Windows

“General HBA and CNA problems” Linux

“General HBA and CNA problems” “UEFI boot problems”

“UEFI boot problems”

“General HBA and CNA problems” “UEFI boot problems”

“General HBA and CNA problems” “BIOS boot problems”

“BIOS boot problems”

“General HBA and CNA problems” “BIOS boot problems”

“BIOS boot problems”

“General HBA and CNA problems” “BIOS boot problems”

All

Windows

6 Brocade Adapters Troubleshooting Guide

53-1001582-01

How to use this chapter

TABLE 2 Isolate adapter problems (continued)

Problem Category OS Specific

“Unable to create more than 126 Virtual (NPIV) ports for HBA”

“Ethernet loopback test problems” “Ethernet network interface

“Ethernet link ports or LOM not coming up on reboot in Linux”

“Loss of adapter hardware address in Linux” “Ethernet network interface

“Loss of adapter IP address in Linux” “Ethernet network interface

“Network stack runs out of heap” “Ethernet network interface

“NIC numbering unexpected on VMware systems” “Ethernet network interface

“Poor network performance” “Ethernet network interface

“VLAN creation and operation problems” “Ethernet network interface

“Teaming or VLAN operations through HCM fail” “Ethernet network interface

“Ping to remote host is failing” “Ethernet network interface

“Teaming errors” “Ethernet network interface

“Loss of sync and loss of signal errors in port statistics” “FCoE and Fibre Channel

“Fabric authentication failures” “FCoE and Fibre Channel

“Adapter is not showing in the fabric” “FCoE and Fibre Channel

“Virtual devices not listed in name server” “FCoE and Fibre Channel

“Adapter not registering with the name server or cannot access storage”

“FCoE link is down (CNAs only)” “FCoE and Fibre Channel

“I/O problem on connected FCoE device (CNAs only)” “FCoE and Fibre Channel

“I/Os are not failing over immediately on path failure in MPIO setup”

“Disk I/O requests cause low throughput and high latency on Linux”

“HBA problems only” All

problems (CNAs only)”

“Ethernet network interface problems (CNAs only)”

problems (CNAs only)”

problems”

“FCoE and Fibre Channel problems”

problems”

“FCoE and Fibre Channel problems”

All

Linux

VMware

Linux Windows

Windows

Windows 2003

All

Windows

All

Windows Linux VMware

Linux

Brocade Adapters Troubleshooting Guide 7 53-1001582-01

General HBA and CNA problems

TABLE 2 Isolate adapter problems (continued)

Problem Category OS Specific

“Disk I/O requests cause low throughput and high latency on VMware”

“CEE is not enabled” “CEE network problems (CNAs

“Failed to connect to agent on host... error when using HCM”

“Unable to completely uninstall HCM” “HCM and HCM Agent problems” Windows

“Time on HCM screens does not match system time” “HCM and HCM Agent problems” Windows

General HBA and CNA problems

This section provides possible resolutions for common problems that could with installed CNAs or HBAs. For additional information on troubleshooting problems on mezzanine or expansion card adapters, refer to “Troubleshooting mezzanine card problems” on page 43.

Adapter not reported under server’s PCI subsystem

The adapter is installed but not visible as a device in the host system’s PCI subsystem.

Verify whether the adapter is visible as a PCI device by executing your host’s operating system command to list PCI devices in the system. For details on this command, refer to the “List PCI Devices” row in Table 5 on page 46. If the adapter is not in the device list, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

“FCoE and Fibre Channel problems”

only)”

“HCM and HCM Agent problems” All

VMware

All

Server 2003

1. Possible Cause: Card not seated in host system connector properly.

Action: Reseat the adapter.

2. Possible Cause: Host system connector issues.

Action: Install an adapter of known working condition to determine whether there is a connector malfunction.

Action: Try installing the adapter into a different slot, if available.

3. Possible Cause: Adapter not compatible with host operating system or connected storage systems.

Action: Verify compatibility by reviewing the Brocade compatibility matrices on the Brocade website at www.brocade.com. Select Products & Solutions, and then select the Compatibility link.

8 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

No adapters reported though BCU adapter --list command

If the adapter does not display when the BCU adapter --list command is initiated, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

1. Possible Cause: Adapter is not reported under server’s PCI subsystem.

Action: Verify if the adapter is visible as a PCI device by executing your host’s operating system command to list PCI devices in the system. For details on this command, refer to the “List PCI Devices” row in Table 5 on page 46.

Action: If the adapter does not appear in the list of PCI devices, refer to “Boot devices not

available in host’s Boot Manager menu” on page 16 for possible causes and recommended

actions.

2. Possible Cause: Adapter driver is not loaded.

Action: Verify if the adapter is visible as a PCI device by executing your host’s operating system command to list PCI devices in the system. For details on these commands, refer to the “List PCI Devices” row in Table 5 on page 46.

Port link is not active

The link between the adapter and switch port does not appear to be active because of adapter LED operation (stand-up adapters only), lack of data over the link, or the BCU port --query or port --list command shows that the link state is down. Refer to “Displaying the port list” on page 86 for more information.

Refer to the following descriptions of possible causes and recommended actions or fixes for the problems:

1. Possible Cause: SFP or cable problems (stand-up adapters).

Action: Ensure that the SFPs and cables are connected properly on both adapter and switch sides. Check for any cable damage.

2. Possible Cause: Switch port is disabled or switch is disabled.

Action: Execute either the Fabric OS switchShow or portShow commands on the attached switch to ensure that the switch or individual port is not disabled or offline. Use appropriate switch commands to enable the port.

3. Possible Cause: Adapter port is disabled.

Action: Verify port state using the HCM Port Properties dialog box or BCU port --list command. Use BCU port --enable command to enable the port.

4. Possible Cause: Adapter’s port speed or topology mismatch with the switch port (HBAs only).

Action: Check the port topology setting on the switch using the Fabric OS portCfgShow command to ensure that Locked L_Port is OFF. Use the portCfgLport command to change the setting to OFF if required.

Action: Check the switch port speed using the Fabric OS portCfgShow command to verify that speed is either AUTO or matches the speed of the attached adapter port (for example, the speed setting for both ports is 4 Gbps).

Action: Check port speed on the adapter with the BCU port --list or port --query commands to display the current and configured speed. Refer to “Displaying port speed” on page 87 and

“Performing a port query” on page 87 for details on using these commands.

Brocade Adapters Troubleshooting Guide 9 53-1001582-01

General HBA and CNA problems

NOTE

5. Possible Cause: Non-Brocade-branded SFP installed (stand-up adapters). If non-Brocade

6. Possible Cause: Firmware failure. In most cases this causes a heartbeat failure, and if

branded SFPs are inserted on the adapter or switch, the port link will not come up.

Action: On the switch, execute the Fabric OS switchShow command to verify that “Mod_Inv” (invalid module) does not display for the port state.

Action: On the adapter, execute the port --list or port --query BCU commands to verify display in the Sfp field. Refer to “Displaying the port list” on page 86 and “Performing a port query” on page 87. If an unsupported SFP is detected, the Sfp field displays “us” (unsupported SFP) for port --list, and the Media field displays “Unsupported SFP” for port --query.

For additional actions and fixes for the port link not coming up, refer to “Verifying Fibre Channel

and CEE links (stand-up adapters)” on page 38.

auto-recovery is enabled, the driver recovers. No corrective action is needed.

Action: If link does not recover and BCU port --list command shows fcoe and eth state is link down, download the latest driver package from the Brocade adapter website (www.brocade.com/hba or www.brocade.com/cna). Remove and reinstall the driver package using instructions in the “Software Installation” chapter of the Brocade Adapters Installation and Reference Manual.

Installer program does not autorun (Windows only)

If the installer program does not automatically run from the CD that you create with the ISO file

containing all supported software installation packages, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

This ISO file can be downloaded from the CNA and HBA websites (www.brocade.com/hba or

www.brocade.com/cna).

Possible Cause: Autorun is not enabled on your system.

Action: Refer to “Installer program does not autorun (Windows only)” on page 10 for more information to isolate this problem.

Host system freezes or crashes

If the BIOS and the operating system recognize the adapter, but the host system freezes or crashes during startup and does not boot, refer to the following descriptions of possible causes and recommended actions to help resolve the problem

Possible Causes:

• Conflicts with adapter port operating speed or topology of attached devices.

• Adapter not compatible with host system.

• Faulty fiber optic cabling and connections (stand-up adapters only).

• Faulty or unseated SFPs or unsupported SFPs (stand-up adapters only).

10 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

NOTE

Action: Disconnect all devices from the adapter, then reboot the host system.

• If the system does not freeze when rebooted and operates correctly, use the following

information to resolve the problem:

a. Check for faulty cable and cable connections.

b. Try rebooting the system without any connectivity to the switch. This will help isolate any

hang caused by switch and device interactions.

c. Reseat SFPs in the adapter (stand-up adapters only). Determine whether the installed

SFPs are faulty by observing LED operation by the adapter ports. If all LEDs are flashing amber, the SFP is invalid and may not be a required Brocade model. You can also verify SFP operation by replacing them with SFPs in a known operating condition. If the problem is resolved after replacement, the original SFP is faulty.

d. Check for conflicts with attached devices. Verify that data speed (1 to 8 Gbps) and

connection topology (for example, point-to-point) for devices attached through the adapter are compatible with settings on the adapter port. Although auto may be set, configuring settings manually on the adapter port and devices may allow connection. Also, note that the adapter only supports point-to-point connection topology. Refer to the Brocade Adapters Administrator’s Guide for procedures to configure adapter ports.

Observe the LEDs on adapter ports (stand-up adapters only). Illuminated LEDs indicate connection, link activity, and connection speed negotiated with the attached device. For the meaning of LED operation, refer to the Brocade Adapters Installation and Reference Manual.

• If the system freezes perform the following tasks:

a. Verify whether the host system firmware supports PCIe specifications listed in the Brocade

Adapters Installation and Reference Manual. If not, download a firmware update to support the adapter.

b. Verify compatibility by reviewing the Brocade compatibility matrices on the Brocade

website at www.brocade.com. Select Products & Solutions, and then select the Compatibility link.

c. On Windows systems, determine when the system freezes during the boot process. If it

freezes as the driver loads, uninstall and reinstall the driver. If it freezes during hardware recognition, uninstall both the driver and adapter, then reinstall both.

d. Remove the adapter and reboot the system. If the system boots, reinstall the adapter.

e. Reseat the adapter in the host system.

f. Uninstall and reinstall the driver.

g. Try installing the adapter into another host system. If the problem does not occur, the

adapter may not be compatible with the original host system. If the problem occurs in the new system, replace the adapter.

Action: Refer to “Verifying Fibre Channel and CEE links (stand-up adapters)” on page 38 for more information to isolate this problem on stand-up adapters.

Brocade Adapters Troubleshooting Guide 11 53-1001582-01

General HBA and CNA problems

NOTE

Operating system errors (blue screen)

If critical errors display for the host system and the system blue screen appears, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Adapter driver not loaded.

Action: Refer to “Adapter driver installation verification” on page 39 for methods to verify driver installation.

Driver event messages appearing in host system log files

If event messages for the adapter driver are appearing in the host system log files, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Various causes and severity levels.

Action: Follow the recommended action in the message.

Action: Resolve critical-level messages and multiple major or minor-level messages relating to the same issue as soon as possible.

Action: For details on event messages, refer to “Logs” on page 56.

BCU version mismatch warning

Output from BCU commands has the following warning message.

WARNING: BCU and Driver versions don’t match !!!

Possible Cause: Installation may be incomplete. Either the BCU or one or more driver instances were not upgraded to the latest version.

Action: Remove the driver package, then reinstall. Refer to the “Software Installation” chapter in the Brocade Adapters Installation and Reference Manual.

Make sure that the server does not reboot before reinstalling the driver. This will corrupt the installation because, without the driver, the boot LUN cannot be located. To safeguard against a reboot, open an additional remote desktop session to the console or terminal during removal and reinstallation. If a reboot occurs, you will be prompted that others are logged on to the system and asked if you want continue with the reboot.

Possible Cause: In Windows systems, this message may be occur if there are redundant entries in the Temporary and Environmental Path string pointing to a previous installation.

Action: For Windows systems, go to My Computer > Properties > Advanced > Environmental Variables. Check for a multiple entry for a previous installation in the Path variable string. For example, if C:\Program Files (x86)\BROCADE\ADAPTER\driver appears twice in the string, there were two x86 driver installations. You can select Edit and remove one of the instances from the string.

12 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

I/O data traffic issues

I/O data traffic issues are occurring, such as an application is not receiving data, FTP problems on an Ethernet network, data is not reaching a target on a Fibre Channel network, or ping failures.

1. Possible Cause: Ethernet traffic problem (CNAs only).

Action: Run the Ethernet loopback serdes test on the suspected Ethernet port using the BCU ethdiag --loopback <port_id> -t serdes command. This tests internal adapter hardware components. If the test passes, suspect the following external problems:

• Faulty fiber

• Faulty sof tware

• Destination host problem

Action: Run the BCU Ethernet external loopback test using the BCU command ethdiag

--loopback <port_id> -t ext (stand-up adapters only). Be sure that a loopback connector is installed in the port. If the serdes or internal loopback test passes, but the external test fails, suspect the following problems:

• Loopback connector not inserted in transceiver

• Faulty SFP or loopback connector.

2. Possible Cause: Fibre Channel or FCoE I/O problems.

Action: Run the loopback serdes test on the suspected Fibre Channel port (HBAs only) or FCoE port (CNAs only) using the BCU diag --loopback <port_id> -t serdes command. If the test passes, suspect the following external problems:

• Faulty fiber

• Faulty sof tware

• Tar get p robl em

Action: Run the BCU external loopback test (stand-up adapters only) using the BCU command diag --loopback <port_id> -text. Be sure that a loopback connector is installed in the port. If the serdes or internal loopback test passes, but the external test fails, suspect the following problems:

• Loopback connector not inserted in transceiver

• Faulty SFP or loopback connector.

Driver installation fails and system cannot be booted

Installation of a new Windows 2008 driver package fails in systems where an adapter is used for booting over SAN and the operating system becomes unbootable.

Possible Cause: Existing driver package that was accessing the remote boot LUN was removed and system was rebooted.

Action: Install the new adapter driver without uninstalling the existing driver. This is the recommended procedure for updating adapter drivers.

Brocade Adapters Troubleshooting Guide 13 53-1001582-01

General HBA and CNA problems

NOTE

If you remove the adapter driver (which requires a system reboot because the adapter was modified), and then reinstall the driver, installation will fail because the operating system cannot update the driver while a system reboot is pending. However, if you reboot the system after removing the driver, the operating system will not come up because the driver accessing the operating system was removed.

Files needed for bfad.sys message appears

If a “Files needed for bfad.sys” message appears on Windows systems when removing a driver, refer to the following possible cause and recommended action to help resolve the problem.

Possible Cause: A “Files needed for bfad.sys” message will occur if you perform the following sequence of steps.

1. Install the driver using the driver installer program (brocade_installer.bat) or Brocade Adapters

2. Uninstall the Brocade adapter using Windows Device Manager.

3. Re-install the driver using the driver installer program (brocade_installer.bat) or Brocade

Software Installer (GUI or command-based application).

Adapters Software Installer (GUI or command-based application).

4. Uninstall the driver using the driver installer program (brocade_installer.bat) program.

Action: To avoid the message when removing a driver, do not uninstall the driver using the Device Manager if you have used Brocade installer driver installer programs to install driver instances. Always use the Brocade installer programs. If only one driver is present in the system, then the Brocade programs also remove the Fibre Channel devices from the Device Manager.

Cannot roll back driver on all adapter instances using Device Manager

If you cannot roll back the driver for all adapter instances using Windows Device Manager, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Installing the driver using the Brocade driver installer program (brocade_installer.bat) or Brocade Adapters Software Installer (GUI or command-based application), then rolling back driver adapter instances using the Device Manager.

Actions: Perform the following actions:

• Install the driver for each adapter instances using the Device Manager, then roll back the driver

using Device Manager.

• Use the driver installer program (brocade_installer.bat) or Brocade Software Installer (GUI or

command-based application) to install or upgrade the driver, then use the Brocade Software Uninstaller to roll back drivers on all adapter instances in one step.

14 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

Errors when installing brocade_driver_linux_<versions>.tar.gz package

If errors occur when installing the no architecture (noarch) brocade_driver_linux_<versions>.tar.gz driver package, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Appropriate distribution kernel development packages are not installed on your host system for the currently running kernel.

Action: If you are installing the brocade_driver_linux_<versions>.tar.gz driver package, the driver module compiles on the system during installation. If driver build errors result when you install the package, verify that the appropriate distribution kernel development packages are installed on your host system for the currently running kernel. These packages should include the gcc compiler and the kernel sources. If these are not installed, you may need to reinstall the operating system before continuing installation. Be sure to “install everything” including the developer library options.

Operating system crashes after invalid HCM operand

The Linux RHEL 4 update 7 system (kernel 2.6.9-78) crashes after an invalid operand from the HCM agent process.

Possible Cause: There may issues with the kernel regarding the system time.

Action: Upgrade the kernel to the latest patch 2.6.9-78.0.5.

UEFI boot problems

This section describes problems that may occur when using the Brocade adapter and unified extensible firmware interface (UEFI) for booting a host system from a remote storage device (boot over SAN). Possible causes and recommended actions to help resolve the problems are provided.

System will not boot over SAN or on local disk in Legacy BIOS mode

On some IBM UEFI based systems, Hard Disk 0 is not present after Legacy Only option in boot manager and system will not boot as local disk or over SAN.

Possible Cause: Hard Disk are not present in the boot manager list.

Action: Follow these steps:

1. Press F1 to enter System Setup during early phase of booting.

2. Select Boot Manager and press Enter.

3. Select Add Boot Option and press Enter.

4. Select Legacy Only and press Enter.

5. Press Esc to return to Boot Manager.

6. Select Change Boot Order and press Enter.

7. Select the existing Boot Order and press Enter.

8. Select Legacy Only and press the + key to promote the boot order to a position above the local device that contains the operating system (typically, this would be above Hard Disk 0). Press Enter.

Brocade Adapters Troubleshooting Guide 15 53-1001582-01

General HBA and CNA problems

9. Confirm that Hard Disk ‘x’ is present in the list. If not, return to step 3 and add Hard Disks to

10. Select Commit Changes and press Enter.

11. Press Esc to return to Boot Manager.

12. Select Reset System and press Enter.

Host not booting from remote LUN

If the host system where the adapter is installed does not boot from the remote boot LUN, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Boot over SAN is not configured correctly.

Action: If booting the host from a remote boot device, verify whether “boot over SAN” configuration is complete and correct. For example, verify the following:

• A zone is created on the attached switch that contains only the PWWN of the storage system

• BIOS or EFI is enabled to support boot over SAN from a specific adapter port.

• BIOS or EFI is configured to boot from a specific LUN.

• The host’s operating system, adapter driver, and other necessary files are installed on the boot

the boot order.

port for the boot LUN and the PWWN of the adapter port.

LUN.

Boot devices not available in host’s Boot Manager menu

Fibre Channel attached boot devices do not appear in the EFI Boot Manager menu or a boot device appears, but it is not functioning. Following are descriptions of possible causes and recommended actions or fixes for the problem.

1. Possible Cause: The adapter is not seated properly in the host system.

Action: Select the EFI Shell from the EFI Boot Menu and use the devices command to determine if EFI has detected the Brocade adapter.

A Brocade adapter will have “Brocade Fibre Channel” in the device name. A CNA will also have “Brocade Ethernet Controller” in a device name.

If the adapter is not listed, perform the following steps:

• Reseat the adapter.

• Replace the adapter with an adapter in known working condition to determine whether

there is a host connector malfunction.

2. Possible Cause: Adapter connector problems in host.

Action: Perform the following steps.

a. Reseat the adapter.

b. Replace the adapter with an adapter in known working condition to determine whether

there is a host connector malfunction.

c. Reinstall the adapter in a different host connector.

16 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

NOTE

3. Possible Cause: The adapter is not compatible with the host operating system or connected storage systems.

Action: Verify compatibility matrices on the Brocade website at www.brocade.com. Select Products & Solutions, and then select the Compatibility link.

4. Possible Cause: No Fibre Channel attached drives are available from the attached Fibre Channel switch.

Action: Check for attached disk devices.

a. Use the devices EFI shell command to display the detected devices.

A Brocade adapter may display as the following.

29 B X - 1 2 8 Brocade Fibre Channel HBA

“29” is the device handle, and will be different in most systems. More than one Brocade adapter may display.

b. Use the EFI shell oh command to display additional information about each Brocade

adapter. This will include any attached Fibre Channel disk devices. For example, you would enter the following for the HBA with device handle 29:

Shell> dh -d 29

The following displays:

29: PciIo ScsiPassThruExt BusSpecificDriverOverride DevPath (..P0A08,300)/Pci(0|0)/Pci(0|0)) ... Managed by : Drv[25] : Brocade Fibre Channel Adapter Bus Driver Drv[26] : SCSI Bus Driver ... Child Controllers : Child[70] : SCSI Disk Device Child[71] : SCSI Disk Device Child[72] : SCSI Disk Device

The SCSI Disk Devices under “Child Controllers” are the LUNs that the Fibre Channel adapter can access.

c. If an expected Fibre Channel attached disk does not appear in the “dh -d” list for a

Brocade adapter, check the cabling, the adapter’s link status LEDs, and the Fibre Channel switch configuration.

The Brocade adapter port may have been disabled with the EFI shell drvcfg command. Use the drvcfg -s shell command to check the enabled status and configuration of the port, including the requested speed. After entering drafted -s, select the appropriate adapter from the Adapter List screen and press Enter to view and modify port properties.

Brocade Adapters Troubleshooting Guide 17 53-1001582-01

General HBA and CNA problems

Driver and operating system installation failure on boot LUN

Installation fails when installing drivers with the driver update disk (DUD) and an installing operating system to a remote LUN being used for booting over SAN.

Possible Cause: The appropriate driver update disk (DUD) may not have been used for the host operating system and platform. For example, if the brocade_adapter_rhel53_x86_64 DUD is used during RHEL 5.3 x86 installation, installation will fail.

Action: Download the appropriate dud for the host operating system from www.brocade.com/hba or

www.brocade.com/cna.

BIOS boot problems

This section describes problems that may occur when using the Brocade adapter and Basic Input/Output System (BIOS) for booting a host system from a remote storage device (boot over SAN). Possible causes and recommended actions that may fix the problems are provided.

Driver and operating system installation failure on boot LUN

When installing drivers using the driver update disk (DUD) and an operating system to a remote LUN being used for booting over SAN, the installation fails.

Action: Verify the DUD for your system using information in Chapter 1 of the Brocade Adapters Installation and Reference Manual. Using the correct DUD, reinstall appropriate drivers and the operating system as necessary.

Target not visible from host

If the storage target configured for containing the boot LUN is not visible from the host, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

1. Possible Cause: No fabric connectivity between adapter and target or target is not online.

Action: Execute the Fabric OS nsAllShow command on the attached switch to verify that the target and the host are online in the fabric and registered in the name server.

2. Possible Cause: The target and the adapter are not on the same zone.

Action: Execute the Fabric OS cfgActvShow command on the attached switch and verify that the host and target are in the same zone (either using domain area members, port area members, or port or node WWNs).

3. Possible Cause: The adapter driver is not loaded.

Action: The adapter driver may not be loaded. Refer to “Adapter driver installation verification” on page 39 for methods to verify driver installation

18 Brocade Adapters Troubleshooting Guide

53-1001582-01

General HBA and CNA problems

4. Possible Cause: There is a problem with the remote port.

Action: Verify that the remote target port (rport) is reporting itself online by comparing rport online and rport offline statistics. Refer to “Remote port statistics” on page 70 for details on displaying these statistics. The rport online counter should be one greater than the rport offline counter. If not, clear the counters and try connecting to the remote port again. Verify the rport online and rport offline statistics again.

Ctlr-B option does not display when booting host

When booting the host, the Ctrl-B option does not display so that you can access the BIOS Configuration menu.

1. Possible Cause: The adapter might not be loaded with the latest adapter boot image (bfa_boot_fw).

Action: Download bfa_boot_fw from the Brocade adapter website (www.brocade.com/hba or

www.brocade.com/cna) and load to option ROM on adapter. For instructions, refer to the “Boot

Code” chapter in the Brocade Adapters Installation and Reference Manual.

2. Possible Cause: Due to memory constraints system BIOS might not be able to execute the Brocade adapter option ROM.

Action: Disable option ROM on several add-in cards installed in system.

No target devices found or link down message displays in Brocade BIOS Configuration menu

“No target devices found or link down” message displays on Brocade BIOS configuration menu during boot device discovery.

1. Possible Cause: There is no fabric connectivity between the Brocade adapter and target, or the target is not online.

Action: Execute the Fabric OS nsAllShow command on the attached switch to verify that the target and the host are online in the fabric and registered in the name server.

2. Possible Cause: The target and the adapter port are not on the same zone.

Action: Execute the Fabric OS cfgActvShow command on the attached switch and verify that the host and target are in the same zone by using either domain area members, port area members, or port or node WWNs.

3. Possible Cause: The link between the adapter port and target is not active yet.

Action: Check that the speeds for the adapter port and the connected switch port match. The best approach is to set both speeds to “auto” or “autonegotiate.”

Unable to boot from the stored boot device settings in the adapter

The host is unable to boot from the boot device settings stored in the adapter. A “No boot LUNs configured” message will display next to the adapter value when booting.

1. Possible Cause: In the Brocade BIOS Configuration Utility, the Boot LUN field In the Adapter Settings screen is set to Auto Discover or First LUN.

Action: Change the Boot LUN setting on the Adapters Settings screen to Flash Values. Refer to the Boot Code chapter in the Brocade Adapters Installation and Reference Manual for details.

Brocade Adapters Troubleshooting Guide 19 53-1001582-01

General HBA and CNA problems

2. Possible Cause: In HCM, the boot option is set to Auto Discovered from Fabric or First Visible

Remote LUNs are not visible to the host

If LUNs from which the host system will boot are not visible from the host system, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

1. Possible Cause: The driver update disk (DUD) used to install the driver and necessary file

2. Possible Cause: Missing or improper storage array LUN mask setting.

LUN in the Boot over SAN dialog box.

Action: Change the boot options to User Configured LUNs in the Boot Over SAN dialog box. Refer to the Boot Code chapter in the Brocade Adapters Installation and Reference Manual for details.

structure on remote LUNs for boot over SAN operation is not correct for the host operating system being installed on the LUN.

Action: Download and install the correct the driver update disk for the OS that is being installed. Download the DUD from the Brocade adapter website (www.brocade.com/hba or

www.brocade.com/cna).

Action: Check LUN mapping and masking using the storage array configuration applications.

3. Possible Cause: Missing or improper storage array LUN mask setting.

Action: Check LUN mapping and masking using storage array configuration tools.

4. Possible Cause: Adapter driver not loaded.

Action: The adapter driver may not be loaded. Refer to “Adapter driver installation verification” on page 39 for methods to verify driver installation.

Cannot boot over SAN from installed CNAs

You cannot boot over SAN from CNAs, but can boot from HBAs installed on the same system.

Possible Cause: A maximum of four boot LUNs are exported to the system. Boot LUNs connected through HBAs take precedence over boot LUNs connected through CNAs. If all four boot LUNs are connected through HBAs, boot over SAN will not occur through installed CNAs.

Action: If you want to boot over SAN through a CNA, disable BIOS on all installed HBAs. To disable BIOS, use BCU commands, the BIOS configuration utility, or HCM. Refer to the “Boot Code” chapter in the Brocade Adapters Installation and Reference Manual for instructions.

Boot from SAN may stop on some Hewlett Packard hosts

The boot process may stop on some Hewlett Packard systems, such as the HP DL180, and the following message displays.

02a2: BMC System Error Log (SEL) Full` /Press F1 to Continue, Press F2 to Setup/

Possible Cause: The System Event Log may become full of erroneous IPMI (intelligent platform management interface) events reported by the system BIOS.

20 Brocade Adapters Troubleshooting Guide

53-1001582-01

HBA problems only

NOTE

Action: Perform the following steps.

1. Boot the server and press F10 when prompted to run BIOS Setup.

2. Select the Advanced menu.

3. Scroll down to IPMI and press Enter.

4. Scroll down to the System Event Log selection and press Enter.

5. At the Clear System Event Log selection, press Enter to toggle between Enable and Disable.

6. Select Enable.

7. P re s s F10 to save the changes and exit the BIOS Setup.

Action: Refer to Hewlett Packard (HP) Customer Advisory Document c01199684 on the HP technical support website for detailed information.

Adapter <port id>: BIOS not installed displays during boot process

An “Adapter <port id>: BIOS not installed” message displays when booting from an adapter.

Possible Cause: Either the boot image is not present in the adapter option ROM or initialization of the adapter failed for some reason.

1. Boot image is not present in the adapters.

2. Adapter might have failed to initialize.

3. Link did not come up.

Action: Bring up the host system either using the Brocade live CD or boot from local disk.

Action: Download the latest boot image zip file (brocade_adapter_boot_fw_<version>.zip) from the Brocade adapter website at www.brocade.com/hba or www.brocade.com/cna. Extract the boot image file and install on all installed adapters. Refer to the “Boot Code” chapter in the Brocade Adapters Installation and Reference Manual for instructions.

HBA problems only

This section provides information for resolving problems more specific to HBA function. For additional information on troubleshooting problems on mezzanine card adapters, refer to

“Troubleshooting mezzanine card problems” on page 43.

Quality of Service performance issues

If enabling Quality of Service (QoS) is causing poor performance or QoS does not appear to be functioning, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

QoS is not supported on CNAs.

Brocade Adapters Troubleshooting Guide 21 53-1001582-01

HBA problems only

1. Possible Cause: One or both of the required licenses are not installed on the edge switch connected to the HBA port:

• Adaptive Networking (AN) license.

• Server Application Optimization (SAO) license.

Action: Enter the bcu qos --query command for the adapter port connected to the switch. If the QoS state is “Unknown” instead of “Online” and total BB-Credits is “zero,” licenses may not be installed. Verify if licenses are installed by executing the Fabric OS licenseshow command on the switch. Install licenses as required.

For more information about Fabric OS commands and QoS support, refer to the Fabric OS Administrator’s Guide.

2. Possible Cause: QoS is not enabled on both switch and adapter port.

Action: Verify if QoS is enabled for an adapter port using the qos -query <port_id> BCU command. Verify if it is enabled on the switch using the islShow command.

Action: Verify zones on the switch using the Fabric OS cfgActvShow command.

3. Possible Cause: QoS zones are not created properly on the switch for high, medium, and low priority targets.

Action: Verify that QoS is configured on the switch using instructions in the Fabric OS Administrator’s Guide.

Quality of Service not functioning

Quality of Service (QoS) is enabled on an adapter port connected to a supported edge switch, but it does not appear to be functioning.

Possible Cause: The following licenses must be installed:

- Adaptive Networking (AN) license.

- Server Application Optimization (SAO) license.

Action: To determine if these licenses are installed on the connected switch, execute the Fabric OS licenseshow command on the switch. If licenses are not installed, install them. For more information about Fabric OS commands and QoS support, refer to the Fabric OS Administrator’s Guide.

Unable to create more than 126 Virtual (NPIV) ports for HBA

If you cannot configure more than 126 N-Port ID Virtualization (NPIV) ports (maximum is 255 for Fibre Channel) refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: The maximum NPIV limit for the connected switch has been reached.

Action: Execute the Fabric OS configure command on the attached switch and change the maximum logins per port parameter under the F_Port login parameters menu to increase the maximum NPIV IDs allowed per port.

22 Brocade Adapters Troubleshooting Guide

53-1001582-01

Ethernet network interface problems (CNAs only)

NOTE

Ethernet network interface problems (CNAs only)

Use the following information to isolate problems that are more specific to CNA function.

Switch command examples used in this section are for the Brocade 8000 Switch.

Ethernet loopback test problems

Errors occur during BCU Ethernet loopback tests.

1. Symptom: Loopback test returns “Check link/cable or SFP” error when executed with -t cable option (.

Possible Cause: Loopback cable not inserted in tested port.

Action: Verify that loopback cable is securely inserted in the port that you are testing.

2. Symptom: Loopback test returns a “port not disabled” error.

Possible Cause: Port is enabled.

Action: Disable the port using the port --disable BCU command before running loopback test.

3. Symptom: Loopback test returns a “port not disabled” error. The error.displays even after disabling the port.

Possible Cause: Network load balancing service is enabled. This will cause the adapter to disable and enable, and the previously configured state (port disable) is lost.

Action: Disable network load balancing and retest.

4. Symptom: The loopback test returns “Device busy - Retry operation” or “diag busy.”

Possible Cause: Other users or sessions are running another instance of loopback tests.

Action: Check if instances of this diagnostic are running using ps -ef for Linux and VMware, and Task Manager for Windows systems.

Action: If instances are still running, wait a few minutes, then again check if the tests done using the ps -ef command or Task Manager.

5. Symptom: The loopback test returns “Missing frame check and replace SFP/cable”.

Possible Cause: The loopback cable was pulled during the test and the test is run in external mode (not serdes mode).

Action: Restart the test with the cable connected.

Brocade Adapters Troubleshooting Guide 23 53-1001582-01

Ethernet network interface problems (CNAs only)

Ethernet link ports or LOM not coming up on reboot in Linux

The host system’s LAN on motherboard (LOM) is not coming up or CNA ports are not visible after rebooting Linux host.

1. Possible Cause: A ifcfg-ethX script is not configured to bring up each LOM and CNA during the system boot process.

Action: Make sure that a script is configured for each CNA and LOM once drivers are installed. Scripts are located in the following directories:

• SLES - /etc/sysconfig/network

• RHEL - /etc/sysconfig/network-scripts

2. Possible Cause: NetworkManager is enabled. There are known issues with NetworkManager managing multiple NICs in some Linux distributions.

Action: Disable NetworkManager.

To check if NetworkManager is running, enter either of the following commands:

• chkconfig --list | grep NetworkManager

• nm-tool

To disable NetworkManager for RHEL 5 systems, enter the following commands.

chkconfig NetworkManager off chkconfig NetworkManagerDispatcher off

To disable NetworkManager for SLES systems, perform the following steps.

a. Open YaST.

b. Select the Network Devices Network Card.

c. On the first screen set the Network Setup Method option to Traditional Method with ifup.

Loss of adapter hardware address in Linux

The ifconfig command displays, HW Addr as 00:00:00:00:00:00.

Possible Cause: The CNA failed to initialize.

Action: Disable the Ethernet I/O controller by entering the BCU ethioc --disable command, then enable the I/O controller by entering the BCU ethioc --enable command.

Loss of adapter IP address in Linux

The IP address set in Linux with the ifconfig command disappears when the adapter goes down or host system reboots.

1. Possible Cause: The IP address was set with the ifconfig command and the adapter is enabled in DHCP (Dynamic Hardware Configuration Protocol) mode.

Action: Configure IP address using system GUI-based networking tools.

2. Possible Cause: The IP address is not configured in the ifcfg-ethX script.

Action: Manually configure IP address in ifcfg-ethX script.

24 Brocade Adapters Troubleshooting Guide

53-1001582-01

Ethernet network interface problems (CNAs only)

Network stack runs out of heap

The network stack on VMware systems is running out of heap space.

Possible Cause: Enabling NetQueue and using jumbo frames has caused the network stack to run out of heap with default values set for netPktHeapMaxSize and netPktHeapMinSize. Leaving default values can result in unpredictable behavior.

Action: Perform the following steps.

1. Log in to the VI Client.

2. Click the Configuration tab for the ESX Server host.

3. Click Advanced Settings.

4. Click VMkernel.

5. Find the corresponding value field for VMkernel.Boot.netPktHeapMaxSize, and enter 128.

6. Find the corresponding value field for VMkernel.Boot.netPktHeapMinSize, and enter 32.

7. C l ic k OK to save the changes.

8. Reboot the system.

NIC numbering unexpected on VMware systems

After installing CNA drivers on VMware systems, NIC numbering is not what is normally expected. For example, instead of vmnic32 or vmnic33, number is vmnic2 and vmnic3.

Possible Cause: CNA hardware was installed before drivers.

Action: When installing a CNAs on a VMware systems, it is advisable to install the driver before the CNA cards so that the NICs will be properly enumerated in the system. To resolve the problem, you must perform the following steps.

1. Uninstall the drivers.

2. Remove the CNA card.

3. Reboot your system without the CNA.

4. Install the drivers.

5. Install the card.

6. Reboot the host system.

Ping to remote host is failing

Pings generated between servers are failing or there is no response from ping.

1. Possible Cause: Ethernet interface on either server is in the following states:

• Administratively down. Running the Linux or VMware ifconfig command shows that the UP

flag is not set.

• Administratively up, but link is down. Running the Linux or VMware ifconfig command

shows that the RUNNING flag is not set.

Brocade Adapters Troubleshooting Guide 25 53-1001582-01

Ethernet network interface problems (CNAs only)

Action: To determine link state, run the ifconfig command for Linux or VMware systems. For Windows systems, run ipconfig /all or use Settings > Network Connections.

Action: For the interface to send and receive packets, both the UP and RUNNING flags must be set.

Action: If pinging a server on a different network. make sure that the route to that host network or that the default gateway is correctly configured.

2. Possible Cause: Other link problems.

Action: Refer to “Port link is not active” on page 9.

3. Possible Cause: IP address and network mask of CNA port are set incorrectly.

Action: Verify and set IP address and network mask if necessary:

• Linux - Run the ifconfig command to determine if port has proper IP address and network

mask and to verify that the link is up.

• Windows - Use Device Manager and network connection tools.

4. Possible Cause: Packets are not being received or stack is dropping packets at remote server due to incorrect IP address set on adapter or incorrect MTU size.

Action: Verify if packets arrived at the remote server using the following commands:

• Linux - Run the tcpdump command.

• Windows - Run the Wireshark application.

Action: Verify MTU size on your system and increase size if necessary. Note that MTU size set on adapter must not be more than MTU size set on attached FCoE switch. To set MTU size on the adapter, refer to the “Adapter Configuration” Chapter in the Brocade Adapters Installation and Reference Manual.

Action: Verify and set IP address and network mask if necessary:

• Linux - Run the ifconfig command to determine if port has proper IP address and network

mask and to verify that the link is up.

• Windows - Use Device Manager and network connection tools.

5. Possible Cause: If hosts with Windows operating systems do not reply to broadcast/multicast ICMP ping commands, this is by design. Windows systems will not reply for security reasons.

Action: Refer to Windows KB137421 on http://support.microsoft.com/kb/137421.

Teaming errors

Ports configured as team members are in an error state and output from the BCU team --query command shows mismatching of teaming parameters. Although you can still create teams, the parameters configured for the first port (primary port) added to a team should match parameters of subsequent ports added to the team or mismatching conflict errors result.

In the following example output from the BCU team --query command, a conflict displays for the port associated with local area connection 6 because its PVID does not match the PVID of the primary port (local area connection 3).

2/0 : Local Area Connection 3 (Primary) (Active) (Linkdown) 3/1 : Local Area Connection 6 )Conflict) Conflict Reason: Mismatching PVIDs configured for adapters

26 Brocade Adapters Troubleshooting Guide

53-1001582-01

Ethernet network interface problems (CNAs only)

NOTE

Note that you can still create teams even though these conflicts exist. However to avoid conflicts, make sure the following parameters match for all ports added to a team:

• Header data split

• Receive side scaling

• Offload parameters

• Port VLAN ID (PVID)

• MTU size

• Link speed

For CNAs, link speed is always 10 Gbps. You cannot change this parameter.

You can view and change these parameters for each port using Windows Device Manager.

1. Run devmgmt.msc to open the Device Manager window.

2. Expand Network Adapters.

An instance of the adapter model should display for each installed adapter port.

3. Right-click an the adapter instance that corresponds to the port where you want to change parameters, and select Properties.

4. Select the Advanced tab.

5. Select the property and change the value as appropriate.

For more details, refer to the “Adapter Configuration” appendix in the Brocade Adapters Installation and Reference Manual.

VLAN creation and operation problems

VLAN creation fails with BCU command or HCM or pass-through VLAN stops working after creating with Device Manager. These problems result when VLANs are created using HCM or BCU commands and also using Device Manager. Follow these guidelines to avoid problems:

• If you need to create a single VLAN and VLANs have not been created using BCU commands or

HCM, you can use Device Manager.

• If you want to configure multiple VLANs, disable the port VLAN created in Device Manager (set

to 0 value), then configure VLANs using HCM or BCU. Refer to the Brocade Adapters Administrator’s Guide for instructions.

1. Symptom: When using BCU commands or HCM, to create VLANs, the initial VLAN fails with an error message.

Possible Cause: Port VLAN was created through Device Manager.

Action: Set port VLANID to 0 in Device Manager and create VLANs using BCU commands or HCM.

2. Symptom: Pass-through VLAN stops working.

Possible Cause: Port VLAN was configured through Device Manager.

Action: Set port VLANID to 0 in Device Manager.

Brocade Adapters Troubleshooting Guide 27 53-1001582-01

Ethernet network interface problems (CNAs only)

3. Symptom: Right-clicking on a VLAN device in Device Manager, then selecting Update does not work.

Possible Cause: The upgrade option for Brocade 10 Gig Ethernet service is not available.

Action: Uninstall and install the service.

4. Symptom: No VLAN operation works except “bcu vlan --list.”

Possible Cause: Port VLAN is configured in Device Manager.

Action: Set port VLANID to 0 in Device Manager.

Enabling and disabling port VLAN in Device Manager

Access the port VLAN configuration in Device Manager using the following steps.

1. Open Device Manager.

2. Expand Network Adapters

An instance of the adapter model should display for each installed adapter port.

3. Right-click an adapter instance and select Properties.

4. Select the Advanced tab.

5. Select VlanID.

6. Set VLANID to 0 to disable or enable by setting an ID number.

Teaming or VLAN operations through HCM fail

When creating a team or VLAN through HCM on Windows 2003 systems, the operation fails and an error may result.

Symptoms: There are several symptoms of this problem:

• VLAN or teaming operations fail through HCM or event 7030 occurs with the following text:

The Brocade HCM is marked as an interactive service. However, the system is configured not to allow interactive services. This service may not function properly.

• The HCM agent is not running in the Windows Services panel, but is running in the Task

Manager.

• When creating or deleting a VLAN through HCM, the operation times out after a few minutes

and a “Failed to connect to Agent on local host” or “Connection failure” message displays. Restarting the HCM Agent causes a 1053 error condition, and the agent stops in the Windows Services panel.

• Restarting the HCM Agent after creating a team through HCM results in a 1053 error.

Possible Cause: Interactive Services is disabled.

Action: Enable NoInteractRiveServices in the Windows 2003 registry. Use the following steps.

1. Run the regedit command.

2. Select NoInteractiveServices and set the value to 0.

NoInteractiveServices is located in the following hierarchy in HKEY_LOCAL_MACHINE:

28 Brocade Adapters Troubleshooting Guide

53-1001582-01

Ethernet network interface problems (CNAs only)

SYSTEM\Current\ControlSet\Control\Windows

Poor network performance

Poor network performance apparent for Windows and Linux systems.

1. Symptom: Checksum offloads are disabled.

Action: For Windows, verify if checksum offload parameters are enabled using the Advanced tab on the Network Adapters > Properties dialog box in Device Manager.

Action: For Linux, run the ethtool -k <interface ID> command. If offload parameters are on, information similar to the following displays in the output.

rx-checksumming: on tx-checksumming: on tcp segmentation offload: on

Action: Checksum offloads should be enabled by default. If not, refer to the “Adapter Configuration” appendix in the Brocade Adapters Installation and Reference Manual.

2. Symptom: Dynamic interrupt moderation is disabled.

Action: For Windows, verify if interrupt moderation is enabled using the Advanced tab on the Network Adapters > Properties dialog box in Device Manager.

Action: For Linux, run the ethtool -c <interface ID> command. If interrupt moderation is enabled, information similar to the following displays in the output.

Coalesce parameters for eth2: Adaptive RX: on TX: off

Action: Interrupt moderation should be enabled by default. If not, refer to the “Adapter Configuration” appendix in the Brocade Adapters Installation and Reference Manual.

3. Symptom: Not all eight lanes of PCIe bus are functioning.

Action: For Linux, run the following command:

lspci –vv –d 1657:0014

If eight lanes are detected, information similar to the following should appear in the command output:

Link: Supported Speed unknown, Width x8, ASPM L0s L1, Port 0 Link: Speed 2.5Gb/s, Width x8

Action: If eight lanes are not detected, try rebooting the system. If this does not fix the problem, contact customer support for your adapter.

Brocade Adapters Troubleshooting Guide 29 53-1001582-01

FCoE and Fibre Channel problems

This section provides resolution of problems related to Fibre Channel and FCoE (CNAs only).

Loss of sync and loss of signal errors in port statistics

If the port is having loss of synchronization and signal errors, refer to the following descriptions of possible causes and recommended actions to help resolve the problem. Learn more about displaying port statistics in “Port statistics” on page 69.

Possible Cause: Possible physical link problem.

Action: Check authentication settings on the switch and adapter. For the switch, execute the authutil --show Fabric Fabric OS command. For the adapter, execute the BCU auth --show command (refer to “Authentication settings” on page 88).

Action: Use the BCU auth --show <port> command on the adapter and Fabric OS authutil --show command on the switch.

Action: Check the shared secret configuration on the attached switch and on the adapter. For the switch, execute the secAuthSecret Fabric OS command. For the adapter, execute the auth –-secret BCU command. Refer to “Authentication settings” on page 88 for details on using the auth-secret command.

Fabric authentication failures

If failures in the authentication process between the adapter in host system and the connected switch occur, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Authenticating configuration is incorrect.

Action: Check authentication settings on the switch and adapter. For the switch, execute the authutil --show Fabric OS command. For the adapter, execute the BCU auth --show command (refer to “Authentication settings” on page 88).

Action: Check the shared secret configuration on the attached switch and adapter. For the switch, execute the secAuthSecret Fabric OS command. For the adapter, execute the auth –-secret BCU command. Refer to “Authentication settings” on page 88 for details on using the auth-secret command.

Adapter is not showing in the fabric

If the adapter does not appear as a Fibre Channel device in the fabric, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: There is a problem in the fabric or a protocol issue between the adapter and a fabric.

Action: Check the fabric statistics. Refer to “Fabric statistics” on page 65 for methods to display fabric statistics for the adapter.

• If counts for FLOGI sent and FLOGI accept fabric statistics do not match, suspect fabric

problem or protocol issue between adapter and fabric.

30 Brocade Adapters Troubleshooting Guide

53-1001582-01

FCoE and Fibre Channel problems

• If fabric offline counts increase and fabric maintenance is not occurring, this may indicate a

serious fabric problem. Refer to your switch troubleshooting guide to isolate and resolve the problem.

Virtual devices not listed in name server

If virtual devices are not listed in the fabric’s name server, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Causes:

• Problem in the fabric or a protocol issue between the adapter and fabric.

• NPIV is not supported or is disabled on the switch.

Action: Check virtual port statistics, such as FDISC sent, FDISC accept, and No NPIV support statistics. Refer to “Virtual port statistics” on page 71 for methods to display virtual port statistics.

Adapter not registering with the name server or cannot access storage

If the adapter is not registering with the name server or cannot access storage, refer to the following descriptions of possible causes and recommended actions to help solve the problem.

1. Possible Cause: Adapter cannot log in to the name server.

Action: Display logical port statistics (refer to “Logical port statistics” on page 68 for details on displaying these statistics). Check for increasing name server port login (NS PLOGI) error rejects and unknown name server port login response (NS login unknown rsp). These errors mean that the adapter most likely cannot log in to the name server.

2. Possible Cause: Adapter has a problem registering with the name server.

Action: Display logical port statistics (refer to “Logical port statistics” on page 68 for details on displaying these statistics). Check for increasing errors of the following types. These indicate that the adapter has a problem registering with the name server:

• Name server register symbolic port name identifier (NS RSPN_ID) errors.

• Name server register symbolic port name identifier response (NS RFT_ID rsp) errors.

• Name server register symbolic port name identifier response rejects (NS RFT_ID rejects).

3. Possible Cause: Adapter has a problem querying the name server for available storage.

Action: Display logical port statistics (refer to “Logical port statistics” on page 68 for details on displaying these statistics). Check for increasing name server get all port ID response (NS GID_FT rsp), rejects (NS_GID FT rejects), or unknown responses (NS_GID FT unknown rsp). This indicates that the adapter has a problem querying the name server for available storage.

FCoE link is down (CNAs only)

The FCoE link is down between the CNA and switch.

1. Possible Cause: The FCoE link is not administratively enabled.

Action: Determine if the link is enabled by entering the BCU port --list command. If the port is administratively disabled, the “port state” field will show Disabled.

Action: Enable the port by entering the BCU port --enable <port_id> command.

Brocade Adapters Troubleshooting Guide 31 53-1001582-01

FCoE and Fibre Channel problems

2. Possible Cause: The FCoE license may not be installed on the connected switch.

Action: Execute the Fabric OS licenseshow command on the connected switch to determine if the license is installed. Install license if not installed. For more information about Fabric OS commands and QoS support, refer to the Fabric OS Administrator’s Guide.

3. Possible Cause: The CEE link is not enabled on the CNA.

Action: Verify that the CEE status using the BCU port –list command is displayed as “CEE Linkup”. If “Linkdown” or “Linkup” displays, refer to “CEE is not enabled” on page 34.

4. Possible Cause: The VLAN to which the FCoE switch front-end port belongs is not FCF-capable.

Action: Verify if VLAN on front-end port is FCF-capable using the appropriate Fabric OS command on the attached switch. Refer to the Fabric-OS Command Reference Manual for more information.

Action: Set the VLAN as FCF-capable using the appropriate Fabric OS commands on the attached FCoE switch. Refer to the Fabric-OS Command Reference Manual for more information.

5. Possible Cause: The FC-MAP on the FCoE switch is not set for a VLAN with FCF capability.

Action: Verify if the FC-MAP on the switch is set for a VLAN with FCF capability using appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for more information.

Action: Set the FC-MAP for a VLAN with FCF capability using the appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for more information.

6. Possible Cause: The FCoE Login group is not created on the FCoE switch, not allowing all VF-Ports to be part of the login group.

Action: Verify if FCoE Login group is created on switch using the appropriate Fabric OS command. Refer to the Fabric OS Command Reference Manual for information.

Action: Create an FCoE login group on the switch using the appropriate Fabric OS command.

7. Possible Cause: PFC (priority flow control), CEE Map, and FCoE Map is not configured correctly

on the FCoE switch.

Action: Refer to “CEE is not enabled” on page 34.

I/O problem on connected FCoE device (CNAs only)

There is an I/O problem on a connected FCoE device.

1. Possible Cause: Link between the CNA and switch is down.

Action: Refer to “FCoE link is down (CNAs only)” on page 31.

Action: Refer to “CEE is not enabled” on page 34.

2. Possible Cause: PFC (priority flow control), CEE Map, and FCoE Map is not configured correctly on the FCoE switch.

Action: Verify configuration using the appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for information.

32 Brocade Adapters Troubleshooting Guide

53-1001582-01

FCoE and Fibre Channel problems

Action: Configure PFC using the appropriate Fabric OS command on the attached switch while in switch configuration mode.

3. Possible Cause: Zoning is configured incorrectly on FCoE switch.

Action: Verify zoning configuration on the attached switch using the appropriate Fabric OS command. Refer to the Fabric OS Command Reference Manual for more information.

I/Os are not failing over immediately on path failure in MPIO setup

When multipath I/O (MPIO) is enabled and input/output operations are not failing over immediately when a path failure occurs, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: Improper driver mpiomode setting.

Action: Execute the port --query <port_id> BCU command and ensure fcpim MPIO mode is enabled (which implies zero Path TOV values) or that fcpim MPIO mode is disabled with the expected “Path TOV” settings (default is 30 seconds).

Disk I/O requests cause low throughput and high latency on Linux

If a high number of I/O requests is causing low throughput and high latency on Linux systems, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: The maximum input/output operations per second are too low on Linux hosts.

Action: Refer to “Linux tuning” on page 93 for suggestions to optimize adapter performance in Linux systems.

Disk I/O requests cause low throughput and high latency on VMware

If a high number of I/O requests is causing low throughput and high latency on VMware systems, refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

Possible Cause: The maximum input/output operations per second are too low on VMware hosts.

Action: Refer to “VMware tuning” on page 95 for suggestions to optimize adapter performance in VMware systems.

Brocade Adapters Troubleshooting Guide 33 53-1001582-01

CEE network problems (CNAs only)

This section provides information to resolve problems of CNA operation on the converged enhanced Ethernet (CEE) network.

CEE is not enabled

The CEE state does not show “CEE Linkup” when you run the BCU port --query command.

1. Possible Cause: The link between the CNA port and switch is down.

Action: Run the cee --query command for the port to gain a better understanding of LLDP attributes, CEE maps, and priority tables configured for the port. Also check the error reason code for the CEE link failure. The error reason code will tell you why the CEE is not enabled or active. If the error reason is “Physical Link down,” refer to “Port link is not active” on page 9 and “Verifying Fibre Channel and CEE links (stand-up adapters)” on page 38.

2. Possible Cause: Adapter did not receive CEE configuration or received invalid CEE configuration from the FCoE switch.

Action: Run the BCU cee --query command for the port to gain a better understanding of LLDP attributes and CEE configuration (such as CEE maps and priority tables) configured for the port. Also check for the error reason code for the CEE link failure. The reason code will tell you why the CEE is not enabled or active. Change or fix the CEE configuration at the switch appropriately based on the displayed error reason code.

Action: Check FCoE switch configuration using the appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for information. Change configuration as necessary using appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for information.

3. Possible Cause: The front-end Ethernet port on the FCoE switch is not configured as “switchport” or is not set to converged mode.

Action: Use the appropriate Fabric OS command on the attached switch to display information about the VLAN interface. Refer to the Fabric OS Command Reference Manual for information.

Action: Configure the FCoE port as a “switchport” using the appropriate Fabric OS command on the attached switch. Refer to the Fabric OS Command Reference Manual for information.

Action: Configure the FCoE port to converged mode using the appropriate Fabric OS command on the attached switch.

HCM and HCM Agent problems

This section provides information to help resolve problems with HCM installation and operation.

Failed to connect to agent on host... error when using HCM

An “Adapter failed to connect to agent on host...” message indicates that the client application cannot connect to the HCM agent listening on the configured port (normally TCP port 34568). Refer to the following descriptions of possible causes and recommended actions to help resolve the problem.

34 Brocade Adapters Troubleshooting Guide

53-1001582-01

HCM and HCM Agent problems

NOTE

1. Possible Cause: If the hcmagent process exited in VMware ESX 3.5 or 4.0, the HCM Agent may be configured to forward events to a remote Syslog host, but the outgoing UDP port 514 is blocked by the ESX firewall.

Action: Perform the following steps.

a. Run the following command to open port 514.

esxcfg-firewall -o 514,udp,out,syslog

b. Start the agent using the following command.

hcmservice start

2. Other Possible Causes:

• The HCM Agent is not running.

• The HCM Agent is not accepting connections on the expected port.

• The HCM Agent is not listening on the expected port.

• Communication between the client and agent is blocked by a firewall preventing access to

the port (usually only a consideration for remote HCM management).

Action: For Linux, Solaris, and VMware systems, perform the following steps to help isolate the problem:

a. Verify that the agent is running by executing the appropriate status command for your

operating system as described in the “Software Installation” chapter in the Brocade Adapters Installation and Reference Manual under “HCM Agent Operations.”

b. If you receive a message that the hcmagent is stopped, restarting the agent should

resolve the problem. To restart, use the appropriate start command for your operating system which is also described in the “Software Installation” chapter in the Brocade Adapters Installation and Reference Manual under “HCM Agent Operations.”

c. Note that one command described in the manual restarts the agent, but the agent will not

restart if the system reboots or the agent stops unexpectedly. Another command restarts the agent, but the agent will restart if the system reboots.

d. Confirm the HCM Agent is responding to requests using the expected user password.

Execute the following command to connect to the HCM Agent and force it to collect the adapter driver supportsave data.

This command is a single line. The localhost can be replaced with a different IP address.

wget --no-check-certificate https://admin:password@localhost:34568/JSONRPCServiceApp/ SupportSaveController.do

If successful, the file SupportSaveController.do (actually a zip format file) will contain the data from the HCM Agent.

e. If you are managing a VMware host system through HCM from a remote system, the host’s

firewall may be blocking TCP/IP port 34568, which allows agent communication with HCM.

Use the following command to open port 34568:

/usr/sbin/esxcfg-firewall-o 34568,tcp,out,https

Use Windows Firewall and Advanced Service (WFAS) to open port 34568.

Brocade Adapters Troubleshooting Guide 35 53-1001582-01

HCM and HCM Agent problems

NOTE

You can change the default communication port (34568) for the agent using procedures in the “Software Installation” chapter in the Brocade Adapters Installation and Reference Manual under “HCM Agent Operations.”

f. If HCM is still unable to connect to the HCM Agent after using the preceding steps, collect

the following data and send to your Support representative for analysis:

• Data collected from the previous step in SupportSaveController.do.

• Data from the HCM application SupportSave feature. Select Tools > SupportSave to

generate a supportsave file. The data file name and location displays when the SupportSave feature runs.

• Adapter agent files on the adapter host (where the HCM Agent is installed). Collect

these files using the command tar cvfz hbafiles.tgz /opt/hba. Output collects to hbafiles.tgz.

• Data collected on the adapter host from the bfa_supportsave feature using the

bfa_supportsave command. Output collects to a file and location specified when the SupportSave feature runs.

Action: For Windows systems, perform the following steps on to help isolate the problem:

a. Verify that the agent is running by executing the appropriate status command for your

operating system described in the Brocade Adapters Installation and Reference Manual. Refer to the section on modifying HCM Agent operation.

b. If you receive a message that the hcmagent is stopped, restarting the agent should

resolve the problem. To restart, use the appropriate start command for your operating system which is also described in the Brocade Adapters Installation and Reference Manual.

Note that one command described in the Brocade Adapters Installation and Reference Manual restarts the agent, but the agent will not restart if the system reboots or the agent stops unexpectedly. Another command restarts the agent, but the agent will restart if the system reboots.

a. If the HCM Agent starts, verify which TCP port the agent is listening on by executing the

following command at the Windows command prompt.

netstat -nao | findstr 34568

Output similar to the following should display.

TCP 0.0.0.0:34568 0.0.0.0:0 LISTENING 1960

The value 1960 in the last column is the process identifier for the Windows process listening on the TCP port. Note that this identifier may be different on your system.

b. Enter the following command to confirm that the process identifier bound to TCP port

34568 is for the hcm.exe process.

tasklist /svc | findstr 1960

The following should display if the identifier from step a is bound to TCP port 34568:

hcm.exe 1960 hcm

36 Brocade Adapters Troubleshooting Guide

53-1001582-01

HCM and HCM Agent problems

NOTE

c. If you are managing a Windows 2008 host system through HCM from a remote system, the

host’s firewall may be blocking TCP/IP port 34568.

Use Windows Firewall and Advanced Service (WFAS) to open port 34568.

You can change the default communication port (34568) for the agent using procedures in the Brocade Adapters Installation and Reference Manual. Refer to the section on modifying HCM Agent operation.

d. If the HCM Agent is running and listening on port 34568 and there are no firewall issues

(as explained in step c), but you get the same “Failed to connect to agent on host..." error when using HCM, collect the following data. Send the following data to your support representative for analysis:

• Copies of output from the commands in step a and step b.

• Files from the output directory created after you execute the support save feature. To

collect these files, execute the BCU bfa_supportsave command. Support data is collected to a file in your system’s tmp directory by default. For more information on using the Support Save feature, refer to “Support Save” on page 48.

• Support data from the HCM application SupportSave feature.

• Build information for the HCM application. Select Help > About in HCM to display the

version, build identification, and build date.

To use the HCM SupportSave feature, select Tools > Support Save in HCM to generate a supportsave file. If HCM cannot connect to the agent, a message displays an error (Support Save could not be collected) and explains that only a basic collection is possible. Messages also display that provide the location of the zip file created.

By default, a zip file is created in the following location.

<user home>\HCM\data\localhost\supportsave

The zip file will have a name similar to the following:

SupportSave_Basic_2008723_0_50_57.zip

Unable to completely uninstall HCM

Removing HCM using Windows Add or Remove Programs causes “Unable to completely uninstall application” error.

Possible Cause: HCM was installed on a host system through the HCM Agent on server system using the web browser.

Action: Use one of the following methods to remove HCM from the host system:

Method 1

1. Enter the following at the command prompt:

Javaws -viewer

The Java Cache Viewer screen displays.

2. Select Host Connectivity Manager, right click, and then select Delete from the menu.

Brocade Adapters Troubleshooting Guide 37 53-1001582-01

Verifying Fibre Channel and CEE links (stand-up adapters)

NOTE

Method 2

1. Enter the following at the command prompt:

Javaws -uninstall

This installs all applications in the Java cache.

Time on HCM screens does not match system time

The time display on HCM screens, such as on the main HCM window and statistics dialog boxes, show time relative to GMT instead of the host system time zone. This is a problem on Windows Server 2003 systems.

Possible Cause: Java virtual machine does not get correct time zone information from the windows registry of the local system and defaults to a time zone relative to the GMT. Possibly the Microsoft timezone.exe tool was launched, which may cause Java applications to return incorrect time and date information.

Action: Use the Time Zone Editor (tzedit.exe utility) to change the time zone settings on a single system. Download this tool from the Microsoft download site at

http://www.microsoft.com/downloads.

Follow these recommendations:

• You must have local administrative credentials to change time zone registry information with

the Time Zone Editor.

• The Time Zone Editor does not provide the capabilities to add the Dynamic DSTregistry

subkeys.

• Obtain the list of time zones that have changed, together with their DST start dates and end

dates. Use the information in the following article located at

http://support.microsoft.com/kb/981128

“A hotfix is available to update the Daylight Saving Time for the “(UTC-04:00) Asuncion”, “(UTC+12:00) Fiji” and “(UTC-04:00) Santiago” time zone for Windows Operating Systems.”

Verifying Fibre Channel and CEE links (stand-up adapters)

Check for link problems on stand-up adapters by observing LED operation for adapter ports. LED operation other than expected or LEDs may indicate link problems. For example, all LEDs flashing amber for a port indicates that an invalid non-Brocade SFP may be installed. For details on adapter LED operation, refer to the “LED Operation” section for your adapter (CNA or HBA) in the Brocade Adapters Installation and Reference Manual. If LEDs do not illuminate to indicate an active link, use appropriate Fabric OS and adapter diagnostic commands and HCM options in Table 3 on page 39. For additional diagnostics commands, refer to Chapter 3, “Tools for Collecting Data” for HCM and BCU commands and the Fabric OS Administrator’s Guide for Fabric OS commands.

Also verify LED operation on a switch port that is connected to an adapter port. Refer to the switch hardware reference manual to analyze LED meaning.

Common link problems can be caused by the following:

• Damaged cables. (Note that damaged cables can also cause errors and invalid data on links.)

38 Brocade Adapters Troubleshooting Guide

53-1001582-01

Adapter driver installation verification

• Cables that are not rated or compatible with adapter port speeds. Refer to cable specifications

in the Brocade Adapters Installation and Reference Manual.

• Faulty switch or adapter SFPs. Verify if an SFP is the problem by connecting a different link to

the adapter port or, if convenient, replace the cable with a cable of known quality. If the errors or invalid data on the link still indicate a cable problem, the SFP may be faulty. Try replacing the SFP.

• SFP issues on the adapter or switch. For example, the SFP may not be compatible with the

adapter, but is compatible with the switch, or vice versa. SCSI retries and time-outs determine communication between the adapter and storage. Dropped packets cause time-outs, and packets can drop because of SFP issues. Run the BCU port -stats command to display port statistics, and look for errors and dropped frames.

Tab le 3 lists HCM options and BCU commands, as well as Fabric OS commands that you can use to

determine link status.

TABLE 3 Tools to determine link status

Application Tool References

HCM • Port Statistics

• Loopback and PCI loopback test

• Fibre Channel ping, echo, and trace route

tests

• Link Beaconing

• Port Properties

• SFP information

BCU

• fcdiag and diag commands.

• Port commands, such as port --stats. port

--list, and port --query.

Switch Fabric OS

• switchShow

• portShow

• portStatsShow

• portErrShow

• fcpProbeShow

• fPortTest

Adapter driver installation verification

Problems with adapter operation may be due to improper hardware or software installation, incompatibility between the adapter and your host system, unsupported SFPs installed on the adapter, improper cable connected to the fabric, or the adapter not operating within specifications. Determine if problems may exist because of these factors by reviewing your installation with information in the Brocade Adapters Installation and Reference Manual listed in Table 4 .

Chapter 3, “Tools for Collecting Data”

• Chapter 3, “Tools for Collecting

Data”

• Fabric OS Administrator’s

Guide

• Fabric OS Troubleshooting and

Diagnostics Guide

TABLE 4 Installation and Reference Manual references

Information Chapter

Hardware and software compatibility information. Product Overview

Software installation packages supported by host operating system and platforms.

Brocade Adapters Troubleshooting Guide 39 53-1001582-01

Product Overview

Adapter driver installation verification

NOTE

TABLE 4 Installation and Reference Manual references (continued)

Information Chapter

Hardware and software installation instructions. Installation

Product specifications. Specifications

Adapter driver packages from Brocade contain the current driver, firmware, and HCM Agent for specific operating systems. Make sure that the correct package is installed for your operating system. Refer to the installation chapter in the Brocade Adapters Installation and Reference Manual.

An out-of-date driver may cause the following problems:

• Storage devices and targets not being discovered by the device manager or appearing

incorrectly in the host’s device manager.

• Improper or erratic behavior of HCM (installed driver package may not support HCM version).

• Host operating system not recognizing adapter installation.

• Operating system errors (blue screen).

If a driver is not installed, try re-installing the driver or re-installing the adapter hardware and then the driver.

You can use HCM and tools available through your host’s operating system to obtain information such as driver name, driver version, and adapter PWWNs.

Confirming driver package installation with HCM

Use the following steps to display the adapter PWWN, driver name and version, firmware name and version, and the BIOS version currently in operation.

1. Launch HCM.

2. Select the adapter in the device tree.

3. Select the Properties tab in the right pane to display the Properties dialog box.

The dialog box displays adapter properties.

Confirming driver package installation in Windows systems

Use the Device Manager to determine driver installation. Verify if the driver is installed and Windows is recognizing the adapter using the following steps.

1. Open the Device Manager.

• For HBAs and CNAs, when you expand the list of SCSI and RAID controllers, an instance of

the adapter model should display for adapter port installed. For example, if two two-port adapters (total of four ports) are installed, four instances of the adapter model display.

• For CNAs only, when you expand Network adapters, an instance of Brocade 10G Ethernet

Adapter should also display for each port of the adapter installed. For example, if two two-port adapters (total of four ports) are installed, four instances of the adapter model display.

40 Brocade Adapters Troubleshooting Guide

53-1001582-01

Adapter driver installation verification

NOTE

If instances of your adapter model do not display, but generic instances flagged with yellow question marks do display under Other Devices, the driver is not installed. For example, Fibre Channel Controller may display as a generic instance for an HBA.

2. Right-click the Brocade adapter model where you are installing the driver.

3. Select Properties to display the Properties dialog box.

4. Click the Driver tab to display the driver date and version. Click Driver Details for more information.

If driver is not installed, try re-installing the driver or re-installing the adapter hardware and then the driver.

Confirming driver package installation in Linux systems

Verify if the adapter driver installed successfully using the following commands:

• # rpm -qa|grep -i bfa

This command prints the names of the Brocade adapter storage driver package (bfa) if installed.

• # rpm -qa|grep -i bna

This command prints the names of the Brocade adapter network driver package (bfa) if installed.

• # lspci

This utility displays information about all PCI buses in the system and all devices connected to them. Fibre Channel: Brocade Communications Systems, Inc. displays for an HBA. Fibre Channel: Brocade Communications Systems, Inc. and Ethernet Controller display for a CNA if driver packages have correctly loaded.

• # lsmod

This command displays information about all loaded modules. If bfa appears in the list, the storage driver is loaded to the system. If bna appears in the list, the network driver is loaded to the system.

• # dmesg

This command prints kernel boot messages. Entries for bfa (storage driver) and bna (network driver) should display to indicate driver activity if the hardware and driver are installed successfully.

• These commands display the location of the driver modules if loaded to the system:

- The following command displays the storage driver module location. The module will have

a bfa prefix.

# modprobe -l bfa

- The following command displays the network driver module location. The module will have

a bfa prefix.

# modprobe -l bna

Brocade Adapters Troubleshooting Guide 41 53-1001582-01

Adapter driver installation verification

NOTE

Confirming driver package installation in Solaris systems

Verify if the adapter driver installed successfully using the following commands:

These commands are not applicable for Brocade 804 adapters.

• pkgchk -nv bfa

This checks for and lists the installed adapter storage driver package files.

• pkginfo -l bfa

This displays details about installed Brocade storage (bfa) adapter drivers. Look for information to display like in the following example. Note that the VERSION may be different, depending on the driver version you installed. The ARCH and DESC information may also be different, depending on your host system platform. If the adapter driver package is installed, bfa_pkg should display with a “completely installed” status.

Storage driver (bfa)

PKGINST: bfa NAME: Brocade Fibre Channel Adapter Driver CATEGORY: system ARCH: sparc&i386 VERSION: alpha_bld31_20080502_1205 BASEDIR: / VENDOR: Brocade DESC: 32 bit & 64 bit Device driver for Brocade Fibre Channel adapters PSTAMP: 20080115150824 INSTDATE: May 02 2008 18:22 HOTLINE: Please contact your local service provider STATUS: completely installed

Confirming driver package installation in VMware systems

Verify if the adapter driver installed successfully using the following commands:

• vmkload_mod -l

This lists installed driver names, R/O and R/W addresses, and whether the ID is loaded. For storage drivers, verify that an entry for bfa exists and that the ID loaded. For network drivers, verify that an entry for bna exists and that the ID loaded.

• cat /proc/vmware/version

This displays the latest versions of installed drivers. For storage drivers look for a bfa entry and related build number. For network drivers, look for a bna entry and related build number.

• rpm -qa|grep -i bfa

This command prints the names of the Brocade adapter storage driver package (bfa) if installed.

• rpm -qa|grep -i bna

This command prints the names of the Brocade adapter network driver package (bna) if installed.

42 Brocade Adapters Troubleshooting Guide

53-1001582-01

Troubleshooting mezzanine card problems

• lspci

This utility displays information about all PCI buses in the system and all devices connected to them. Brocade Communications Fibre Channel displays for an HBA. Brocade Communications Fibre Channel and Ethernet Controller display for a CNA if driver packages have correctly loaded.

Troubleshooting mezzanine card problems

As mezzanine adapters are installed in blade servers as part of a blade system enclosure, problems may occur for different reasons than for adapters installed in standard host systems. This section outlines some general methods for isolating causes of problems with card operation. For more information, please see troubleshooting and maintenance information published for your specific blade server, server blade, and blade system enclosure. Also, refer to the support location on the manufacturer’s website for your blade server and blade system enclosure.

If problems exist with adapter operation on a blade server, verify the following:

• The blade server is turned on.

• The adapter is installed in the appropriate connector on the blade server. On some blade

servers, connectors may only support a specific adapter type. Refer to your blade sever documentation for help.

• The blade system enclosure is configured for adapter operation. Refer to your blade system

enclosure and documentation for blade system enclosure components for help.

• The blade server on which the adapter is installed is correctly configured for adapter operation

and is correctly installed in the blade system enclosure. Refer to your blade server and blade system enclosure documentation for help.

• All modules or blades in that support adapter operation are installed in the appropriate

enclosure bays and correctly configured. Refer to the documentation for your blade system enclosure for help.

• You are using the latest device drivers, firmware, and BIOS for the blade server and other

components in the blade system enclosure that support adapter operation.

Additional references for isolating problems

Refer to the following publications and to chapters in this manual for gathering information to further isolate and resolve adapter problems.

• Chapter 3, “Tools for Collecting Data” in this manual

Contains procedures to perform adapter diagnostics, display adapter statistics and display event logs, and collect data for troubleshooting using BCU commands, HCM options, Fabric OS commands, and your host system commands.

• Fabric OS Administrator’s Guide

Provides detailed information on features available on Brocade storage area network (SAN) products, and how to configure and administer these products

• Fabric OS Command Reference Manual.

Provides detailed descriptions of command line interface commands to help system administrators and technicians operate, maintain, and troubleshoot Brocade SAN products.

Brocade Adapters Troubleshooting Guide 43 53-1001582-01

Additional references for isolating problems

• Fabric OS Troubleshooting and Diagnostic Guide.

Provides help with isolating problems in other Brocade SAN components.

• Your host’s operating system documentation and help system.

Provides details on commands for gathering information and isolating problems.

44 Brocade Adapters Troubleshooting Guide

53-1001582-01

Chapter

Tools for Collecting Data

In this chapter

•For detailed information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

•Data to provide support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

•Data collection using host system commands . . . . . . . . . . . . . . . . . . . . . . . 46

•Data collection using BCU commands and HCM . . . . . . . . . . . . . . . . . . . . . 48

•Data collection using Fabric OS commands (Brocade switches only). . . . . 53

•Adapter event messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

•Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

•Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

•Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

•Collecting BIOS data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

•Collecting LLDP data (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

•Collecting SFP data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

•Collecting port data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

•Collecting teaming information (CNAs only) . . . . . . . . . . . . . . . . . . . . . . . . . 87

•Authentication settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

•QoS settings (HBAs only). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

•Target rate limiting settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

•Persistent binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

For detailed information

This chapter provides basic instruction on tools useful for gathering information to isolate adapter problems. For more detailed information on using these tools, refer to the following publications:

• Brocade Adapters Administrator’s Guide.

The following chapters in this guide cover adapter HCM and BCU monitoring and diagnostics tools:

- Monitoring

- Diagnostics

- Brocade Command Utility (BCU)

Brocade Adapters Troubleshooting Guide 45 53-1001582-01

Data to provide support

NOTE

• Fabric OS Troubleshooting and Diagnostics Guide

This guide provides detailed information on collecting troubleshooting information and isolating general SAN problems between the Brocade switch, host systems, and storage systems.

• Fabric OS Command Reference Manual

Fabric OS diagnostic and monitoring commands for Brocade switches.

• Your host system’s operating system user and administrator’s guides.

Host system diagnostics, logs, and system monitoring tools.

Data to provide support

When problems occur requiring support assistance, provide a detailed description of the problem, as well as output collected from the following HCM and BCU tools:

• Support Save

• Diagnostics

• Port logs

• Port statistics and properties

• Adapter properties

• Host operating system error logs

Data collection using host system commands

Tab le 5 describes commands common to each supported operating system that you can use to

gather information for troubleshooting problems. For details on these commands, refer to your system’s online help and documentation.

Output from all of these commands is captured using the Support Save feature.

TABLE 5 Host system data collection commands

Task Linux Windows VMware Solaris

Listing PCI devices lspci -vv In Windows registry

Listing installed HW details lsdev msinfo32.exe

Displaying process information

location HKEY_LOCAL_MACHINE \SYSTEM\CurrentContro lSet\Enum\PCI devcon find pci\*

Click the plus sign(+) next to Components to view hardware details.

ps -efl, top Windows Task Manager,

tasklist.exe

lspci -vv, esxcfg-info -w prtdiag -v, prtconf -pv

esxcfg-info -a prtdiag -v, prtconf -pv

ps -efl, top ps -efl, top

46 Brocade Adapters Troubleshooting Guide

53-1001582-01

Data collection using host system commands

TABLE 5 Host system data collection commands (continued)

Task Linux Windows VMware Solaris

Displaying memory usage top, vmstat -m Windows Task Manager,

tasklist.exe

Monitoring performance iostat, vmstat, sar Windows Task Manager,

perfmon.exe

Listing driver modules lsmod driverquery vmkload_mod -l modinfo

Checking for Brocade Fibre Channel adapter (BFA) driver module

Checking for Brocade network (BNA) driver module

Displaying driver information

Locating system log messages

NOTE: For more

information, refer to

“Host system logs”

on page 56.

Showing operating system distribution info

Locating BFA configuration file

Locating BFA device file /dev/bfa* Windows Registry

lsmod | grep bfa driverquery /v | findstr

bfad

lsmod | grep bna driverquery /v | findstr

bnad

• Use lsmod

command for general driver information.

• Use ethtool

options to query network driver information and settings.

dmesg, /var/log/message*

(SuSE) cat /etc/SuSE-release, (RedHat) cat /etc/redhat-release

/etc/bfa.conf Windows Registry

On Device Manager Right-click storage controller or network adapter instances, select Properties, then select Driver tab.

System Category in Windows Event Viewer (eventvwr.exe)

systeminfo.exe cat /etc/vmware-release uname -a, cat

(HKEY_LOCAL_MACHINE \SYSTEM\CurrentContro lSet\Services\bfad\Para meters\Device), adapter Flash

(HKEY_LOCAL_MACHINE \HARDWARE\DEVICEMA P\Scsi\Scsi Port x)

top, vmstat -m vmstat -s

vmstat, VM Performance: esxtop [first type 'v', 'e' then enter vm# in the list down], Disk Performance: esxtop [type 'v' then 'd'].

vmkload_mod -l | grep bfa modinfo | grep bfa

vmkload_mod -l | grep bfa NA

• For general driver

information use vmkload_mod.

iostat -nx 1 5, vmstat, mpstat, sar

Use modinfo options for bna or bfa driver.

• For network driver

information, use esxcfg-nics.

/var/log/vmkernel* /var/log/vmkwarning*/proc /vmware/log

/var/log/message*

/etc/bfa.conf /kernel/drv/bfa.conf

/opt/brocade/adapter/bfa/ bfa.conf

dmesg, /var/adm/message*

/etc/release

• (Release 1.0) -

/devices/pci*/pci*/ fibre-channel@0:dev ctl,

• (Release 1.1 and

later) /devices/pci*/pci*/ bfa@0:devctl

Brocade Adapters Troubleshooting Guide 47 53-1001582-01

Data collection using BCU commands and HCM

TABLE 5 Host system data collection commands (continued)

Task Linux Windows VMware Solaris

Locating BNA device file /dev/bna* Windows Registry

(HKEY_LOCAL_MACHINE \HARDWARE\DEVICEMA P\Scsi\Scsi Port x)

Verifying network interface parameters, such as link status, IP address, and subnet mask.

Ethernet statistics ethtool -S

Ethernet link status ethtool

ifconfig ipconfig

Settings > Network Connections

netstat ethtool -S

<interface_name>

netstat esxcfg-nics -l N/A

interface_name>

/dev/bna*

ifconfig N/A

<interface_name>

Data collection using BCU commands and HCM

• (Release 1.0) -

/devices/pci*/pci*/ fibre-channel@0:dev ctl,

• (Release 1.1 and

later) /devices/pci*/pci*/ bfa@0:devctl

N/A

You can collect a variety of information on installed Brocade adapters, such as firmware version installed, operational status, port speed, WWN, PCI data, configuration data, flash status, and other details for troubleshooting using BCU commands, HCM menu options, Data Center Fabric Manager (DCFM), and host operating system commands.

Support Save

The Support Save feature is an important tool for collecting debug information from the driver, internal libraries, and firmware. You can save this information to the local file system and send to support personnel for further investigation. Use one of the following options to launch this feature:

• For HCM, launch Support Save through the Tools menu.

• For DCFM use the Techni cal S uppor t Save dialog box.

• For BCU, enter bfa_supportsave at your operating system prompt (direct mode) or at the BCU>

command prompt using the BCU shell mode. For details on using BCU commands, refer to the Brocade Adapters Administrator’s Guide.

• Through your internet browser (Internet Explorer 6 or later or Firefox 2.0 or later), you can

collect bfa_supportsave output if you do not have root access, do not have access to file transfer methods such as FTP and SCP, or do not have access to the Host Connectivity Manager (HCM).

• A bfa_supportsave collection can also occur automatically for a port crash event.

Launching Support Save in HCM collects HCM application information.

48 Brocade Adapters Troubleshooting Guide

53-1001582-01

Data collection using BCU commands and HCM

NOTE

Launching Support Save through BCU, DCFM, and during a port crash event collects the following information:

• Adapter model and serial number

• Adapter firmware version

• Host model and hardware revision

• All support information

• Adapter configuration data

• All operating system and adapter information needed to diagnose field issues

• Information about all adapters in the system

• Firmware and driver traces

• Syslog message logs

• Windows System Event log .evt file

• HCM-related engineering logs

• Events

• Adapter configuration data

• Environment information

• Data .xml file

• Vital CPU, memory, network resources

• HCM (logs, configuration)

• Driver logs

• Install logs

• Core files

• Details on the Ethernet interface, including IP address and mask (CNAs only)

• Status and states of all adapter ports, including the Ethernet, FCoE, and CEE ports (CNAs only)

• CEE status and statistics (CNAs only)

• Network driver information, Ethernet statistics, offload parameters, and flow control coalesce

parameters (CNAs only)

• Ethernet offload, flow control, and coalescing parameters (CNAs only)

Before collecting data through the Support Save feature, you may want to disable auto-recovery on the host system. This is because when adapters are reset after an auto-recovery from a failure, traces initiated before the failure may be lost or overwritten.

To disable auto-recovery, use the following commands:

• For Linux and VMware, use the following command.

insmod bna.ko bnad_ioc_auto_recover=0

• For Windows, use the Registry Edit tool (regedt32) or the BCU drvconf --key command.

Following is the drvconf ---key command.

bcu drvconf --key ioc_auto_recover --val 0

• For Solaris, edit /kernel/drv/bfa.conf using the following:

Brocade Adapters Troubleshooting Guide 49 53-1001582-01

Data collection using BCU commands and HCM

ioc_auto_recover=0

Initiating Support Save through HCM

Launching the Support Save feature in HCM collects HCM application data. Initiate Support Save by selecting Tool > Support Save.

Messages display during the Support Save operation that provide the location of the directory where data is saved. If you are initiating Support Save from a remote management station and receive a warning message that support files and Agent logs could not be collected, the HCM Agent is unavailable on the remote host. Select Tool > Backup to backup data and configuration files manually.

Support data is collected to a file in your system’s tmp directory by default.

For more information and additional options for using this feature, refer to the Brocade Adapters Administrator’s Guide.

Initiating Support Save through DCFM

To capture technical support and event information for installed adapters, complete the following steps.

1. Select Monitor > Technical Support > SupportSave.

The Technical SupportSave dialog box displays.

2. Click the Generate Now tab.

3. Click the Hosts tab.

4. Right-click in the Available Hosts table and select Expand All.

5. Select the host where adapters are installed, and click the right arrow to move them to the Selected Switches and Hosts table.

5. Click OK on the Technical SupportSave dialog box.

6. Click OK on the confirmation message.

To view Support Save information use the following steps.

1. Select Monitor > Technical Support > View Repository.

The Repository dialog box displays.

2. Select the Hosts tab to view technical support information on hosts.

3. Click View to view the repository in an Internet browser window.

The technical support information displays in an Internet browser window.

Initiating Support Save through BCU commands

Launching the Support Save feature using the bfa_supportsave command collects information for the selected adapter. For information on entering BCU commands, refer to “Collecting adapter data

using BCU commands” on page 52.

The bfa_supportsave command creates and saves the supportsave data to the following locations:

50 Brocade Adapters Troubleshooting Guide

53-1001582-01

Data collection using BCU commands and HCM

NOTE

• The /tmp directory on Linux and Solaris systems.

• The current directory for Windows systems.

Following are examples of other parameters you can use with this command:

• bfa_supportsave <dir> - Creates and saves the supportsave data under a directory name that

you provide.

• bfa_supportsave <dir> <ss_file_name> - Creates and saves the supportsave data under a

directory and file name that you provide. If the directory already exists, it will be overwritten.

If specifying a directory, make sure that the directory does not already exist to prevent overwriting the directory. Do not just specify a driver (such as C:) or C:\Program Files.

Messages display as the system gathers information. When complete, an output file and directory display. The directory name specifies the date when the file was saved.

For more information and additional options for using this feature, refer to the Brocade Adapters Administrator’s Guide.

Initiating Support Save through the internet browser

Launching Support Save through the internet browser collects information for adapters installed on a selected host system. Use the following steps to launch this feature.

1. Open an Internet browser and type the following URL:

https://localhost:34568/JSONRPCServiceApp/SupportSaveController.do

where localhost is the IP address of the host system from which you want to collect the bfa_supportsave information.

2. Log in using the factory default user name (admin) and password (password). Use the current user name and password if they have changed from the default,

The File Download dialog box displays, prompting you to save the file.

3. Click Save and navigate to the location where you want to save the Support Save file.

4. Save the file, but rename with a “zip” extension. For example,

supportSaveController.zip.

5. Open the file and extract contents using any compression utility program.

supportSaveController.do

Initiating Support Save through a port crash event

If the port crashes and triggers a port crash event, Support Save data is collected at a system-wide level. An Application Log message is generated with the following message.

Port Crash Support Save Completed

Port crash events have a CRITICAL severity and you can view the details in the Master Log and Application Log tables in HCM. For more information on these logs, refer to “HCM logs” on page 57.

Brocade Adapters Troubleshooting Guide 51 53-1001582-01

Data collection using BCU commands and HCM

NOTE

Support Save differences

Following are differences in data collection for the HCM, BCU, and browser applications of bfa_supportsave:

• BCU - Collects driver-related logs, HCM Agent information, and configuration files.

• DCFM - Collects only driver-related logs and configuration files.

• Browser - Collects driver-related and HCM Agent logs and configuration files.

• HCM - Collects HCM application data, driver information, HCM Agent logs, and configuration

files.

HCM Master logs and Application logs are saved when Support Save is initiated through HCM, but not through the BCU command.

Collecting adapter data using HCM

Select an adapter from the HCM device tree and click the Properties tab in the right pane to display basic information about the adapter. Depending on whether you select an HBA or CNA, the following displays:

• The Host Bus Adapter (HBA) Properties panel displays HBA and driver information about the

adapter, such as its WWN, vital product data (VPD), manufactured date code (MDC), engineering date code (EDC), power rating (PW), disabled or enabled status, model number, temperature, maximum speed supported, installed driver name, driver version, firmware version, hardware path, and BIOS version. This panel also displays PCI information for the adapter, such as vendor ID, device ID, current number of lanes, and PCI generation.

• The Converged Network Adapter (CNA) Properties panel displays CNA and driver information

about the adapter, such as the MAC address, model number, enabled or disabled status, maximum speed supported, hardware path, serial number, temperature, installed driver name, driver version, firmware version, and BIOS version. This panel also displays PCI information for the adapter, such as vendor ID, device ID, current number of lanes, and PCI generation.

Use the adapter Properties panel in HCM to display information about the adapter. To display this panel, use the following steps.

1. Launch HCM.

2. Select an adapter in the device tree

3. Click the Properties tab in the right pane.

Collecting adapter data using BCU commands

BCU runs in two modes:

• Direct mode, where you type a BCU command at your operating system command prompt. For

example:

bcu fabric --stats <port_id>

• Shell mode, accessed using the bcu shell. For example:

bcu> fabric --stats <port_id>

52 Brocade Adapters Troubleshooting Guide

53-1001582-01

Data collection using Fabric OS commands (Brocade switches only)

To access the shell mode, type bcu shell.

To exit out of shell mode, type exit.

Use the BCU adapter command to list and query available adapters seen by the driver.

The bcu adapter --list command lists all adapters on the system with a brief summary of information such as model number, serial number, and adapter number. Enter the following command:

adapter --list

where:

list Lists all adapters in the system. For each adapter in the system, a brief

information line displays containing the adapter type, model number, serial number, and hardware path.

The adapter --query command displays adapter information, such as the current version of the adapter, adapter type, adapter (chip revision) and driver (fw version), maximum port speed, model information, serial number, number of ports, PCIe information, pwwn, nwwn, hardware path, flash information (such as firmware version), BIOS version, EFI version, current power mode, and port information (PWWN, NWWN, beacon status).

adapter –-query <ad_id>

where:

ad_id ID of the adapter for which you want to query.

Data collection using Fabric OS commands (Brocade switches only)

Use the following Fabric OS commands on attached Brocade switches to gather information and help isolate connectivity and other problems between the adapter, switch, and storage ports. For details on using these commands, refer to the Fabric OS Command Reference Manual.

• authUtil

Use this command to display and set local switch authentication parameters.

• cfgShow

Use this command to display zone configuration information for the switch. You can use command output to verify target ports (by port WWN) and LUNs that are intended to be accessible from the adapter.

• fcpProbeShow

Use this command to display the Fibre Channel Protocol daemon (FCPd) device probing information for the devices attached to a specified F_Port or FL_Port. This information includes the number of successful logins and SCSI INQUIRY commands sent over this port and a list of the attached devices.

• nsShow

Use this command to display local NS information about all devices connected to a specific switch. This includes information such as the device PID, device type, and the port and node WWN.

• zoneshow

Brocade Adapters Troubleshooting Guide 53 53-1001582-01

Adapter event messages

Use this command without parameters to display all zone configuration information (both defined and enabled).

• portErrShow

Use this command to display an error summary for all switch ports.

• portLogShow

Use this command to display the port log for ports on a switch.

• portLogShowPort

Use this command to display the port log for a specified switch port.

• portPerfShow

Use this command to display throughput information for all ports on the switch.

• portStatsShow

Use this command to display hardware statistics counters for a specific switch port.

• portShow

Use this command to display information and status of a specified switch port, including the speed, ID, operating state, type, and WWN.

• SecAuthSecret

Use this command to manage the DH-CHAP shared secret key database used for authentication. This command displays, sets, and removes shared secret key information from the databases.

• sfpShow

Use this command to display detailed information about specific SFPs installed in a switch.

• show vlan brief

Displays information about a VLAN interface on the switch.

• show cee maps

Displays information about the configured CEE maps in the switch.

• switchShow

Use this command to display switch and port information. Output may vary depending on the switch model. Use this information to determine the fabric port WWN and PID connected to an adapter port. Also display topology, speed, and state of each port on the switch.

Adapter event messages

When applicable events occur during adapter operation, the adapter driver generates event messages. These messages are captured in your host system logs. These messages are also captured in an agtEvent.log file by the HCM and displayed in the HCM master log. Note that message display may differ in your host system log and the HCM master log, however messages may contain the following information:

• Message ID

• Description

• Severity level

54 Brocade Adapters Troubleshooting Guide

53-1001582-01

Adapter event messages

NOTE

• Event category

• Cause of event

• Recommended action

• Date and time event occurred

For details of all driver event messages, refer to Appendix A, “Adapter BIOS and Event Message

Reference”.

Message details are also contained in HTML files, which load your system when you install the adapter driver. You can view these HTML files using any internet browser application. Tab le 6 provides the default location where these message files install for each supported operating system.

TABLE 6 Message catalog location

Operating System Catalog Location

Linux /opt/bfa

VMware /opt/bfa

Solaris /opt/bfa

Windows aen.zip loaded to your driver installation

directory. Unzip this file to obtain all message catalog files.

Tab le 7 lists the message file names and content for the message files.

TABLE 7 Event message files

Event Catalog File Content

bfa_aen_adapter.html Adapter events, such as adapter added or

removed

bfa_aen_audit.html Audit events, such as authentication enabled or

disabled for base port

bfa_ethport.html Base port Ethernet events, such as the Ethernet

link up and link down.

bfa_aen_ioc.html I/O controller (IOC) events.

bfa_aen_itnim.html Initiator-target nexus events.

bfa_aen_lport.html Logical port events.

bfa_aen_port.html Physical base port events.

Brocade Adapters Troubleshooting Guide 55 53-1001582-01

NOTE

Logs

TABLE 7 Event message files

Event Catalog File Content

bfa_aen_rport.html Remote port (R_Port) events.

hba_error_codes.doc List of error codes and meanings for the

following events:

• Adapter - events relating to the adapter

• Physical port

• L_Port - logical port

• R_Port - remote initiator or target port

• ITNIM - initiator target nexus

• Audits

• IOC - I/O controller

• Ethernet port

Complete content for adapter driver event messages is included in Appendix A, “Adapter BIOS and

Event Message Reference”.

Logs

Event and error messages that occur during adapter, driver, and HCM operation are important tools for isolating and resolving problems. These messages provide descriptions of an event or problem, severity, time and date of the event, and in some cases, cause and recommended actions. Messages are captured in logs available through HCM, BCU commands, and through host system commands. Monitoring events and errors in these logs allows early fault detection and isolation on a specific adapter.

Host system logs

Brocade adapter event messages are captured in host system log files. All messages related to the Brocade adapter are identified in these logs by BFA (Brocade fabric adapter), BNA (Brocade network adapter), and BFAL (Brocade fabric adapter library]. Table 8 describes the logs for each supported operating system where adapter event messages display and how to view these logs.

TABLE 8 System event Logs

Operating System Log Name Location Viewing Message Log

Solaris Syslog /var/adm/messages dmesg command

Windows Event Log Not applicable

Linux Messages Log /var/log/message dmesg command

VMware

Messages Log /var/log/message* ,

/var/log/vmkernel*, /var/log/vmkwarning*,

/proc/vmware/log

• System category in Event

Viewer (eventvwr.exe)

• bcu winlog command

dmesg command

1. Supported on Windows systems only.

2. For ESX Server Console operating system. For Guest system, refer to information in Windows or Linux.

56 Brocade Adapters Troubleshooting Guide

53-1001582-01

Logs

Syslog support

You can configure the HCM Agent to forward events to a maximum of three system log destinations using the Syslog option on the HCM Configure menu. These events will display in the operating system logs for systems such as Solaris and Linux. For procedures to configure syslog destinations, refer to the Brocade Adapters Administrator’s Guide.

HCM logs

You can view data about adapter operation through HCM logs that display in HCM. These logs display on the bottom of the HCM main window. Click the Master Log or Application Log to toggle between the following logs:

• The Master Log displays informational and error messages during adapter operation. This log

contains the severity level, event description, date and time of event, and the function that reported the event (such as a specific adapter port or remote target port).

• The Application Log displays informational and error messages related to user action in HCM,

discovery, or HCM application issues.

Master Log

The Master Log displays event information in seven fields:

• Sr No.

Sequence number assigned to the event when it occurred, in ascending order.

• Severity

Event severity level (informational, minor, major, or critical).

- Critical-level messages indicate that the software has detected serious problems that will

eventually cause a partial or complete failure of a subsystem if not corrected immediately. For example, IO controller heartbeat failure is a critical error.

- Major messages represent conditions that do not impact overall system functionality

significantly.

- Minor messages highlight a current operating condition that should be checked or it might

lead to a failure.

- Information-level messages report the current non-error status of the system components,

for example, the online and offline status of a port.

• WWN

World-Wide Name of adapter where the event occurred.

• Category

The category or type of event. Categories define the component where events occur:

- ADAPTER - Events pertaining to the adapter.

- CEE - Events pertaining to Converged Enhanced Ethernet.

- ETHPORT - Events pertaining to the Ethernet port.

- IOC - Events pertaining to the I/O controller.

Brocade Adapters Troubleshooting Guide 57 53-1001582-01

NOTE

Logs

- IP over FC - Events pertaining to IP over Fibre Channel.

- VLAN - Events pertaining to a virtual LAN.

- PORT - Events pertaining to a physical port.

- LPORT - Events pertaining to a specific logical port (one logical port always exists per

physical port).

- RPORT - Events pertaining to a specific remote port (could be an initiator or target).

- ITNIM - Events pertaining to an initiator-target nexus.

- RSVD - Reserved.

- AUDIT - Audit events subcategory.

• Subcategory of main category.

• Event description, Date, and Time

Brief description of event and date and the time when the event occurred.

Complete content of adapter event messages is provided in Appendix A, “Adapter BIOS and Event

Message Reference”.

You can block events from display in the Master Log by severity, category, and WWN of adapter using the Master Log Filter dialog box. To display this dialog box, click the Filter button in the Master Log section of the main HCM screen. Select areas that you want to filter and click OK.

Application Log

The Application Log displays all application-related informational and error messages, as well as the following attributes:

• Date and time the message occurred.

• Severity of the message.

• Description of the message.

• The Agent IP address.

Logging levels adjustment

Adjust the logging level for related adapter logs using the following BCU commands and HCM options. By adjusting the logging level, you can control the number and type of messages that are captured the log.

For greater detail on adjusting logging levels, refer to the Brocade Adapters Administrator’s Guide.

Adjusting the adapter event logging level

Specify the number of event messages logged by the host system log for the storage driver using this BCU command.

58 Brocade Adapters Troubleshooting Guide

53-1001582-01

Port logging level

Adjust logging level for port logs using BCU commands and HCM.

Adjusting the port logging level through HCM Adjust port logging level through HCM using the following steps.

1. Select Configure > Basic Port Configuration from HCM.

The Basic Port Configuration dialog box displays.

2. Select a value from the Port Logging Level list.

Supported values are Log Critical, Log Error, Log Warning, and Log Info.

3. Click OK to save the changes and close the window.

Adjusting the port logging level through BCU Adjust port logging level using this BCU command.

bcu log –-level <port_id> [<level>]

where:

Logs

port_id The ID of the port for which you want to set the log level. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

level Critical | Error | Warning | Info

Specifies the severity level. Error is the default setting. If no level is specified, the current log level displays.

Adjusting the Ethernet logging level through HCM (CNAs only)

This is available for CNAs only. Adjust the logging level for each port for Ethernet driver messages through HCM using the following steps.

1. Select an Ethernet port from the HCM device tree.

2. Select Configure > Basic Port Configuration.

The Basic Port Configuration dialog box displays.

3. Select a value from the Eth Logging Level list.

Supported values are Log Critical, Log Error, Log Warning, and Log Info.

4. Click OK to save the changes and close the window.

Adjusting the Ethernet logging level through BCU

Adjust port logging level for the Ethernet driver using this BCU command.

ethlog --level <port_id> [<Critical | Error | Warning | Info>]

where:

Brocade Adapters Troubleshooting Guide 59 53-1001582-01

Statistics

<port_id> ID of port for which you want to change the logging level. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

level Critical | Error | Warning | Info

Specifies the severity level. Error is the default setting. If no level is specified, the current log level displays.

Adjusting the HCM logging level

Adjust logging levels for the following logs using HCM:

• Agent Communication Log, where all messages are exchanged between the HCM application

and the HCM Agent.

• HCM Debug Log, where messages are logged locally.

To adjust the logging level, use the following steps.

1. Select Configure > HCM Logging Levels.

2. Select a level on the Agent Communication Log and HCM Debug Log lists.

Statistics

Values are Trace, Debug, Info, Warning, Error, and Fatal.

You can access a variety of statistics using BCU commands and HCM. Use these statistics to monitor adapter performance and traffic between the adapter and LUNs and isolate areas that impact performance and device login.

You can display statistics for the following:

• Adapter ports

• Authentication activities

• CEE

• Ethernet

• Ethernet I/O controller

• Fibre Channel over Ethernet (FCoE)

• IO controller (IOC)

• Virtual ports (vport)

• Link Layer Discovery Protocol (LLDP)

• Logical ports (lport)

• Remote ports (rport)

• Fibre Channel Protocol (FCP) initiator mode

• Fabric

• Tar get s

• Teaming

• Security authentication

60 Brocade Adapters Troubleshooting Guide

53-1001582-01

Statistics

• VLAN

This section provides an overview of these statistics and how to access them. For more detail, refer to the Brocade Adapter’s Adapters Administrator’s Guide.

Authentication statistics

Use BCU commands and HCM to display statistics related to transmitted and received DH-CHAP attempts for a selected port. You can display statistics such as the following:

• The number of times security authentication failed

• The number of times security authentication succeeded

• The number of rejected transmitted Fibre Channel authentication attempts (Tx Auth Rjts)

• The number of transmitted Fibre Channel authentication negotiation attempts (Tx Auth Negs)

• The number of completed Fibre Channel authentication negotiation attempts (Tx Auth Dones)

• The number of transmitted DH-CHAP replies (Tx DHCHAP)

• The number of transmitted DH-CHAP challenge attempts (Tx DHCHAP Challenges)

• The number of times a transmitted Fibre Channel authentication attempt was successful (Tx

DHCHAP Successes)

• The number of rejected received Fibre Channel authentication attempts (Rx Auth Rjts)

• The number of received Fibre Channel authentication negotiation attempts (Rx Auth Negs)

• The number of completed received Fibre Channel authentication attempts (Rx Auth Dones)

• The number of received DH-CHAP challenge attempts (Rx DHCHAP Challenges)

• The number of received DH-CHAP replies (Rx DHCHAP Replies)

• The number of times a received DH-CHAP challenge was successful (Rx DHCHAP Successes)

Displaying statistics through BCU

Use the following BCU command to display CEE statistics.

bcu auth --stats <port_id>

where:

<port_ID> The ID of the Ethernet port. This could be the PWWN, port hardware path, or

user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

Displaying statistics through HCM

Display the Authentication Statistics dialog box using the following steps.

1. Select an HBA port from the device tree.

2. Select Configure > FC_SP > Authentication Statistics.

Brocade Adapters Troubleshooting Guide 61 53-1001582-01

Statistics

CEE statistics (CNAs only)

Use BCU commands and HCM to display converged enhanced Ethernet (CEE) statistics, such as the following:

• Logical link layer discovery protocol (LLDP) frames transmitted, received, timed out, discarded,

with error, type-length-values (TLVs) discarded, and TLVs unrecognized.

• Data center bridging capability exchange (DCBX) TLVs unrecognized, negotiation fails, remote

configurations changed, TLVs received and invalid, status up and down, and received invalid configurations.

Displaying statistics through BCU

Use the following BCU command to display CEE statistics.

bcu cee --stats <port_id>

where:

<port_ID> The ID of the Ethernet port. This could be the PWWN, port hardware path, or

user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

Displaying statistics through HCM

Display the CEE Statistics dialog box using the following steps.

1. From the device tree, select a physical port of a CNA.

2. Select Monitor > Statistics > CEE Statistics.

CEE query (CNAs only)

Query the CEE information on a selected port and display such information as LLDP attributes and the priority group table.

bcu cee --query <port_id>

where:

<port_ID> The ID of the Ethernet port. This could be the PWWN, port hardware path, or

user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

Ethernet statistics (CNAs only)

Use BCU commands and HCM to display total numbers of the following statistics:

• Tran s mitted unic ast octets

• Transmitted unicast frames

• Transmitted unicast VLANs

• Tran s mitted mult i cast o c tets

62 Brocade Adapters Troubleshooting Guide

53-1001582-01

• Transmitted multicast frames

• Transmitted multicast VLANs

• Transmitted broadcast octets

• Transmitted broadcast frames

• Transmitted broadcast VLANs

• Transmitted errors

• Transmitted VLAN filters

• Transmitted filter MAC source addresses

• Received unicast octets

• Received unicast frames

• Received unicast VLANs

• Received multicast octets

• Received multicast frames

• Received multicast VLANs

• Received broadcast octets

• Received broadcast frames

• Received broadcast VLANs

• Received frame drops

• Received packets

• Transmitted packets

• Received bytes

• Transmitted bytes

• Linux NetIf queue stops

• Linux NetIf queue wakups

• Linux TSO IPv4 packets

• Linux TSO IPv6 packets

• Linux errors

• TCP checksum offloads

• UDP checksum offloads

• Checksum help requests

• Checksum help errors

• Hardware statistics updates

Statistics

Displaying statistics through BCU

Display Ethernet statistics using the following BCU command.

ethport --stats <port_id>

where:

Brocade Adapters Troubleshooting Guide 63 53-1001582-01

Statistics

<port_ID> The ID of the Ethernet port. This could be the PWWN, port hardware path, or

user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

Displaying statistics through HCM

To display the Ethernet Statistics dialog box, use the following steps.

1. Select an Ethernet port from the device tree.

2. Select Monitor > Statistics > Eth Statistics from the main menu.

Right-click the Ethernet port and select Statistics > Eth Statistics from the list.

The Ethernet Statistics dialog box at the host level displays.

Ethernet IOC statistics (CNAs only)

Use HCM options and BCU to display statistics relevant to the Ethernet IO controller (IOC). A variety of statistics display, such as the following:

• Mailbox interrupts

• Enable and disable events

• Heartbeat failures

• Firmware boots

• Ethernet ICO statistic timeouts

• Checksum help errors

• VLAN transmit and receive transactions

You can also select options to keep running data, set the polling frequency, start polling data, and reset statistics.

Displaying statistics through HCM

To display Et hernet IOC stat istics use the following steps.

1. Select an Ethernet port from the device tree.

2. Select Monitor > Statistics > Eth IOC Statistics from the main menu.

Right-click the Ethernet port and select Statistics > Eth IOC Statistics from the list.

The Eth IOC Statistics dialog box at the host level displays.

64 Brocade Adapters Troubleshooting Guide

53-1001582-01

Statistics

Displaying statistics through BCU

Use the BCU ethioc command to display Ethernet input/output controller (IOC) statistics such as serial number, model, PWWN, NWWN, firmware version, operating status, vendor ID, IOC type, interrupt coalescing status, and interrupt delay value.

ethioc --query <port_id> ethioc --stats <port_id> ethioc --statsclr <port_id>

--query Displays attributes of the Ethernet IOC.

port_id Specifies the ID of the Ethernet port for which you will display attributes. This

could be the PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

--stats Displays the Ethernet IOC statistics.

port_id Specifies the ID of the Ethernet port for which you will display the statistics.

--statsclr Clears the Ethernet IOC-level statistics.

port_id Specifies the ID of the Ethernet port for which you will reset the statistics.

FCoE statistics (CNAs only)

Use BCU and HCM to display statistical information related to a selected Fibre Channel over Ethernet (FCoE) port. Statistics include the number of transmitted and received packets and transmitted and received bytes. You can also select options to continue running data, configure polling frequency, and start polling.

Displaying FCoE statistics through HCM

To display FCoE statistics through HCM, use the following steps.

1. Select an FCoE port from the device tree.

2. Select Monitor > Statistics > FCoE Statistics from the device tree.

Right-click the FCoE port and select Statistics > FCoE Statistics from the list.

The FCoE Statistics dialog box at the host level displays.

Fabric statistics

Use BCU and HCM to display statistics for fabric login (FLOGI) activity and fabric offlines and onlines detected by the port. Use these statistics to help isolate fabric login problems. Examples include number of FLOGIs sent, FLOGI response errors, FLOGI accept errors, FLOGI accepts received, FLOGI rejects received, unknown responses for FLOGI, allocation waits prior to sending, the number of delayed fabric login allocations, FLOGIs received, and incoming FLOGIs rejected.

Following are two examples of how to use these statistics for troubleshooting:

Brocade Adapters Troubleshooting Guide 65 53-1001582-01

Statistics

• If the adapter is not showing in the fabric, check the FLOGI sent and FLOGI accept statistics. If

the counts do not match, the switch or fabric may not be ready to respond. This is normal as long as it does not persist. If the problem persists, this could indicate a problem in the fabric or a protocol issue between the adapter and fabric.

• If fabric offline counts increase and fabric maintenance is not being done, this may indicate a

serious fabric problem. Slow fabric performance or hosts unable to address storage could also be seen.

Displaying fabric statistics through BCU

Use the fabric --stats command to display fabric statistics.

fabric -–stats <port_id>

where:

port_id ID of the adapter port for which you want to display statistics. This could be

the PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

Displaying fabric statistics through HCM

Use the Fabric Statistics dialog box to monitor a variety of port data.

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Statistics > Fabric Statistics.

IOC statistics

Use BCU and HCM to display port-level statistics for the I/O controller through the BCU and HCM. The I/O controller refers to the firmware entity controlling the port. The following types of IOC statistics are displayed:

• IOC driver

• IOC firmware

• Firmware IO

• Firmware port FPG

• Firmware port PHYSM

• Firmware port LKSM

• Firmware port SNSM

66 Brocade Adapters Troubleshooting Guide

53-1001582-01

Statistics

Displaying IOC statistics through BCU

Use the ioc command to display IOC statistics.

ioc -–stats --query <ioc_id>

where:

stats Displays IOC statistics.

query Displays IOC attributes.

ioc_id ID of the IOC controller for which you want to display attributes.

The ioc --query command displays such attributes as adapter serial number, model number, PWWN, firmware and BIOS version, vendor identification, IOC type, and interrupt coalescing status.

Displaying IOC statistics through HCM

Use the IOC Statistics dialog box to monitor a variety of port data.

1. Launch HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Statistics > IOC Statistics.

FCP initiator mode statistics

Use HCM and BCU to display FCP IM module statistics for each initiator target nexus (ITN). Statistics display such as I/Os waiting for circular queue space, number of I/O context requests, I/O abort requests, number of task management I/O context requests, I/O completions with OK status, number of successful firmware I/O underrun operations, number of successful firmware I/O overrun operations, aborted I/O requests, I/O timeouts, I/O selection timeouts, I/O protocol errors, and host I/O abort requests.

Displaying FCP initiator mode statistics through BCU

Use the fcpim command to display FCP initiator mode statistics and attributes.

fcpim --stats <port_id> --query <port_id> <rpwwn [-l <lpwwn>]

where:

stats Displays FCP initiator mode statistics.

query Displays FCP initiator mode attributes.

port_id ID of the port for which you want to display statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

lpwwn Logical PWWN. This is an optional argument. If the -l lpwwn argument is not

specified, the base port is used.

rpwwn Remote PWWN.

Brocade Adapters Troubleshooting Guide 67 53-1001582-01

Statistics

Displaying FCP IM module statistics through BCU

Use the fcpim command to display FCP initiator mode (FCP IM) module statistics.

bcu fcpim --modstats <port_id>

where:

modstats Displays FCP IM module statistics.

port_id ID of the port for which you want to display statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

Displaying FCP IM mode statistics through HCM

Use the following steps to display the FCP IM Statistics dialog box:

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Statistics > FCP IM Module Statistics.

Right-click a port from the device tree and select Statistics > FCP IM Module Statistics.

Logical port statistics

Use HCM and BCU to display logical port statistics for the following:

• Name server (NS) port logins (plogin) activity

• Register symbolic port name (RSPN_ID) identifier activity

• Register FC4 type identifier (RFT_ID) activity

• Get all port ID requests for a given FC4 type (NS_GID_FT) activity

• Retries

• Timeouts

Use these statistics to help determine if the adapter is not registering with the name server or cannot access storage. Following are examples of how these statistics indicate these problems:

• If name server port login (NS PLOGI) error rejects and unknown name server port login

response (NS login unknown rsp) errors increase, then the adapter most likely cannot log in to the name server.

• If name server register symbolic port name identifier (NS RSPN_ID) or name server register

symbolic port name identifier response (NS RFT_ID rsp) errors or rejects (NS RFT_ID rejects) are increasing, the adapter has a problem registering with the name server.

• If name server get all port ID response NS GID_FT rsp), rejects (NS_GID FT rejects), or unknown

responses (NS_GID FT unknown rsp) are increasing, the adapter has a problem querying the name server for available storage.

68 Brocade Adapters Troubleshooting Guide

53-1001582-01

Statistics

Displaying logical port statistics through HCM

Display logical port statistics by selecting Monitor > Statistics > Logical Port Statistics.

Right-click a logical port from the device tree and select Logical Port Statistics.

Displaying logical port statistics through BCU

Use the lport --stats command to display logical port statistics.

lport –-stats <port_id> [-l lpwwn]

where:

port_id ID of the port for which you want to display statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

lpwwn Logical port world wide name for which you want to display statistics. This is

an optional argument. If the -l lpwwn argument is not specified, the base port is used.

rpwwn Remote port world wide name for which you want to display statistics.

Port statistics

Use BCU and HCM to display a variety of port statistics. Following is an overview of HBA and CNA statistics:

• For HBAs, statistics include transmitted and received frames and words, received loop

initialization primitive (LIP) event counts, error frames received, loss of synchronization, link failure and invalid CRS counts, end of frame (EOF) errors, and encoding non-frame errors. Use these statistics to isolate link and frame errors. For example, loss of synch and loss of signal errors indicate a physical link problem. To resolve these problems, check cables, SFPs on adapters or switch, and patch panel connections.

• For CNAs, statistics include total transmit and receive counts for different sizes and types of

frames. Data is included on 64-byte to 1519-1522-byte, multicast, broadcast, control, jabber, drop, control, FCS error, alignment error, code error, pause MAC control, zero pause MAC control, FCoE pause MAC control, and zero pause MAC control frames.

Displaying statistics through BCU

Use the port --stats BCU command to display statistics for a specified adapter port.

port --stats <port_id>

where:

port_id ID of the port for which you want to display statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

Brocade Adapters Troubleshooting Guide 69 53-1001582-01

Statistics

Displaying statistics through HCM

Use the Port Statistics dialog box to monitor a variety of port data. Launch this dialog box using the following steps:

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Statistics > Port Statistics.

Remote port statistics

Remote port statistics can help isolate end-to-end login problems. Use HCM and BCU to display statistics for the following:

• Port login (PLOGI) activity

• Authentication and discovery (ADISC) activity

• Logout (LOGO) activity

• RCSNs received

• Process logins (PRLI) received

• Hardware abstraction layer (HAL) activity

• Remote port speed capability (RPSC)

As an example of using these statistics for troubleshooting, if the host cannot see the target, you can verify that the remote port (rport) is reporting itself online by comparing the rport offline and rport online statistics. The rport online counter should be one greater than the rport offline counter. If not, clear the counters and retry connecting to the remote port. Verify the rport online and rport offline statistics again.

Displaying target statistics through HCM

Launch the Target Statistics dialog box using the following steps to display target statistics.

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Statistics > Remote Port Statistics > Target Statistics.

Displaying remote port statistics through BCU

Use the rport --stats command to display remote port statistics.

rport –-stats <port_id> <rpwwn> [-l lpwwn]

where:

port_id ID of the port for which you want to display rport statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

lpwwn Displays the logical PWWN. This is an optional argument. If the -l lpwwn

argument is not specified, the base port is used.

70 Brocade Adapters Troubleshooting Guide

53-1001582-01

Statistics

rpwwn Displays the remote port’s PWWN.

Quality of Service statistics (HBAs only)

Use HCM and BCU to display quality of service (QoS) statistics for individual ports. You can display statistics for fabric login (FLOGI) activity, exchange link parameter (ELP) activity, and received QOS registered state change notifications (RSCNs).

Displaying QoS statistics through HCM

Use the QOS Statistics dialog box to display QoS statistics.

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Monitor > Remote Port Statistics > QOS Statistics.

Displaying QoS statistics through BCU

Use the qos --stats command to display remote port statistics.

vport --stats <port_id> vpwwn

where:

port_id ID of the port for which you want to display rport statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

Virtual port statistics

Use HCM and BCU to display logical port statistics for fabric discovery (FDISC) activity, logouts (LOGO) activity, NPIV support, number of fabrics online and offline, and fabric cleanups.

Use these statistics to isolate NPIV login problems. Following are examples of what to check if virtual devices are not listed in the name server:

• If FDISC sent and FDISC accept statistics do not match, the fabric or switch may not be ready

for data transmission. This is normal as long as it does not persist. If it does persist, there may be a problem in the fabric or a protocol issue between the adapter and fabric. Note that in this case FDISC retries also increase.

• Check the No NPIV support statistics to verify that NPIV is supported and enabled on the

switch.

Displaying virtual port statistics through HCM

Display statistics by selecting Monitor > Statistics > Virtual Port Statistics.

Right-click a virtual port on the device tree and select Virtual Port Statistics.

Brocade Adapters Troubleshooting Guide 71 53-1001582-01

Statistics

Displaying virtual port statistics through BCU

Use the vport --stats command to display statistics.

vport --stats <port_id> vpwwn

where:

port_id ID of the port for which you want to display rport statistics. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

vpwwn Displays the statistics for the virtual port by its WWN. If no PWWN is specified,

the information provided is for the base vport.

VLAN Statistics (CNAs only)

The HCM and BCU display statistics related to a specific VLAN, such as VLAN ID, VLAN name, number of transmit and receive bytes, length of time between byte transmission and reception, and correction status. You can also use options to set the polling frequency, start polling, and reset statistics.

Displaying VLAN statistics through HCM

To display the VLAN Statistics dialog box, use the following steps.

1. Select an Ethernet port from the device tree.

2. Select Monitor > Statistics > VLAN Statistics from the main menu.

The VLAN Statistics dialog box displays.

Displaying VLAN statistics through BCU

Use the vlan --query command to display VLAN statistics (Windows systems only).

vlan --query <vlan_id> {<port_id>|<team_name>}

where:

vlan_id Specifies the VLAN identifier. The range for the VLAN ID is 1 to 4094.

port_id Specifies the Ethernet port associated with the VLAN. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

team_name Specifies the team name.

72 Brocade Adapters Troubleshooting Guide

53-1001582-01

Diagnostics

NOTE

Diagnostics

Diagnostics, available through BCU commands and HCM, evaluate the integrity of adapter hardware and end-to-end connectivity in the fabric. All of these diagnostics can be used while the system is running.

Be sure to disable the port before running any type of port diagnostics.

It is advisable to not perform other operations on the adapter while running HCM or BCU diagnostics.

Beaconing

Initiate beaconing on a specific adapter port to flash the port LEDs and make it easier to locate the adapter in an equipment room.

Initiate link beaconing to flash the LEDs on a specific adapter port and the LEDs on a connected switch port to verify the connection between adapter and switch. When you initiate link beaconing, commands are sent to the other side of the link. When the remote port receives these commands, that port’s LEDs flash. The remote port sends a command back to the originating port. When that port receives this command, the port’s LEDs flash.

To initiate link beaconing, this feature must be available on the connected switch.

Toggle beaconing on and off and set beaconing duration using the BCU or HCM.

Enabling beaconing through BCU

Use the diag --beacon command to enable beaconing for a specific adapter port and end-to-end beaconing.

diag --beacon <port_id> -e <on | off> [<duration>]

where:

port_id ID of the port for which you want to enable beaconing. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

-e End-to-end beaconing.

duration Length of time between blinks.

Use the fcdiag --linkbeacon command to enable link beaconing.

fcdiag --linkbeacon <portid> {on | off} [<duration>]

where:

Brocade Adapters Troubleshooting Guide 73 53-1001582-01

Diagnostics

port_id ID of the port for which you want to run a link beacon test. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

on | off Toggle on or off. If turned on, you can specify duration.

duration Length of time between blinks.

Enabling beaconing through HCM

Enable link and port beaconing using the following steps.

1. Launch the HCM.

2. Select the base adapter port from the device tree window.

3. Click Configure > Beacon.

4. Click the Beacon Link check box, the Beacon Port check box, or both.

Internal and external loopback tests

Use the BCU or the HCM to perform a loopback test for a specific port. Loopback tests require that you disable the port. The following loopback tests are available:

• Internal

Random data patterns are sent to the adapter port, then returned without transmitting through the port. The returned data is validated to determine port operation. Errors may indicate a failed port.

• External

For this test, a loopback connector is required for the port. Random data patterns are sent to the adapter port. The data transmits from the port and then returns. The returned data is validated to determine port operation. Errors may indicate a failed port.

Performing loopback tests through BCU

Use the diag --loopback BCU command test to verify port function through a loopback test.

bcu diag --loopback <port_id> [-t <loopback_type>][<duration>][-s <speed>] [-c <frame_count>] [-p <pattern>]

where:

port_id ID of the port for which you want to run the test. This could be the PWWN, port

hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

loopback type Type of loopback test. Possible values are internal, external, and serdes.

speed For HBAs only. For 8 Gbps adapter, this is 2, 4, or 8. For 4 Gbps adapter, this

is 1, 2 or 4.

duration Length of time between blinks.

frame count Integer from 0 to 4,294,967,295. Default is 8192.

74 Brocade Adapters Troubleshooting Guide

53-1001582-01

Diagnostics

NOTE

-p pattern Hex number. Default value is A5A5A5A5.

Performing loopback tests through HCM

Use the Hardware Tests tab on the Diagnostics dialog box to perform a loopback test.

1. Launch the HCM.

2. Select Configure > Diagnostics.

3. Click the Hardware Tests tab.

4. Select Loopback Test.

You can modify the following test parameters

• Subtest - The three options are Internal, Serdes, and External.

• Link Speed (HBAs only) - For 8G adapter, 2G, 4G, and 8G. For 4G adapter, 1G, 2G, and 4G.

• Frame Count - Integer from 0 to 4,294,967,295. Default value is 8192.

• Data Pattern - Hex value. Default value is A5A5A5A5.

5. Click Start.

Ethernet port loopback test (CNAs only)

Use BCU commands and HCM to test the Ethernet data path from the host to serdes or external loopback based on your selection. You must disable the port before testing and use a loopback connector for the test.

For 64-bit platforms only, you cannot perform Ethernet loopback tests on the port unless a VLAN is first created for the port.

Performing Ethernet loopback tests through HCM

Using the Ethernet Tests tab on the Diagnostics dialog box, you can run Ethernet loopback tests, and set test parameters such as external subtests, serdes subtests, link speed, frame count, test cycle, and data pattern to test. Results display on the bottom of the tab as the test commences.

1. Launch the HCM.

2. Select Configure > Diagnostics.

3. Click the Ethernet Tests tab.

4. Select Loopback Test.

Brocade Adapters Troubleshooting Guide 75 53-1001582-01

NOTE

Diagnostics

Performing Ethernet loopback tests through BCU

Use the ethdiag --loopback command to run an Ethernet loopback test.

bcu ethdiag --loopback <port_id> [-t <loopback_type] [-c <frame_count>] [-p pattern]

where:

port_id ID of the port for which you want to run the test. This could be the PWWN, port

hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

-t loopback-type Specifies the loopback type. Possible values are serdes, and ext.

-c frame count Specifies the frame count.

-p pattern Specifies the pattern (must be one hex word).

You must disable the port before running a loopback test. Use BCU port --disable <port_ID> command.

PCI loopback test

Use BCU commands or HCM to perform a PCI loopback test for a specific port. In this test, a data pattern is sent from the host to adapter firmware through the PCI bus. The returned data is validated to determine PCI operation.

You must disable the port before you run loopback tests.

Performing PCI loopback tests through BCU

Use the diag --pciloopback BCU command to perform a PCI loopback test.

diag --pciloopback <port_id> [-p <pattern>] [-c <frame_count>]

where:

port_id ID of the port from which you want to run the test. This could be the PWWN,

port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

pattern Specifies the data test pattern. Must be at least one hexadecimal word.

frame count Specifies the frame count as an integer from 0 to 4,294,967,295.

Performing PCI loopback tests through HCM

Use the Hardware Tests tab on the Diagnostics dialog box to perform a PCI loopback test as follows.

1. Launch the HCM.

2. Select Configure > Diagnostics.

76 Brocade Adapters Troubleshooting Guide

53-1001582-01

Diagnostics

NOTE

3. Click the Hardware Tests tab.

4. Select PCI Loopback Test.

You can modify the following parameters:

• Frame count: Specifies the frame count as an integer from 0 to 4,294,967,295.

• Data pattern: Specifies the data test pattern. Must be at least one hexadecimal word.

• Test cycle: The number should be positive and the default is 1.

5. Click Start.

Memory test

Use the BCU or the HCM to perform a memory test for the adapter.

Performing the memory test disables the adapter.

Performing a memory test through BCU

Use the diag --memtest command to test the adapter’s memory blocks.

diag --memtest <ad_id>

where:

ad_id ID of the adapter.

Performing a memory test through HCM

Use the Hardware Tests tab on the Diagnostics dialog box as follows to perform a memory test.

1. Launch the HCM.

2. Select Configure > Diagnostics.

3. Click the Hardware Tests tab.

4. Select Memory Test.

5. Specify the test cycle using a positive number.

6. Click Start.

Pinging Fibre Channel end points

Use the BCU and HCM to ping a Fibre Channel end point from an adapter port to determine the basic connectivity to the remote port and monitor network latency.

Ping testing is not supported on Solaris systems.

Brocade Adapters Troubleshooting Guide 77 53-1001582-01

Diagnostics

Issuing a ping command to end points through BCU

Use the fcdiag --fcping BCU command to test the connection to a Fibre Channel end point.

fcdiag -–fcping <port_id> <rpwwn> [-l lpwwn]

where:

port_id ID of the adapter port from which you want to ping the remote port. This could

be the PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

rpwwn Remote PWWN that you want to ping.

lpwwn Logical PWWN. 0 indicates the base port.

Issuing a ping command to end points through HCM

Use the FC Protocol Tests tab on the Diagnostics dialog box to test the connection to Fibre Channel end points. Use the following steps to ping end points.

1. Launch the HCM.

2. Select Configure > Diagnostics to display the Diagnostics dialog box.

3. Click the FC Protocol Tests tab.

4. Select FC Ping Test.

5. Select the adapter port and target that you wish to ping.

6. Enter a test cycle if applicable.

7. C l ic k Start.

Adapter temperature

Use the BCU diag --tempshow command to read the adapter’s temperature sensor registers.

diag --tempshow <ad_id>

where:

ad_id ID of adapter.

Queue test

Use BCU commands and the HCM to send a health check message from the host to the firmware through message queues memory mapped over the PCI.

78 Brocade Adapters Troubleshooting Guide

53-1001582-01

Diagnostics

Performing a queue test through BCU

Use the diag --queuetest command to perform a queue test.

bcu diag --queuetest <port_id> [-q <queue_number>

where:

port_id Adapter port number. This could be the PWWN, port hardware path, or

user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1, you would use 1/1 as the port identification.

-q queue_number Specifies the CPE queue number from 0 to 3. If the queue number is not

specified, all queues are tested.

Performing a queue test through HCM

Use the Hardware Tests tab on the Diagnostics dialog box as follows to perform a queue test.

1. Launch HCM.

2. Select Configure > Diagnostics

3. Click the Hardware Tests tab.

4. Select Queue Test.

5. Click Start.

Trace route

Use the BCU and HCM to trace the SAN path between the adapter and remote end point.

Tracing the route through BCU

Use the fcdiag --traceroute BCU command to trace the route between end points.

fcdiag -–fctraceroute <port_id> <rpwwn> [-l lpwwn]

where:

port_id ID of the port from which you want to trace the route. This could be the

PWWN, port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 use 1/1 as the port identification.

rpwwn Remote PWWN to which you want to trace the route.

lpwwn Logical PWWN. 0 indicates the base port.

Tracing the route through HCM

Use the FC Trace Route test to trace the route between the adapter port and a Fibre Channel target port. To run this test, use the following steps:

1. Launch the HCM.

2. Select Configure > Diagnostics to display the Diagnostics dialog box.

Brocade Adapters Troubleshooting Guide 79 53-1001582-01

Diagnostics

3. Click the FC Protocol Tests tab.

4. Select FC Trace Route.

5. Select the adapter port and target for which you wish to trace the route.

6. Enter a test cycle if desired.

7. C l ic k Start.

Echo test

Use the BCU and HCM to initiate an echo test between the adapter port and a Fibre Channel end point. This sends an ECHO command and response sequence between the adapter port and target port to verify connection with the target.

Performing an echo test through BCU

Use the fcdiag --fcecho BCU command to initiate an echo test between the adapter and remote port.

fcdiag –-fcecho <port_id> <rpwwn> [-l lpwwn]

where:

port_id ID of the port for which you want to perform the test. This could be the PWWN,

port hardware path, or user-specified port name. This could also be the adapter-index/port-index. For example, to specify adapter 1, port 1 you would use 1/1 as the port identification.

rpwwn Remote port WWN to which the echo command is being sent.

lpwwn Logical port WWN. 0 indicates the base port.

Performing an echo test through HCM

Use the Echo Test feature as follows to initiate an echo test between the adapter port and a Fibre Channel end point.

1. Launch the HCM.

2. Select Configure > Diagnostics to display the Diagnostics dialog box.

3. Click the FC Protocol Tests tab.

4. Select Echo Test.

5. Select the adapter port and target port for the test.

6. Enter a test cycle if applicable.

7. C l ic k Start.

80 Brocade Adapters Troubleshooting Guide

53-1001582-01

Brocade Communications Systems CNA 1010, HBA 415, CNA 1020, HBA 425, HBA 804 Troubleshooting Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

About this Document

In this chapter

How this document is organized

Supported CNA hardware and software

Supported HBA hardware and software

What’s new in this document

Document conventions

Notice to the reader

Additional information

Providing details for support

Document feedback

Introduction to Troubleshooting

In this chapter

How to use this manual for troubleshooting

Gathering problem information

Isolating Problems

In this chapter

How to use this chapter

General HBA and CNA problems

Adapter not reported under server’s PCI subsystem

No adapters reported though BCU adapter --list command

Port link is not active

Installer program does not autorun (Windows only)

Host system freezes or crashes

Operating system errors (blue screen)

Driver event messages appearing in host system log files

BCU version mismatch warning

I/O data traffic issues

Driver installation fails and system cannot be booted

Files needed for bfad.sys message appears

Cannot roll back driver on all adapter instances using Device Manager

Errors when installing brocade_driver_linux_<versions>.tar.gz package

Operating system crashes after invalid HCM operand

UEFI boot problems

BIOS boot problems

HBA problems only

Quality of Service performance issues

Quality of Service not functioning

Unable to create more than 126 Virtual (NPIV) ports for HBA

Ethernet network interface problems (CNAs only)

Ethernet loopback test problems

Ethernet link ports or LOM not coming up on reboot in Linux

Loss of adapter hardware address in Linux

Loss of adapter IP address in Linux

Network stack runs out of heap

NIC numbering unexpected on VMware systems

Ping to remote host is failing

Teaming errors

VLAN creation and operation problems

Teaming or VLAN operations through HCM fail

Poor network performance

FCoE and Fibre Channel problems

Loss of sync and loss of signal errors in port statistics

Fabric authentication failures

Adapter is not showing in the fabric

Virtual devices not listed in name server

Adapter not registering with the name server or cannot access storage

FCoE link is down (CNAs only)

I/O problem on connected FCoE device (CNAs only)

I/Os are not failing over immediately on path failure in MPIO setup

Disk I/O requests cause low throughput and high latency on Linux

Disk I/O requests cause low throughput and high latency on VMware

CEE network problems (CNAs only)

CEE is not enabled

HCM and HCM Agent problems

Failed to connect to agent on host... error when using HCM

Unable to completely uninstall HCM

Verifying Fibre Channel and CEE links (stand-up adapters)

Time on HCM screens does not match system time

Adapter driver installation verification

Confirming driver package installation with HCM

Confirming driver package installation in Windows systems

Confirming driver package installation in Linux systems

Confirming driver package installation in Solaris systems

Confirming driver package installation in VMware systems

Troubleshooting mezzanine card problems