Emulex Adapters User Manual

Troubleshooting and

Maintenance Manual

for LightPulse® Adapters

P003409-01A Rev A Critical Connectivity Solutions

Information furnished by Emulex is believed to be accurate and reliable. However, no responsibility is assumed by Emulex for its use; or for any infringements of patents or other rights of third parties which may result from its use. No license is granted by implication or otherwise under any paten t, copyrigh t or rela ted ri ghts of Emulex .

Emulex, AutoPilot Installer, BlockGuard, cLAN, FabricStream, FibreSpy , Giganet, HBAnyware, InSpeed, IntraLink, LightPulse, MultiPulse, SAN Insite, SBOD and Vixel are registered trademarks, and AutoPilot Manager, Critical Connectivity Solutions, EZPilot, SLI and VMPilot are trademarks of Emulex. All other brand or product names referenced herein are trademarks or registered trademarks of their respective companies or organizations.

Emulex provides this manual “as is” without any warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Emulex may make improvements and changes to the product described in this manual at any time and without any notice. Em ulex assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties that may result. Periodic changes are made to information contained herein; although these changes will be incorporated into new editions of this manual, Emulex disclaims any undertaking to give notice of such changes.

Emulex, 3333 Susan Street Costa Mesa, CA 92626

Troubleshooting and Maintenance Manual for LightPulse Adapters Page ii

Overview .................................................................................................................1

Be Informed, Stay Informed, Stay Current.........................................................1

Driver Updates ..................................................................................................1

Drivers for Windows.................................................................................... 1

Drivers for Linux..........................................................................................2

Drivers for Solaris .......................................................................................2

Drivers for HP-UX .......................................................................................2

Drivers for Netware.....................................................................................2

AutoPilot Installer........................................................................................2

Firmware Updates.............................................................................................2

The Importance of Compatibility........................................................................ 3

Emulex Configuration Parameters.....................................................................3

How Emulex Device Driver Configuration Parameters Work.......................3

Emulex Utility Applications ................................................................................ 3

Troubleshooting.......................................................................................................4

Observing the Problem...................................................................................... 4

Isolating the Problem......................................................................................... 5

SAN Components .......................................................................................5

Server Components.................................................................................... 5

Checklists..........................................................................................................6

Connectivity Checklist................................................................................. 6

Installation Checklist ................................................................................... 8

Upgrade Checklist.......................................................................................8

Configuration Checklist...............................................................................9

Tools to Collect Data ....................................................................................... 10

Driver Tools............................................................................................... 10

The HBAnyware Utility .............................................................................. 10

Other ways to Collect Data........................................................................10

Common Problems.......................................................................................... 11

Hardware Issues....................................................................................... 11

Link Down................................................................................................. 11

Can Not See Devices or Drives................................................................. 12

Out-of-Date Driver Version ........................................................................ 12

Out-of-Date Firmware Version...................................................................13

LP21000 Problems.......................................................................................... 13

LP21000 Checklist.................................................................................... 13

LP21000 Symptoms.................................................................................. 13

When to Contact Emulex.......................................................................................14

Emulex Web Site and Emulex Support..................................................................15

Emulex Support Page .....................................................................................15

Contact Emulex After Hours, Weekends or Holidays ................................17

LED Reference Information ................................................................................... 18

Firmware Operation and Port Activity LEDs ....................................................18

Common LED States.......................................................................................18

Normal Link Up ......................................................................................... 18

Link-Down or adapter waiting....................................................................19

Heart-beat indication................................................................................. 19

LP21000 LEDs.......................................................................................... 19

LED Table ................................................................................................. 20

Troubleshooting and Maintenance Manual for LightPulse Adapters Page iii

Overview

Be Informed, Stay Informed, Stay Current

In addition to staying informed and current, these steps are a great first defense when troubleshooting any Emulex adapter software and hardware issue.

• Read the documentation on the Emulex Web site. From the main Emulex Web site, click Support. Each adapter, driver, firmware and boot code has documentation posted to the Web site.

• Use the Emulex knowledgebase. From the main Emulex Web site, click Support, then click knowledgebase.

• Check for known issues. In addition to the Emulex Web site, the README.txt file included in the distribution kit contains information describing what is included, how to install it, how to get started, a change history and any known issues.

• Verify that the current adapter driver is installed. Check with your account vendor for the current supported version since it may vary from the Emulex current version.

• Check the status indicator light emitting diodes (LED)s. See “Common LED States” on page 18.

• Verify that the adapter firmware version is up-to-date per the original equipment manufacturer (OEM) or storage vendor. If necessary , load and update firmware using the Emulex Offline utility; (see the Offline Utility Manual for Emulex Utilities).

• Verify that you have current switch firmware.

• Verify that cluster software and other storage and third-party applications are up-to-date.

• Check with disk and tape vendors for known issues.

Driver Updates

To update your driver, download the latest Emulex driver and utilities from the Emulex Web site. From the main Emulex Web site, click Support, then click the link for your operating system (OS).

Drivers for Windows

• The drivers for Windows include support for Windows Server 2008, Windows Server 2003, Windows 2000 Server and Windows NT (No future versions of drivers for Windows 2000 Server or Windows NT are planned.)

• The driver kit down load includes the base driver and the HBAnyware utility.

• If you are run ning Windows Ser ver 2008, Windows Ser ver 2003 or Wind ows 2000 Server,

the driver kit download includes AutoPilot Installer update drivers and utilities and configure adapters, drivers and utilities.

, which enables you to quickly install or

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 1

Drivers for Linux

• The driver for Linux supports kernels for Open Source drivers.

• The driver d ownload includ es the base dr iver, and if it is not included in the applications kit,

the application helper module install script.

• The applic at i on s ki t i nc lu d e s t h e H BA n yw a re u ti l it y and the application helper module.

• The SMI-S Provider kit for Re dHat includes the SMI-S adapter Pro vider, compliant with the

SMI-S 1.0.2 specification and certified against the SNIA-CTP v1.0.2.23 suite.

• The SMI-S Provider kit for SLES-8 is pre-packaged with OpenPegasus CIMOM v2.3.2, as a

turnkey SMI-S agent for ease of installation and deployment.

Drivers for Solaris

• The lpfc driver for Solaris supports several versions of 32-bit and 64-bit OSs. The driver kit includes the lpfc base driver and the HBAnyware utility. Links to the Solaris’ SFS FCA drivers are provided as well.

Drivers for HP-UX

• The driver for HP-UX supports PA-RISC and the 64-bit Itanium processor family. The driver kit download includes the base driver and the lputil utility.

Drivers for Netware

• The driver for NetWare supports several versions of NetWare. The downloads include the base driver only (no utilities).

AutoPilot Installer

• If applicable, instructions for AutoPilot Installer are in the quick installation manual and the driver manual.

Firmware Updates

Download the latest Emulex firmware from the Emulex Web site. Click Support to go to the Support page, then click the link for your adapter model. A page appears with downloads for firmware, boot code, drivers and utilities. Download the firmware, and then use the HBAnyware utility to update the firmware

on the adapter.

Note: The LP21000 features an Intel 10Gb Media Access Control (MAC) device.

You must ensure that the correct driver version is loaded for the device to function properly. You must load the In tel driver separately from the other software drivers and packages. See the Intel Web site for Intel drivers.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 2

The Importance of Compatibility

Emulex works very closely with storage and system providers to ensure optimal performance of supported systems. As part of that work, both Emulex and the storage and system providers test specific hardware configurations and specific versions of firmware, drivers and othe r software for all the components of a storage area network (SAN). This testing ensures that each supported configuration performs as indicated and operates successfully in an enterprise installation. Compatibility matrices are available on the Emulex Web site (www .emulex.com/interop/index.jsp) that provide information on which configurations, firmware/software, servers, storage, adapters, switches and other components properly function together.

It is critical to verify that all the components of the SAN, including hardware and firmware/drivers/ software for each component are supported. Storage and system providers’ web sites are the best sources for this information.

Emulex Configuration Parameters

Each Emulex driver has a number of configuration parameters that allow the customization of the behavior of the driver. Because these parameters can significantly change the behavior of the Emulex product, great care must be taken in the setting or changing of the parameters. Storage and system providers put forth great effort in determining what particular configuration parameters must be set to in order to provide optimal storage area SAN performance.

Note: For best results, never change the driver configuration parameters from their

default values or the values specified by the storage and systems provide rs. Doing so can alter performance of the SAN and place the SAN in a configuration that is not supported, or cause it to operate improperly.

How Emulex Device Driver Configuration Parameters Work

Emulex configuration parameters are stored in the server into which the adapter is installed. The device driver acts upon these parameters. Emulex configuration parameters are NOT stored in the adapter flash memory. Unlike other SCSI and Fibre Channel (FC) adapters, Emulex does not store any configuration information such as timers, FC topology or link speed in the ad apter fla sh. When an Emulex adapter is installed into a server, there is no separate configuration step required for the adapter. Proper driver configuration is generally all that is required. The exception to this is if you boot from SAN. In this case, you must use the adapter BIOS utility to configure the boot device. The adapter m ust be told which FC device contains the boot volume, and that information is stored in the flash. These settings are only in effect at boot time. Once the operating system (OS) loads the device driver, most settings of the driver are used and the adapter BIOS no longer functions.

Emulex Utility Applications

Emulex provides a variety of utility applications that include diagnostic capabilities. Documentation is available at http://www.emulex.com/support/. These utilities are:

• The HBAnyware utility

• The stand-alone utilities

• DOSLpCfg

• Win LpCfg

• LinLpCfg

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 3

Troubleshooting

1. Observe the problem.

2. Isolate the problem.

Observing the Problem

Take a step back and think about what is being seen. Ask the following questions:

• If the SAN has been working, what has changed? Apart from a catastrophic hardware failure, SANs typically do not just stop working – something has changed. For example, a commonly overlooked SAN change is a switch firmware upgrade. Even something as seemingly innocuous as an upgrade to disk drive firmware in a RAID storage system can have un expecte d effects on a SAN.

• What is the observed behavior compared to the expected behavior? For example, a planned storage fail-over during system maintenance takes eight minutes to occur, but was expected to complete in two minutes. Observe behavior on two levels:

• The over all problem. F or example, users expe rienced an ou tage of eight minutes.

• The exact p roblem. For e xample , the storage co ntroller is r eporting an error.

• Is the expected behavior supported by storage an d system prov iders? For example, is a manually initiated path fail-over of two minutes supported by the storage and path management software providers? A storage provider can only support two-minute failovers when their internal Redundant Array of Independent Disks (RAID) controller cache is disabled.

• What are the exact symptoms? Make a list. Examples:

• Mouse po inter stopp ed moving for 30 seconds imm ediately after failover was initiated, then went to an hour glass until the failover completed.

• Path man age men t software reported errors in the system erro r log 60 seconds after the failover was initiated.

• Adapter FC link to switch dropped and did not recover immediately after failover was initiated.

• Is the problem repeatable? If yes, can it be repeated on a non-production test system? Collecting information such as system error logs is often needed. Since production syste ms ar e not generally set up to collect this information in normal operation, it is important to be able to configure the system to collect data and recreate the problem on a non-production test system.

• What do the LEDs on the adapter indicate? Check the adapter and switch LEDs to determine the status. If the LEDs stop flashing or flash an error code, this indicates the adapter may need to be returned to Emulex for repair. See “LED Reference Information” on page 18.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 4

Isolating the Problem

Once the problem is observed, determine where the problem originates. Eliminate problem sources on a coarse level, is the problem in the SAN or in the server? Determining this reduces the size and scope of the troubleshooting effort. If the SAN can be eliminated, only the server and its components need to be examined. This topic discusses common problems related to the SAN and the server.

SAN Components

SAN-related problems can include the switch, storage and cabling.

•Cables

• Bad cables. Noise generates many CRC or other invalid data on the FC link. This translates into slower performance on disks and failed backups on tapes.

• Old cables. 4Gb/s F C cannot run a s far as 2Gb/s FC with the same cable.

• Switch

• Imprope r zoning. This causes d evices to be improperly disco vered and th e SAN does not recover from disturbances.

• Link problems. T hese include wron g speed, wrong topo logy or no link at all.

• Storage

• Improperly configured storage systems. Emulex has determined that this is the number one cause of SAN performance problems. Examples of improperly configured stora ge systems include not enough spindles dedicated to the busiest Logical Units (LUNs) or too many hosts talking to a particular storage port.

Server Components

Server problems can include the adapter and its firmware, adapter device drivers, adapter management software, OS SCSI stack drivers and path management software.

• Adapter firmware

• If this is not qua lified by th e storage an d systems providers, un predictable behavio rs could result.

• Outdated firmware. Outdated firmware can ca use link proble ms (wrong spe ed, wrong topology or no link at all). Tape backups can fail when using the FCP-2 Sequence Level Recovery feature which is enabled automatically if the tape system supports it.

• Unsupported dual-channel adapters. Some dual-channel adapters are not supported by certain systems.

• Bad adapter. If the system stops responding, install the adapter in another server to determine if the problem is the adapter.

• Adapter device drivers

• Unqu alified drive r version. The ve rsion must b e qualified by the storage and system s providers. A version can be installed and supported by the storage or system provider, but provider compatibility specifies an older driver version is required for a particular configuration.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 5

• Incorrectly se t configuratio n parameters. If the driver parameters are set incorr ectly, erratic behavior can occur such as very long failure timeouts, storage being overrun by too many concurrent commands and failures in path management software that rely on specific Emulex driver settings.

• Incorr ect target bindin g. If persistent b in ding is not correct, devices that are expected to show in the OS storage management tool are not present.

• Adapter management software

• Manage ment software package does not match th e driver versio n. If the package does not match the driver version, erratic behavior can occur with the management standard HBA API, firmware downloads and functions of the HBAnyware utility. For best results, use the HBAnyware utility version packaged with the driver.

• OS SCSI stack drivers

• Wrong OS patch level. An incorrect level can effect system performance. OS providers often patch elements of the SCSI stack to correct problems and improve performance. OS providers generally maintain a knowledgebase, search there if you are experiencing a problem. OS patches can require coordinated changes in low-level device drivers from the adapter makers to work properly. For example, Emulex specifies (on the Emulex Web site and in Emulex documentation) what version of Microsoft's STORPORT.SYS is required for a particular version of Emulex's Storport Miniport driver, ELXSTOR.SYS.

• Path management software

• Unsup ported configuration. An un supported configuratio n or incorrect SAN zo ning an d/ or configuration can result in unexpected behavior.

• Unsup ported ada pter drive r. An unsupported driver can result in software that does not operate properly. Tight integration between software and drivers is critical. The correct version of the driver must be used for proper operation.

• Unsup ported ada pter drive r configuration. Unsu pporte d driver settin gs can result in failovers that take too long or can be triggered unnecessarily (these problems can be caused by incorrectly configured timers in the adapter driver). Correct setting s in the adapter driver specified by the path management software provider must be used.

Checklists

Connectivity Checklist

After setting up a new SAN or reconfiguring an existing SAN, connectivity problems occasionally occur. Storage volumes (devices) may not appear in the OS, although the storage management tool is ready for use. The devices could be marked offline or not present at all. Below is a checklist that can be used to help solve connectivity problems.

• Check for a good FC link. Verify that there is a good link at the desired speed. Most Emulex adapters have LED lights that provide the link status and current speed of each port. The LED states for each adapter are documented in each adapter's User Manual and are also av ailable at: www.emulex.com/ts/index.jsp If the LEDs are not visible, use the HBAnyware utility to show link status or use the remote port's utility to examine the link state. For example, if the Emulex adapter is connected to a switch port, use the switch's utility tools to see the link state.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 6

• Check switch zoning. Large switched fabrics are complicated to set up and maintain. Zoning is a key element of a good configuration, and it is one of the most complicated to ma nage. An improperly zoned fabric can cause a variety of problems that are not always directly attributable to zoning. For example, a small disturbance on a large fabric with no zoning can cause connectivity problems because all the devices on the fabric will be trying to resolve the disturbance at the same time which overloads the switch. It is important to verify that the device that is not being discovered is properly zoned.

• Verify device discovery. Use the HBAnyware utility to examine devices discovered by the adapter and driver. In nearly all OSs, the device driver for the FC adapter is responsible for discovering all the devices in a SAN. These devices must then be assigned SCSI T arget IDs and mapped to the OS. The HBAnyware utility provides the unique ability to see all the devices on the SAN that the device driver discovered, regardless of whether the devices where mapped to the OS. This provides a way to determine if the device that is having problems is visible on the SAN.

• Check that the Automap driver parameter is ON or there are valid persistent binding entries. In order for the OS to present, all of the expected devices in its storage management tool must be mapped to SCSI Target IDs so the OS can communicate with the devices. As previously mentioned, the device driver performs the mapping function that assigns a remote FC device to a SCSI Target ID so the OS can communicate with the device. Nearly all OSs do not know how to directly communicate with a FC device. They must use the SCSI Target ID to communicate across the SAN. Emulex device drivers have several options on how this mapping is performed.

• Emulex de vice drivers can be configured to automatically m ap the FC de vices to SCSI Target IDs.

• Emulex de vice drivers can be configured to map certain FC devices to corre spondin g SCSI Target IDs to maintain a known configuration of SCSI Target devices. This manual mapping process is done using the HBAnyware utility and typically links a FC World Wide Name value to a SCSI Target ID value. This is called persistent binding.

The device driver must have correct persistent binding information or the Automap configuration parameter must be enabled to allow the device driver to map the devices automatically.

• Check storage configuration. When storage devices are presented in the OS storage management tool, they are typically described as volumes or disks. The adapter device drive r maps discovered storage ports on a SAN as SCSI Targets and the Logical Units (LUNs) are discovered and managed by the OS, not the adapter driver. If LUNs are configured on a particular storage controller port and the port is not discovered by the adapter, all the LUNs associated with that port will not be available for use.

Many storage systems provide security by requiring that each storage controller port be configured with the host server World Wide Names that will communicate with that storage port. The storage controller will not allow a server with unknown World Wide Names to communicate with LUNs on the storage system. Further, some storage systems allow LUNs to be allocated to particular servers based on World Wide Name. If the storage controller with this security and LUN mapping feature is not properly configured, the controller will allow the server adapter port to login and communicate with the storage controller, however, the storage controller will not present any LUNs to the server. This means that the adapter driver will complete its discovery and map the Targets to the OS, but the OS will not find LUNs with which to communicate.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 7

Installation Checklist

This checklist covers installation of drivers and utilities. In addition to this checklist, verify with your storage and system providers for any specific instruction they may provide. Some OEM vendors will repackage drivers and utilities.

• Verify recommended driver, firmware, and boot versions from your storage and system providers. Many storage and system providers qualify specific adapter models, driver, and firmware releases. Using qualified storage and system provider solutions will ensure you have a supported SAN configuration.

• Verify releases for storage and system providers that have a page on the Emulex Web site. If the storage or system provider is not listed, contact them to verify their preferred driver releases. If the storage and system provider does not have preference, Emulex recommends the latest driver and firmware releases. Latest releases ensure the latest enhancem ents and bug fixes. For latest driver and utilities, refer to the Emulex Web site.

• Review the prerequisites and compatibility sections in the driver user manual. This will ensure that you meet the minimal requirements for each driver you are installing. The manual lists supported firmware versions and OS releases. It may contain other information regarding tested SAN environments.

• Follow the installation instructions in the Emulex driver and utilities manuals. The manual has step-by-step instructions that can be followed for easy installation of the driver. It is available at the above URL and on the same page as the driver download. Utilities are packaged with the driver kits or listed separately on the driver page. Use the same utilities bundled with the driver kit. Running older utilities with newer drivers can render some utility features inoperative.

Upgrade Checklist

• Upgrade adapter firmware. Old adapter firmware run ning with a newer drive r can produce unexpected behavior from the adapter. Always use the version recommended by your storage or system provider. Use the latest released version if they have no preference as stated above. In either case, make sure the version you intend to use is the same or newer than the minimum recommended version specified in the driver manual (see the Prerequisites and Compatibility section). Whenever possible, ensure the adapter firmware has been updated to the appropriate version prior to updating drivers.

• Upgrade adapter boot code. Adapter boot code allows the ada pter to act as a ho st boot path device. It is not required if the OS is loaded from an internal hard disk. The exception to this is the Emulex SBus adapters, as they do require boot code. Consult with your storage and system providers to see if they require boot code installed in the adapter and if so, which ve rsion. Boot code manuals are located on the same page as the firmware images. Use the same utilities to upgrade adapter boot code as you would to upgrade adapter firmware.

• Upgrade the driver and utilities. Utilities are packaged with the driver kits or listed separately on the driver page. It is important when upgrading a driver, upgrade the utilities associated with it as well. Running older utilities with newer drivers can render some utility features inopera tive. Refer to the driver manual for specific upgrade procedures of the driver and utilities. So me utilities and their manuals must be downloaded and installed separately from the driver install. Check the driver download page for the specifics of each. They may be different for each OS.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 8

Configuration Checklist

• Verify recommended driver, firmware, and boot versions from your storage and system provider. Many providers qualify specific adapter models, driver and firmware releases that they

support. Using qualified provider solutions will ensure you have a supported SAN configuration.

• Verify releases for storage and system providers that have a page on the Emulex Web site. If the provider is not listed, verify any preferred driver releases. If the provide r does not have a preference, Emulex recommends the latest driver and firmware releases. Latest releases ensure the latest enhancements and bug fixes.

• Review the Prerequisites and Compatibility sections in the driver manual. This ensures that you meet the minimal requirements for each installed driver. The driver manual lists supported firmware versions and OS releases. The driver manual may also contain other information regarding tested SAN environments.

• Verify preferred settings with your storage and system providers. Many providers require particular Emulex driver settings for communicating with their storage. Verify with your providers any configuration changes they recommend. When performing configuration changes to the drivers and utilities, be sure to have the providers verify that the change will not affect communications to your storage devices.

• Review procedures and settings in the driver and utilities manuals. Each OS has its own system of configuring drivers. In some cases, the tools to configure the Emulex driver are the same between OSs. On some OSs, only manual configuration through text files is available. Review the manual to ensure familiarity with the process, prior to making configuration changes.

• Emulex utilities have few configurations that can be changed. See the driver and utilities manuals for configuration changes available for the utility you are using.

• Driver parameters can be different between OSs. Be familiar with the parameters to make the configuration process easier, especially if multiple OSs are used.

• Some settings are dynam ic and take effect immediately, while others require the OS to be rebooted. Note which settings you are changing and when a reboot is necessary.

• Assume that you need server downtime to perform changes. Changing parameters on a production machine is not recommended unless downtime has been schedu led.

• Verify driver settings are set correctly. Defaults are optimized for normal operation and Emulex recommends default settings in most cases. By default, all Emulex drivers perform automatic mapping of FC nodes to target IDs (SCSI IDs). If SAN changes occur, the target mapping may not stay consistent for each FC node between reboots of the OS. Use Persistent binding or target mapping to insure target IDs do not change between server reboots.

Note: For SAN disk drives (storage arrays) on Windows, persistent binding is not

necessary. Windows uses the drive signature on each drive to determine the correct drive letter. Persistent binding is necessary for SAN tape devices on Windows.

• Configure the driver using instructions in the driver and utilities manuals. Manual links are available on the same page as the driver download. The HBAnyware utility manual has parameter lists with minimum and maximum values for each setting. If a parameter can be changed with tools other than the HBAnyware utility, p arameter information is in the HBAnyware utility and the driver manuals. Review this parameter list before you perform changes to insure values are not out of range or invalid. If changes require a reboot, sche dule downtime fo r performing the changes.

Caution:

Do not make configuration changes to a production server unless downtime has been scheduled. These changes can cause issues with access to the SAN storage, thereby causing unexpected server downtime.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 9

Tools to Collect Data

Emulex provides tools to collect data about a problem or behavior with Emulex products. Sometimes, more importantly, these tools provide information about general SAN operation, even if the problem appears to be outside of Emulex's domain.

Driver Tools

All Emulex OS device drivers can log additional events beyond critical errors. This can be useful in troubleshooting. By default, Emulex drivers log only severe err ors, but can be configured to log more events. Examples of events: FC timer expiration, RSCN reception, SCSI errors (such as Check Condition errors, Busy or Queue Full), and FC errors (to detect bad cables or tran sceivers). Emulex considers this type of event logging a powerful troubleshooting tool.

Note: Drivers for Linux only: Emulex's provides a "grab" tool to collect pertinent adapter,

driver and system information for processing by Emulex and stores it in a tarball file. This tarball file is available for download from the Support section of the Emule x Web site.

The HBAnyware Utility

• HBAnyware GUI interface. The HBAnyware utility can show all targets and LUNs an adapter port has detected on the SAN. Problems such as fabric zoning and array configuration can be detected. Detect persistent binding problems: if the target shows in the HBAnyware utility and is not seen in the OS storage management tool, the persistent binding entries should be checked.

• HBAnyware port statistics. All FC ports contain link quality data (Link Error Status Block). This structure counts events such as link transitions, CRC errors received, invalid transmission words received and other statistics. These counters can detect bad cables or ports or misbehaving devices. The HBAnyware utility can display these counters for selected Emulex adapter ports.

• Version Check ing. Ensure that all versions of Emulex adapters, firmware, drivers, and software are correct. This is critical to ensuring a stable SAN. The HBAnyware utility can collect and display this data all in one place.

• Diagnostic Dump. If necessary, use the HBAnyware utility to keep a record of all the attributes of a server's adapter installation The HBAnyware Diagnostic Dump feature collects attributes such as adapter models, firmware, driver, and software versions, driver configuration parameters and stores it in a standard text file. The dump stops the selected adapter port and dumps the adapter memory to a file for analysis by Emulex. This is not often required, but can be useful when it is needed. Emulex Technical Support can advise when the adapter memory dump file is required.

• Diagnostic tests. Use the HBAnyware utility to test the adapter port hardware and firm ware. These tests are similar to the tests used in Emulex manufacturing and can help identify malfunctioning adapter hardware.

Other ways to Collect Data

Other SAN component manufactures provide tools to collect additional data. For example most fabric switches provide ways to dump the port and switch configurations which can provide the switch firmware revision, detailed information about zones, the display the port error counters for each switch port.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 10

Common Problems

Hardware Issues

Symptoms that indicate you may need to return the adapter to Emulex for repair:

• Host system (server) does not pass power-on self test (POST).

• The server does not boot.

• LEDs on the adapter stop flashing or flash an error code.

• The bus has incorrect power.

• Hardware errors are logged in the event log or message file.

• There are onboard parity errors.

• There are Peripheral Component Interconnect (PCI) parity errors.

• There are firmware “traps”.

• A physical FC interface problem looks similar to a bad cable but follows the adapter.

• A high error count is reported on the Statistics tab of the HBAnyware utility. Errors include cyclic redundancy check (CRC) errors, frames out of order, bad frames (a bad cable can cause problems of this type) or bad adapter optics.

See page 16 for information on returning your adapter for repair.

Link Down

Symptoms that indicate a link down:

• Firmware operation and port activity indicate that the link is down. LEDs are visible through openings in the adapter’s mounting bracket and indicate the conditions and results of the POST function. A link-down is indicated when the green LED flashes and the yellow LED is off (see Table 1 on page 20).

• If the adapter has a power LED, the power LED indicates that power is off. Some adapter models have a separate green 3.3V power indicator LED (see page 19).

• Topology and link speed are improperly matched for the adapter and the devices connected to it. If there is a mismatch, the link does not come up. If you make change s to the topology or link speed parameters, you may need to reboot the system.

• The boot code is improperly loaded, en able d o r co nf igur ed . If boot code is enabled, the link is reset any time the driver is loaded. If the boot code is not enabled, the link does not come up. If boot code brings the link up, the link goes down when the adapter is reset.

• Link status is other than operational. Use the HBAnyware utility to determine the link state. Click on the General tab and look at the Link Status field. If the status is other than operational, the link is down.

• The cable is not working. Use a cable from a known working connection to test. Isolate the problem to the adapter, cable or connected device.

• If you have a loopback connector, the adapter fails the external loopback tests. If you have isolated the problem to the adapter, run an external loopback tests using the HBAnyware utility or the Offline Utility (see the Offline Utility Manual for Emulex Utilities).

• The adapter optics do not match the cabling. If the adapte r is one type (single mo de or multimode) and your cabling is the other type, the link will not come up.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 11

Can Not See Devices or Drives

Possible reasons why you can not see devices or drives:

• Some devices are not powered on. Some adapter have a separate green 3.3V power indicator LED (see page 19).

• The LEDs indicate that the link is down. On an adapter, a link-down is indicated when the green LED flashes and the yellow LED is off (see Table 1 on page 20). On a switch, the LED states vary depending on the switch vendor. See the switch vendor’s documentation.

• The cable is not working. Use a cable from a known working connection to test. Isolate the problem to the adapter, cable or connected device.

• The driver is improperly installed or loaded. Verify that a compatible driver is completely installed and loaded. Some devices may require a specific device driver installed on the host.

• Automap is turned off. The first time the machine boots with automap off, the devices are not mapped to the OS. See the HBAnyware Utility User Manual for more information where applicable.

• On the FC Port d river, automap is off by default. Enable it using the HBAnyware utility.

• On th e Storport Miniport and SCSIport Miniport drivers, automap is on by default. Change it

with the HBAnyware utility.

• On the driver fo r Sola ris, autom ap is on by defau lt. Change it using the HBAnyware u tility.

• On the driver fo r Linux, lpfc_automap is on by default. Chan ge it using the HBAnywa re

utility.

• On the driver fo r HP-UX, automa p is on by default an d automatically assigns SCSI IDs by

World Wide Port Name (WWPN). Three parameters (scan-down, fc-bind and automap) control how the Emulex driver maps FC devices. Change parameters at the command line. See the driver user manual for more information.

• The targets are not discovered. Use the HBAnyware utility to determine discovered targets.

• To find the number of targets, click on Discovered Elements in the discovery-tree.

• To view information specific to a target, sort by host name and click on the Target Attributes tab.

• The switch is configured incorrectly. The adapter must be in the proper zone to see the target device.

• The array is configured incorrectly. Check settings in the RAID system security settings. Verify that the array knows the adapter world wide node name (WWNN).

Out-of-Date Driver Version

These symptoms indicate you may need to update your driver. For more information, see page 1.

• Target devices appear incorrectly in device manager.

• The Emulex driver operated correctly before cluster services were installed, but now that the server has rebooted, the cluster will not start or there are connection time-outs. Verify

with the software vendor that you have the latest versions of software and that you are running Emulex drivers and firmware that are qualified for that software.

• Problem with third-party management software. Verify with the software vendor that you have the latest versions of software and that you are running Emulex drivers and firmware qualified for that software.

• An operating system error (blue screen) occurs.

• Tape operation problems.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 12

Out-of-Date Firmware Version

Symptoms that indicate you may need to update your firmware. For more information, see page 2.

• The FC link fails to come up or stay up.

• Mailbox time-out errors in the OS event log or the UNIX message file.

• One or more LEDs have stopped flashing.

• The system stops responding.

• The system has lower than expected performance.

LP21000 Problems

LP21000 Checklist

Check CEE Switch Configuration. Typical CEE environments will install two Converged Ethernet

Adapters in the hosts. Verify that the 10 Gigabit link is operational at the host and at the 10 Gigabit port on the DCE switch. Check FCoE parameters on the DCE switch and ensure no changes were made to the original settings. The HBAnyware utility (4.0 or later) allo ws you to ch eck CEE config uration parameters on a CNA such as firmware revision, pause type, FCoE priority, and so forth.

Check Switch Zoning. If you can see the LP21000 WWNN in the FC switch name server, it can be zoned to storage. All zoning at this point is similar and can be man aged the sa me as an ad apter. Verify driver, firmware, and boot versions from your storage and system provider.

LP21000 Symptoms

Symptoms such as POST failures and PCI problems for the LP21000 are usually simi lar to Emulex adapters. Exceptions are problems that occur with incompatible Convergence Enhanced Ethernet (CEE)/bridge and firmware for the LP21000 family. Ensure the correct firmware versions are installed. Also ensure that you have the correct firmware version installed for the Media Access Control MAC

device.

Note: The LP21000 features an Intel 10Gb MAC device. Ensure that the correct

driver version is loaded for the device. You must load the Intel driver separately from the other software drivers and packages. See the Intel Web site for Intel drivers.

• 10Gb Link down. Unlike FC adapters, the Converged Network Adapter (CNA) is connected to a lossless Ethernet fabric via 10 gigabit Enhanced Ethernet. The CNA has an additional green LED for the 10GbE link and indicates proper connection to the Data Center Ethernet switch. For normal operation, this LED will blink. When this LED is off, a problem exists between the CNA and FCoE switch.

• FC Link Down. If the 10GbE connection to the FCoE switch is lost or the DCE switch is not properly configured, both 10GbE and the green FC LED will blink. The yellow LED will be off.

• Can’t See Devices or Drives. If you determine that the 10 GbE link is operational, check all FC connections including ISL links between the FCoE switch and the FC switch and between the FC switch and the storage array. If storage devices are missing, the CNA FC green LED may not indicate this condition and the 10 GbE LED may continue to show a good link between CNA and FCoE switch. Check zoning on the FC switch. Verify that the Data Center Ethernet is properly set up for FCoE traffic.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 13

When to Contact Emulex

Contact Emulex if you still experience problems after you:

• Review the checklists in this manual and follow any suggestions for basic troubleshooting, such as updating driver or firmware.

• Verify the environment is supported.

If the adapter is a standard Emulex model, use the online knowledgebase located on the support section of www.emulex.com for resolutions to the problem. The knowledgebase is available 24/7.

If the knowledgebase does not provide an adequate solution, contact Technical Support. When contacting Technical Support, the information you need to have available will vary according to the type of problem. However, for all OSs you need the following information:

• OS version and patch level.

• Adapter models in use.

• Firmware version installed on each adapter.

• FCODE/OpenBoot/BootBIOS version installed on the adapter.

• Driver version and the HBAnyware utility version installed on each affected host.

• Listings or screen shots of any errors that are related to the issue.

• Any changes or updates made to the SAN, OS, adapter firmware or driver lately.

• A SAN map, if available.

• The vendor make and model of storage in use.

In addition, the following information is needed for these specific OSs:

• For Linux (at a minimum)

• A copy of /va r/log/me ssages and an y logs the Emu lex installer scripts provide.

• If possible: the tarball output of the Emule x System Gra b Diagnostic Tool found on the Linux support section of the Emulex Web site.

• For Solaris with the lpfc driver

• A copy of /va r/adm/messages.

• A copy of /kernel/drv/lpfc.conf.

• A copy of /kernel/drv/st.conf (for tape).

• A copy of /ke rnel/drv/sd.conf (for disk).

• For Solaris with the SFS FCA driver:

• The versions of the SFS packages or patches installed.

• A copy of /kernel/drv/eml xs.conf.

• For Windows:

• A copy of th e System Event log.

• For the HBAnyware utility, a copy of the Applications Event log.

•For HP-UX:

• A copy of th e output of comma nd 'ioscan -fkn'.

• For Netware:

• A captur e of all messages when the se rver is started (if possible).

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 14

Emulex Web Site and Emulex Support

Emulex Support Page

The Emulex Support page contains links to the following:

• Storage and system supplier-qualified and supported adapters

• Emulex released driver, downloads and documentation

• Contact support (OEM and Emulex e-mail and phone information)

• E-mail support (Technical Support Request Form)

• Emulex knowledgebase

• Online support

• Product registration

• Return product information

• Warranty information

• Legacy Product documentation (adapters, switches and hubs)

• SR-1215/BR-2401 Support (Emulex bridges and Emulex routers)

• Interoperability/Compatibility (links to compatible products)

The Emulex Support page is part of the Emulex Web site at www.emulex.com.

• Storage and System Supplier- Qualified and Suppor ted adapters . Most suppliers market and sell Emulex products under their own respective brands and qualify Emulex drivers, software and documentation for these products. The Support page provides qualified drivers, software and documentation as well as support information for suppliers of Emulex products.

• Emulex Released Driver Downloads and Documentation.The Emulex Support page contains downloads and documentation for several driver types, boot code and firmware for each Emulex adapter model.

• Contact support (OEM and Emulex). This page contains OEM Web site and phone information, Emulex SAN switch EMEA e-mail information and Emulex adapters and embedded switches phone and e-mail information.

• E-mail support (Technical Support Request Form). E-mail support does not require a password, but does require a serial number for the supported product.

• Emulex Knowledgebase. The searchable Emulex knowledgebase is an on-g oing effort to provide you with answers to questions regarding all facets of Emulex products. Information is provided for older (legacy) products as well as for current products. Knowledgebase topics are stored by knowledgebase solution ID (solution number). A link to the Emulex knowledgebase is available from the Support page. Search for information using the knowledgebase:

• By a word o r phrase: Con duct a gener al search b y entering one wo rd (such as “Storport”) in the Keywords field or by entering multiple words and choosing Any as the match method or conduct a specific se arch by enter ing multiple words and choosing All as the match method.

• By the solution ID: Ente r a 1-5 digit solutio n ID in the Solution ID field (you must know the solution ID to conduct this search).

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 15

• Online Support. This password protected site allows you to open new cases and ch eck the status of existing cases by clicking the online support link on the Emulex Support page. Report problems, documentation issues, web site navigation difficulties or status of an RMA. Request general information, recommend enhancements or escalate a problem. The online support system is available on all non-holiday work days, Monday through Friday, 7:00 AM to 4:00 PM (Pacific Time). If you have after-hours holiday or weekend issues, telephone Emulex Technical Support.

• Product Registration. Use this page to register your product online. You need the following information to register your product:

• Custom er name

• E-mail add ress

• Fax number

• Company name

• Contact name

• Addre ss (street, city, state and zip code)

• Produ ct model o r part number

• Produ ct serial n umber

• Purcha se date

• Product Repair and Return. Whether you contact Emulex through the Emulex Web site, by email or by telephone, gather the following informatio n befo re you contact Emulex:

• Bill-to an d ship-to company and a ddress

• Model num ber

• Part nu mber requ ired on all products to be returned

• Serial num ber required on all products to be returne d

• Purcha se date requ ired for a ll cLAN products

• Description of failure

Note: Model and serial numbers are located on bar code labels on the produ ct

itself. Record the information from the bar code label and not the packaging.

Get an RMA for 10 Products or Fewer. To return your product to Emulex for repair or replacement, you must obtain authorization using the RMA request form for up to 10 units. Return products one of these ways:

RMA form. Use this online Return Material Authorization form to return up to 10 units. E-mail service. Send an e-mail message to the Emulex service department. Check on the Status of an RMA for 10 Products or Fewer. Call the Emulex Support Services

Repair Hotline to check on the status of an existing RMA: In the Americas, telephone: 800-752-9068, select option 2 and the n optio n 3 In Europe, telephone: (44) 1189-772929 In Asia, telephone (call the Emulex 800 number, Monday through Friday, between 7:00 AM to

4:00 PM (Pacific Time) In China, Philippines: 00 + 1+ 800-752-9068, select option 2 and then optio n 3 In Japan, South Korea: 001 + 1 + 800-752-9068, select option 2 and then option 3 In Taiwan: 002 + 1 + 800-752-9068, select option 2 and then option 3

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 16

Return More than 10 Products or Check RMA Status. E-mail or phone to check repair status

or return more than 10 products.

Contact Emulex AfterHours, Weekends or Holidays

Phone if you have after-hours, holiday or weekend issues.

The Americas and in Asia:

• E-mail address:

tech.support@emulex.com

• Telephone: 800-752-9068, option 2.

Europe:

• E-mail address:

tech.support@emulex.com

• Telephone: (44) 1189-772929

• Warranty Information. A printable .PDF of the Emulex standard limited warranty.

• Legacy Product documentation. This page contains downloads and documentation for legacy products, including Emulex adapters, switches and hubs.

• SR-1215/BR-2401 Support (Emulex bridges and Emulex rou ters). This password protected site allows you to access Technical Support specifically for bridges and routers.

• Interoperability/Compatibility (links to compatible products). Select from the many companies that have tested and approved Emulex FC adapters and CNAs. Filter information by manufacturer model, adapter or CNA model, data rate, bus type or OS.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 17

LED Reference Information

Firmware Operation and Port Activity LEDs

Emulex adapters have a POST test. Each port has a set of green and yellow LEDs (visible through openings in the adapter's mounting bracket) that indicate the conditions and results of the POST.

• Green = firmware operation

• Yellow = port activity

Adapter LEDs also identify possible problems. For more information on LED states, see Table 1 on page 20.

LEDs can help you locate:

• Bad cables

• Bad transceivers

• Bad switches or hub ports

LEDs can help you determine whether:

• The switch or hub is on

• The driver is loaded

• The driver reset the adapter

Common LED States

Although the adapter LED has many possible states, the three most common are:

• Normal, that is, the link is up

• Link-down or adapter waiting

• Heart-beat indication

Normal Link Up

A normal link up means that the driver is loaded and talking to the switch. Everything is operational. 1 Gb/s link up:

• For 1Gb/s adapters, a normal link up state is indicated with a solid green LED and a slow blinking (1 Hz) yellow LED.

• For 2 Gb/s adapters, a solid green LED and a slow blinking yellow LED.

• For 4 Gb/s adapters, a solid green LED and a single blinking yellow LED.

2 Gb/s link up:

• For 2 Gb/s adapters, a solid green LED and a fast blinking yellow LED.

• For 4 Gb/s adapters, a solid green LED and a double blinking yellow LED.

• For 8 Gb/s adapters, a solid green LED and a fast double blinking yellow LED.

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 18

4 Gb/s link up:

• For 4 Gb/s adapters, a solid Green LED and triple blinking (4 Hz) yellow LED.

• For 8 Gb/s adapters, a solid green LED and a fast triple blinking yellow LED.

8 Gb/s link up:

• For 8 Gb/s adapters, a solid Green LED and fast quadruple blinking (4 Hz) yellow LED.

Link-Down or adapter waiting

• The difference between slow and fast blinking LEDs is obvious. A slow green flashing LED with the yellow off can means either:

• The ada pter just powered on and is waiting for the driver to give it instructions.

• The link is do wn. If the lin k was up and the cable was pulled or switch turned off, then the adapter will return to this state.

Heart-beat indication

• If adapter is ope rational, a t least one of the LEDs will blink at som e rate .

• No valid state if the yellow LED is solid or off.

Power Indicator LEDs

Some adapter models also have a separate green 3.3V power indicator LED.

• Adapter models LP9402DC, LP1050, LP1050DC, LP10000 and LP100 00 have an LED located on the upper back corner of the adapter (not at the mounting bracket). If it is on, the adapter has power.

• Adapter model LP9002C has an additional LED on the mounting bracket labeled, “Power”. If this LED is lighted, the cPCI bus slot is supplying 3.3V power to the adapter.

LP21000 LEDs

The LP21000 uses an LED scheme that supports Emulex CNAs.

• Green (Ethernet Link)

ON solid = Link up ON/OFF intermittent = Activity OFF = No Link

• Yellow (Adapter status; standard Emulex adapter)

• Green (Adapter status; standard Emulex adapter)

Troubleshooting and Maintenance Manual for LightPulse Adapters Page 19

LED Table

Note: All states except link rate apply to both HBAs and CNAs. The 4 Gb/s link rate is

applicable to both HBAs and CNAs. The 2 Gb/s and 8 Gb/s link speed apply to HBAs only.

Table 1: LED Reference Table

Yellow LED Green LED State

Off Off Wake-up failure (dead board) On Off POST failure (dead board) Slow blink Off Wake-up failure monitor Fast blink Off POST failure Flashing Off POST process in progress Off On Failure while functioning On On Failure while functioning 8-Gb/s adapters:

2 blinks 3 blinks 4 blinks

On On On

2-Gb/s link rate - normal, link is up 4-Gb/s link rate - normal, link is up 8-Gb/s link rate - normal, link is up

4-Gb/s adapters: 1 blink 2 blinks 3 blinks

2-Gb/s adapters: Slow blink Fast blink

Off Slow blink Normal - link down or not started Slow blink Slow blink Offline for download Fast blink Slow blink Restricted offline mode (waiting for restart) Flashing Slow blink Restricted offline mode (test active)