Installation and Configuration .......................................................................................................................8
Figure 13 Port Properties............................................................................................................................15
Figure 14 Disable the IDE Devices, HS20 and HS40.................................................................................16
Figure 30 linux nofb.....................................................................................................................................26
Figure 31 Unmount the Diskette Image......................................................................................................26
Figure 32 Boot Failure for RH 3.0...............................................................................................................28
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
5
IBM BladeCenter Deployment Guide
1 Introduction
1.1 Overview
The IBM BladeCenter Deployment Guide is intended to present the general steps necessary to
successfully deploy IBM blades in the IBM®
steps required for setting up remote boot on IBM Blades. The information provided in this document is
based on technical observations.
Due to the breadth of content related to this topic, this guide has now been split into 3 parts.
1. Management Module – information specific to the initial setup and ongoing usage of the
Management Module.
2. Embedded Switches – information related to the configuration of the embedded Ethernet and
Fibre channel switches.
3. Blades – blade configuration and installation information with a primary focus of installing Linux in
a boot to SAN environment.
Remote boot, root boot, or boot from SAN, is the name referred to the server configuration where the
server operating system is installed on a logical drive (LUN) that does not reside inside the server
chassis. This document will describe in detail the process that one must go through to setup a remote
boot for HS20 or HS40 blade servers. The blade servers utilize the IBM BladeCenter Fibre Channel
Expansion Card connected to the BladeCenter Fibre Channel Switch Modules via the BladeCenter
backplane. The switch modules in turn are connected to the core fibre channel switches that host the
SAN Fabric.
This document is a supplement the IBM®
BladeCenter products. In addition to the publications being provided with the BladeCenter products, they
also are available in Portable Document Format (PDF). The latest versions of the publications are
available from the IBM web site at
http://www.ibm.com/pc/support/.
BladeCenter™. Its primary focus is to detail the
BladeCenter™ publications provided with the
BladeCenter publications include the following:
• IBM
planning information for the IBM BladeCenter products. It describes the BladeCenter components,
explains deployment and installation considerations, and provides worksheets that you can use to
determine the configuration, power, weight, and cabling requirements for your BladeCenter unit.
• IBM
information about your BladeCenter unit, including information about features, how to configure your
BladeCenter unit, and how to get help. It is provided with the BladeCenter unit and is on the IBM
BladeCenter Documentation CD.
BladeCenter Type 8677 Planning and Installation Guide: This manual provides physical
BladeCenter Type 8677 Installation and User’s Guide. This document contains general
• IBM BladeCenter Management Module User Guide: This guide contains information about
configuring the management module and managing components installed in BladeCenter chassis.
• IBM xSeries and BladeCenter Server Management (SG24-6495): This document covers the
hardware side of the IBM systems management solution including the BladeCenter management
module. It is available for download from the IBM Web site at http://www.ibm.com/redbooks/.
1.2 BladeCenter Subsystems
The various BladeCenter Subsystems include:
• Server Blades - provide the processors, memory, hard disk(s) and firmware of an industry standard
server. In addition, they will normally incorporate keyboard, video, and mouse interfaces, and an
onboard service processor.
• Management Modules – incorporate the chassis-level management processor and
Keyboard/Video/Mouse (KVM) access for the blades.
• Switch Modules - provide multiple high-speed Ethernet ports in Switch Module slots 1 and 2 for use
by all of the Server Blades to interconnect internally as well as to external network devices. Switch
Module slots 3 and 4 are used for Fibre Channel or other switch types and connect to the Server
Blades as well as external devices.
• Power - Four modular power supplies provide primary and backup power to all BladeCenter
subsystems. Power supplies 1 and 2 are responsible for powering blade 1 through 6 plus all of the
BladeCenter components such as the switches, management modules, media tray, etc, and power
supplies 3 and 4 are responsible for power blades 7 through 14.
• Cooling - Dual, modular blowers provide forced-air cooling of BladeCenter components and
subsystems.
• Media Tray - Removable media subsystem that is shared by all blades. Includes CD/DVD and floppy
media.
Figure 1 BladeCenter Chassis – Rear View
1.3 Notes
This is currently a living document and will continue to grow as we have more experience with IBM Blades
in the boot-to-SAN environment. This document is intended as a personal productivity tool. It is not
intended to be comprehensive, and is provided for guidance only, on an 'as is basis' without warranty of
any kind. Please be aware that its contents have not been certified by IBM.
If you are installing Linux, please pay close attention to the issues listed in Section 4.5.
1.3.1 Contributors
We would like to acknowledge the contributions of the following people:
1. Richard Gebhardt, IBM Lab Services – Appendix E
Diskette Image
2. Fred Rabert, Nortel Networks – Assistance with Section on Nortel switches plus BroadCom Driver
installation instructions.
, portions of Section 4 Installation Process for Red Hat Linux.
Modify or Create the Post-install Driver
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
7
IBM BladeCenter Deployment Guide
Installation and Configuration
2 Preparatory Steps
The BladeCenter Chassis components should be prepared by updating the firmware and assigning the
appropriate IP addresses to the management and switch modules. The IP addresses of the various
modules as well as the userids and passwords for each will be needed during the installation. A sample
worksheet can be found in
The operating systems that are supported in BladeCenter boot from SAN configurations are Linux
RedHat Advanced Server 2.1, 3.0 and 4.0; Microsoft Windows 2000 (W2K) Advanced Server, Microsoft
Windows 2000 Server, Microsoft Windows 2003 (WS03) Enterprise Edition, Web Edition and Server.
Note: Since the storage subsystem provides multiple paths to the server, multiple views
of the same logical disk will be presented. Multipathing software controls these multiple
views by interposing between the operating system and the hardware. Since the
multipathing software is not available for the initial boot, only one path may be presented
to the server for the initial boot. This process is described in detail later in this document.
2.1 Network Configuration
The two switch modules provide redundant paths to each of the server blades for reaching other server
blades within the same chassis or for communication to external devices via the network infrastructure.
Figure 2 BladeCenter internal network) The Ethernet switch modules can be managed via telnet or a
(
Web interface. Both the Web and telnet interfaces can be started by accessing the switch directly or
starting a session from the management module’s Web interface.
Note: The external ports on the switch must be enabled, and enabled for management as well, to
manage the switch directly without going through the management module. Because the management
interface is on VLAN 4095 and cannot be moved, a separate interface from the management interface
must be created with an IP address valid on the production network to manage the switch directly.
Note: For switch communication through the management module external Ethernet port, the switch
module internal network interface and the management module internal and external interfaces must be
on the same subnet.
2.2 BladeCenter Management Module
To use the Management Module to perform tasks in the coming sections remotely, first do the following:
Open a browser and input the IP address of the MM. The default IP address of the external port of the
management module is 192.168.70.125, the default subnet mask is 255.255.255.0, and the default
hostname is MMxxxxxxxxxxxx where xxxxxxxxxxxx is the burned-in MAC address. It is possible to
connect directly into the management module by using a crossover cable between a laptop and the
Ethernet port of the management module. Change the ip address of the laptop to an address on the same
subnet as the management module. 192.168.70.124 is a good choice.
• The logon screen will display, and you will be prompted to log onto the MM web server. The default
user ID and password are USERID and PASSW0RD (0 is a zero). These are case sensitive.
• Once you enter your user ID and password, you will presented with a welcome window that displays
where you are connected and, for security purposes, where you set the duration that your connection
can be idle before being automatically disconnected.
• Click <Continue>. The BladeCenter Management and Configuration window opens. From the
Management and Configuration program main menu, select settings that you want to view or change.
2.3 Remote Control Hints and Tips
Note: All of the subsequent tasks described in the document can be performed either from
direct KVM connection at the BladeCenter or via the Management Module remote console. In
most cases, instructions will be provided assuming access via the Management Module.
1. Sun JVM Version 1.4.2 is the minimum required for remote control function. The Microsoft JVM is not
supported. It may work some of the time, but Remote Disk tends to act flaky with the MSJVM. To
check this in Internet Explorer, go to Tools-Options, and then click on the Advanced tab.
Figure 3 JVM Selection in IE
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
9
IBM BladeCenter Deployment Guide
2. Repeating Keystrokes - in order to remove the repeating keystroke problem, you need to check the
"Disable key repeats" box and click Save.
Figure 4 Disable key repeats
3. Start a Remote Control of the target blade by clicking on Remote Control under Blade Tasks.
Ensure that the media tray and KVM selections are for the target blade)
4. The Remote Control window is a pop up window. If you have a popup blocker installed, Remote
Control will be blocked, so make sure popups are allowed for the management module IP. Note:
Remote Control does work with Firefox, so if you are having trouble with the popup blocker in
Windows XP SP2, Firefox may be a viable alternative.
5. The Remote Control window includes a tool bar (
Figure 6) with “sticky keys” for passing “ctrl”,
“alt”, and others keystrokes to the target system. This is particularly important when trying to enter
setup screens, such as the QLogic setup which is accessed by hitting “ctrl-Q”. Click on the sticky
“ctrl” then hit Q when you see the message during POST.
6. If there is no keyboard response in Remote Control, confirm that the Remote Control window has
focus. Click on the keyboard widget in the toolbar. There is also a known issue, fixed in the 73G
release of the Management Module firmware, where the “ctrl” or the “alt” keys become stuck on.
Clicking the icons on the toolbar twice should clear this state and may restore keyboard response.
Figure 6 Remote Control Window
7. If, during a Linux installation, you see the message “No Video Available” (shown in
most likely “nofb” was not specifying as a boot option. See Section
4.3 for more information.
Figure 6) then
8. When switching from managing one chassis to another, it is recommended that you close your
browser after logging off from the first chassis, especially if you were using Remote Control in the first
session. Otherwise, you may encounter errors with the Remote Disk applet (middle frame,
Figure 5)
in the second session such as “remotedrive.dll is not accessible” “exception:
java.lang.NullPointerException.” “Initialization error. Library not installed” “Unsatisfied Link Error –
Applet com.ibm.asm.remotedisk.RemoteDisk not inited.”
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
11
IBM BladeCenter Deployment Guide
You have logged off the web interface.
This window will attempt to close in 5 seconds.
If a confirmation dialog is displayed, click "Yes" or "OK".
To maintain security, you must also manually close any
other web browser windows that might be open. Otherwise,
any previously entered user IDs and passwords will
remain available in the temporary browser cache.
Figure 7 Logoff and Close the Browser Messages
9. Unexpected DLL version number error when starting Remote Control.,
Initializing RemoteDisk v2.0
Attempting to load remotedrive.dll
Remote Disk library successfully loaded.
DLL reported: 12. Expected: 13
Unexpected DLL version number.
If you encounter this error when the remote control window is opened, exit Internet Explorer, find the copy
of remotedrive.dll on your system (most likely in c:\Program Files\Internet Exploder) and delete it.
2.4 Embedded Fibre Channel Switches
2.4.1 SAN Configuration
Shown in Figure 8 is an example of a dual path SAN configuration that we will be using as a reference in
this document. Two McData directors steer traffic through separate fabrics to the IBM BladeCenters,
which use Qlogic HBAs (Host Bus Adapters). Separate fabrics allow full redundancy in the storage path.
The IBM Blade Server boots to drives physically located on the Hitachi subsystem over the SAN. Qlogic
HBAs must be specifically configured to allow remote booting. See diagram 1 for a visual representation
of this architecture.
Here are the steps necessary to disable ports on the QLogic switch. This can be done either via a telnet
session or a browser session.
2.4.2.1 Using Telnet
1. Start a telnet session with the IP address of the QLogic Switch Module.
2. After logging in, enter the command ”show port” to view the current status of the ports.
Admin Operational Login Config Running Link Link
Port State State Status Type Type State Speed
---- ----- ----------- ------ ------ ------- ----- ---- Ext1:0 Online Online LoggedIn GL F Active 1Gb/s
Ext2:15 Online Offline NotLoggedIn GL Unknown Inactive Auto
Bay1 Online Online LoggedIn F F Active 2Gb/s
Bay2 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay3 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay4 Online Online LoggedIn F F Active 2Gb/s
Bay5 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay6 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay7 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay8 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay9 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay10 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay11 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay12 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay13 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay14 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Figure 9 Port Status Online
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
13
IBM BladeCenter Deployment Guide
3. Enter admin mode by entering “admin start”
4. Disable the target port by entering the command “set port x state offline” where x is the
number of the port. For this example, we used “4”
5. Confirm the status by entering “show port”again.
QLogic: USERID> admin start
QLogic (admin): USERID> set port 4 state offline
QLogic (admin): USERID> show port
Admin Operational Login Config Running Link Link
Port State State Status Type Type State Speed
---- ----- ----------- ------ ------ ------- ----- ---- Ext1:0 Online Online LoggedIn GL F Active 1Gb/s
Ext2:15 Online Offline NotLoggedIn GL Unknown Inactive Auto
Bay1 Online Online LoggedIn F F Active 2Gb/s
Bay2 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay3 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay4 Offline Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay5 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Bay6 Online Offline NotLoggedIn F Unknown Inactive 2Gb/s
Figure 10 Port Status Offline
2.4.2.2 Using SANSurfer GUI
1. Start the SANSurfer GUI. If the fabric has not already been discovered, click Fabric-Add Fabric
and enter the IP address of the target QLogic switch in the window that will pop up - That should
open SAN Browser.
2. Click the handle in the left pane to drop down the switch - highlight it and the faceplate will appear
Parameters Recommended settings
BIOS Disabled
Frame size 2048
Loop reset delay 5 (minimum)
Adapter hard loop ID Disabled
Hard loop ID 0
Execution throttle 100
Fast command posting Enabled
>4 GB addressing Disabled (for 32-bit systems)
LUNs per target 0
Enable LIP reset No
Enable LIP full login Yes
Enable target reset Yes Note: Enable LIP reset, Enable LIP full login, and Enable target reset
parameters control the behavior of the adapter when Windows tries to do a
SCSI bus reset. You must perform a target reset to make cluster failovers
work. Use the SCSI bus device reset option to clear SCSI reservations.
Login retry count 20 (minim um)
Port down retry count 20 (minimum)
Driver load RISC code Enabled
Enable database updates No
Disable database load No
IOCB allocation 256
Extended error logging Disabled
Table 1 Recommended settings for the QLogic QLA23xx adapter card for Windows 2000 or 2003
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
15
IBM BladeCenter Deployment Guide
Blade Configuration and Installation
3 Configuring the Blades for Remote Boot
The next few steps require BIOS configuration. To enter the BIOS, follow these steps:
1. Logon to the Management Module and select Power/Restart under Blade Tasks.
2. Select the blade to be restarted, then at the bottom of the page click Restart Blade and “Ok” when
prompted.
3. Start a Remote Control of the target blade by clicking on Remote Control under Blade Tasks.
Ensure that the media tray and KVM selections are for the target blade)
4. Enter the Blade BIOS by pressing F1 or clicking the F1 button at the appropriate time during POST.
3.1.1 Disabling IDE devices
Since the BladeCenter servers will boot to drives located on the SAN, the locally attached drives will not
be used. To boot to the Qlogic HBAs, the local IDE controllers for the IBM BladeCenter should be
disabled. To disable the IDE devices in the Blade Server BIOS, use the following steps from within the
BIOS setup program:
1. Select Devices and IO ports from the main menu
2. Select IDE configuration menu. Select Disable to disable the Primary and Secondary IDE controllers.
Note: in some instances, you may see the following error:
I9990303 No Boot device found
If you encounter this error, then spin up delay should be enabled on the HBA. See Section
Configuring the Qlogic HBA for details.
3.2
Figure 13 Disable the IDE Devices, HS20 and HS40
3.1.2 Check the Boot Order
• From main BIOS screen, access Start Options, then Startup Sequence to access the boot order
and insure the devices are listed in the following order:
Device Startup Order
CD-Rom First Startup Device
Diskette Drive 0 Second Startup Device
Hard Disk 0 Third Startup Device
Network Fourth Startup Device
3. Adjusting the boot sequence for HS40 blades is slightly different. Use the arrow keys to scroll to each
startup device, then hit enter and select the appropriate value in the popup window as shown in
Figure 15 HS40 Boot Sequence.
Figure 15 HS40 Boot Sequence
4. Press ESC to exit out of the BIOS and select the option to save changes.
Note: For of HS40 blades, make sure that EFI is at the end of the boot order if present.
5. If you have a lot of blades to update in a single blade chassis, the boot order changes may be made
through the Management Module and applied to any or of all of blades in the chassis as a group.
Figure 16 Changing boot order from Management Module) Since EFI appears as an unknown
(See
device for HS40s, it is recommended that the HS40 boot sequence be modified in BIOS program.
Version 3.0, 3/10/2006 http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100564
The BladeCenter Deployment Guide is intended as a personal productivity tool. It is not intended to be comprehensive and is provided for
guidance only, on an 'as is basis' without warranty of any kind. Please be aware that its contents have not been certified by IBM.
17
Loading...
+ 37 hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.