Broadcom HT1000, RAIDCore BC4000 Series User Manual

User Manual
BC4000
RAIDCore™ BC4000 Series RAID
Controllers
16215 Alton Parkway • P.O. Box 57013 • Irvine, CA 92619-7013 • Phone: 949-450-8700 • Fax: 949-450-8710 02/13/06
BC4000-UM100-R
REVISION HISTORY
Revision Date Change Description
BC4000-UM100-R 02/13/06 Initial release; describes version 2.1 of XelCore™ software and the
BC4000 Series of hardware controllers (which includes HT-1000 controllers).
Broadcom Corporation
P.O. Box 57013
16215 Alton Parkway
Irvine, CA 92619-7013
© 2006 by Broadcom Corporation
All rights reserved
Printed in the U.S.A.
Broadcom®, the pulse logo, Connecting everything®, and the Connecting everything logo are among the trademarks of Broadcom Corporation and/or its affiliates in the United States, certain other countries and/or the EU. Any other trademarks or trade names mentioned are the property of their respective owners.
This user guide (including, without limitation, the Broadcom component(s) identified herein) is not designed, intended, or certified for use in any military, nuclear, medical, mass transportation, aviation, navigations, pollution control, hazardous substances management, or other high risk application. BROADCOM PROVIDES THIS USER GUIDE "AS­IS", WITHOUT WARRANTY OF ANY KIND. BROADCOM DISCLAIMS ALL WARRANTIES, EXPRESSED AND IMPLIED, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
User Manual BC4000 Series RAID Controllers
02/13/06
TABLE OF CONTENTS
Section 1: Introduction........................................................................................................1
Audience ....................................................................................................................................................... 1
Obtaining Acrobat ........................................................................................................................................ 1
Contacting Broadcom.................................................................................................................................. 1
Section 2: Overview of RAID Concepts .............................................................................2
Understanding Arrays ................................................................................................................................. 2
Overview of Array Types ............................................................................................................................. 2
Expanding Disk Capacity Online ................................................................................................................ 3
Migrating RAID Levels Online..................................................................................................................... 3
Performance Considerations ...................................................................................................................... 4
Reliability Considerations ........................................................................................................................... 4
Flexibility and Expansion Considerations ................................................................................................. 5
Multiple Types......................................................................................................................................... 5
Future Expansion .................................................................................................................................... 5
Operating System Considerations ............................................................................................................. 5
Section 3: General Product Information............................................................................6
Overview of Features ................................................................................................................................... 6
BC4000 Series Model Numbers .................................................................................................................. 7
BC485x and BC445x Controllers ............................................................................................................ 7
BC4810 and BC4410 Controllers............................................................................................................ 8
HT-1000 Controllers................................................................................................................................ 9
Licensing Level Differences........................................................................................................................ 9
Detailed Feature List .................................................................................................................................. 10
Section 4: Installing BC4000 Series Controllers.............................................................12
Overview of the Installation Process ....................................................................................................... 12
System Requirements........................................................................................................................... 12
BC4000 Series Kit Contents ................................................................................................................. 13
Installation Steps................................................................................................................................... 13
Installing and Cabling BC4000 Hardware ................................................................................................ 15
Installing the BC4000 Controller ........................................................................................................... 15
Installing Hard Disk Drives and Cables (BC4000) ................................................................................ 17
Document BC4000-UM100-R Page iii
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Preparing Disks and Arrays for New OS Installs .....................................................................................19
Initializing Disks from the BIOS .............................................................................................................19
Creating Arrays from the BIOS..............................................................................................................20
Windows: Installing Drivers and Applications.........................................................................................22
Transferring Drivers to Floppy Disk (Windows) .....................................................................................22
Installing the BC4000 Series Drivers (Windows)...................................................................................22
New Installation of Windows 2000, XP, or 2003 ...........................................................................23
Existing Installation of Windows 2000, XP, or 2003 .......................................................................24
Installing the Management Suite ...........................................................................................................27
Linux: Installing Drivers and Applications...............................................................................................28
Transferring Drivers to Floppy Disk (Linux)...........................................................................................28
Installing the BC4000 Series Drivers (Linux).........................................................................................29
New Install of Red Hat and Fedora Core 1 Linux...........................................................................29
New Install of Fedora Core 2, 3, and 4 Linux.................................................................................30
New Install of SuSE Linux..............................................................................................................30
Existing Install of Red Hat or Fedora Core 1, 2, 3, or 4 Linux........................................................31
Existing Install of SuSE Linux.........................................................................................................32
Installing the bcadm Application............................................................................................................33
Installing the Online Help.......................................................................................................................33
Installing the bc_winraid Application......................................................................................................34
Booting from a Legacy Disk ......................................................................................................................35
Section 5: Updating the Drivers, Firmware, and Applications...................................... 36
Updating Components for the BC4000 Series .........................................................................................36
Updating the BC4000 Series Components for Windows.......................................................................37
Updating the BC4000 Series Driver for Linux........................................................................................39
Updating the BC4000 Series BIOS Image (Linux) ................................................................................40
Updating the BC4000 Driver for a Linux Errata Kernel..........................................................................41
Creating a Bootable Floppy Disk ..............................................................................................................42
Page iv Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Section 6: General Concepts for Managing Arrays and Disks ......................................43
RAID Minimum Disk Requirements .......................................................................................................... 43
Using Legacy Disks ................................................................................................................................... 43
Understanding Array and Disk States...................................................................................................... 44
Array States .......................................................................................................................................... 44
Disk States............................................................................................................................................ 45
Rescanning for Changes in State ......................................................................................................... 45
Starting and Stopping Tasks .................................................................................................................... 46
Working with Spares.................................................................................................................................. 47
About Sparing Options .......................................................................................................................... 47
Using Distributed Sparing ..................................................................................................................... 47
Using Dedicated Sparing ...................................................................................................................... 49
Using Global Sparing ............................................................................................................................ 49
Section 7: Using the BIOS Configuration Utility.............................................................50
When to Use the BIOS Configuration Utility............................................................................................ 50
Understanding the Color Code in the BIOS............................................................................................. 50
Initializing Disks from the BIOS................................................................................................................ 51
Creating Arrays from the BIOS ................................................................................................................. 52
Deleting Arrays from the BIOS ................................................................................................................. 54
Swapping Arrays from the BIOS............................................................................................................... 55
Hiding or Unhiding Arrays from the BIOS ............................................................................................... 56
Viewing Disk Details from the BIOS ......................................................................................................... 56
Viewing Array Details from the BIOS ....................................................................................................... 57
Rescanning All Channels from the BIOS ................................................................................................. 57
Changing Controller Options From the BIOS.......................................................................................... 58
Continue Booting from the BIOS .............................................................................................................. 59
Section 8: Using RAIDConsole.........................................................................................60
Starting the RAIDConsole Management Application.............................................................................. 60
Using Password Protection ...................................................................................................................... 62
Using the Help Menu.................................................................................................................................. 64
Document BC4000-UM100-R Page v
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Reviewing the Interface..............................................................................................................................65
The Array Status Window......................................................................................................................65
Array View Elements .............................................................................................................................66
Disk List Elements.................................................................................................................................67
Array List Elements ...............................................................................................................................67
I/O Status Bar Elements........................................................................................................................68
Working with Disks.....................................................................................................................................69
Initializing Disks .....................................................................................................................................69
Rescanning Disks..................................................................................................................................69
Changing Cache Settings for Disks.......................................................................................................69
Changing Disk View Settings ................................................................................................................70
Using LEDs to Identify Disks .................................................................................................................72
Working with Arrays...................................................................................................................................72
Creating and Formatting Arrays ............................................................................................................72
Creating Arrays Larger Than 2 TB ........................................................................................................75
Naming Arrays.......................................................................................................................................83
Using LEDs to Identify Arrays................................................................................................................83
Transforming Arrays..............................................................................................................................84
Copying Arrays......................................................................................................................................85
Using the Un-Link Command.........................................................................................................89
Accessing Additional Space ..................................................................................................................90
Hiding and Unhiding Arrays...................................................................................................................90
Preparing to Physically Remove an Array ............................................................................................. 91
Deleting Arrays......................................................................................................................................91
Recovering Arrays.................................................................................................................................92
Changing Cache Settings for Arrays.....................................................................................................92
Changing the Priority Level of a Task....................................................................................................93
Interrupting or Cancelling a Task...........................................................................................................93
Checking for Consistency......................................................................................................................94
Scheduling a Consistency Check ..........................................................................................................95
Scanning an Array in the Background ...................................................................................................95
Splitting Mirrored Arrays ........................................................................................................................96
Split a Mirror Using the Default Settings ........................................................................................96
Split a Mirror Using the Advanced Split Option ..............................................................................96
Page vi Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Adding or Removing Distributed Spares............................................................................................... 97
Adding or Removing Dedicated Spares................................................................................................ 97
Adding or Removing Global Spares...................................................................................................... 98
Removing Spares ................................................................................................................................. 98
Working with Options ................................................................................................................................99
Viewing or Changing the License Level................................................................................................ 99
Changing Your Password Settings ..................................................................................................... 102
Changing the Interface View............................................................................................................... 103
Setting Small Icons...................................................................................................................... 103
Using Expand On Click................................................................................................................ 103
Editing Text Columns in the Disk List .......................................................................................... 104
Editing the Physical View of the Disks ................................................................................................ 104
Viewing Logs....................................................................................................................................... 105
Setting Event Notifications .................................................................................................................. 106
Setting up Remote Management ........................................................................................................ 108
Adding a Remote System ................................................................................................................... 109
Updating the Controller Firmware....................................................................................................... 110
Reading SMART Information from Disks............................................................................................111
Turning on Advanced Options ............................................................................................................ 113
Stopping and Restarting the RAIDCore Service ................................................................................. 113
Section 9: Using bcadm..................................................................................................114
What is bcadm?........................................................................................................................................ 114
Understanding Query Output.................................................................................................................. 115
bcadm Controller List Elements.......................................................................................................... 115
bcadm Disk List Elements................................................................................................................... 115
bcadm Array List Elements ................................................................................................................. 116
Viewing Help from the Command Line................................................................................................... 117
Document BC4000-UM100-R Page vii
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Managing Controllers, Arrays, and Disks: bcadm --manage................................................................118
Options ................................................................................................................................................118
--query, -q.....................................................................................................................................118
--query-all, -qa..............................................................................................................................118
--add-spare, -as............................................................................................................................119
--remove-spare, -rs.......................................................................................................................119
--remove-all-spares, -ras..............................................................................................................119
--add-global-spare, -ags ...............................................................................................................119
--remove-global-spare, -rgs..........................................................................................................120
--cache-array, -ca.........................................................................................................................120
--cache-disk, -cd...........................................................................................................................121
--check-start, -cs...........................................................................................................................121
--check-end, -ce ...........................................................................................................................122
--check-bitmap-start, -cbs.............................................................................................................122
--check-bitmap-end, -cbe .............................................................................................................122
--scan-array, -sa...........................................................................................................................123
--task, -t........................................................................................................................................123
--priority, -p...................................................................................................................................124
--initialize disk, -id.........................................................................................................................124
--blink, -b ......................................................................................................................................125
--prepare-to-remove, -ptr..............................................................................................................125
--recover-array, -rcv......................................................................................................................126
--rescan, -rsc ................................................................................................................................126
--hide, -h.......................................................................................................................................126
--unhide, -uh.................................................................................................................................127
--unlink, -ul....................................................................................................................................127
--name, -n.....................................................................................................................................127
--smart-get, -sg............................................................................................................................. 128
--flash, -f.......................................................................................................................................128
--array, -a......................................................................................................................................128
--controller, -ct .............................................................................................................................129
--disk, -d .......................................................................................................................................129
--verbose, -v .................................................................................................................................129
Page viii Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Creating New Arrays: bcadm --create .................................................................................................... 130
Syntax................................................................................................................................................. 130
Options................................................................................................................................................ 131
--sub-member <num>, -sub <num>............................................................................................. 131
--spare-disk <list>, -sp <list>........................................................................................................ 131
--size <size_mb>, -s <size_mb>..................................................................................................131
--no-sync, -ns............................................................................................................................... 131
--d-spare, -ds ............................................................................................................................... 131
--cache <r, rw, w, nc>, -ca <r, rw, w, nc>..................................................................................... 131
--max-size, -ms............................................................................................................................ 131
--name "name", -n "name" ........................................................................................................... 131
--priority <1...10>, -p <1...10>...................................................................................................... 132
--preferred-id <64bit_hex>, -pid <64bit_hex> .............................................................................. 132
--scan-array, -sa .......................................................................................................................... 132
--leave-existing-data, -led ............................................................................................................ 132
--disk, -d....................................................................................................................................... 132
Examples ............................................................................................................................................ 132
Deleting Arrays: bcadm --delete ............................................................................................................. 133
Syntax................................................................................................................................................. 133
Options................................................................................................................................................ 133
Examples ............................................................................................................................................ 133
Transforming Arrays: bcadm --transform.............................................................................................. 134
Syntax................................................................................................................................................. 134
Options................................................................................................................................................ 135
--sub-member <num>, -sub <num>............................................................................................. 135
--size <size_mb>, -s <size_mb>..................................................................................................135
--same-size, -ss ........................................................................................................................... 135
--d-spare, -ds ............................................................................................................................... 135
--max-size, -ms............................................................................................................................ 135
--priority <1…10>, -p <1…10>..................................................................................................... 135
--array, -a..................................................................................................................................... 135
--disk, -d....................................................................................................................................... 135
Examples ............................................................................................................................................ 136
Document BC4000-UM100-R Page ix
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Copying Arrays: bcadm --create-copy....................................................................................................136
Syntax.................................................................................................................................................. 136
Options ................................................................................................................................................137
--sub-member, -sub......................................................................................................................137
--spare-disk, -sp ...........................................................................................................................137
--size, -s........................................................................................................................................137
--d-spare, -ds................................................................................................................................137
--unlink, -ul....................................................................................................................................137
--cache, -ca ..................................................................................................................................137
--max-size, -ms.............................................................................................................................138
--name, -n.....................................................................................................................................138
--priority, -p...................................................................................................................................138
--scan-array, -sa...........................................................................................................................138
--array, -a......................................................................................................................................138
--disk, -d .......................................................................................................................................138
Examples.............................................................................................................................................138
Splitting Mirrored Arrays: bcadm --split.................................................................................................139
Syntax.................................................................................................................................................. 139
Options ................................................................................................................................................139
--no-hide, -nh................................................................................................................................139
--default, -def................................................................................................................................140
--array, -a......................................................................................................................................140
--disk, -d .......................................................................................................................................140
Examples.............................................................................................................................................140
Following or Monitoring Arrays and Disks: bcadm --follow.................................................................141
Linux Syntax and Options....................................................................................................................141
--test-mail, -tm ..............................................................................................................................141
--mail-address <to_email_address>, -ma <to_email_address> ................................................... 141
--from_user <from_email_address>, -fu <from_email_address> .................................................142
--program <program>, -p <program> ..........................................................................................142
--delay <seconds>, -d <seconds> ................................................................................................ 142
--scan, -s ......................................................................................................................................142
Page x Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Windows Syntax and Options ............................................................................................................. 142
--test-mail, -tm.............................................................................................................................. 143
--mail-address <to_email_address>, -ma <to_email_address>................................................... 143
--mail-server <email_server>, -ms <email_server>..................................................................... 143
--from_user <from_email_address>, -fu <from_email_address>................................................. 143
--program, <program>, -p <program>.......................................................................................... 143
--delay <seconds>, -d <seconds> ............................................................................................... 143
--scan, -s...................................................................................................................................... 143
Examples ............................................................................................................................................ 144
Viewing or Changing License Levels: bcadm --license........................................................................ 145
Syntax................................................................................................................................................. 145
Options................................................................................................................................................ 145
--display, -d.................................................................................................................................. 145
--update <license_key>, -u <license_key> .................................................................................. 145
Example .............................................................................................................................................. 145
Retrieving and Updating the License Key........................................................................................... 146
Example bcadm Usage with Output ....................................................................................................... 147
Section 10: Compliance Information..............................................................................158
FCC Compliance Statement .................................................................................................................... 158
Industry Canada Compliance Statement ............................................................................................... 158
CE Compliance Information .................................................................................................................... 159
L'Information de Conformite de la CE ................................................................................................. 159
CE-befolgungInformationen ................................................................................................................ 159
Index..................................................................................................................................161
Document BC4000-UM100-R Page xi
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Page xii Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
LIST OF FIGURES
Figure 1: RAIDCore Hardware and Software Installation Process Flow......................................................... 14
Figure 2: BC4000 Series Controller Showing LED Connectors and Port Locations....................................... 15
Figure 3: BC4000 Series Controller Ports....................................................................................................... 16
Figure 4: LED Connector Pinout ..................................................................................................................... 17
Figure 5: BC4000 Series Cable Connection Between Controller and Hard Disk............................................ 18
Figure 6: The Array Status Window................................................................................................................ 65
Document BC4000-UM100-R Page xi
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
LIST OF TABLES
Table 1: Key to Number of Channels and Supported Functionality Level
Indicated by the BC4000 Series Model Number................................................................................ 7
Table 2: BC4000 Series License Levels.......................................................................................................... 9
Table 3: Feature List for Different License Levels of the BC4000 Series ...................................................... 10
Table 4: System Requirements ..................................................................................................................... 12
Table 5: Minimum Disks Required for Each RAID Level ............................................................................... 43
Table 6: Event Priority Levels...................................................................................................................... 106
Document BC4000-UM100-R Page xiii
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Section 1: Introduction
This manual describes the installation and operation of Broadcom’s RAIDCore™ BC4000 Series RAID controllers, which includes HT-1000 controllers, as well as the use of RAIDCore XelCore™ RAID management software.
AUDIENCE
The intended audience is system administrators and experienced users who are familiar with RAID and storage configuration, and who also have a general understanding of one of the following operating systems:
Microsoft Windows
Linux
For specific supported versions of each operating system, see the list given on the Broadcom website at the following web address:
http://www.broadcom.com/storage
OBTAINING ACROBAT
The documentation provided for the BC4000 Series RAID controllers is in the *.pdf Acrobat reader format. To obtain the latest version of Acrobat from Adobe Systems, Inc., use the following web address:
http://www.adobe.com/products/acrobat/readermain.html
CONTACTING BROADCOM
You can contact Broadcom by phone, e-mail, or regular mail.
Phone: 603-324-1555
Web Address:
For more information about RAIDCore BC4000 Series RAID controllers:
http://www.broadcom.com/storage
E-mail:
For service and support assistance: support@raidcore.com
For sales questions: sales@raidcore.com
For documentation feedback: documentation@raidcore.com
Postal Address:
Broadcom Corporation, Inc. 71 Spit Brook Road, Suite 304 Nashua, NH 03060
.
Document BC4000-UM100-R Introduction Page 1
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Section 2: Overview of RAID Concepts
UNDERSTANDING ARRAYS
Disk arrays are several disks that are grouped together in various organizations to improve either the performance or the reliability of a computer’s storage system. Because some array types enhance performance while others improve reliability, and because some array types enhance both, it is important to consider your needs when planning an array configuration.
The BC4000 Series RAID controller supports various array types. For small systems with four or less drives, RAID1 and RAID5 are appropriate choices. For larger systems with more drives available, RAID10 and RAID50 may be the appropriate choices. The unique ability of the RAIDCore controller to provide online expansion to other array types such as RAID10 and RAID50 across multiple drives and controllers becomes extremely valuable when expanding storage is a requirement.
Note: It is highly recommended that you review this documentation in its entirety before configuring arrays. Some of the advanced features of this controller (such as Online Capacity Expansion, Online RAID Level Migration, sparing options) need to be understood by the user before arrays are created.
OVERVIEW OF ARRAY TYPES
The BC4000 Series RAID controller supports the following Redundant Array of Independent Disk (RAID) array types. Note that the list of supported array types is affected by a controller’s license level.
Vo lum e: The controller treats one or more disks or unused space on a disk as a single array. Volume provides the ability to concatenate storage from various drives regardless of the size of the space on those drives. Volume is useful in scavenging space on drives unused by other arrays. Volume does not provide any performance or data redundancy benefit.
RAID0: RAID0, or striping, provides the highest performance but no data redundancy. Data in the array is striped (distributed) across several physical drives. RAID0 arrays are useful for holding information such as the operating system paging file, where performance is extremely important but redundancy is not.
RAID1: RAID1, or mirroring, mirrors data on a partition of one disk to another. RAID1 is useful when there are only two disks available and data integrity is more important than storage capacity.
RAID1n: RAID1n, or n-way mirroring, mirrors the data stored in one hard drive to several hard drives. This array type provides superior data redundancy because there are three or more copies of the data, and is useful for creating exact copies of an array for backup purposes. However, this array type is expensive, in both performance and the amount of disk space necessary to create the array type.
RAID10: RAID10 is also known as RAID(0+1) or striped mirror sets. This array type combines mirrors and stripe sets. RAID10 allows multiple drive failures, up to 1 failure in each mirror that has been striped. This array type offers better performance than a simple mirror because of the extra drives. RAID10 requires twice the disk space of RAID0 to offer redundancy.
RAID10n: RAID10n stripes multiple n-way mirror sets. RAID10n allows multiple drive failures per mirror set, up to n-1 failures in each mirror set that has been striped, where n is the number of drives in each mirror set. This array type is useful in creating exact copies of an array's data using the split command. This array type offers better random read performance than a RAID10 array, but uses more disk space.
Page 2 Overview of RAID Concepts Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
RAID5: RAID5, also known as a stripe with parity, stripes data as well as parity across all drives in the array. Parity information is interspersed across the drive array. In the event of a failure, the controller can restore the lost data of the failed drive from the other surviving drives. This array type offers exceptional read performance as well as redundancy. In general, write performance is not an issue due to the tendency of operating systems to perform many more reads than writes. This array type requires only one extra disk to offer redundancy. For most systems with four or more disks, this is the correct choice as array type.
RAID50: RAID50, also known as striped RAID5 sets, intersperses parity information across each RAID5 set in the array. This array type offers good read performance as well as redundancy. A 6-drive array provides two striped 3-drive RAID5 sets. Generally, RAID50 is useful in very large arrays, arrays with 10 or more disks. Like the RAID1n and RAID10n array types, RAID50 can handle multiple disk failures.
For the minimum disk requirements for each type of array, see “Expanding Disk Capacity Online” on page 3. See also “Performance Considerations” on page 4 to learn how to maximize the performance of your arrays.
EXPANDING DISK CAPACITY ONLINE
Online capacity expansion (OCE) allows you to add any number of disks to an array at any time and continue to access the array data while it is being redistributed.
To increase the size and organization of an array, transform the array. You can also use the transform function as an integral part of the system backup and recover strategy through the use of the RAID1, RAID10, RAID1n, and RAID10n array types. For more information on transforming arrays from RAIDConsole, see “Transforming
Arrays” on page 84. For more information on transforming arrays from bcadm, see “Transforming Arrays: bcadm --transform” on page 134.
MIGRATING RAID LEVELS ONLINE
With Online RAID Level Migration (ORLM), you can easily move from one RAID level to another. While the migration is taking place, the data is accessible and protected to the lowest protection of either the source RAID level or the destination RAID level.
The transform feature can be used to increase the size and organization of an array. It can also be used as an integral part of the system backup and recover strategy through the use of the RAID1, RAID10, RAID1n, and RAID10n array types. To do this from RAIDConsole, see “Transforming Arrays” on page 84, and from bcadm, see “Transforming Arrays: bcadm --transform” on page 134.
Document BC4000-UM100-R Expanding Disk Capacity Online Page 3
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
PERFORMANCE CONSIDERATIONS
One of the main advantages of using RAID is increased performance. With RAID, performance is based on four elements: the number and organization of disks in an array, the caching attributes being used for the array, the application workload, and the PCI bus speed.
Spindles: RAID increases performance by putting more disks to work and by buffering data for the host. Many disks can transfer data at greater than 50 MegaBytes per second, and the BC4000 Series RAID controllers can aggregate this bandwidth in an almost linear fashion as more of the same drives are included in an array.
Caching: The BC4000 Series RAID controller software can also be configured to provide read- and write- back caching, if desired. Write-back caching has a large effect on most workloads, but should be used with caution.
Workload: When configuring an array, workload is probably the most important performance variable. Most applications do many more reads than writes, so the best performance will be obtained with array types like RAID0, RAID10, and RAID5.
PCI Bus Speed: Performance bottlenecks can be caused by the type of PCI bus you are using. The maximum theoretical performance of a PCI 32-bit 33-MHz bus is 132 MegaBytes per second. Performance increases to 512 MegaBytes per second with a PCI 64-bit 66-MHz bus.
The best possible performance with the fewest bottlenecks on PCI-X systems can be obtained by using a PCI-X 64-bit 133-MHz bus, which offers 1024 MegaBytes per second.
RELIABILITY CONSIDERATIONS
Improved reliability is the other main argument for using RAID. Reliability is enhanced through data redundancy and backup.
Redundancy. RAID1, RAID10, RAID1n, RAID 10n. RAID5, or RAID50 are necessary for redundancy. With redundancy, both capacity and performance are sacrificed for reliability. It should be noted, however, that with the BC4000 Series RAID controllers, extremely high performance is obtained even with redundant type arrays.
Backup. The controller’s ability to transform arrays and split mirrors can be used as part of your backup strategy. For example, you may want to create a RAID1 or RAID10 array and periodically transform these into RAID1n and RAID10n arrays. These RAID1n and RAID10n arrays can then be split into the original array and a backup array. As part of a hot fallback strategy, the backup array can be kept online and hidden from the OS or remain visible, or it can be removed and stored as a backup device.
When a boot array is split, the original and the copy are exact copies with the same labels. This means that when using the Linux operating system, Broadcom recommends that you keep the split copy hidden during boot time to avoid boot-time issues with duplicate labels.
Page 4 Performance Considerations Document BC4000-U M100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
FLEXIBILITY AND EXPANSION CONSIDERATIONS
Before configuring an array, consider the following points to enhance the flexibility of your RAID system.
MULTIPLE TYPES
Most RAID controllers do not allow you to change a chosen array type, create more than one type of array for each controller, or to create more than one array per physical disk. With the BC4000 Series RAID controller, you can create different types of arrays on the same disk to adapt each array to the I/O that it processes. Furthermore, depending on the array capacity and redundancy level, you can then transform an existing array to another RAID type if the type of array you are using is not the optimal type for your application. You can also build different arrays with different characteristics for different applications.
FUTURE EXPANSION
When creating arrays, consider whether you will need to expand your disk capacity in the future. If you anticipate the need to expand a file system:
On Windows, format your arrays with NTFS. Microsoft provides a utility (diskpart.exe) that will dynamically extend an NTFS file system onto any unused adjacent space. Note also that using a single partition per array will make expansion much easier. The diskpart.exe utility version you need depends on which version of Windows you are running. The diskpart.exe utility can be found on the CD for some versions of Windows or on the Microsoft website (www.microsoft.com) for others. Use the correct version for your operating system.
On Linux operating systems, be sure to use an expandable file system. Because the BC4000 Series RAID controller is limited to eight arrays, if you need a large number of logical volumes, you may want to use a logical volume manager, such as LVM.
OPERATING SYSTEM CONSIDERATIONS
One final and important array planning consideration is whether to extend the RAID benefits to your system disk and operating system, as well as to your data disks. For example, if you install the operating system on a RAID1 array (mirror), split the array into two volumes and then hide one of the resulting volumes, you can easily recover your operating system should your system disk fail. Alternatively, you can improve operating system performance by installing the OS on a RAID10 array and then splitting it and hiding one of the RAID0 arrays (like a hot spare).
To install the operating system on an array and then boot from that array, see “Initializing Disks from the BIOS”
on page 51.
Broadcom Corporation
Document BC4000-UM100-R Flexibility and Expansion Considerations Page 5
BC4000 Series RAID Controllers User Manual
02/13/06
Section 3: General Product Information
OVERVIEW OF FEATURES
The RAIDCore BC4000 Series RAID controllers and XelCore™ software support various array types. For small systems with four or less drives, RAID1 and RAID5 are appropriate choices. For larger systems with more drives available, RAID10 and RAID50 may be the appropriate choices. The unique ability of the controller to provide online expansion to other array types such as RAID10 and RAID50 across multiple drives and controllers becomes extremely valuable when expanding storage is a requirement.
The licensing level that you purchase determines which set of features you receive.The list of enterprise-class RAIDCore features available with the BC4000 Series RAID Controller includes:
Online RAID Level Migration (ORLM): With online RAID level migration, users can easily move from one RAID level to another. While the migration is taking place, the data is accessible and protected to the lowest protection of either the source RAID level or the destination RAID level.
Online Capacity Expansion (OCE): With online capacity expansion, users can have a maximum of 32 disks in an array at any time and continue to access their data while it is being redistributed.
Controller Spanning: A unique feature that allows arrays to be created across controllers so that very wide arrays can be created.
Array Hiding: An array can be hidden from the operating system so neither software nor users can see or access it. This is a key element in creating secure data backups.
Mirror Splitting: A user can split a mirrored array so that the volumes can be worked with independently.
Drive Roaming: With drive roaming, SATA cables can be disconnected from their drives and shuffled
without confusing the controller (this is performed offline). The controller detects which disks belong to which arrays.
Cache Support for Arrays: Various array-caching options are supported: read, read with write-back, write-back, and no cache.
Cache Support for Disks: Various disk-caching options are supported: no cache, disk read ahead + write-back, disk read ahead, and disk write-back.
Background Array Initialization: Background initialization allows a redundant array to be used immediately. If a disk is lost prior to completion of the create, no data is lost.
Sparing Support: For enhanced data integrity, several types of disk sparing are supported: distributed, dedicated, and global sparing.
Multiple Array Types Per Drive: Support for multiple array types per drive allows the administrator to create different array types using the same disks.
SMART: Self-Monitoring Analysis and Reporting Technology is a set of advanced diagnostics that monitor the internal operations of disk drives and provide early warning of any potential problems.
Page 6 General Product Information Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
BC4000 SERIES MODEL NUMBERS
BC485X AND BC445X CONTROLLERS
The meaning of each element of the BC4000 Series controller model number is illustrated in Table 1.
Table 1: Key to Number of Channels and Supported Functionality Level
Indicated by the BC4000 Series Model Number
Broadcom Family Identifier Ports License Level Dash Interface
BC 4—BC4000 Series 8—Eight ports
4—Four ports
52—Highest license level 50—Medium license level 10—Lowest license level
- 02—HBA, PCI-X, Host-based
00—Reserved
First and Second
Broadcom Corporation
Characters:
First Digit:
Second Digit
Third and Fourth Digits:
Final Digits:
Current models are Serial ATA and support up to RAID50. This means the only variable model number elements are the functionality level and the number of channels. The third and fourth numerals in the RAIDCore controller model number reflect the functionality level that is licensed. The controller has three functionality
,
levels: level 52
level 50 and level 10.
4 = BC4000 Series: SATA RAID only
The number of ports on the controller
8 = Eight ports
4 = Four ports
License Level; see Table 3 on page 10
52 = Full lIcense level: RAID0, RAID , RAID5, RAID10, RAID50, RAID1N, RAID10N, and Volume (JBOD), plus all features, including Online RAID Level Migration (ORLM), Online Capacity Expansion (OCE), controller spanning, distributed sparing, and mirror splitting
50 = RAID0, RAID1, RAID5, RAID10, and Volume (JBOD); features do not include Online RAID Level Migration (ORLM), Online Capacity Expansion (OCE), controller spanning, distributed sparing, and mirror splitting
10 = RAID0, RAID1, RAID10, and Volume (JBOD); features do not include Online RAID Level Migration (ORLM), Online Capacity Expansion (OCE), controller spanning, distributed sparing, and mirror splitting
Dash
-
Interface
02 = Host Bus Adapter (HBA), PCI-X, Host-based only
00 = Reserved
The RAIDCore BC4000 Series RAID controllers support either four or eight storage channels. As shown in
Table 1, the second numeral indicates the number of channels on the controller. For example, the BC4450-02
is a 4-channel controller that supports level 50 features, and the BC4852-02 is an 8-channel controller that supports level 52 features.
Document BC4000-UM100-R BC4000 Series Model Numbers Page 7
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
The following advanced RAID levels are
RAID50
RAID1N
RAID10N
The BC4450 and BC4850 only support the following RAID levels: RAID0, RAID1, RAID5, RAID10, and Volume (JBOD).
The following features are
Controller spanning; this means that disk arrays cannot be shared between two or more BC4000 Series controllers.
Online Capacity Expansion (OCE)
Online RAID Level Migration (ORLM)
Distributed sparing
Mirror splitting
not
supported by the BC4450 and BC4850:
not
supported by the BC4450 and BC4850:
BC4810 AND BC4410 CONTROLLERS
The BC4410 and BC4810 controllers are sold as entry-level products that do not support the full complement of features supported by the advanced BC4000 Series RAID controllers, the BC4452 and BC4852. These features and capabilities can be subsequently added to the BC4410 and BC4810 controllers without changing the hardware. Both controllers use the same XelCore software as the rest of the BC4000 Series.
not
The following advanced RAID levels are
RAID 5
RAID50
RAID1N
RAID10N
The BC4410 and BC4810 only support the basic RAID levels: RAID 0, RAID 1, RAID 10, and Volume (JBOD).
not
The following features are
Controller spanning
Online Capacity Expansion (OCE)
Online RAID Level Migration (ORLM)
Distributed sparing
Mirror splitting
For more information about these features, see Section 3: “General Product Information” on page 6 and
Section 6: “General Concepts for Managing Arrays and Disks” on page 43 of this document.
Upgrading the BC4450/BC4850 or BC4410/BC4810 controller to advanced RAID levels and features can be done with a simple license download, described in “Licensing Level Differences” on page 9.
supported by the BC4410 and BC4810:
supported by the BC4410 and BC4810:
Page 8 BC4000 Series Model Numbers Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
HT-1000 CONTROLLERS
The HT-1000 is an I/O controller for HyperTransport™-based server platforms. It incorporates all of the RAIDCore BC4000 Series functionality directly through XelCore.
Boards equipped with the HT-1000 are considered part of the BC4000 Series and use the same drivers as all other BC4000 Series controllers.
All HT-1000 controllers use the highest RAIDCore licensing level.
LICENSING LEVEL DIFFERENCES
The license level defines the RAID functionality supported by a specific RAIDCore controller model. Three license levels are supported by the RAIDCore software. Various features are enabled when different license levels are purchased. Licenses are keyed to the module serial number that is embedded in the hardware, with a unique serial number for each individual controller.
The BCXX10, BCXX50, and BCXX52 license levels and associated descriptions are shown in Table 2.
Table 2: BC4000 Series License Levels
License Level Description
10 RAID(0, 1, 10, Volume)
50 RAID(0, 1, 5, 10, Volume)
52 RAID(0, 1, 5, 10, 1n, 10n, 50, Volume), Online RAID Level Migration (ORLM), Online
Capacity Expansion (OCE), controller spanning, distributed sparing, and mirror splitting
The licensing level can be viewed from the BIOS banner or the BIOS Configuration Utility. After booting to your operating system, you can view the license level and/or modify license keys with either RAIDConsole (see
“Viewing or Changing the License Level” on page 99) or the bcadm application (see “Viewing or Changing License Levels: bcadm --license” on page 145).
Contact your local distributor to purchase license key upgrades for your RAIDCore controllers.
Document BC4000-UM100-R Licensing Level Differences Page 9
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
DETAILED FEATURE LIST
Table 3: Feature List for Different License Levels of the BC4000 Series
License Level
Features BCXX52/HT-1000 BCXX50 BCXX10
SATA channels
1
Online Capacity Expansion (OCE) Yes No No
Online RAID Level Migration (ORLM) Yes No No
Controller spanning
1
RAID Levels Supported
RAID 0, 1, 0+1, Volume Yes Yes Yes
RAID 5 Yes Yes No
RAID 1n, 0+1n, 50 Yes No No
2
Split
Drive roaming
3
Cache Support
No cache Yes Yes Yes
Read cache Yes Yes Yes
Read with write-back cache Yes Yes Yes
Write-back Yes Yes Yes
Background array Initialization Yes Yes Yes
Highest PCI support 64-bit 133 MHz
Create and delete without reboot Yes Yes Yes
Zero create option Yes Yes Yes
Copy arrays Yes Yes Yes
Background consistency check Yes Yes Yes
Scheduled consistency checks Yes Yes Yes
Restore (rebuild) priority Yes Yes Yes
Sparing Support
Dedicated Yes Yes Yes
Global Yes Yes Yes
Distributed
Multiple array type per drive
Touched region logging
4
5
6
E-Mail event notification Yes Yes Yes
NT Event log integration Yes Yes Yes
Instant create support RAID 1,1n,0+1,0+1n,5,50
Hot swap support
8
7
Remote management Yes Yes Yes
4, 84, 84, 8
Yes No No
Yes No No
Yes Yes Yes
(PCI-X)
64-bit 133 MHz (PCI-X)
64-bit 133 MHz (PCI-X)
Yes No No
Yes Yes Yes
Yes Yes Yes
Yes Yes Yes
Yes Yes Yes
Page 10 Detailed Feature List Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
Table 3: Feature List for Different License Levels of the BC4000 Series
License Level
Features BCXX52/HT-1000 BCXX50 BCXX10
Delayed spin-up support Yes Yes Yes
Swap array Yes Yes Yes
BIOS Support
Create Yes Yes Yes
Boot Yes Yes Yes
INT13 control
Operating System Support
Windows
Windows
2000, Windows Server 2003 Yes Yes Yes
XP Yes Yes Yes
Linux (see Table 4 on page 12) Yes Yes Yes
Notes:
9
Yes Yes Yes
1 Arrays can be created across multiple controllers allowing the creation of very wide arrays.
2 This feature allows the administrator to split a mirror into multiple arrays. It is used for array
backup and duplication.
3 Drive roaming allows arrays to be moved between different controllers and systems.
4 Distributed sparing is a technique where the space that would normally be assigned to a
dedicated spare is instead spread across all the disks in an array. It has the advantage of keeping all disks in an array active, increasing performance and also alleviating the problem of a silent failure of a dedicated or global spare.
5 Multiple array types per drive allow the administrator to create different array types on the
same drives. For example: The administrator wants data redundancy for the user data, and creates a RAID5 set using part of the disks data. At the same time the administrator wants performance for the swap spaces, and creates a RAID0 array using the rest of the disks space. This feature is also useful in collecting unused space from different capacity disks.
6 This feature increases data integrity for redundant array types by logging areas of an array
that have been written to. In the event of a system crash, the logged areas consistency is checked and/or corrected. Without this feature, data corruption may occur.
7 Arrays can be instantly created and used by skipping the background consistency check. For
certain types of redundant arrays this is a viable option and has no data integrity drawbacks. A consistency check can always be done at a later time. For RAID5 and RAID50 sets, if the initialization is skipped, the arrays will not be redundant until a consistency check is performed.
8 Disks can be added to the system and to an array while the system is operating.
9 INT13 support can be turned on or off from the BIOS configuration utility. If on, you can boot
from an OS installed on a BC4000 Series controller. If not booting from a BC4000 Series controller, Broadcom recommends setting INT13 support to OFF
Document BC4000-UM100-R Detailed Feature List Page 11
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Section 4: Installing BC4000 Series Controllers
OVERVIEW OF THE INSTALLATION PROCESS
This section is designed to quickly get the controller up and running using default settings, which for most users are the optimal settings. For instructions on how to modify any of the default settings, see Section 7: “Using
the BIOS Configuration Utility” on page 50 and either Section 8: “Using RAIDConsole” on page 60 or Section 9: “Using bcadm” on page 114.
To upgrade an existing BC4000 Series RAID controller, see Section 5: “Updating the Drivers, Firmware, and
Applications” on page 36.
SYSTEM REQUIREMENTS
Before attempting to install a BC4000 Series RAID controller, be sure the system meets the requirements described in Table 4.
Table 4: System Requirements
Item Requirements
Processor x86, 32-bit or 64-bit compatible processor greater than 500 MHz
Memory 256 MB minimum
Hard disk At least one SATA disk; the number of disks depends on the number, type, and capacity of arrays to be
created.
Available Slots
Operating System
One PCI or PCI-X slot per controller (PCI-X 133/100/66, PCI-64 66/33, PCI-32 66/33)
Windows 2000 (Professional, Server, Advanced Server) with Service Pack 4 or later
Windows XP (Professional Edition–64-bit and 32-bit, Home Edition, Media Edition)
Windows Server 2003 (Enterprise Edition, Standard Edition, Web Edition) (64-bit and 32-bit)
Note:
Microsoft .NET Framework 1.1 is required for all Windows installations.
Red Hat
Red Hat Enterprise Linux 4 (64-bit and 32-bit)
Red Hat Fedora™ Core 1 and 2 (32-bit only)
Red Hat Fedora Core 3 and 4 (64-bit and 32-bit)
SuSE™ Linux Professional 9.1 and 9.2 (32-bit only)
SuSE Linux Professional 9.3 (64-bit and 32-bit)
SuSE Linux Enterprise Server 9 (64-bit and 32-bit)
SuSE Linux Enterprise Server 10 (64-bit and 32-bit)
Visit the Broadcom website at: http://www.broadcom.com/storage additional Linux support.
Linux 9 and Red Hat Enterprise Linux 3 (32-bit only)
for Linux errata kernel support and
Page 12 Installing BC4000 Series Controllers Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
BC4000 SERIES KIT CONTENTS
When a RAIDCore BC4000 Series RAID controller is received, the package should contain the following items:
BC4000 Series RAID controller
BC4000 Series RAID controller media CD
2U-compliant bracket
Serial ATA cables, one per port
If any of these items are missing or damaged, contact your dealer or distributor.
Caution! Before installing the controller in an existing system, back up any critical data. Failure to follow this accepted system management practice could result in data loss.
INSTALLATION STEPS
The major installation steps are given below and are depicted in Figure 1 on page 14:
1. Install the controller or controllers (see “Installing the BC4000 Controller” on page 15).
2. Install the hard disk drives and cables (see “Installing Hard Disk Drives and Cables (BC4000)” on page 17).
3. Initialize the disks and creating a single array or multiple arrays using the BIOS (see “Preparing Disks and
Arrays for New OS Installs” on page 19.)
If existing disks (referred to as “legacy” disks) are being used, this step can be skipped. See “Booting from a
Legacy Disk” on page 35 for details on using existing disks.
4. Transfer the drivers to a floppy disk. There are different procedures for Windows and Linux users.
Windows users: See “Transferring Drivers to Floppy Disk (Windows)” on page 22. If a new operating system is not being installed in the BC4000 Series RAID array, this step can be skipped.
Linux users: See “Transferring Drivers to Floppy Disk (Linux)” on page 28.
5. Install the RAIDCore drivers and, if required, the new OS. There are different procedures for Windows and Linux users.
Windows users: See “Installing the BC4000 Series Drivers (Windows)” on page 22.
Linux users: See “Installing the BC4000 Series Drivers (Linux)” on page 29.
Caution! All components (firmware, drivers and applications) must be installed when upgrading to Version 2.0 of XelCore. This means that a version 1.x driver will not work with version 2.x of the software and vice versa.
6. Install the RAIDCore applications. There are different procedures for Windows and Linux users.
Windows users: See “Installing the Management Suite” on page 27.
Linux users: See “Installing the bcadm Application” and “Installing the Online Help” on page 33.
Document BC4000-UM100-R Overview of the Installation Process Page 13
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
Install Controller
Install Cables & Disks
Use BIOS to Initialize Disks & Create Arrays
Windows or Linux?
Windows Linux
Existing OS or New Install?
Copy Drivers to
New OS
Install
Existing OS Install
Floppy Disk
Copy Drivers to
Floppy Disk
Install OS & Drivers
Using “New OS”
Install Procedure
Install RAIDCore
Management Suite
Existing OS or New Install?
New OS
Install
Install Driv ers
Using “Existing OS”
Install Procedure
Install OS & Drivers
Using “New OS”
Install Procedure
Install Driv ers
Using “Existing OS”
Install Procedure
Install RAIDCore
Management Suite
Done
Figure 1: RAIDCore Hardware and Software Installation Process Flow
Existing OS Install
Page 14 Overview of the Installation Process Document BC4000-UM100-R
Broadcom Corporation
User Manual BC4000 Series RAID Controllers
02/13/06
INSTALLING AND CABLING BC4000 HARDWARE
Begin by installing the BC4000 Series RAID controller board and the disks to be used in the system, and then cable them together.
INSTALLING THE BC4000 CONTROLLER
The controller fits into any available 32-bit PCI or 64-bit PCI-X slot on the motherboard. RAIDCore BC4000 Series RAID controllers support all PCI variants from 32-bit/33-MHz through 64-bit/133-MHz (PCI-X). For maximum performance, installing the controller in a PCI-X slot is recommended.
Up to four controllers can be installed in a system provided there are available PCI/PCI-X slots. Controllers with different licensing levels can be mixed in a system, but the functionality of the lowest license level is used. The process for physically installing multiple controllers is the same as for installing a single controller.
A RAIDCore BC4000 Series controller is shown in Figure 2.
LED Connectors
Port 0 Port 1
Port 2 Port 3
Port 4 Port 5
Port 7
Por t 6
Figure 2: BC4000 Series Controller Showing LED Connectors and Port Locations
Caution! A BC4000 Series RAID controller, like every other electronic part of a computer, can be
damaged by static electricity. Be sure that you are properly grounded. Broadcom recommends that a grounded antistatic strap be worn and/or that something grounded be held, and that the computer be unplugged before installing the controller.
Document BC4000-UM100-R Installing and Cabling BC4000 Hardware Page 15
Broadcom Corporation
BC4000 Series RAID Controllers User Manual
02/13/06
To install the BC4000 Series RAID controller:
1. Turn off the computer and any connected peripheral devices.
2. Unplug the power cord from the computer.
3. Remove the cover of the computer and the slot cover of an available PCI slot.
4. If desired, attach LED cables from the inside of the computer to the LED connectors on the RAIDCore
controller, shown in Figure 2 (see also “Installing Hard Disk Drives and Cables (BC4000)” on page 17).
After the LED cables are attached, and the system is powered up, the BIOS can be used as a diagnostic tool to verify that the cables are correctly connected. Each LED lights up when the drive associated with the LED in the BIOS is selected.
Notes:
The LED connectors are in the J2 position on the controller.
When the controller is held as shown on Figure 2, the LED 1 connector for port 0 is on the left,
and the LED 8 connector for port 7 is on the right.
The ground pins are in the lower row, and the power pins are in the upper row. If a light does not work, try reversing the LED cable connection.
Until the BIOS loads, the LEDs are not indicative of disk connectivity.
5. Insert the controller in an available slot.
Note: Depending on the system, this may require changing the bracket to a 2U-compliant bracket.
Be careful to align the bus connector of the controller with the selected PCI slot on the motherboard. Push the board down firmly until it is seated properly.
6. Secure the controller to the computer chassis with a bracket screw.
Port 3
Por t 1
Por t 0
Por t 2
Port 6
Por t 5
Por t 4
Por t 7
Page 16 Installing and Cabling BC4000 Hardware Document BC4000-U M100-R
Figure 3: BC4000 Series Controller Ports
Broadcom Corporation
Loading...
+ 150 hidden pages