Kontron CP6923 User Manual

CP6923 Manual ID

2.2 Revision Index

10 May, 2007 Date of Issue

CP6923 USER GUIDE

6U CPCI Ethernet Switch

with 24 Channels

Page ii CP6923 User Guide

Revision History

Imprint

Kontron AG may be contacted via the following:

North America EMEA

Kontron Canada, Inc. Kontron Modular Computers GmbH

616 Curé Boivin Sudetenstrasse 7

Boisbriand, Québec 87600 Kaufbeuren

J7G 2A7 Canada Germany

Tel: [phone] [phone]

[phone]

Fax: [phone] [phone]

E-mail: [e-mail] [e-mail]

For further information about Kontron AG, our products or services, please visit our Internet
web site: www.kontron.com

Disclaimer

Copyright © 2006 Kontron AG. All rights reserved. All data is for information purposes only and not guaranteed for
legal purposes. Information has been carefully checked and is believed to be accurate; however, no responsibility
is assumed for inaccuracies. Kontron and the Kontron logo and all other trademarks or registered trademarks are
the property of their respective owners and are recognized. Specifications are subject to change without notice.

Publication Title: CP6923: 6U CPCI Ethernet Switch with 24 channels

ID Number: CP6923

Rev.

Index

Brief Description of Changes Date of Issue

1.0 Preliminary Issue Oct., 2006

2.0 Initial Issue March, 2007

2.1 Minor changes in Chapter 3, Hardware Description and Chapter
4, Software Description

22 March, 2007

2.2 Minor changes to Preface and Chapter 1 10 May, 2007

CP6923 Preface

Page iii CP6923 User Guide

Table of Content

Revision History .........................................................................................................ii

Imprint ........................................................................................................................ii

Disclaimer ..................................................................................................................ii

Proprietary Note ...................................................................................................... viii

Trademarks ............................................................................................................. viii

Environmental Protection Statement ....................................................................... viii

Explanation of Symbols ............................................................................................ix

For Your Safety .........................................................................................................ix

Safety Instructions .................................................................................................x

Special Handling and Unpacking Instructions ......................................................xi

General Instructions on Usage ............................................................................ xii

Two Year Warranty .................................................................................................. xiii

Chapter

1

1. Introduction .................................................................................................. 1 - 2

1.1 Product Overview .................................................................................... 1 - 2

1.1.1 CP6923 Features ............................................................................ 1 - 2

1.1.2 General compliances ...................................................................... 1 - 4

1.1.3 Hot Swap Capability ....................................................................... 1 - 4

1.2 Technical Specification ........................................................................... 1 - 5

1.3 Software Support ..................................................................................... 1 - 7

Chapter

2

2. Installation ................................................................................................... 2 - 2

2.1 Safety Requirements ............................................................................... 2 - 2

2.2 CP6923 Initial Installation Procedures .................................................... 2 - 3

2.3 Standard Removal Procedures ............................................................... 2 - 4

2.4 Software Installation ................................................................................ 2 - 5

CP6923 Preface

Page iv CP6923 User Guide

Chapter

3

3. Hardware Description .................................................................................. 3 - 2

3.1 Fabric Switch ........................................................................................... 3 - 3

3.1.1 PCI Interface ................................................................................... 3 - 3

3.1.2 Port Mapping .................................................................................. 3 - 3

3.1.3 Standard Fabric Links ..................................................................... 3 - 5

3.1.4 Extended Fabric Uplinks ................................................................. 3 - 5

3.1.5 Copper Uplinks ............................................................................... 3 - 5

3.1.6 1000BASE-xX SFP Uplinks ............................................................ 3 - 6

3.1.7 10GBASE-xX XFP Uplinks ............................................................. 3 - 7

3.2 CPU and Memory .................................................................................... 3 - 8

3.2.1 PCI Interface ................................................................................... 3 - 8

3.2.2 SDRAM ........................................................................................... 3 - 8

3.2.3 FLASH ............................................................................................ 3 - 8

3.2.4 Fast Ethernet Management Interface ............................................. 3 - 8

3.2.5 RS232 Management Interface ........................................................ 3 - 9

3.3 IPMI ....................................................................................................... 3 - 11

3.3.1 PM ................................................................................................ 3 - 11

3.3.2 FUM .............................................................................................. 3 - 11

3.4 Power Supply and Hot Swap ................................................................ 3 - 12

3.4.1 Power Up and Power Down Sequencing ...................................... 3 - 12

3.4.2 Reset ............................................................................................ 3 - 12

3.5 Display Elements ................................................................................... 3 - 13

3.6 CompactPCI Bus Connector ................................................................. 3 - 15

3.6.1 CompactPCI Connectors J1 and J2 Pinouts ................................ 3 - 16

3.6.2 CompactPCI Rear I/O Connectors J3-J5 and Pinouts .................. 3 - 17

Chapter

1

4. Software Description ................................................................................... 4 - 2

4.1 Supported RFCs ..................................................................................... 4 - 2

CP6923 Preface

Page v CP6923 User Guide

4.1.1 Management ................................................................................... 4 - 2

4.1.2 Switching ........................................................................................ 4 - 3

4.1.3 Routing ........................................................................................... 4 - 4

4.1.4 QoS ................................................................................................. 4 - 5

4.1.5 Multicast .......................................................................................... 4 - 5

4.2 Supported MIBs ...................................................................................... 4 - 5

4.2.1 Enterprise MIB ................................................................................ 4 - 5

4.2.2 Switching Package MIBs ................................................................ 4 - 5

4.2.3 Routing Package MIBs ................................................................... 4 - 6

4.2.4 QoS Package MIBs ........................................................................ 4 - 6

4.2.5 Multicast package MIBs .................................................................. 4 - 6

4.2.6 SNMP MIBs .................................................................................... 4 - 7

4.3 Bootloader ............................................................................................... 4 - 7

4.3.1 Power On Self Test ......................................................................... 4 - 7

4.4 IPMI Firmware ......................................................................................... 4 - 9

4.4.1 Sensor Data Record (SDR) .......................................................... 4 - 10

4.4.2 Board FRU Information ................................................................. 4 - 11

4.4.3 PM Firmware Code ....................................................................... 4 - 12

4.4.4 LEDs ............................................................................................. 4 - 12

4.5 Firmware Administration ....................................................................... 4 - 13

4.5.1 Startup Configurations .................................................................. 4 - 14

4.5.2 Updating Firmware ....................................................................... 4 - 15

4.5.3 Updating IPMI ............................................................................... 4 - 17

Appendix

A

A. Getting Help ................................................................................................... A - 2

CP6923 Preface

Page vi CP6923 User Guide

List of Tables

1-1 CP6923 Main Specifications ...................................................................... 1 - 5

1-2 CP6923 Software Specification ................................................................. 1 - 7

3-1 PCI-ID ........................................................................................................ 3 - 3

3-2 Port Mapping .............................................................................................. 3 - 4

3-3 Port Mapping .............................................................................................. 3 - 4

3-4 Copper Uplink RJ45 Pin Assignment ......................................................... 3 - 5

3-5 Copper Uplink LEDs Signification .............................................................. 3 - 6

3-6 SFP Connectors Pin Assignment ............................................................... 3 - 6

3-7 XFP Connectors Pin Assignment ............................................................... 3 - 7

3-8 FE Management RJ45 Connector Pin Assignment ................................... 3 - 9

3-9 FE Management LEDs Signification .......................................................... 3 - 9

3-11 Serial console terminal cable interface: RJ45 Female to DB9 Female .... 3 - 10

3-10 Serial Port (RJ45) Pin Assignment .......................................................... 3 - 10

3-12 Hot Swap LED Signification ..................................................................... 3 - 14

3-13 PM Status LED Signification .................................................................... 3 - 14

3-14 Standard Fabric Interface LED Signification ............................................ 3 - 14

3-15 Extendes Fabric Interface LED Signification ............................................ 3 - 14

3-16 Copper Uplink LEDs Signification ............................................................ 3 - 15

3-17 FE Management LEDs Signification ........................................................ 3 - 15

3-18 Connector J1 Pinout ................................................................................ 3 - 16

3-19 Connector J2 Pinout ................................................................................ 3 - 17

3-20 Connector J3 Pinout ................................................................................ 3 - 17

3-21 Connector J4 Pinout ................................................................................ 3 - 18

3-22 Connector J5 Pinout ................................................................................ 3 - 19

4-1 POST routines and error codes ................................................................. 4 - 8

4-2 POST Boot Steps ....................................................................................... 4 - 8

4-3 CP6923 sensors ...................................................................................... 4 - 10

4-4 Health LED state ...................................................................................... 4 - 12

4-5 Activity LED state ..................................................................................... 4 - 12

4-6 FLASH Partition Scheme (64MB) ............................................................ 4 - 13

CP6923 Preface

Page vii CP6923 User Guide

List of Figures

3-1 Functional Block Diagram CP6923 ............................................................ 3 - 2

3-2 Front Panels of the CP6923 .................................................................... 3 - 13

CP6923 Preface

Page viii CP6923 User Guide

Proprietary Note

This document contains information proprietary to Kontron AG. It may not be copied or trans­mitted by any means, disclosed to others, or stored in any retrieval system or media without
the prior written consent of Kontron AG or one of its authorized agents.

The information contained in this document is, to the best of our knowledge, entirely correct.
However, Kontron AG cannot accept liability for any inaccuracies or the consequences thereof,
or for any liability arising from the use or application of any circuit, product, or example shown
in this document.

Kontron AG reserves the right to change, modify, or improve this document or the product de­scribed herein, as seen fit by Kontron AG without further notice.

Trademarks

Kontron AG and the Kontron logo are trade marks owned by Kontron AG, Germany. In addition,
this document may include names, company logos and trademarks, which are registered trade­marks and, therefore, proprietary to their respective owners.

Environmental Protection Statement

This product has been manufactured to satisfy environmental protection requirements where
possible. Many of the components used (structural parts, printed circuit boards, connectors,
batteries, etc.) are capable of being recycled.

Final disposition of this product after its service life must be accomplished in accordance with
applicable country, state, or local laws or regulations.

CP6923 Preface

Page ix CP6923 User Guide

Explanation of Symbols

For Your Safety

Your new Kontron product was developed and tested carefully to provide all features necessary
to ensure its compliance with electrical safety requirements. It was also designed for a long
fault-free life. However, the life expectancy of your product can be drastically reduced by im­proper treatment during unpacking and installation. Therefore, in the interest of your own safety
and of the correct operation of your new Kontron product, you are requested to conform with
the following guidelines.

CE Conformity

This symbol indicates that the product described in this manual is in compli­ance with all applied CE standards. Please refer also to the section “Applied
Standards” in this manual.

Caution, Electric Shock!

This symbol and title warn of hazards due to electrical shocks (> 60V) when
touching products or parts of them. Failure to observe the precautions indi­cated and/or prescribed by the law may endanger your life/health and/or
result in damage to your material.

Please refer also to the section “High Voltage Safety Instructions” on the fol­lowing page.

Warning, ESD Sensitive Device!

This symbol and title inform that electronic boards and their components are
sensitive to static electricity. Therefore, care must be taken during all han­dling operations and inspections of this product, in order to ensure product
integrity at all times.

Please read also the section “Special Handling and Unpacking Instructions”
on the following page.

Warning!

This symbol and title emphasize points which, if not fully understood and
taken into consideration by the reader, may endanger your health and/or
result in damage to your material.

Note...

This symbol and title emphasize aspects the reader should read through
carefully for his or her own advantage.

CP6923 Preface

Page x CP6923 User Guide

Safety Instructions

Warning!

All operations on this device must be carried out by sufficiently skilled person­nel only.

Warning!

Do not connect a switch port to a telephone line.

Warning!

For intallation in a Hot-Plug system, observe the safety instructions specific to
the system. Read the relevant documentation.

Caution, Electric Shock!

High voltages are present inside the chassis when the unit’s power cord is
plugged into an electrical outlet. Turn off system power, turn off the power
supply, and then disconnect the power cord from its source before removing
the chassis cover. Turning off the system power switch does not remove
power to components.

Caution, Laser Light!

Laser light from fiber-optic transmission cables and components can damage
your eyes. The laser components plugged into the switch are Class 1 laser
components. Class 1 laser is considered incapable of producing damaging
radiation levels during normal operation or maintenance.
To avoid damaging your eyes and to continue safe operation in case of
abnormal circumstances:

• Never look directly into the outlets of fiber-optic transmission compo-

nents or fiber-optic cables with unprotected eyes.

• Never allow fiber-optic transmission path to operate until all the connec-

tions have been made.

• Always fit protective plugs to any unused ports of the switch.

CP6923 Preface

Page xi CP6923 User Guide

Special Handling and Unpacking Instructions

Do not handle this product out of its protective enclosure while it is not used for operational pur­poses unless it is otherwise protected.

Whenever possible, unpack or pack this product only at EOS/ESD safe work stations. Where
a safe work station is not guaranteed, it is important for the user to be electrically discharged
before touching the product with his/her hands or tools. This is most easily done by touching a
metal part of your system housing.

It is particularly important to observe standard anti-static precautions when changing mezza­nines, ROM devices, jumper settings etc. If the product contains batteries for RTC or memory
back-up, ensure that the board is not placed on conductive surfaces, including anti-static plas­tics or sponges. They can cause short circuits and damage the batteries or conductive circuits
on the board.

ESD Sensitive Device!

Electronic boards and their components are sensitive to static electricity.
Therefore, care must be taken during all handling operations and inspections
of this product, in order to ensure product integrity at all times.

CP6923 Preface

Page xii CP6923 User Guide

General Instructions on Usage

In order to maintain Kontron’s product warranty, this product must not be altered or modified in
any way. Changes or modifications to the device, which are not explicitly approved by Kontron
AG and described in this manual or received from Kontron’s Technical Support as a special
handling instruction, will void your warranty.

This device should only be installed in or connected to systems that fulfill all necessary techni­cal and specific environmental requirements. This applies also to the operational temperature
range of the specific board version, which must not be exceeded. If batteries are present their
temperature restrictions must be taken into account.

In performing all necessary installation and application operations, please follow only the in­structions supplied by the present manual.

Keep all the original packaging material for future storage or warranty shipments. If it is neces­sary to store or ship the board please re-pack it as nearly as possible in the manner in which it
was delivered.

Special care is necessary when handling or unpacking the product. Please, consult the special
handling and unpacking instruction on the previous page of this manual.

CP6923 Preface

Page xiii CP6923 User Guide

Two Year Warranty

Kontron AG grants the original purchaser of Kontron’s products a TWO YEAR LIMITED HARDWARE

WARRANTY

as described in the following. However, no other warranties that may be granted or

implied by anyone on behalf of Kontron are valid unless the consumer has the express written
consent of Kontron AG.

Kontron AG warrants their own products, excluding software, to be free from manufacturing
and material defects for a period of 24 consecutive months from the date of purchase. This war­ranty is not transferable nor extendible to cover any other users or long-term storage of the
product. It does not cover products which have been modified, altered or repaired by any other
party than Kontron AG or their authorized agents. Furthermore, any product which has been,
or is suspected of being damaged as a result of negligence, improper use, incorrect handling,
servicing or maintenance, or which has been damaged as a result of excessive current/voltage
or temperature, or which has had its serial number(s), any other markings or parts thereof al­tered, defaced or removed will also be excluded from this warranty.

If the customer’s eligibility for warranty has not been voided, in the event of any claim, he may
return the product at the earliest possible convenience to the original place of purchase, togeth­er with a copy of the original document of purchase, a full description of the application the
product is used on and a description of the defect. Pack the product in such a way as to ensure
safe transportation (see our safety instructions).

Kontron provides for repair or replacement of any part, assembly or sub-assembly at their own
discretion, or to refund the original cost of purchase, if appropriate. In the event of repair, re­funding or replacement of any part, the ownership of the removed or replaced parts reverts to
Kontron AG, and the remaining part of the original guarantee, or any new guarantee to cover
the repaired or replaced items, will be transferred to cover the new or repaired items. Any ex­tensions to the original guarantee are considered gestures of goodwill, and will be defined in
the “Repair Report” issued by Kontron with the repaired or replaced item.

Kontron AG will not accept liability for any further claims resulting directly or indirectly from any
warranty claim, other than the above specified repair, replacement or refunding. In particular,
all claims for damage to any system or process in which the product was employed, or any loss
incurred as a result of the product not functioning at any given time, are excluded. The extent
of Kontron AG liability to the customer shall not exceed the original purchase price of the item
for which the claim exists.

Kontron AG issues no warranty or representation, either explicit or implicit, with respect to its
products’ reliability, fitness, quality, marketability or ability to fulfil any particular application or
purpose. As a result, the products are sold “as is,” and the responsibility to ensure their suit­ability for any given task remains that of the purchaser. In no event will Kontron be liable for
direct, indirect or consequential damages resulting from the use of our hardware or software
products, or documentation, even if Kontron were advised of the possibility of such claims prior
to the purchase of the product or during any period since the date of its purchase.

Please remember that no Kontron AG employee, dealer or agent is authorized to make any
modification or addition to the above specified terms, either verbally or in any other form, written
or electronically transmitted, without the company’s consent.

CP6923 Introduction

Page 1 - 1 CP6923 User Guide

Introduction

Chapter

1

CP6923 Introduction

Page 1 - 2 CP6923 User Guide

1. Introduction

1.1 Product Overview

The Kontron CP6923 is a PICMG 2.16 compliant Standard Fabric 6U CompactPCI managed
Gigabit Ethernet Switch with 24 channels.

Four different variants of the CP6923 are available:

CP6923-C (Copper)

In addition to the 20 backplane links (19 node boards and one fabric board), four GbE switch
ports are available on the front plate via RJ45 connectors.

CP6923-O (Optical)

The four front plate ports are implemented as SFP cages, two 10 GbE ports are accessible via
XFP cages. SFP and XFP modules are not included with the CP6923-O.

CP6923-R (Rear)

All 24 GbE ports connect to the backplane (four additional extended node board links).

CP6923-E (Extended)

Like the CP6923-R but for operation in extended temperature range of -40 to +85°C.

1.1.1 CP6923 Features

The board is composed of the following building blocks:

• Ethernet Infrastructure

• Ethernet Switch Controller and Memory

•IPMI

• Power Supply

Ethernet Infrastructure

• Broadcom StrataXGS®III Chip BCM56500: 24 Port Gigabit Ethernet Layer-3 Switch
(CP6923-C, CP6923-E)

• BCM56502: 24 Port Gigabit Ethernet Layer-3 Switch with 2 10GbE Uplinks (CP6923­O)

• PCI 32b/66MHz Management IF.

• Line rate switching for all packet sizes and conditions.

• Supports 19x Link Ports to Node boards [10/100/1000Base-T via external Copper PHY]

• Supports 1x Link Port to Fabric board [10/100/1000Base-T via external Copper PHY].

CP6923 Introduction

Page 1 - 3 CP6923 User Guide

• Supports optional 4x 10/100/1000Base-T Copper based uplinks to front (CP6923-C).

• Supports optional 4x 1GbE SFP uplinks to front (CP6923-O).

• Supports optional 2x 10GBase-R XFP uplinks to front (CP6923-O).

• Supports optional 4x Link Ports to extended Node boards [10/100/1000Base-T via exter­nal Copper PHY] to rear (CP6923-R and CP6923-E)

• Broadcom BCM5488 OctalPHY for copper based links

• Broadcom BCM8704 single 10GbE PHY for uplinks (CP6923-O)

Ethernet Switch Controller and System Memory

• Socketless PowerPC IBM PPC405GPr 400 MHz

• Used for switch provisioning, diagnostics and protocol handling

• 256 MBytes SDRAM 133 MHz

• 64 MBytes FLASH

IPMI

• Renesas H8S2166 16-bit µController (PM)

• PICMG 2.9 / IPMI 1.5 compliant

• Renesas R5F21144 Firmware Update Manager (FWUM)

• 512 kByte Flash (Boot Image)

• 64 kByte PROM (FRU)

• Voltage, current monitoring with ADC of PM

• Temperature monitoring with LM83

Power Supply

• 5V and 3.3V only board, no 12V or -12V required

• IPMB_PWR or 5V used for 3.3V PM (generated by LDO)

• Hot Swap support

• PoL converter with 2.5V, 1.8V, 1.25V and 1.0V

Miscellaneous

• Management connection to Switch Controller (CLI, SNMP) via RS232 and Fast Ethernet
via RJ45 connectors.

• Management connection to Switch Controller via RS232 on J4.

• Serial EEPROM for FRU storage (serial ID).

• Board support package includes Montavista Linux, device drivers, protocol stacks and di­agnostics.

• Metalwork compliant to PICMG® 2.0 6U/4HP (233.35 mm by 160 mm) / (20.32 mm typ.)

CP6923 Introduction

Page 1 - 4 CP6923 User Guide

1.1.2 General compliances

The CP6923 is conform to the following specifications:

• PICMG® 2.0 R3.0 CompactPCI® Specification, as amended by ERN 2.0-3.0-002;

• PICMG® 2.1 R2.0 CompactPCI® Hot Swap Specification;

• PICMG® 2.5 R1.0 CompactPCI® Computing Telephony Specification;

• PICMG® 2.9 R1.0 CompactPCI® System Management Specification;

• PICMG® 2.10 R1.0 Keying of CompactPCI® System Management Specification;

• PICMG® 2.16 R1.0 Sep. 5, 2005 Packet Switching Backplane Specification

• Intelligent Platform Management Interface Specification V1.5

• IEEE 802.3, 2000

• IEEE 802.3ae 2002

• Small Form-factor Pluggable (SFP) Transceiver MultiSource Agreement (MSA), Sept.
14th, 2000

• SFF Committee, INF-80077i, rev. 4.0, 10 Gigabit Small Form Factor Pluggable Module
(XFP)

• ANSI/Vita 30.1-2002 American National Standard for 2mm Connector Equipment Prac­tice on Conduction Cooled Euroboards (option).

1.1.3 Hot Swap Capability

The board supports Full Hot Swap capability as required by PICMG 2.1 R2.0. It can be removed
or inserted in a powered slot. Please refer to the PICMG 2.1 R2.0 specification for additional
details.

CP6923 Introduction

Page 1 - 5 CP6923 User Guide

1.2 Technical Specification

Table 1-1: CP6923 Main Specifications

CP6923 SPECIFICATIONS

Mechanical • 6U/4HP form factor mechanically compliant to PICMG 2.0

• 233.35 mm x 160 mm

• Weight: 600g typ

Power Requirements

• Maximum: 50W

• Operating Voltage: 5V (5A max), 3.3V (7A max)

Temperature

• Operating: 0°C to +55°C (standard version) and -40°C to +85°C for temperature

extended version (CP6923-E)

Humidity Designed to meet or exceed the following:

• Bellcore GR63, Section 4.1

• Operating: 15%-90% (non-condensing) at 55°C (131°F)

• Non-Operating: 5%-95% (non-condensing) at 40°C (104°F)

Altitude Designed to meet or exceed the following:

• Operating: 4000 m (13123ft)

• Non-operating: 15000 m (49212 ft)

Vibration Designed to meet or exceed the following:

• Bellcore GR-63, Section 4.4

• Operating: 1.0G, 5-500Hz each axis

• Non-operating: 0.5G, 5-50Hz, 3.0G, 50-500Hz each axis

Shock Designed to meet or exceed the following:

• DIN/IEC 60068-2-27

• MIL-STD-810E, Method 516

• 30G, half-sine 11ms, each axis

Safety Designed to meet or exceed the following:

• UL 60950-1

• EN 60950

• LVD 73/23/EEC

• Denan Law

EMC Designed to meet or exceed the following:

• FCC 47 CFR Part 15, Subpart B

• EN55022, EN55024

• EN 300 386

Reliability

• MTBF: > 198,000 hours @ 30°C/86°F (Telcordia SR-332, Issue 1)

CP6923 Introduction

Page 1 - 6 CP6923 User Guide

LEDs

• IPMB Status LED

• Hot Swap LED

• “Healthy“ and “Activity“ LED (LED 1 and 2, green)

• Gigabit Ethernet Link Port Status LED

• 20 LEDs (status indication for standard fabric interfaces)

• PowerPC 10/100BASE-T Management LEDs

• Green LED: Link/Activity

• Amber LED: Speed

Board Management

• based on IPMI 1.5

• FRU Management

• Sensors (Voltage, Current, Temperature, Fuse)

• Status and Alerting

• Hot Swap Management

• Local SEL

Table 1-1: CP6923 Main Specifications (Continued)

CP6923 SPECIFICATIONS

CP6923 Introduction

Page 1 - 7 CP6923 User Guide

1.3 Software Support

The following table contains information related to software supported by the CP6923.

Table 1-2: CP6923 Software Specification

CP6923 SPECIFICATIONS

General • Reliable field upgrades for all software components

• Dual boot images with roll-back capability

• Management via SNMP and Command Line Interface

• System access via TELNET, SSH and serial line

• Hot-Swap support (IPMI)

Ethernet/Bridging

• Static link aggregation (IEEE 802.3ad) on any port combination

• Classic and rapid spanning tree algorithms shall be supported (IEEE 802.1D,

IEEE 802.1w)

• Quality Of Service on all ports (IEEE 802.1p)

• Full Duplex operation and flow control on all ports (IEEE 802.3x)

• Static MAC filtering

• Port Authentication (IEEE 802.1X)

• Auto negotiation of speeds and operational mode on all external GE interfaces

as well as on all base fabric interfaces

• Layer 2 multicast services using GARP/GMRP (IEEE 802.1p)

• VLAN support including VLAN tagging (IEEE 802.3ac), dynamic VLAN registration

with GARP/GVRP (IEEE 802.1Q) and Protocol based VLANs (IEEE 802.1v)

• Double VLAN tagging

• Port Mirroring

IP Routing

• Redundancy of routing functionality using a second switch hub board

• IPv4 Forwarding on all base channels and connected uplink ports

• Quality of service according to the DiffServ standards

• ARP for all routable interfaces

• ICMP for all routable interfaces

• OSPF routing protocol version 2

• RIP routing protocol version 2

• VRRP (virtual router redundancy protocol) for transparent fail over of default routers

• IGMP snooping

QoS

• CoS (Class of Service )

• DifffServ (Differentiated Services)

• ACL (Access Control List)

CP6923 Introduction

Page 1 - 8 CP6923 User Guide

IP Multicast

• DVMRP

• PIM-DM

• PIM-SM

• IGMP (Internet Group Message Protocol) v2 and v3

• IGMP Proxy

Applications

• NTP client for retrieving accurate time and date information

• DHCP server

• Onboard event management

• Test and trace facilities

• POST (power on self tests) diagnostics

• Standards based SNMP implementation supporting SNMP v1, v2 and v3

for monitoring and management purposes

• Persistent storage of configuration across restarts

• Support for retrieving and installing multiple configurations

Supported MIBS

• For a list of supported MIBs please see “Supported MIBs” on page 4 - 5

Bootloader u-boot Version 1.1.2

• POST

• multi image support

• loadable bootimage via network (bootp/tftp)

• reliable field upgradable

• H/W protected

• KCS interface to PM

• serial console support

Operating System

• MontaVista Linux Professional Edition 3.1

Table 1-2: CP6923 Software Specification (Continued)

CP6923 SPECIFICATIONS

Installation

Chapter

2

Page 2 - 1 CP6923 User Guide

CP6923 Installation

CP6923 Installation

Page 2 - 2 CP6923 User Guide

2. Installation

The CP6923 has been designed for easy installation. However, the following standard precau­tions, installation procedures, and general information must be observed to ensure proper in­stallation and to preclude damage to the board, other system components, or injury to
personnel.

2.1 Safety Requirements

The following safety precautions must be observed when installing or operating the CP6923.
Kontron assumes no responsibility for any damage resulting from failure to comply with these
requirements.

Warning!

Due care should be exercised when handling the board due to the fact that the
heat sink can get very hot. Do not touch the heat sink when installing or
removing the board.

In addition, the board should not be placed on any surface or in any form of stor­age container until such time as the board and heat sink have cooled down to
room temperature.

ESD Equipment!

This CompactPCI board contains electrostatically sensitive devices. Please
observe the necessary precautions to avoid damage to your board:

• Discharge your clothing before touching the assembly. Tools must be dis­charged before use.

• When unpacking a static-sensitive component from its shipping carton,

do not remove the component's antistatic packing material until you are
ready to install the component in a computer. Just before unwrapping the
antistatic packaging, be sure you are at an ESD workstation or grounded.
This will discharge any static electricity that may have built up in your
body.

• When transporting a sensitive component, first place it in an antistatic

container or packaging.

• Handle all sensitive components at an ESD workstation. If possible, use
antistatic floor pads and workbench pads.

• Handle components and boards with care. Do not touch the components
or contacts on a board. Hold a board by its edges or by its metal mounting
bracket.

• Do not handle or store system boards near strong electrostatic, electro-

magnetic, magnetic, or radioactive fields.

CP6923 Installation

Page 2 - 3 CP6923 User Guide

2.2 CP6923 Initial Installation Procedures

The following procedures are applicable only for the initial installation of the CP6923 in a sys­tem. Procedures for standard removal and hot swap operations are found in their respective
chapters.

To perform an initial installation of the CP6923 in a system proceed as follows:

1. Ensure that the safety requirements indicated in section 2.1. are observed.

2. Ensure that the board is properly configured for operation in accordance with application
requirements before installing. For information regarding the configuration of the CP6923
refer to the CLI Reference Manual.

3. To install the CP6923 perform the following:

a. Ensure that no power is applied to the system before proceeding.

b. Carefully insert the board into the slot designated by the application requirements for

the board until it makes contact with the backplane connectors.

c. Using both ejector handles, engage the board with the backplane. When the ejector

handles are locked, the board is engaged.
d. Fasten the front panel retaining screws.
e. Connect all external interfacing cables to the board as required.
f. Ensure that the board and all required interfacing cables are properly secured.

Warning!

Failure to comply with the instruction below may cause damage to the
board or result in improper system operation.

Warning!

Care must be taken when applying the procedures below to ensure that
neither the CP6923 nor other system boards are physically damaged by
the application of these procedures.

Warning!

DO NOT push the board into the backplane connectors. Use the ejector
handles to seat the board into the backplane connectors.

CP6923 Installation

Page 2 - 4 CP6923 User Guide

4. The CP6923 is now ready for operation.

2.3 Standard Removal Procedures

To remove the board proceed as follows:

1. Ensure that the safety requirements indicated in section 2.1. are observed.

2. Ensure that no power is applied to the system before proceeding.

3. Disconnect any interfacing cables that may be connected to the board.

4. Unscrew the front panel retaining screws.

5. Disengage the board from the backplane by first unlocking the board ejection handles
and then by pressing the handles as required until the board is disengaged.

6. After disengaging the board from the backplane, pull the board out of the slot.

7. Dispose of the board as required.

Warning!

During power-up, the 3.3V input power supply must be able to provide a
minimum peak current of 10A to the CP6923. This applies for each
CP6923 in a given systems.

Failure to comply with the above warning may result in damage to or
improper operation of the CP6923.

Warning!

Care must be taken when applying the procedures below to ensure that
neither the CP6923 nor system boards are physically damaged by the
application of these procedures.

Warning!

Due care should be exercised when handling the board due to the fact
that the heat sink can get very hot. Do not touch the heat sink when
changing the board.

CP6923 Installation

Page 2 - 5 CP6923 User Guide

2.4 Software Installation

The CP6923 comes as a pre-installed system with all necessary OS, filesystem, drivers and
applications factory-installed with default configurations.

Updating the Software with new operating system or applications or new versions is provided
by a dedicated update mechanism, which is described in Chapter 4.

Hardware Description

Chapter

3

Page 3 - 1 CP6923 User Guide

CP6923 Hardware Description

CP6923 Hardware Description

Page 3 - 2 CP6923 User Guide

3. Hardware Description

This chapter describes the board specific items of the CP6923.

Figure 3-1: Functional Block Diagram CP6923

OPTICA L

Optio n

COPPER

Optio n

SFP

REAR Op t io n

OPTICAL Option

IPMC

IPMB 1

IPMB 0

256MB

SDRAM

+ECC

32MB

FLA SH

(16bit)

J5

J4

J3

J2

J1

XF P

XF P

RJ45

RJ45

PPC405GPr

400MHz

24 Port

Laye r 2/3

Fab ri c

Sw itch
BCM56500
BCM56502

10GbE

PHY

BCM8704

10GbE

PHY

BCM8704

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

Pow er :

Ho t Sw ap

3.3V, 5V

CPLD

10/100

Bas e - T

PHY

BLUE LED

Handle

Switch

JTAG

RS2 32

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

PCI

32bit

66MHz

Back-

plane

Link

State

LED

RED LED

Reset

Har d w ar e

Monitor

HX5020NL
Dual M ag .

HX5020NL
Dual M ag .

RJ45

RJ45

RJ45

RJ45

BCM

5488S

Octal

BCM

5488S

Oct al

BCM

5488S

Octal

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

H5020NL

Dual M ag .

SFP

SFP

SFP

CP6923 Hardware Description

Page 3 - 3 CP6923 User Guide

The main building blocks of the CP6923 are:

• Fabric Switch

• CPU and Memory

•IPMI

• Power Supply with Hot Swap

3.1 Fabric Switch

The fabric switch is composed of a Broadcom Strata XGSIII 24 port GbE multilayer switch and
three Broadcom BCM5488S octal 10/100/1000BASE-T PHYs. Two Broadcom BCM8704
10GbE PHYs are used in the CP6923-O variant. 20 Gigabit Ethernet Links are connected to
the backplane, the four remaining GbE ports are used as 10/100/1000BASE-T Uplink ports
(CP6923-C), SFP optical links (CP6923-O) or 4 additional extended fabric links (CP6923-R
and CP6923-E). The Broadcom Strata XGSIII switch supports SGMII to the PHYs. and SER­DES interfaces to the SFP. Two types of Broadcom Strata XGSIII are currently defined:

• BCM56502, with advanced Layer 3 switching features for IPv4 and IPv6 routing (full fea­tured) and 2 10GbE ports.

• BCM56500, with advance Layer 3 switching features for IPv4 and IPv6 routing (full fea­tured).

For details on the Broadcom Strata XGSIII features please refer to the corresponding
datasheets.

3.1.1 PCI Interface

The PCI interface is a 32bit/66MHz system to control the fabric switch. The internal PCI arbiter
of the PowerPC 405 GPr is used. There is no connection to the backplane.

The fabric switch has the following PCI-ID:

3.1.2 Port Mapping

The switch-port-numbers and MDIO-addresses are numbered from 1 to 24. This is concurrent
with the port identification in the CLI (0/1 to 0/24).

Table 3-1: PCI-ID

PCI-Slot IDSEL Device

1 0 Fabric Switch: Broadcom BCM 56502 or BCM56500

CP6923 Hardware Description

Page 3 - 4 CP6923 User Guide

Table 3-2: Port Mapping

CLI ID CP6923-C CP6923-O CP6923-R/CP6923-E

Interface LED Interface LED Interface LED

0/1 FL08 8 FL08 8 FL08 8
0/2 FL07 7 FL07 7 FL07 7
0/3 FL06 6 FL06 6 FL06 6
0/4 FL05 5 FL05 5 FL05 5
0/5 FL04 4 FL04 4 FL04 4
0/6 FL03 3 FL03 3 FL03 3
0/7 FL02 2 FL02 2 FL02 2
0/8 FL01 1 FL01 1 FL01 1
0/9 FL11 11 FL11 11 FL11 11
0/10 FL10 10 FL10 10 FL10 10
0/11 FL09 9 FL09 9 FL09 9
0/12 FL_f F FL_f F FL_f F
0/13 Gig.Ethernet20 On RJ45 SFP20 SFP 20 FL20 20
0/14 Gig.Ethernet21 On RJ45 SFP21 SFP 21 FL21 21
0/15 Gig.Ethernet22 On RJ45 SFP22 SFP 22 FL22 22
0/16 Gig.Ethernet23 On RJ45 SFP23 SFP 23 FL23 23
0/17 FL19 19 FL19 19 FL19 19
0/18 FL18 18 FL18 18 FL18 18
0/19 FL17 17 FL17 17 FL17 17
0/20 FL16 16 FL16 16 FL16 16
0/21 FL15 15 FL15 15 FL15 15
0/22 FL14 14 FL14 14 FL14 14
0/23 FL13 13 FL13 13 FL13 13
0/24 FL12 12 FL12 12 FL12 12
0/25 - - XFP1 XFP 1 - ­0/26 - - XFP2 XFP 2 - -

Note...

• CP6923-E and CP6923-R variants support Fabric Links 20 to 23 to the

backplane

• CP6923-C supports Fabric Links 20 to 23 on the Frontpanel (RJ45,

1000Base-T)

• CP6923-O supports Fabric Links 20 to 23 on the Frontpanel (SFP,

1000Base-X) and 2 10GbE Uplinks via XFP

CP6923 Hardware Description

Page 3 - 5 CP6923 User Guide

3.1.3 Standard Fabric Links

The board supports 19x 10/100/1000BASE-T Standard Fabric links and one 10/100/
1000BASE-T link to the redundant CompactPCI Switch in the system (F-Link).

The pin assignment for J3 and J5 corresponds to the PICMG 2.16, please refer to Chapter 3.6
for a detailed pin assignment.

3.1.4 Extended Fabric Uplinks

The CP6923-R and CP6923-E variants support 4x10/100/1000BASE-T Extended Fabric links
(FL20 to FL23) .

The pin assignment for J4 corresponds to the PICMG 2.16, please refer to Chapter 3.6 for a
detailed pin assignment.

3.1.5 Copper Uplinks

The CP6923-C supports four 10/100/1000BASE-T RJ45 fabric switch uplinks to the front panel.
The switch is connected to the RJ45 connectors with integrated magnetics and status LEDs on
the front panel via external PHYs.

The RJ45 connectors have the following pin assignment (MDI-X hub mode).

Table 3-4: Copper Uplink RJ45 Pin Assignment

Contact MDI-X

1BI_DB+

2BI_DB-

3BI_DA+

4 BI_DD+

5 BI_DD-

6BI_DA-

7 BI_DC+

8 BI_DC-

Green Yellow/Green

1

8

CP6923 Hardware Description

Page 3 - 6 CP6923 User Guide

3.1.6 1000BASE-xX SFP Uplinks

The CP6923-O supports four Gigabit Ethernet SFP fabric switch uplinks to the front panel. The
SFPs uplink ports are according the Small Form-factor Pluggable (SFP) Transceiver
MultiSource Agreement (MSA), Sept. 14th, 2000 (see also Application Note CP6923-AN­#0002-Optical Modules.pdf).

The fabric switch controls the SFPs via an I²C bus. The SFP connectors have the following pin
assignment:

Table 3-5: Copper Uplink LEDs Signification

Green LED (Link/Activity)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

Green/Yellow LED (Speed)

OFF 10BASE-T

YELLOW ON 100BASE-Tx

GREEN ON 1000BASE-T

Table 3-6: SFP Connectors Pin Assignment

Signal Contact Contact Signal

GND 1 20 GND

TX_FAULT 2 19 TD-

TX_DIS 3 18 TD+

MODDEF2 4 17 GND

MODDEF1 5 16 3.3V TX

MODDEF0 6 15 3.3V RX

R_SEL 7 14 GND

LOS 8 13 RD+

GND 9 12 RD-

GND 10 11 GND

CP6923 Hardware Description

Page 3 - 7 CP6923 User Guide

3.1.7 10GBASE-xX XFP Uplinks

The CP6923-O supports two 10 Gigabit Ethernet XFP Fabric Switch uplinks to the front panel.
The fabric switch is connected via a BCM8704 10GbE transceiver to XFP connectors.

The BCM8704 is managed from the Fabric Switch XMDIO management interface (MDC/
MDIO).

The XFP uplink ports are according the SFF Committee, INF-80077i, rev. 4.0, 10 Gigabit Small
Form Factor Pluggable Module (XFP). XFP Modules with -5.0V are not supported (see also Ap­plication Note CP6923-AN-#0002-Optical Modules.pdf).

The XFP connectors have the following pin assignment.

Caution!

Do not look into the laser beam!
The SFP modules are fitted with a class 1 or 1M laser. To avoid possible
exposure to hazardous levels of invisible laser radiation, do not exceed max­imum ratings.

Table 3-7: XFP Connectors Pin Assignment

Signal Contact Contact Signal

GND 1 30 GND

NC 2 29 TD+

MOD_DE_SEL 3 28 TD-

INTERRUPT# 4 27 GND

TX_DIS 5 26 GND

5.5V 6 25 REFCLK-

GND 7 24 REFCLK+

3.3V 8 23 GND

3.3V 9 22 1.8V

SCL 10 21 PDWN/RST

SDA 11 20 1.8V

MOD_ABS 12 19 GND

MOD_NR 13 18 RD+

RX_LOS 14 17 RD-

GND 15 16 GND

CP6923 Hardware Description

Page 3 - 8 CP6923 User Guide

3.2 CPU and Memory

The CPU is an IBM PowerPC 405 GPr 400MHz 32-bit RISC processor with 16KB D-cache.

3.2.1 PCI Interface

The PCI interface is a 32bit/66Hz system to control the on-board fabric switch. The internal PCI
arbiter of the PowerPC 405 GPr is used.

The virtual PCI slot 1 (IDSEL 0) is assigned to the Fabric Switch.

3.2.2 SDRAM

The CPU operates with a 133MHz and 32 bit SDRAM interface. Five 256Mbit devices, soldered
directly onto the PCB, provide 256 Mbyte of SDRAM plus 64 MByte for ECC.

3.2.3 FLASH

The CPU has 64Mbyte of Flash Memory. It is connected to the 16 bit peripheral data bus of the
PowerPC 405 GPr.

The upper sectors containing the bootloader are write-protected. Jumper FWPD disables the
protection.

3.2.4 Fast Ethernet Management Interface

The internal Fast Ethernet MAC of the PowerPC 405 GPr is used as a management interface.
A Fast Ethernet PHY transceiver and a RJ45 Connector with integrated magnetics and two
LED's, located on the front panel, are used to complete the network interface.

The default setting of the PHY is to operate in auto-negotiation enabled mode, 10/100, Full or
Half duplex. The LEDs indicate Link/Activity (LED: Green) and Speed (LED: Yellow).

Caution!

Do not look into the laser beam!
The XFP modules are fitted with a class 1 or 1M laser. To avoid possible
exposure to hazardous levels of invisible laser radiation, do not exceed maxi­mum ratings.

CP6923 Hardware Description

Page 3 - 9 CP6923 User Guide

The standard RJ45 connector has the following pin assignment:

3.2.5 RS232 Management Interface

The UART0 interface of the PowerPC is connected to the front plate serial RJ45 port and alsoto
the backplane connector J4. This allows accessing the RS232 management port using an ap­propriate RIO module.

Table 3-8: FE Management RJ45 Connector Pin Assignment

Pin

Function

1TX+

2TX-

3RX+

4NU

5NU

6RX-

7NU

8NU

Table 3-9: FE Management LEDs Signification

Green LED (Link/Activity)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

Yellow LED (Speed)

OFF 10BASE-T

ON 100BASE-Tx

green yellow

1 8

CP6923 Hardware Description

Page 3 - 10 CP6923 User Guide

External connection is established with a straight through Ethernet cable and a RJ45 (female)
to SubD (female) adapter if required. The adapter is described in the following table.

Table 3-11: Serial console terminal cable interface: RJ45 Female to DB9 Female

The pin assignment of the RIO RS232 connection is included in Chapter 3.6.

Table 3-10: Serial Port (RJ45) Pin Assignment

Signal Pin

RTS 1
DTR 2
TXD 3
GND 4
GND 5
RXD 6
DSR 7
CTS 8

8

1

RJ45 Female RJ45 Pin

Number

Signal Connected Description DB9 Pin

Number

DB9 Female

1 RTS Y Request To Send 8
2 DTR Y Data Terminal Ready 6
3 TXD Y Transmit 2
4 GND N Ground ­5 GND Y Ground 5
6 RXD Y Receive 3
7 DSR Y Data Set Ready 4
8 CTS N Clear To Send 7

- RI N Ring Indicator
(Not used)

9

Front View

- CD N Carrier Detect
(Not used)

1

Front View

CP6923 Hardware Description

Page 3 - 11 CP6923 User Guide

3.3 IPMI

The CP6923 board supports an intelligent hardware management system, based on the Intel­ligent Platform Management Interface Specification 1.5. The hardware management system
provides the ability to manage the power, cooling and interconnect needs of intelligent devices;
to monitor events; and to log events to a central repository intelligent FRU (Field Replaceable
Unit) contain a Peripheral Manager.

The main building blocks of the IPMI architecture of the CP6923 are:

• PM Peripheral Manager

• FUM Firmware Update Manager

• CPLD Complex Programmable Logical Device

3.3.1 PM

The Peripheral Manager is a 16-bit microcontroller for IPMI applications and it is compliant to
IPMI version 1.5 specification. The microcontroller has large on chip memory of 512 Kbyte
Flash and 40Kbyte SRAM. It provides six I²C interfaces to have access to the dedicated BMC
A and B and the on board peripheral devices such as SEEPROM, temperature sensor and oth­ers.

One LPC interface for communication between PM and PPC over KCS protocol is implemented
by CPLD. By default the PM is in reset until the FUM is alive.

A requirement for the PM is to monitor voltages and currents. The PM has 8 A/D converter in­puts for monitoring.

3.3.2 FUM

The Firmware Update Manager (FUM) is a microcontroller with embedded 16Kbyte data flash
ROM and 1Kbyte RAM.

The FUM is responsible for field upgrades, rollbacks and watchdog functions of the PM. The
microcontroller provides four SPI compatible memory devices. Each memory device has
256kbyte memory space. Two devices contain the current firmware for the PM and the other
two devices can be programmed with the new firmware for the PM. The FUM is able to update
the PM during the normal operation of the board. In the case of a fault during the update pro­cess, the FUM can reconfigure the PM with the last version of firmware that is kept in its mem­ory space. The FUM is also the watchdog timer for the PM. There are several control signals
to supervise the PM.

CP6923 Hardware Description

Page 3 - 12 CP6923 User Guide

3.4 Power Supply and Hot Swap

The CP6923 is supplied by the 3.3V and the 5.0V from the backplane. The PM on the CP6923
operates also with the IPMB_PWR from the backplane, when no other supply rails are avail­able.

The CP6923 supports Hot Swap according PIGMG® 2.1 R2.0. All supply voltages from the
backplane are enabled with a predefined ramp-up time. The inrush current is limited by Hot­Swap controllers.

3.4.1 Power Up and Power Down Sequencing

The CP6923 supports Hot Swapping and could be inserted into a live backplane. In this case
the longest 3.3V and the longest 5V pins from the backplane are used to supply the manage­ment devices CPLD and PM.

When the board is totally seated into the slot and the handle is closed, an active BRD_SEL#
signal triggers the CPLD to start the payload power up sequence.

The power down sequence is initiated by opening the handle switch. When the payload voltag­es are disabled, the CPLD turns on the blue LED. The board can be removed.

3.4.2 Reset

The reset chain is based on seven elements. The first element in the chain is the voltage supply
monitor, followed by the CPLD, FUM, IPMC, Payload voltage, PPC and finally the Fabric Inter­face.

The reset switch will perform a reset on the CPU when pressed for less than 1 second and a
complete board reset (including IPMI) when pressed for more than 2 seconds.

CP6923 Hardware Description

Page 3 - 13 CP6923 User Guide

3.5 Display Elements

Figure 3-2: Front Panels of the CP6923

CP6923-C

CP6923-0

CP6923-R, CP6923-E

CP6923 Hardware Description

Page 3 - 14 CP6923 User Guide

The tables below describe the Hot Swap and PM status LEDs:

The front panel provides 20 LEDS for status indication for the standard fabric interfaces. The
color is green with the following states:

The front panel provides 4 LEDS for status indication for the extended fabric interfaces or the
SFP optical connectors. The color is green with the following states:

Table 3-12: Hot Swap LED Signification

Blue LED (Hot Swap LED)

ON Ready for hot swap

OFF Normal Operation

Table 3-13: PM Status LED Signification

Green LED 1 and Green LED 2

Healthy ON = PM operational

Activity ON = PM activity

Table 3-14: Standard Fabric Interface LED Signification

Green LED (Standard Fabric Interface)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

Table 3-15: Extendes Fabric Interface LED Signification

Green LED (Extended Fabric Interface)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

CP6923 Hardware Description

Page 3 - 15 CP6923 User Guide

The RJ45 for 10/100/1000BASE-T Uplinks provide integrated LEDs; green and green/yellow:

The reset switch will perform a reset on the CPU when pressed for less than 1 second and a
complete board reset (including IPMI) when pressed for more than 2 seconds.

3.6 CompactPCI Bus Connector

The complete CompactPCI connector configuration comprises five connectors named J1 to J5.
Their functions are as follows:

• J1, J2: management, IPMB and power. PCI is not supported.

• J3, J4 and J5 have rear I/O interface functionality, providing GbE to the backplane or RIO
module and an RS232 interface to a RIO module

The board supports signaling voltages V(I/O) of either 3.3 V or 5 V. No keying is required on J1
which designates universal V(I/O).

Table 3-16: Copper Uplink LEDs Signification

Green LED (Link/Activity)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

Green/Yellow LED (Speed)

OFF 10BASE-T

YELLOW ON 100BASE-Tx

GREEN ON 1000BASE-T

Table 3-17: FE Management LEDs Signification

Green LED (Link/Activity)

OFF Port did not perform linkup

ON Port performed linkup but no activity

BLINKING Port performed linkup and there is activity

Yellow LED (Speed)

OFF 10BASE-T

ON 100BASE-Tx

CP6923 Hardware Description

Page 3 - 16 CP6923 User Guide

3.6.1 CompactPCI Connectors J1 and J2 Pinouts

The CP6923 is provided with two 2 mm x 2 mm pitch female CompactPCI bus connectors, J1
and J2.

Table 3-18: Connector J1 Pinout

Pin Row A Row B Row C Row D Row E Row F

25 V_5V_CPCI NC NC V_3V3_CPCI V_5V_CPCI GND
24 NC V_5V_CPCI V_IO_CPCI NC NC GND
23 V_3V3_CPCI NC NC V_5V_CPCI NC GND
22 NC GND V_3V3_CPCI NC NC GND
21 V_3V3_CPCI NC NC NC NC GND
20 NC GND V_IO_CPCI NC NC GND
19 V_3V3_CPCI NC NC GND NC GND
18 NC GND V_3V3_CPCI NC NC GND
17 V_3V3_CPCI IPMB0_SCL IPMB0_SDA GND NC GND
16 NC GND V_IO_CPCI NC NC GND
15 V_3V3_CPCI NC NC CPCI_BD_SEL# NC GND
14

Key Area

13
12
11 NC NC NC GND NC GND
10 NC GND V_3V3_CPCI NC NC GND
9NC NC NC GND NC GND
8 NC GND V_IO_CPCI NC NC GND
7NC NC NC GND NC GND
6 NC GND (1) V_3V3_CPCI NC NC GND
5 NC NC CPCI_PCI_RST# GND NC GND
4 V_5V_IPMB_PWR CPCI_HEALTHY# V_IO_CPCI NC NC GND
3 NC NC NC V_5V_CPCI NC GND
2 NC V_5V_CPCI NC NC NC GND
1 V_5V_CPCI NC NC NC V_5V_CPCI GND

CP6923 Hardware Description

Page 3 - 17 CP6923 User Guide

3.6.2 CompactPCI Rear I/O Connectors J3-J5 and Pinouts

The CP6923 conducts all backplane I/O signals connectors J3, J4 and J5. These are the GbE
interfaces and the rear RS232 interface.

For accessing these I/O signals with a rear I/O module, a special backplane is necessary. The
CP6923 is compatible with all standard 6U CompactPCI passive backplanes with rear I/O sup­port on the system slot.

Table 3-19: Connector J2 Pinout

Pin Row A Row B Row C Row D Row E Row F

22 CPCI_GA[4] CPCI_GA[3] CPCI_GA[2] CPCI_GA[1] CPCI_GA[0] GND
21 NC NC NC NC NC GND
20 NC NC NC NC NC GND
19 NC NC IPMB1_SDA IPMB1_SCL IPMB_ALERT# GND
18 NC NC NC NC NC GND
17 NC NC NC NC NC GND
16 NC NC NC NC NC GND
15 NC NC NC NC NC GND
14 NC NC NC NC NC GND
13 NC NC NC NC NC GND
12 NC NC NC NC NC GND
11 NC NC NC NC NC GND
10 NC NC NC NC NC GND
9NC NC NC NC NC GND
8NC NC NC NC NC GND
7NC NC NC NC NC GND
6NC NC NC NC NC GND
5NC NC NC NC NC GND
4NC NC NC NC NC GND
3NC NC NC NC NC GND
2NC NC NC NC NC GND
1NC NC NC NC NC GND

Table 3-20: Connector J3 Pinout

Pin Row A Row B Row C Row D Row E Row F

19 CPCI_SA[4] CPCI_SA[3] CPCI_SA[2] CPCI_SA[1] CPCI_SA[0] GND
18 FL_DA24_f+ FL_DA24_f- GND FL_DC24_f+ FL_DC24_f- GND
17 FL_DB24_f+ FL_DB24_f- GND FL_DD24_f+ FL_DD24_f- GND
16 FL_DA8+ FL_DA8- GND FL_DC8+ FL_DC8- GND
15 FL_DB8+ FL_DB8- GND FL_DD8+ FL_DD8- GND
14 FL_DA7+ FL_DA7- GND FL_DC7+ FL_DC7- GND
13 FL_DB7+ FL_DB7- GND FL_DD7+ FL_DD7- GND

CP6923 Hardware Description

Page 3 - 18 CP6923 User Guide

12 FL_DA6+ FL_DA6- GND FL_DC6+ FL_DC6- GND
11 FL_DB6+ FL_DB6- GND FL_DD6+ FL_DD6- GND
10 FL_DA5+ FL_DA5- GND FL_DC5+ FL_DC5- GND
9 FL_DB5+ FL_DB5- GND FL_DD5+ FL_DD5- GND
8 FL_DA4+ FL_DA4- GND FL_DC4+ FL_DC4- GND
7 FL_DB4+ FL_DB4- GND FL_DD4+ FL_DD4- GND
6 FL_DA3+ FL_DA3- GND FL_DC3+ FL_DC3- GND
5 FL_DB3+ FL_DB3- GND FL_DD3+ FL_DD3- GND
4 FL_DA2+ FL_DA2- GND FL_DC2+ FL_DC2- GND
3 FL_DB2+ FL_DB2- GND FL_DD2+ FL_DD2- GND
2 FL_DA1+ FL_DA1- GND FL_DC1+ FL_DC1- GND
1 FL_DB1+ FL_DB1- GND FL_DD1+ FL_DD1- GND

Table 3-21: Connector J4 Pinout

Pin Row A Row B Row C Row D Row E Row F

25 FL_DA20+ FL_DA20- GND FL_DC20+ FL_DC20- GND
24 FL_DB20+ FL_DB20- GND FL_DD20+ FL_DD20- GND
23 FL_DA21+ FL_DA21- GND FL_DC21+ FL_DC21- GND
22 FL_DB21+ FL_DB21- GND FL_DD21+ FL_DD21- GND
21 FL_DA22+ FL_DA22- GND FL_DC22+ FL_DC22- GND
20 FL_DB22+ FL_DB22- GND FL_DD22+ FL_DD22- GND
19 FL_DA23+ FL_DA23- GND FL_DC23+ FL_DC23- GND
18 FL_DB23+ FL_DB23- GND FL_DD23+ FL_DD23- GND
17 NC NC NC NC NC GND
16 NC NC NC NC NC GND
15 NC NC NC NC NC GND
14

Key Area

13
12
11 NC NC NC NC NC GND
10 NC NC NC NC NC GND
9 NC NC NC RTM_TXD# RTM_RXD# GND
8NC NCNCNCNC GND
7NC NCNCNCNC GND
6NC NCNCNCNC GND
5NC NCNCNCNC GND
4NC NCNCNCNC GND
3NC NCNCNCNC GND
2NC NCNCNCNC GND
1NC NCNCNCNC GND

Table 3-20: Connector J3 Pinout (Continued)

Pin Row A Row B Row C Row D Row E Row F

CP6923 Hardware Description

Page 3 - 19 CP6923 User Guide

Note…

RTM_TXD# (driven by CP6923) and RTM_RXD# (driven by RIO module) are
the two-pin RS232 rear I/O interface.

Note...

FL20 to FL23 are only supported on CP6923-R and CP6923-E.

Table 3-22: Connector J5 Pinout

Pin Row A Row B Row C Row D Row E Row F

22 FL_DA19+ FL_DA19- GND FL_DC19+ FL_DC19- GND
21 FL_DB19+ FL_DB19- GND FL_DD19+ FL_DD19- GND
20 FL_DA18+ FL_DA18- GND FL_DC18+ FL_DC18- GND
19 FL_DB18+ FL_DB18- GND FL_DD18+ FL_DD18- GND
18 FL_DA17+ FL_DA17- GND FL_DC17+ FL_DC17- GND
17 FL_DB17+ FL_DB17- GND FL_DD17+ FL_DD17- GND
16 FL_DA16+ FL_DA16- GND FL_DC16+ FL_DC16- GND
15 FL_DB16+ FL_DB16- GND FL_DD16+ FL_DD16- GND
14 FL_DA15+ FL_DA15- GND FL_DC15+ FL_DC15- GND
13 FL_DB15+ FL_DB15- GND FL_DD15+ FL_DD15- GND
12 FL_DA14+ FL_DA14- GND FL_DC14+ FL_DC14- GND
11 FL_DB14+ FL_DB14- GND FL_DD14+ FL_DD14- GND
10 FL_DA13+ FL_DA13- GND FL_DC13+ FL_DC13- GND
9 FL_DB13+ FL_DB13- GND FL_DD13+ FL_DD13- GND
8 FL_DA12+ FL_DA12- GND FL_DC12+ FL_DC12- GND
7 FL_DB12+ FL_DB12- GND FL_DD12+ FL_DD12- GND
6 FL_DA11+ FL_DA11- GND FL_DC11+ FL_DC11- GND
5 FL_DB11+ FL_DB11- GND FL_DD11+ FL_DD11- GND
4 FL_DA10+ FL_DA10- GND FL_DC10+ FL_DC10- GND
3 FL_DB10+ FL_DB10- GND FL_DD10+ FL_DD10- GND
2 FL_DA9+ FL_DA9- GND FL_DC9+ FL_DC9- GND
1 FL_DB9+ FL_DB9- GND FL_DD9+ FL_DD9- GND

Software Description

Page 4 - 1 CP6923 User Guide

CP6923 Software Description

Chapter

4

CP6923 Software Description

Page 4 - 2 CP6923 User Guide

4. Software Description

Software on the CP6923 includes the following parts:

• Bootloader

• initrd (including rootFS, kernel and application software)

•IPMI FW

The Software accomplishes operation of the switching hardware and is therefore also refer­enced as firmware. It is preinstalled on the system and can only be updated by a dedicated
update procedure. This manual only describes bootloader, its self tests and IPMI Firmware and
introduces the update procedure.

For additional information of system configuration using CLI commands refer to documentation
“ CP6923 CLI Reference Manual”.

4.1 Supported RFCs

The Software supports the following standards and RFCs.

4.1.1 Management

• RFC 826 - ARP

• RFC 854 - Telnet

• RFC 855 - Telnet Option

• RFC 1155 - SMI v1

• RFC 1157 - SNMP

• RFC 1212 - Concise MIB Definitions

• RFC 1867 - HTML/2.0 Forms w/ file upload extensions

• RFC 1901 - Community based SNMP v2

• RFC 2068 - HTTP/1.1 protocol as updated by draft-ietf-http-v11-spec-rev-03

• RFC 2246 - The TLS Protocol, Version 1.0

• RFC 2271 - SNMP Framework MIB

• RFC 2295 - Transparent Content Negotiation

• RFC 2296 - Remote Variant Selection; RSVA/1.0 State Management "cookies"

• RFC 2346 - AES Ciphersuites for Transport Layer Security

• RFC 2576 - Coexistence between SNMP v1,v2 & v3

• RFC 2578 - SMI v2

• RFC 2579 - Textual Conventions for SMI v2

• RFC 2580 - Conformance statements for SMI v2

• RFC 2818 - HTTP over TLS

• RFC 3410 - Introduction and Applicability Statements for Internet Standard Management
Framework

• RFC 3411 - An Architecture for Describing SNMP Management Frameworks

• RFC 3412 - Message Processing and Dispatching (December 2002)

CP6923 Software Description

Page 4 - 3 CP6923 User Guide

• RFC 3413 - SNMP Applications (December 2002)

• RFC 3414 - User-based Security Model (December 2002)

• RFC 3415 - View-based Access Control Model (December 2002)

• RFC 3416 - Version 2 of SNMP Protocol Operations (December 2002)

• RFC 3417 - Transport Mappings (December 2002)

• RFC 3418 - MIB for the Simple Network Management Protocol.

• RFC 3635 - Definition of Managed Objects for Ethernet-like Interface Types

• HTML 4.0 Specification - December, 1997

• Java & Java Script 1.3

• SSL 3.0 & TLS 1.0

• SSH 1.5 & 2.0

• Draft-ietf-secsh-transport-16 - SSH Transport Layer Protocol

• Draft-ietf-secsh-userauth-17 - SSH Authentication Protocol

• Draft-ietf-secsh-connect-17 - SSH Connection Protocol

• Draft-ietf-secsh-architecture-14 - SSH Protocol Architecture

• Draft-ietf-secsh-publickeyfile-03 - SECSH Public Key File Format

• Draft-ietf-secsh-dh-group-exchange-04 - Diffie-Hellman Group exchange for the SSH
Transport Layer Protocol

• Configurable Management VLAN ID

• Industry Standard CLI

4.1.2 Switching

• IEEE 802.3ac - VLAN Tagging

• IEEE 802.3ad - Link Aggregation with Static LAG and LACP support

• IEEE 802.1S - Multiple Spanning Tree

• IEEE 802.1W - Rapid Spanning Tree

• IEEE 802.1D - Spanning Tree

•GARP

• GVRP - Dynamic VLAN Registration

• GMRP - Dynamic L2 Multicast Registration

• IEEE 802.1Q - Virtual LANs with Port Based VLANs

• IEEE 802.1v - Protocol based VLANs

• IEEE 802.1p - Ethernet Priority with User Provisioning & Mapping

• IEEE 802.1X - Port Authentication

• IEEE 802.3x - Flow Control

• IGMP Snooping

• Port Mirroring

• Broadcast Storm Recovery

• Static MAC Filtering

• Double VLAN / vMAN Tagging

• Jumbo Frames

• IPv6 Classification APIs

•XMODEM

CP6923 Software Description

Page 4 - 4 CP6923 User Guide

• RFC 768 - UDP

• RFC 783 - TFTP

• RFC 791 - IP

• RFC 792 - ICMP

• RFC 793 - TCP

• RFC 951 - BOOTP

• RFC 1321 - Message Digest Algorithm (MD5)

• RFC 1534 - Interoperation between BOOTP and DHCP

• RFC 2030 - Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI

• RFC 2131 - DHCP Client

• RFC 2131 - DHCP Server

• RFC 2132 - DHCP Options and BOOTP Vendor Extensions

• RFC 2865 - RADIUS Client

• RFC 2866 - RADIUS Accounting

• RFC 2868 - RADIUS Attributes for Tunnel Protocol Support

• RFC 2869 - RADIUS Extensions

• rfc2869bis - RADIUS support for EAP

• RFC 3176 - InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and
Routed Networks

• RFC 3396 - Encoding Long Option in the Dynamic Host Configuration Protocol
(DHCPv4)

• RFC 3580 - 802.1X RADIUS Usage Guidelines

• Draft-ietf-magma-snoop-11.txt - Considerations for IGMP and MLD Snooping Switches

4.1.3 Routing

• Weighted Static Routes

• RFC 819 - Domain Naming Convention for Internet User Applications

• RFC 826 - Ethernet ARPRFC 894 - Transmission of IP Datagrams over Ethernet Net­works

• RFC 896 - Congestion Control in IP/TCP Networks

• RFC 919 - Broadcasting Internet Datagrams

• RFC 922 - Broadcasting Internet Datagrams in the presence of subnets

• RFC 950 - Internet Standard Subnetting Procedure

• RFC 1027 - Using ARP to implement transparent subnet gateways

• RFC 1058 - RIP v1

• RFC 1256 - ICMP Router Discovery Messages

• RFC 1321 - Message Digest Algorithm (MD5)

• RFC 1519 - CIDR

• RFC 1723 - RIP v2

• RFC 1765 - OSPF Database Overflow

• RFC 1812 - Requirements for IP Version 4 Routers

• RFC 2328 - OSPF v2 w/ Equal Cost Multipath support

• RFC 3046 - DHCP/BootP Relay

• RFC 3101 - OSPF with NSSA support

• RFC 3768 - Virtual Router Redundancy Protocol (VRRP)

• Route Redistribution across RIP and OSPF

• VLAN Routing

CP6923 Software Description

Page 4 - 5 CP6923 User Guide

4.1.4 QoS

• Bandwidth Policing (Min and Max; per port/per VLAN)

• Committed Information Rate (CIR)

• Maximum Burst Rate (MBR)

• Per Port (Interface)

• Per VLAN

• Filtering (L3/L4 Access Lists)

• IP Classification - 6 Tuple Classification

• RFC 2474 - DiffServ Definition

• RFC 2475 - DiffServ Architecture

• RFC 2597 - Assured Forwarding PHB

• RFC 3246 - An Expedited Forwarding PHB

• RFC 3260 - New Terminology and Clarifications for DiffServ

4.1.5 Multicast

• RFC 1112 - Host Extensions for IP Multicasting (IGMPv1)

• RFC 2236 - IGMPv2

• RFC 2362 - PIM-SM

• IP Multicast Traceroute

• RFC 2365 - Administratively Scoped Multicast

• RFC 3376 - Internet Group Management Protocol, Version 3

• Draft-ietf-pim-v2-dm-03 - PIM-DM

• Draft-ietf-idmr-dvmrp-v3-10 - DVMRP

• Draft-iet.magma-mgmd-mib-03.txt

• Static Configuration of RP

4.2 Supported MIBs

The Software supports the following MIBs.

4.2.1 Enterprise MIB

• Support for all managed objects not contained in standards based MIBs.

4.2.2 Switching Package MIBs

• LAG-MIB: The Link Aggregation module for managing IEEE 802.3ad

• RFC 1213 - RFC1213-MIB: Management Information Base for Network Management of
TCP/IP-based internets: MIB-II

• RFC 1493 - BRIDGE-MIB: Definitions of Managed Objects for Bridges (dot1d)

• RFC 1643 - Ethernet-like -MIB

• RFC 2620 - RADIUS Accounting MIB

• RFC 2674 - P-BRIDGE-MIB: The Bridge MIB Extension module for managing Priority
and Multicast Filtering, defined by IEEE 802.1D-1998.

CP6923 Software Description

Page 4 - 6 CP6923 User Guide

• RFC 2674 - Q-BRIDGE-MIB: The VLAN Bridge MIB module for managing Virtual Bridged
Local Area Networks

• RFC 2737 - ENTITY-MIB: Entity MIB (Version 2)

• RFC 2819 - RMON Groups 1,2,3 & 9

• RFC 2863 - IF-MIB: The Interfaces Group MIB using SMIv2

• RFC 3291 - Textual Conventions for Internet Network Addresses

• RFC 3635 - Etherlike-MIB: Definitions of Managed Objects for the Ethernet-like Interface
Types

• RADIUS-ACC-CLIENT-MIB: RADIUS Accounting Client MIB

• RADIUS-AUTH-CLIENT-MIB: RADIUS Authentication Client MIB

• IANA-ifType-MIB

• IEEE802.1 - PAE-MIB: Port Access Entity module for managing IEEE 802.1X.

• IEEE 802.3AD MIB: IEEE8021-AD-MIB

4.2.3 Routing Package MIBs

• Draft-ietf-ipv6-rfc2096-update-07.txt: IP Forwarding Table MIB

• IANA-ADDRESS-FAMILY-NUMBERS-MIB: The MIB module defines the AddressFami­lyNumbers textual convention.

• RFC 1724 - RIPv2-MIB: RIP Version 2 MIB Extension

• RFC 1850 - OSPF-MIB: OSPF Version 2 Management Information Base

• RFC 1850 - OSPF-TRAP-MIB: The MIB module to describe traps for the OSPF Version
2 Protocol.

• RFC 2787 - VRRP-MIB: Definitions of Managed Objects for the Virtual Router Redundan­cy Protocol

• DVMRP-STD-MIB: Distance-Vector Multicast Routing Protocol MIB

• IANA-RTPROTO-MIB: IANA IP Route Protocol and IP MRoute Protocol Textual Conven­tions

4.2.4 QoS Package MIBs

• RFC 3289 - DIFFSERV-MIB: Management Information Base for the Differentiated Ser­vices Architecture

• RFC 3289 - DIFFSERV-DCSP-TC MIB: Management Information Base for the Textual
Conventions used in DIFFSERV-MIB

4.2.5 Multicast package MIBs

• RFC 2932 - IPMROUT-MIB: IPv4 Multicast Routing MIB

• draft-ietf-magma-mgmd-mib-03: MGMD MIB, includes IGMPv3 and MLDv2.

• RFC 2934 - PIM-MIB: Protocol Independent Multicast MIB for IPv4

• FASTPATH-MULTICAST-MIB: FASTPATH Flex Multicast Routing Private MIB

CP6923 Software Description

Page 4 - 7 CP6923 User Guide

4.2.6 SNMP MIBs

• RFC 1907 - SNMPv2-MIB: The MIB module for SNMPv2 entities

• SNMP-COMMUNITY-MIB: This MIB module defines objects to help support coexistence
between SNMPv1, SNMPv2 and SNMPv3.

• SNMP-FRAMEWORK-MIB: The SNMP Management Architecture MIB

• SNMP-MPD-MIB: The MIB for Message Processing and Dispatching

• SNMP-NOTIFICATION-MIB: The Notification MIB Module

• SNMP-TARGET-MIB: The Target MIB Module

• SNMP-USER-BASED-SM-MIB: The management information definitions for the SNMP
User-based Security Model.

• SNMP-VIEW-BASED-ACM-MIB: The management information definitions for the
View-based Access Control Model for SNMP.

4.3 Bootloader

On the CP6923 Switch board, the bootloader ‘u-boot‘ (universal bootloader) is used. The boot­loader initializes the main components of the board like CPU, SDRAM, serial lines etc. for op­eration. After this, kernel and application are started from flash.

4.3.1 Power On Self Test

4.3.1.1 Test Routines

Upon power on or system reset, the bootloader performs a set of Power On Self Tests (POST)
to check the integrity of specific components. Components where a POST is available are:

• SDRAM

• PPC405 serial line

• PPC405 I2C

• PPC405 FE

In the case that a POST fails, a POST error code is written into the postcode high byte register
of the onboard CPLD. The boot process is not stopped as there are good chances the board
can boot even in case of POST errors. The postcode high byte register is also accessible by
the PM which can report error codes to a separate management instance. Thus more compre­hensive diagnostic tests could be started.

CP6923 Software Description

Page 4 - 8 CP6923 User Guide

The following table shows a list of available POST routines including POST error codes.

4.3.1.2 Boot Steps

In addition to the Power On Self Tests described above, the bootloader logs the board startup
sequence in the postcode low byte register. A postcode value is written each time a step in the
start sequence has been completed successfully. The postcode stored is also accessible by
the PM. In the case that an error occurs during execution of a step, the boot sequence is
stopped because a fatal error has occurred with great likelihood. In this case, a management
instance can read the last postcode written via the PM and thus determine where the fatal error
has occurred.

A list of defined postcodes is shown in the table below.

Table 4-1: POST routines and error codes

Device Test POST Error Code

SDRAM Data bus - walking 1 test PCW_DLINE

SDRAM Address bus - walking 1 test PCW_ALINE

SDRAM Memory - read/write test PCW_MEM

PPC405 UART Serial loopback teststring PCW_SERIAL

PPC405 I2C Bus scan for devices from I2C_ADDR_LIST PCW_I2C

PPC405 FE Phy access PCW_ETH1

PPC405 FE Phy loopback test using special Ethernet test frame PCW_ETH2

KCS KCS READY signal test KCSCTL

Table 4-2: POST Boot Steps

POST Step Code Value Boot Step

PC_INIT 0x00 Initial PC, EBC has been set up

PC_BINIT 0x01 Board early init (interrupt settings)

PC_CLOCKS 0x02 Get system clocks

PC_TIMEB 0x03 Init timebase

PC_ENVINIT 0x04 Init environment

PC_BAUD 0x05 Init baudrate

PC_SERIAL 0x06 Init UART

PC_CPU 0x07 Check CPU

PC_PHY 0x08 Setup PHY

PC_I2C 0x09 Init I2C

PC_INITRAM 0x0A Init SDRAM controller and SDRAM

PC_TESTRAM 0x0B Test SDRAM

PC_INITSEQ 0x0F Board init sequence completed

PC_INITBOARD 0x10 Board init ok, stack set up ok, board info struct set up

PC_RELOC 0x11 Relocation completed

CP6923 Software Description

Page 4 - 9 CP6923 User Guide

4.4 IPMI Firmware

The PPC communicates with the Peripheral Manager (PM) using the Keyboard Controller Style
(KCS) interface. The bootloader is able to communicate with the PM, e.g. for POST error log­ging purposes and fault resilient purposes.

The memory subsystem of the PM consists of an integrated flash memory to hold the PM op­eration code and integrated RAM for data. The field replaceable unit (FRU) inventory informa­tion is stored in the nonvolatile memory on an EEPROM connected via a local I2C interface to
the PM microcontroller. It is possible to store up to 4 Kbytes within the FRU inventory informa­tion. Communication over IPMB bus to the ShMC ensures that ‘post-mortem’ logging informa­tion is available even if the main processor becomes disabled.

The PM provides several I2C bus connections. Two are used as the redundant IPMB bus con­nections to the backplane, one for the Board Sensors and one is for local EEPROM storage.

If an IPMB bus fault or PM failure occurs, IPMB isolators are used to switch and isolate the
backplane/system IPMB bus from the faulted Hub Board. If possible, the PM activates the re­dundant IPMB bus to re-establish system management communication to report the fault.

The onboard DC voltage, current, and temperature sensors are monitored by the PM micro­controller continuously. The PM will log an event into the ShMC’s System Event Log (SEL) if
any of the thresholds are exceeded.

To increase the reliability of the Hub board management subsystem, an external watchdog su­pervisor for the PM is implemented. The PM strobes the external watchdog at two-second in­tervals to ensure continuity of operation of the board’s management subsystem. If the PM
ceases to strobe the watchdog supervisor for more than six seconds, the watchdog isolates the
PM from the IPMBs and resets the PM. The watchdog supervisor does not reset the payload
power and the restart of the PM will not affect the payload. The external watchdog supervisor
is not configurable and must not be confused with the IPMI v1.5 watchdog timer commands.

PC_TRAP 0x18 Setup trap handler

PC_FLASH 0x19 Flash OK

PC_CPU2 0x1A Init higher level parts of CPU

PC_RELOCENV 0x1B Relocation of environment Ok

PC_BDINFO 0x1C Fill missing fields of bdinfo

PC_PCI 0x1D PCI configuration done

PC_DEVICES 0x1E Device init done

PC_JUMPTABLE 0x1F Jumptable init done

PC_CONSOLE 0x20 Console init done

PC_MAIN 0x2F Enter main loop

PC_START_OS 0x3F Pass control to OS, leave bootloader

Table 4-2: POST Boot Steps (continued)

POST Step Code Value Boot Step

CP6923 Software Description

Page 4 - 10 CP6923 User Guide

This external watchdog of the PM is implemented in a second microcontroller. This Firmware
Upgrade Manager (FUM) is responsible for monitoring the PM and for managing the PM fail
safe firmware upgrade process. The FUM keeps two PM Firmware code images in two external
SEEPROM memories. If a failure occurs during firmware upgrade, the FUM will automatically
rollback to the last known working PM firmware image.

4.4.1 Sensor Data Record (SDR)

Every sensor on the Board is associated with a Sensor Data Record (SDR). Sensor Data
Records contain information about the sensor’s identification such as sensor type, sensor
name, sensor unit. SDR also contain the configuration of a specific sensor such as thresh­old/hystheresis and event generation capabilities that specifies sensor behaviour. Some field
of the sensor SDR are configurable through IPMI v1.5 commands and are set to built-in initial
value.

All SDRs can be queried using Device SDR commands. Board sensors that have been imple­mented are listed below (“Sxx” stands for Sensor of board in slot Nr. xx)..

Table 4-3: CP6923 sensors

IPMI Sensor Name Unit

Scanning Enabled Under

Power State

Sxx: Watchdog discrete On/Off

Sxx: IPMB0 Link discrete On/Off

Sxx: SEL State discrete On/Off

Sxx: IPMI Info-1 discrete On/Off

Sxx: IPMI Info-2 discrete On/Off

Sxx: IniAgent Err discrete On/Off

Sxx: FW Upg Mng discrete On/Off

Sxx: Temp PHY 1 degrees C On/Off

Sxx: Temp PHY 2 degrees C On/Off

Sxx: Temp PHY 3 degrees C On/Off

Sxx: Temp Board degrees C On/Off

Sxx: Vcc 5.0v Volts On

Sxx: Icc 5.0v Amps On

Sxx: Icc 3.3v Amps On

Sxx: Vcc 3.3v Volts On

Sxx: Vcc 2.5v Volts On

Sxx: Vcc 1.8v Volts On

Sxx: Vcc 1.25v Volts On

Sxx: Vcc 1.0v Volts On

Sxx: Vcc 5.0v CPCI Volts On/Off

Sxx: Vcc 3.3v CPCI Volts On/Off

CP6923 Software Description

Page 4 - 11 CP6923 User Guide

4.4.1.1 IPMB Link Sensor

The CP6923 provides two IPMB links to increase communication reliability to the shelf manag­er and other IPM devices on the IPMB. These IPMB links work together for increased through­put where both busses are actively used for communication at any time. A request might be
received over IPMB Bus A, and the response is sent over IPMB Bus B. Any requests that time
out are retried on the redundant IPMB bus. In the event of any link state change, the events are
written to the CP6923 SEL. The PM monitors the bus for any link failure and isolates itself from
the bus if it detects that it is causing errors on the bus. Events are sent to signal the failure of
a bus or, conversely, the recovery of a bus.

4.4.2 Board FRU Information

This FRU information contains the IPMI defined Board and Product Information areas that hold
the part number and serial number of the board and the Multirecord Information Area that con­tains the PICMG defined Point to Point Information records.

The Internal Use Area is preallocated to 384 bytes and is free for customer use. This FRU in­formation responds to FRU ID #0, which is the ID for the PM.

Sxx: Vcc 3.3v CPLD Volts On/Off

Sxx: Icc 3.3v CPLD Amps On/Off

Sxx: Vcc 5.0v IPMB Volts On/Off

Sxx: Icc 5.0v IPMB Amps On/Off

Sxx: SFP-1 Status discrete On/Off

Sxx: SFP-2 Status discrete On/Off

Sxx: SFP-3 Status discrete On/Off

Sxx: SFP-4 Status discrete On/Off

Sxx: XFP-1 Status discrete On/Off

Sxx: XFP-2 Status discrete On/Off

Sxx: Board Reset discrete On/Off

Sxx: Boot Cycle discrete On/Off

Sxx: Handle Switch discrete On/Off

Table 4-3: CP6923 sensors (continued)

IPMI Sensor Name Unit

Scanning Enabled Under

Power State

CP6923 Software Description

Page 4 - 12 CP6923 User Guide

4.4.3 PM Firmware Code

PM firmware code is organized into boot code and operational code, both are stored in a flash
module. Upon an PM reset, the PM executes the boot code and performs the following:

1. Self test to verify the status of its hardware and memory.

2. Calculates a checksum of the operational code.

3. Communicates with the Firmware Upgrade Manager (FUM) in order to inform the PM
watchdog that the current PM firmware is suitable for execution.

Upon successful verification of the operational code checksum, the firmware will jump to the
operational code.

4.4.4 LEDs

For LED positions on the front plane refer to Chapter 3, section, Display Elements.

4.4.4.1 Health LED (Green LED)

The CP6923 Switch Board supports a Health LED mounted on the front panel. The following
states are possible:

4.4.4.2 Activity LED (Green LED)

The CP6923 supports a green Activity LED mounted on the front panel. The position of this
LED is near the bottom handle beside th ehealth LED. This heartbeat LED blinks every 4 sec­onds

Table 4-4: Health LED state

LED state Description

ON normal state when board is in operation, PM is running.

Off PM is not in operational status

Blinking The FUM is programming the PM due to firmware update or rollback

Table 4-5: Activity LED state

LED state Description

Blinking Heartbeat of PM

CP6923 Software Description

Page 4 - 13 CP6923 User Guide

4.5 Firmware Administration

A running CP6923 system requires – after the bootloader has passed control to the kernel –
the kernel itself, the root file system (initrd), the FASTPATH switching application and a config­uration file for the switch. These software components, together with the PM image, make up
the CP6923 firmware.

The flash holding the software is divided into 7 partitions. There are partitions to store two boot­loader and two Softare Images integrating initrd, kernel and applications. Two partitions are re­served for the bootloader environments, one partition is used to hold up to nine images of the
switching application (depending on the size of the images) and one partition is used to hold
up to 99 configuration setting files. The partition scheme of the flash is shown below:

The firmware update procedure is designed to provide a failsafe capability to update IPMI FW
and Software settings separately. Each combination of these components can be used as a
startup configuration, though they must be compatible to each other. Please allways follow
Kontron documentation for all your upgrades. Software versions provided with an official re­lease are known to work together.

In the following, the CLI commands to setup, change and activate startup configuration and the
CLI commands necessary to perform firmware upgrades are described. The CLI commands
described below are executed in the privileged mode of the CLI hierarchy, which is entered by
executing the ‘enable’ command. Please refer to the “ CP6923 CLI Reference Manual“ for more
information regarding the CLI commands and the way to use them.

Table 4-6: FLASH Partition Scheme (64MB)

Partition Size MTD

U-Boot Loader primary image (write protected) 256KB 6

U-Boot Loader secondary image, updatable (optional) 256KB 5

U-Boot primary environment 256KB 4

U-Boot secondary environment 256KB 3

Software Image secondary partition (Image_B) 28MB 2

Software Image primary partition (Image_B) 28MB 1

JFFS Configuration File Partition

CONFIGURATION text file F

CONFIGURATION text file 01

CONFIGURATION text file ...

CONFIGURATION text file 99

CONFIGSLOTS file containing multiple combination for selection
through CLI

7MB 0

CP6923 Software Description

Page 4 - 14 CP6923 User Guide

4.5.1 Startup Configurations

A startup configuration is a combination of a software image and a configuration file for the
switch. The IPMI FW image is not part of the configuration and is updated separately. Several
configurations (up to 99) can be defined but only one is active at a time. To display the currently
available startup configurations, the CLI command ‘show’ is used.

(Ethernet Fabric) #show startupconfig startup
ACT NR SYSTEM BASE-CONFIG

--- -- ------------ --------------­ F A HEAD 0.00 F factory
*A 01 A HEAD 0.00 01 standard
(Ethernet Fabric) #

In this example, two startup configurations are available. Startup configuration 01 is the current­ly active configuration which is indicated by the ‘*’ in the first column. This configuration consists
of:

• Software version HEAD 0.00 which is located in initrd partition A

• Switch configuration file which is stored in slot 01 of the configuration partition.

The user can define other startup configurations and combine images and configuration files
as needed. To set up a new startup configuration, it is necessary to change into the CLI’s con­figuration mode using the ‘configure’ command. In this mode, the ‘startupslot’ command is used
as shown in the following example:

(Ethernet Fabric) (Config)#startupslot 2 config 1 initrd A

Successfully set startup slot 2
You may mark it active now and reboot to use the new configuration.

(Ethernet Fabric) (Config)#exit

(Ethernet Fabric) #

(Ethernet Fabric) #show startupconfig startup
ACT NR SYSTEM BASE-CONFIG

--- -- ------------ --------------­ F A HEAD 0.00 F factory
*A 01 A HEAD 0.00 01 standard
02 A HEAD 0.00 01 standard
(Ethernet Fabric) #

A new startup configuration has been added to the list combining the software images given in
the previous command.

The startup configuration created before can be deleted by entering exactly the same com­mand string preceeded by ‘no’:

(Ethernet Fabric) (Config)#no startupslot 2 config 1 initrd A

CP6923 Software Description

Page 4 - 15 CP6923 User Guide

The ‘show’ command can also be used to display available versions of initrd and configurations
or all of them. The syntax is:

show startupconfig startup|initrd|config|all

The configuration mode is also used to activate one of the available startup configurations. The
syntax is:

startupslot 2 activate [once]

This command would activate the startup configuration 2 upon next system reset. The optional
parameter ‘once’ would start configuration 2 only once after the next system reset and start the
previously active configuration 1 on following resets. This is used to implement the failsafe up­grade procedure described below.

4.5.2 Updating Firmware

The firmware is updated using the CLI. To get a reliable and failsafe update procedure, the fol­lowing precautions must be fulfilled:

• Two independent partitions for the initrd, one holding initrd A and the other one holding
initrd B. The active system is either A or B. See the flash partition scheme shown above
for more detail.

• One Time Boot capability: After having updated the inactive kernel and root file system,
the new system has to be started by rebooting the board. In the case that the update has
installed an inoperable system which would cause the board to hang when booting, the
next board reset must restart the previous known good version of the initrd. This is
achieved by programming the bootloader environment appropriately.

• Redundant bootloader environment sectors: When the system is updated, the bootloader
environment must be changed to be able to start the updated version. The bootloader
environment sector is stored twice in flash, one active version and one backup version in
case the active version is deleted during update due to power loss or similar errors. In
this case, the redundant environment would cause the bootloader to start the previous
known good version of kernel and root file system.

A software update of the CP6923 Switch Board is done by performing the following steps:

1. Download initrd image into the appropriate slot of the flash memory. Ensure that the cur­rently active image will not be overwritten.

2. Select a configuration for the switch for the new software release. This can be done by
choosing the factory default or by storing the running system configuration into flash.

3. Create a startup configuration by combining the slots with the update image and the con­figuration slot for the switch.

4. Activate the selected startup configuration for One Time Boot.

5. Restart the board.

6. Activate the new startup configuration permanently

CP6923 Software Description

Page 4 - 16 CP6923 User Guide

A software release for the CP6923 consists of one software package, that icludes kernel, initrd
and application software. The package is a tar archive containing the image itself and a MD5
checksum file for consistency check. The name of the package file is arbitrary but the file
names in the archive must not be altered.

When performing a firmware update, the software package is loaded from a remote TFTP serv­er in the first step. This is done by means of the frontpanel FE port of the Hub board. To load a
software package via TFTP into a specified slot, the CLI command ‘download’ is used:

(Ethernet Fabric) #download initrd tftp://192.168.50.5/initrd.pkg B

Downloading image, this may take a while...
Successfully transferred kernel image tftp://192.168.50.5/initrd.pkg to slot B
You may mark it active now and reboot to use the new kernel image.

(Ethernet Fabric) #

This downloads the specified software package file via TFTP and writes the image into the initrd
partition of the specified slot (B). The MD5 checksum of the image is checked before writing it
into flash. It is important not to overwrite the slot containing the currently active software, oth­erwise the One Time Boot mechanism does not make sense.

After the software image has been downloaded into flash, the configuration slot for the switch
must be selected. In case that no appropriate configuration the switch is available, the factory
default configuration should be used for the startup configuration of the new software release.
Alternatively, the running configuration settings of the switch can be stored on flash using the
following CLI command:

(Ethernet Fabric) #copy system:running-config nvram:startup-config slot 3

This operation may take a few minutes.
Management interfaces will not be available during this time.

Are you sure you want to save? (y/n) y

Configuration Saved!

(Ethernet Fabric) #

Finally, a startup configuration containing the slot of the new software release and the configu­ration slot must be selected or a new startup configuration must be created as described above.
After this step has been completed, the startup configuration is activated only for the next boot
and the board is rebooted. This is done with the ‘startupslot’ CLI command in the configure
mode:

(Ethernet Fabric) (Config)#startupslot 3 activate once

This command enables the startup configuration 3 only for the next system restart. In the case
that the board hangs due to a corrupted software image, this will be detected and the board is
automatically rebooted with the previous known good startup configuration. This way, a failsafe
upgrade of the CP6923 software is possible.

CP6923 Software Description

Page 4 - 17 CP6923 User Guide

To restart the board with the new startup configuration, the CLI command ‘reload’ is used:

(Ethernet Fabric) (Config)#reload

If the new startup configuration is considered functional after the reboot, it must be activated
permanently, or else the one that is still active will be used again:

(Ethernet Fabric) (Config)#startupslot 3 activate

4.5.3 Updating IPMI

Updating the IPMI firmware is different from updating the other software parts as updating is
done directly when invoking the download command. In the case that the update procedure
fails or the update image is corrupted, the PM will be able to restart all the same by means of
its rollback functionality. The IPMI Firmware file is stored in the result/ppc405/firmware path of
the release directory tree. To update the IPMI firmware, the CLI command ‘download’ is used:

(Ethernet Fabric) #download ipmifw tftp://192.168.50.5/ipmi.pkg

Flashing a new IPMI firmware will disable the IPMI Controller for some minutes.
Are you sure to update the IPMI firmware? (y/n)y
...
(Ethernet Fabric) #

CP6923 Getting Help

Page A - 1 CP6923 User Guide

Getting Help

A

ppendix

A

CP6923 Getting Help

Page A - 2 CP6923 User Guide

A. Getting Help

If, at any time, you encounter difficulties with your application or with any of our products, or if
you simply need guidance on system setups and capabilities, contact our Technical Support at:

North America EMEA

Tel.: [phone] Tel.: [phone]

Fax: [phone] Fax: [phone]

If you have any questions about Kontron, our products, or services, visit our Web site at:
www.kontron.com

You also can contact us by E-mail at:

North America: [e-mail]

EMEA: [e-mail]

Or at the following address:

North America EMEA

Kontron Canada, Inc. Kontron Modular Computers GmbH

616 Curé Boivin Sudetenstrasse 7

Boisbriand, Québec 87600 Kaufbeuren

J7G 2A7 Canada Germany

CP6923 Getting Help

Page A - 3 CP6923 User Guide

RETURNING DEFECTIVE MERCHANDISE

Before returning any merchandise please do one of the following:

• Call

1. Call our Technical Support department in North America at [phone] or in EMEA
at [phone]. Make sure you have the following on hand: our Invoice #, your
Purchase Order # and the Serial Number of the defective unit.

2. Provide the serial number found on the back of the unit and explain the nature of your
problem to a service technician.

3. The technician will instruct you on the return procedure if the problem cannot be solved
over the telephone.

4. Make sure you receive an RMA # from our Technical Support before returning any mer­chandise.

• Fax

1. Make a copy of the request form on the following page.

2. Fill it out.

3. Fax it to us at: North America [phone], EMEA [phone]

• E-mail

1. Send us an e-mail at: [e-mail] in North America and at: orderprocess­[e-mail] in EMEA. In the e-mail, you must include your name, your
company name, your address, your city, your postal/zip code, your phone number, and
your e-mail. You must also include the serial number of the defective product and a de­scription of the problem.

CP6923 Getting Help

Page A - 4 CP6923 User Guide

WHEN RETURNING A UNIT

• In the box, you must include the name and telephone number of a person, in case further
explanations are required. Where applicable, always include all duty papers and in-

voice(s) associated with the item(s) in question.

• Ensure that the unit is properly packed. Pack it in a rigid cardboard box.

• Clearly write or mark the RMA number on the outside of the package you are returning.

• Ship prepaid. We take care of insuring incoming units.

North America EMEA

Kontron Canada, Inc. Kontron Modular Computers GmbH

616 Curé Boivin Sudetenstrasse 7

Boisbriand, Québec 87600 Kaufbeuren

J7G 2A7 Canada Germany

Return to

Manufacturer

Authorization Request

Contact Name:

__________________________________________________________

Company
Name:

__________________________________________________________

Street
Address:

__________________________________________________________

City: ________________________ Province/State: _______________________

Country: ________________________ Postal/Zip Code: _______________________

Phone
Number:

________________________ Extension: _______________________

Fax Number: ________________________ E-Mail: _______________________

Serial Number Failure or Problem Description

P.O. #

(if not under warranty)

Fax this form to Kontron’s Technical Support department

in North America at [phone]

or in EMEA at [phone]