Sun Microsystems Fire F6800, Fire F3800, Fire F4810, Fire F4800 Quick Reference Manual

Sun Microsystems, Ltd

Citygate, Cross Street

Sale

Manchester

M33 7JF, UK

Sun Conﬁdential: Internal Only Brian Jackson, July 2005

Sun Fire™ 3800 - 6800

Quick Reference Guide

Sun Conﬁdential: Internal Only

Please

Recycle

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Sun, Sun Microsystems, the Sun Logo, Sun Fire, and UltraSPARC III are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.

The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.

U.S. Government approval might be required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Sun Conﬁdential: Internal Only

Please

Recycle

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.

Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.

Sun, Sun Microsystems, le logo Sun, Sun Fire, et UltraSPARC III sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.

Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.

UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.

L’interfaces d’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sun reconnaît les efforts de pionniers de Xerox pour larecherche et le développement du concept des interfaces d’utilisation visuelle ou graphique pour l’industrie de l’informatique. Sun détient une licence non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licence couvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui en outre se conforment aux licences écrites de Sun.

L’accord du gouvernement américain est requis avant l’exportation du produit.

LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

Sun Conﬁdential: Internal Only

Please

Recycle

Sun Conﬁdential: Internal Only

Table of Contents

Additional Resources ........................................................................... 9

Sun Fire 3800, 4800, 4810, and 6800 Platforms.................................10

Sun Fire 3800............................................................................... 11

Sun Fire 4800............................................................................... 12

Sun Fire 4810............................................................................... 13

Sun Fire 6800............................................................................... 14

Conceptual Overview..........................................................................15

System Board ASICs............................................................................16

System Boards, Physical......................................................................17

CPU & Memory Configuration..........................................................18

System Boards summary ....................................................................21

I/O Board ASICs..................................................................................22

PCI I/O Physical ..................................................................................23

cPCI I/O Physical ................................................................................24

I/O Board Summary...................................................................24

Sun Fireplane Switch Boards..............................................................25

How it all bolts together......................................................................26

System Controller Boards ...................................................................27

Nvram, nvci & SC flashproms ...........................................................28

System Controller Clock Circuit ........................................................29

5.13.x firmware features......................................................................30

5.14.x firmware features......................................................................31

5.15.x firmware features......................................................................32

5.16.x firmware features......................................................................33

5.17.x firmware features......................................................................34

5.18.x firmware features......................................................................35

Service Mode ........................................................................................36

System Controller failover..................................................................37

Resetting the System Controller Board.............................................38

Maintenance Buses - I2C & FRU-ID ..................................................39

ID Board ................................................................................................40

Fan Trays and Blower Assemblies ....................................................41

DC Power Distribution........................................................................42

Sun Conﬁdential: Internal Only

Sun Fire 3800-6800 Quick Reference Guide

DC Power Distribution, Sun Fire 6800..............................................43

System LEDs .........................................................................................44

AC Power Distribution........................................................................45

RTU/RTS Connectors & indicators...................................................46

Installing the Administration Console..............................................47

System Controller Interconnect Panel......................................47

Accessing the Platform Shell ..............................................................48

Using the Serial Port ...................................................................48

Using the telnet Command.........................................................48

Using the ssh Command.............................................................48

RSA & DSA Authentication.......................................................49

Regenerating SSH host keys using ssh-keygen ...............................50

Introducing Platform Management...................................................51

Configuring The Platform...................................................................53

System Controller Configuration .............................................53

Platform Configuration..............................................................53

Managing Platform time-of-day (TOD)............................................55

Configuring an sntp server for the System Controllers .................56

setupplatform .......................................................................................57

Rebooting the system controller ...............................................58

Setting up the loghost..........................................................................59

Edit syslog.conf on the admin workstation.............................59

syslog Parsing Tools ...................................................................59

Powering On and Powering Off System Components...................60

Firmware Upgrades.............................................................................61

Do you need to flashupdate?..........................................................61

flashupdate............................................................................................63

ftp server setup.....................................................................................64

Edit /etc/passwd & /etc/shadow.................................................64

Run the shell script from the ftpd man page...................................64

ftp server setup at Solaris 9 .............................................................65

mkdir /export/ftp/pub and copy the patch into it ........................65

flashupdate - Command Examples ...................................................66

Replacing an SC in a single SC configuration..................................71

Replacing an SC in a dual SC platform (5.13.x & above) ...............73

Managing Shell Passwords.................................................................74

Overriding Shell Passwords...............................................................75

Environmental and Platform Monitoring ........................................76

Segments and Domains.......................................................................77

Definitions of Segments and Domains..............................................78

Segments (also known as Partitions)........................................78

Domains........................................................................................78

Domain IDs ...........................................................................................79

Segmenting the Platform ....................................................................80

Recommendation ........................................................................80

Enabling Dual-Partition Mode..................................................80

Sun Conﬁdential: Internal Only

Assigning boards to Domains............................................................81

Setupdomain.........................................................................................83

Showdomain.........................................................................................84

Domain Access Control List (ACL)...................................................85

Configuring ACLs.......................................................................85

Viewing ACLs .............................................................................85

Backup and Restore a System Controller .........................................86

Updating SC MAC addresses ...................................................86

dumpconfig command...............................................................86

restoreconfig Command ............................................................87

Example - relative & absolute pathnames...............................88

Starting and Stopping a Domain .......................................................89

Starting the domain - Example ..........................................................90

show-post-results & probe-scsi-all ...........................................97

Resetting a hung domain ....................................................................99

reset ................................................................................................99

showresetstate ................................................................................99

Connection refused, console busy ...................................................100

Navigating Between Shells...............................................................101

Escape sequence .................................................................................102

showescape ................................................................................102

setescape.....................................................................................102

Sun Fire Device Tree..........................................................................103

Sun Fire Physical Device Mapping..................................................105

Node Mapping ..........................................................................105

System Board Mapping............................................................105

Processor/Memory Controller AIDs (hex in brackets) .......106

I/O Mapping .............................................................................106

Schizo AIDs (hex in brackets)..................................................107

Eight-Slot PCI I/O Assembly Slot Assignments ...........................108

Six-Slot cPCI I/O Assembly Slot Assignments .............................109

Four-Slot cPCI I/O Assembly Slot Assignments ..........................110

Setdefaults...........................................................................................111

NVRAM Domain Configuration Variables....................................112

Displaying Domain Configuration Variables .......................112

Changing System Configuration Variables...........................113

Device Aliases.....................................................................................114

Change a Device Alias..............................................................114

Auto-diagnosis Engine (ADE) Case Study.....................................116

Re-assigning HostID/MAC Address.....................................121

Re-setting component health status .......................................124

Sun Conﬁdential: Internal Only

Sun Fire 3800-6800 Quick Reference Guide

Sun Conﬁdential: Internal Only

Additional Resources

Additional resources – The following references provide additional

details on the topics discussed in this quick reference guide:

● Sun Fire Systems Site Planning Guide, 805-7365-xx

● Sun Fire System Overview Manual, 805-7362-xx

● Sun Fire Systems Installation Guide, 817-2353-xx

● Sun Fire Systems Service Manual, 805-7363-xx

● Sun Fire 3800-6800 Troubleshooting Manual, 805-7364-xx

● Sun StorEdge™ D240 Media Tray Installation, Operation, and Service

Manual, 806-4211-xx

Refer to the Platform Administration Manual and System Controller Commands Reference Manual speciﬁc to the ﬁrmware you are using.

For an exhaustive list of all Sun Fire manuals, go to:

http://www.sun.com/products-n-solutions/hardware/docs/Servers/

● SUN SYSTEM HANDBOOK available from http://sunsolve

● James Hsiehs “Best Practice” blueprints

http://www.sun.com/solutions/blueprints/0901/sunfireconfig.pdf http://www.sun.com/solutions/blueprints/1001/sunfire-admin.pdf http://www.sun.com/blueprints/1002/817-0080-10.pdf

● Peter Gonscherowskis & Tricia Wittsacks Auto Diagnosis blueprint

http://www.sun.com/blueprints/0404/817-6212.pdf

● Sun Fire Quick Reference Guide, latest rev available at the following link

http://sale-wts.uk/~brianja/

Sun Fire 3800, 4800, 4810, and 6800 Platforms

Sun Conﬁdential: Internal Only 10 Sun Fire 3800 - 6800 Quick Reference Guide

Sun Fire 3800, 4800, 4810, and 6800 Platforms

There are four models of Sun Fire platforms as shown below. They are Midframe servers built around UltraSPARC™-III processors and PCI or hot-swappable cPCI I/O.

They have been developed under the codename of serengeti. They have gone through a number of name changes since their inception; they are included below for your reference:

● K6 > Serengeti SP > Serengeti-8 > Sun Fire 3800

● K12 > Serengeti MD > Serengeti-12 > Sun Fire 4800

● R12 > Serengeti ME > Serengeti-12i > Sun Fire 4810

● R24 > Serengeti DS > Serengeti-24 > Sun Fire 6800

● All the Sun Fire 3800 - 6800 range require Solaris 8 Update 4, S8U4

(Solaris 8 4/01) as a minimum.

● Memory ﬁgures on pages 11 - 14 assume 1GB DIMMs, which are

only supported on 900MHz and faster system boards.

Sun Fire 3800 Sun Fire 4800 Sun Fire 4810 Sun Fire 6800

Sun Fire 3800, 4800, 4810, and 6800 Platforms

Sun Conﬁdential: Internal Only

Sun Fire 3800

The Sun Fire 3800 platform is a service-provider platform that is a Sun Fire cabinet or rack-mounted, highly available, compact system.

Up to three Sun Fire 3800 systems can be mounted in a Sun Fire cabinet.

The Sun Fire 3800 has the following features:

● Support for 2 to 8 UltraSPARC™-III/III Cu processors, mounted on

two system boards.

● Up to 64 Gbytes of error checking and correction (ECC) memory

● Twelve hot-swappable compact peripheral component interconnect

(cPCI) slots mounted on two I/O boards.

● Support for up to two domains in either one or two segments.

● Three power supplies providing N+1 DC power redundancy.

● Four Fan Trays providing N+1 cooling redundancy.

● The Sun Fire Rack has N+1 AC input redundancy, assuming the

customer has two separate supplies into his facility. (See page 45)

Front view

Sun Fire 3800, 4800, 4810, and 6800 Platforms

Sun Conﬁdential: Internal Only 12 Sun Fire 3800 - 6800 Quick Reference Guide

Sun Fire 4800

The Sun Fire 4800 is a highly available, ﬂexible standalone or rackmounted system.

The Sun Fire 4800 has the following features:

● Support for 2 to 12 UltraSPARC™-III/III Cu processors, mounted on

three system boards

● Up to 96 Gbytes of ECC memory.

● Sixteen PCI I/O slots mounted on two I/O boards, or eight hot-

swappable cPCI slots mounted on two I/O boards or a combination of an eight-slot PCI and four-slot cPCI.

● Support for up to two domains in either one or two segments.

● Three power supplies providing N+1 DC power redundancy.

● Three Fan Trays providing N+1 cooling redundancy.

Sun Fire 3800, 4800, 4810, and 6800 Platforms

Sun Conﬁdential: Internal Only

Sun Fire 4810

The Sun Fire 4810 is a highly available system for specialized environments.

● Environments with limited depth, which require 100% front

accessibility, for example, telco providers, ISPs etc

● Military vehicles (trucks, ships, and so on)

The Sun Fire 4810 has the following features:

● Support for 2 to 12 UltraSPARC™-III/III Cu processors, mounted on

three system boards.

● Up to 96 Gbytes of ECC memory.

● Sixteen PCI I/O slots mounted on two I/O boards, or eight hot-

swappable cPCI slots mounted on two I/O boards or a combination of an eight-slot PCI and four-slot cPCI.

● Support for up to two domains in either one or two segments.

● N+1 DC power redundancy using three power supplies, N+1 cooling

redundancy using three Fan Trays, N+1 AC redundancy in the rack. (See page 45)

Front view Rear view

Sun Fire 3800, 4800, 4810, and 6800 Platforms

Sun Conﬁdential: Internal Only 14 Sun Fire 3800 - 6800 Quick Reference Guide

Sun Fire 6800

The Sun Fire 6800 is a highly available, large-scale system.

The Sun Fire 6800 has the following features:

● Support for 2 to 24 UltraSPARC™-III/III Cu processors, mounted on

six system boards.

● Up to 192 Gbytes of ECC memory.

● Thirty-two PCI I/O slots or 16 hot-swappable cPCI slots or

combinations of PCI and cPCI, mounted on four I/O boards.

● Support for up to four domains. Three or four domains require two

segments. One or two domains require one segment, although there are advantages with creating two domains in two segments.

● The Sun Fire 6800 has two Power Grids.

● Each Power Grid can have N+1 AC power redundancy, assuming

the customer has two separate supplies into his facility. (See page 45)

● There is N+1 DC power redundancy in each Power Grid.

● Four Fan Trays providing N+1 cooling redundancy.

Front view Rear view

Conceptual Overview

Sun Conﬁdential: Internal Only

Conceptual Overview

The Sun Fire platforms are organized into multiple administrative/service layers.

Each layer provides a set of tools that allow you to conﬁgure, monitor, service, and administer various aspects of the platform.

Refer to the Sun Fire Systems Site Planning Guide, part number 805-7365-xx:

● Dimensions, clearance requirements & weight

● Power requirements

● BTUs/hr & cooling requirements

● Power factor

● Connector type

Platform shell

OpenBoot PROM

Environment

Applications

Platform hardware

Domain A shell

Solaris

Operating

OpenBoot PROM

Environment

Applications

Domain B shell

Solaris

Operating

OpenBoot PROM

Environment

Applications

Domain C shell

Solaris

Operating

OpenBoot PROM

Environment

Applications

Domain D shell

Solaris

Operating

System Board ASICs

Sun Conﬁdential: Internal Only 16 Sun Fire 3800 - 6800 Quick Reference Guide

System Board ASICs

The system board CPUs and memory are conﬁgured in pairs, (two pairs per system board maximum) interconnected with a dual cheetah data switch (DCDS). The DCDS provides a data path between the CPUs and the L1 Sun Fireplane switch.

The system board L1 (level 1) Sun Fireplane switching logic consists of an address repeater (AR) and 4 x data switch (DX) ASICs.

Each system board contains a system data controller (SDC). The SDC has two functions; it multiplexes the console bus connections to allow all system boards and Sun Fireplane switch boards to communicate to each of the system controllers.

It is also the brains of the operation behind the data crossbar. If a data packet is considered as a train, the DXs are like the tracks and the SDC is like the signal box; the SDC routes the data.

System Boards, Physical

Sun Conﬁdential: Internal Only

System Boards, Ph ysical

The Sun Fire system board houses either two or four UltraSPARC™-III Cu 900 MHz, 1050 MHz or 1200 MHz processors, or UltraSPARC™-III 750MHz processors.

Each processor supports two physical banks of memory.

Each memory bank consists of four dual inline memory modules (DIMMs) which gives, 8 x DIMMS per CPU (See page 18)

Memory capacity can also be mixed but not within the same bank.

The processor’s L2-SRAM (also known as Ecache) consists of two cache modules mounted on their own pcb’s, each module being 4MB in size, giving 8MB per CPU.

Neither the CPU modules or the L2-SRAMs are ﬁeld replacable.

Mixing CPU speeds of 900MHz, 1050MHz and 1200MHz within a domain or platform is supported.

750MHz CPUs cannot be included in mixed-speed conﬁgurations.

CPU & Memory Configuration

Sun Conﬁdential: Internal Only 18 Sun Fire 3800 - 6800 Quick Reference Guide

CPU & Memory Conﬁguration

Each CPU processor has two corresponding physical DIMM banks.

Each physical bank is split into two logical banks

It is possible that a processor might not have any DIMMs installed in its corresponding DIMM bank, but a populated DIMM bank must have a corresponding CPU processor installed in order for that memory to be utilised.

The system board supports the following DIMM sizes:

● 256MB DIMM Part number 501-5401

● 512MB DIMM Part number 501-5030

● 1GB DIMMs, part numbers 501-6109 & 540-5086 are only supported

on 900MHz and faster system boards.

CPU

DIMM 3, bank 0

processor 2

processor 3

processor 0

processor 1

DIMM 3, bank 1

DIMM 2, bank 0

DIMM 2, bank 1

DIMM 1, bank 1

DIMM 0, bank 0

DIMM 1, bank 0

DIMM 0, bank 1

CPU & Memory Configuration

Sun Conﬁdential: Internal Only

CPU and Memory Configuration (cont)

Below is a layout of a system board. Descriptions of physical and logical banks follow:

6800b-sc0:SC> showcomponent /N0/SB2

Component Status Pending POST Description

--------- ------ ------- ---- ----------/N0/SB2/P0 enabled - pass UltraSPARC-III+, 900MHz, 8M ECache /N0/SB2/P1 enabled - pass UltraSPARC-III+, 900MHz, 8M ECache /N0/SB2/P2 enabled - pass UltraSPARC-III+, 900MHz, 8M ECache /N0/SB2/P3 enabled - pass UltraSPARC-III+, 900MHz, 8M ECache /N0/SB2/P0/B0/L0 enabled - pass 2048M DRAM /N0/SB2/P0/B0/L2 enabled - pass 2048M DRAM /N0/SB2/P0/B1/L1 enabled - pass 2048M DRAM /N0/SB2/P0/B1/L3 enabled - pass 2048M DRAM /N0/SB2/P1/B0/L0 enabled - pass 2048M DRAM /N0/SB2/P1/B0/L2 enabled - pass 2048M DRAM /N0/SB2/P1/B1/L1 enabled - pass 2048M DRAM /N0/SB2/P1/B1/L3 enabled - pass 2048M DRAM /N0/SB2/P2/B0/L0 enabled - pass 2048M DRAM /N0/SB2/P2/B0/L2 enabled - pass 2048M DRAM /N0/SB2/P2/B1/L1 enabled - pass 2048M DRAM /N0/SB2/P2/B1/L3 enabled - pass 2048M DRAM /N0/SB2/P3/B0/L0 enabled - pass 2048M DRAM /N0/SB2/P3/B0/L2 enabled - pass 2048M DRAM /N0/SB2/P3/B1/L1 enabled - pass 2048M DRAM /N0/SB2/P3/B1/L3 enabled - pass 2048M DRAM

6800b-sc0:SC>

Each cpu has 2 physical banks (Even & Odd) with 4 DIMMS each

B0: Group 0 is the Even bank (Note the Even bank DIMMS end with a "0") B1: Group 1 is the Odd bank (Note the Odd bank DIMMS end with a "1")

Each physical bank has 2 logical banks

L0 is the front of all 4 DIMMS of the physical bank B0 L2 is the back of all 4 DIMMS of the physical bank B0

L1 is the front of all 4 DIMMS of the physical bank B1 L3 is the back of all 4 DIMMS of the physical bank B1

CPU & Memory Configuration

Sun Conﬁdential: Internal Only 20 Sun Fire 3800 - 6800 Quick Reference Guide

CPU and Memory Configuration (cont)

P B D (Proc, bank, dimm)

J14600 1 0 3 J14601 1 1 3 J14500 1 0 2 J14501 ---> CPU B 1 1 2 ---> CPU P1 J14400 1 0 1 J14401 1 1 1 J14300 1 0 0 J14301 1 1 0

J13600 0 0 3 J13601 0 1 3 J13500 0 0 2 J13501 ---> CPU A 0 1 2 ---> CPU P0 J13400 0 0 1 J13401 0 1 1 J13300 0 0 0 J13301 0 1 0

J16600 3 0 3 J16601 3 1 3 J16500 3 0 2 J16501 ---> CPU D 3 1 2 ---> CPU P3 J16400 3 0 1 J16401 3 1 1 J16300 3 0 0 J16301 3 1 0

J15600 2 0 3 J15601 2 1 3 J15500 2 0 2 J15501 ---> CPU C 2 1 2 ---> CPU P2 J15400 2 0 1 J15401 2 1 1 J15300 2 0 0 J15301 2 1 0

System Boards summary

Sun Conﬁdential: Internal Only

System Boards summary

System board lay out is clearly marked on the cabinets.

However, remember that in the F6800, the System boards are numbered non-sequentially from left to right:

SB0 SB2 SB4 SB1 SB3 SB5

platform

Number of System Boards

Slot Number Location

F6800 Six SB0 through SB5 Front

F4810 Three SB0, SB2, and SB4 Front

F4800 Three SB0, SB2, and SB4 Rear

F3800 Two SB0 and SB2 Front

SB0

SB2 SB4

SB5

SB3

SB1

I/O Board ASICs

Sun Conﬁdential: Internal Only 22 Sun Fire 3800 - 6800 Quick Reference Guide

I/O Board ASICs

The PCI I/O board is powered by two Schizo I/O bridges. On the centreplane side, each Schizo interfaces to the Sun Fireplane interconnect. On the I/O side, each Schizo contains two PCI buses. The “A” bus has enhanced PCI (EPCI) logic supporting one 66-MHz, 64-bit, 3.3VDC card. The “B” bus has standard PCI logic supporting three 33-MHz, 64-bit, 5VDC cards.

Each PCI I/O board contains a system data controller (SDC). Again, the SDC multiplexes the console bus connections and acts as the brains behind the movement of data packets across the Fireplane switch.

Just like the System board, the PCI I/O board also contains Sun Fireplane switching logic in the form of address repeater (AR) and data switch (DX) ASICs, which connects to the level 2 switches.

SDC

SBBC

SRAM

FPROM

PCI

BootBus

Address

Console bus Control

Data

Data route

Schizo

slot 7(66 MHz) slot 6 (33 MHz) slot 5 (33 MHz) slot 4 (33 MHz)

slot 3 (66 MHz) slot 2 (33 MHz) slot 1 (33 MHz) slot 0 (33 MHz)

PCI I/O Physical

Sun Conﬁdential: Internal Only

PCI I/O Physical

The Sun Fire platforms support PCI cards and compact PCI cards, mounted on I/O boards.

The F6800 & F4800/4810 platforms support an 8-slot PCI board (illustrated below), and a 4-slot cPCI I/O board.

Note – There are 6-slots which run at 33MHz and 2-slots which run at

33/66 MHz, balanced across two schizo ASICs.

Balance the PCI cards across schizo ASICs, and make sure you do not tie up a 33/66 MHz slot with a 33 MHz card unless absolutely necessary.

Note also that due to the power regulator on the I/O board, only short PCI cards can be ﬁtted in slot 0 & 1.

cPCI I/O Physical

Sun Conﬁdential: Internal Only 24 Sun Fire 3800 - 6800 Quick Reference Guide

cPCI I/O Physical

Below is an illustration of the 6-slot cPCI board is used in the F3800.

The F3800 only supports a 6-slot cPCI board.

The 4-slot cPCI board is very similar to the layout below.

I/O Board Summary

platform

Number of I/O boards

Slot number Location

F6800 Four IB6 IB7 IB8 IB9 Rear

F4810 Two IB6 IB8 Front

F4800 Two IB6 IB8 Rear

F3800 Two IB6 IB8 Front

Sun Fireplane Switch Boards

Sun Conﬁdential: Internal Only

Sun Fireplane Switch Boards

The Sun Fireplane switches are used for data and address transfers across boards.

They are alternatively known as level 2 (L2) switches and repeaters, although the preferred name is the Sun Fireplane switch.

In the F3800, the equivalent of two Sun Fireplane switch board is integrated into an active centerplane.

platform Fireplane switch numbering Location

F6800 RP2, RP3, RP0, RP1 Rear

F4810 RP0, RP2 Front

F4800 RP0, RP2 Rear

F3800 The equivalent of two Sun Fireplane

switch boards are integrated into the active centerplane

Centerplane

How it all bolts together

Sun Conﬁdential: Internal Only 26 Sun Fire 3800 - 6800 Quick Reference Guide

How it all bolts together

The Sun Fire system architecture is designed with two levels of Sun Fireplane switches. One switch level is incorporated into the system boards and I/O boards. The second switch level is implemented as separate boards (In the F3800, the second level switches are incorporated in the centerplane).

Transactions originating from the ﬁrst level switches (on the system and I/O boards) are sent to the second level switches. The second level switches direct the transactions to the correct board destination.

Note: The address “bus” is broadcast based, to ensure cache coherency. The data crossbar is packet-switched; the DX asics provide the connections and the SDC sets up the routing.

CPU

DCDS

MEM

DCDS

System boards

Centerplane

Data Address

I/O boards

Schizo

CPU

Memory

and

CPU

Memory

and

L2 Interconnect

L1 Interconnect

System Controller Boards

Sun Conﬁdential: Internal Only

System Controller Boards

There are three versions of the system controller:

F3800 System Controller 501-5563 F48x0 and F6800 System Controller 501-5407. F4800 and F6800 System Controller V2 501-6631

They are all electrically identical except for minor changes to control the DC-DC converters and the F3800’s system controller has a cooling fan. See page 34 for SC V2 details.

Each system controller contains a console bus hub (CBH) which connects the system controller to the console bus which, in turn, is connected to the system boards, I/O boards, and Sun Fireplane switch boards.

Each system board, I/O board, and Sun Fireplane switch board contains an SDC. The SDC multiplexes the two console bus connections to allow each system board to communicate with the currently active system controller.

Note – With ﬁrmware 5.11.x and 5.12.x the console arbitration was limited;

SSC0 was always the main and SSC1 was always the spare.

Starting with 5.13.x ﬁrmware, either SC can be the main

CPU

SBBC

SDC

System board

I/O board

SBBC

SDC

Schizo1

Schizo0

SDC

SBBC

BootBus

PCI

SC1

SC0

CBH

SDC

PCI

CPU

Console Arbitration (note)

Sun Fireplane switch

Nvram, nvci & SC flashproms

Sun Conﬁdential: Internal Only 28 Sun Fire 3800 - 6800 Quick Reference Guide

Nvram, nvci & SC ﬂashproms

The system controller has a 100-MHz MicroSPARC IIep, 32 Mbytes of DRAM (128 MB on the SC V2), a serial interface and a 10/100Mbit ethernet interface.

The SC has its own power-on self test, called SCPOST, and its own operating system, the RTOS (sometimes referred to as vxworks), which is held in a 2 MB ﬂashPROM on-board. (This is the sgrtos.flash ﬁle in the ﬂashupdate patch).

The SC also has SPOST and its own application called the ScApp which is held in an 8 MB ﬂashPROM on-board. (This is the sgsgsc.flash ﬁle in the ﬂashupdate patch).

The SC maintains a centralized TOD, hostids and MAC addresses and its own host setups in an NVRAM chip.

It also maintains information for four domains in an NVCI chip

Use the dumpconfig command to save the contents as <scname>.tod and <scname>.nvci

SC (microSPARC II ep)

SCPOST & RTOS

32 MB DRAM (V1)

PCI (33 MHz)

RIO

10/100-baseT enet

Ebus

SBBC

CBH

. . . . . . .

PROM Bus

SC error

FPGA

SRAM

Miscellaneous registers

TOD

NVRAM

nvci

ScApp

FPROM

16552

tty serial

16552

SC serial

ttya,ttyb

Serial to other SC

14x Console buses

Error

Clks ctrl

Clocks

Fireplane

I/O PANEL

Temp.

Volt.

lines

Reset lines

Console bus

I2C buses

Global

I2C buses

. . . . . .

I2C muxes

Local I2C Bus

Glue FPGA

SEEPROM

FPROM

128 MB DRAM (V2)

System Controller Clock Circuit

Sun Conﬁdential: Internal Only

System Controller Clock Circuit

The system controllers use a programmable clock. The clock generation circuitry manufactures a base clock of 75 MHz.

In normal operation, each system controller sends its clocks to the phase lock loop (PLL) circuitry in the clock control logic of the other system controller. The clock control logic arbitrarily selects one of the System controllers as the source of the active clocks for the platform.

Assuming you have both SCs present, during the boot sequence of the SSC, you will get a messages indicating that clock failover is enabled.

Jun 06 10:32:23 4800a-sc0 Platform.SC: Clock failover enabled.

If one of the system controller board clock circuits fails, you will see a message stating:

Jun 06 10:32:23 4800a-sc0 Platform.SC: Clock failover disabled.

The PLL circuitry in the other system controller will detect the failure and present its clocks to the platform. The platform will source the clocks from the remaining system controller without interruption to domain operation.

A platform only needs one system controller. The other is there to provide a redundant clock source to the platform, and to facilitate system controller failover.

PLL

SC0

SC1

75MHz to

75MHz

the platform

5.13.x firmware features

Sun Conﬁdential: Internal Only 30 Sun Fire 3800 - 6800 Quick Reference Guide

5.13.x ﬁrmware features

5.13.x ﬁrmware has the following new features:

● Full System Controller (SC) failover

The main SC is identiﬁed as <scname>:SC The spare SC is identiﬁed as <scname>:sc

● Improved diagnostics & recovery.

● Firmware compatibility

New Commands:

● setfailover

● showfailover

Modiﬁed Commands:

● password

● setupplatform

● setupdomain

● showboards

● showdomain

● showplatform

● showresetstate

Service mode:

● Password protected. To get a password, go to https://sfservicepass.sfbay/

where you will need to login with your employee ID and LDAP password.

Note – Refer to the System Controller Command Reference Manual and

Platform Administration Manual shipped with ﬁrmware patch

112494-0x, in which the new and modiﬁed commands are documented.

+ 96 hidden pages

Sun Microsystems Fire F6800, Fire F3800, Fire F4810, Fire F4800 Quick Reference Manual

Specifications and Main Features

Frequently Asked Questions

User Manual