Supermicro AOC-SLG3-4E2P User Manual

Product Brief

General Features

• State-of-the-art switch fabric

− Sharing I/Os among multiple
hosts

− Low latency TWC

• Any port can be a host port or
Downstream (device) Port

• Works with standard PCIe end­points and hosts – and software,
as well as with existing application
software

• MSI-X support

• Allows flexible fabric topologies

Key Advantages

• PCI Express Switches

− 12 to 97 Lanes with Integrated
on-chip SerDes

− 5 to 25 Independent ports

−Designate any Port as the
Upstream Port

− Low-power SerDes (under 90
mW per Lane)

− Device-Specific Relaxed Ordering

− Port configuration

− Dedicated management port
for mCPU

PEX9700 Series Switch Chips

Managed PCI Express Switches Based on
ExpressFabric® Technology

Converge Servers and IO controllers with PCIe

• Create cost-eective high-availability hyperscale systems by enabling
communication between in-rack hosts and endpoints using PCIe

• Simplify connectivity while providing the highest PCIe switching
performance available for data center servers, storage, and networks

• Reduce latency, system complexity, and power consumption
by up to 50% in data-intensive environments

• Take advantage of industry-first features for most demanding hyper­converged, NVMe and rack scale systems

Broadcom PEX9700 switches allow customers to build high performance, low
latency, scalable, cost-eective PCI Express-based fabrics. The switches enable
I/O sharing with standard SR-IOV or multifunction capability, allowing multiple
hosts or Nodes to reside on a single PCIe-based network. Hosts communicate
through Tunneled Window Connection (TWC), a special low latency host-to­host communication capability for short packets.

Shared I/O Using Standards

PEX9700 switches allow the Virtual Functions (VFs) of SRIOV endpoints
(such as an Broadcom MegaRAID® SAS controller) to be shared and
assigned to multiple hosts concurrently. Each host can enumerate its
assigned functions using standard BIOS and OS software and use them with
unmodified vendor-supplied drivers. The use of standard system software
minimizes software support costs.

Software-Defined Fabric

The switches are built on a hybrid hardware/software platform that oers high
configurability and flexibility in regards to the number of hosts, end-points,
and PCIe slots. The critical pathways have direct hardware support, enabling
the fabric to oer non-blocking, line speed performance with features such
as I/O sharing. The solution is completed by management processor that
communicates with platform management via API and/or CLI. The solution
oers an innovative approach to setup and control, making use of an o­chip management CPU (mCPU) to initialize the PEX9700 switch, configure
the routing tables, handle errors, Hot-Plug events, and enable the solution to
extend the capabilities without modifying the system software.

Tunneled Window Connection (TWC)

TWC allows short messages to be sent from one host to another in a very low
latency manner, and without the overhead associated with DMA.

PEX9700 Series Switch Chips

Product Brief

Key Advantages (Continued)

− x4, x8, or x16, depending on
Port configuration; x4 can
down-train to x1 and x2 width

− Configurable through serial
EEPROM, I2C, SMBus, and/or
Host port

• Standards Compliant

− PCI Express Base Specification,
r3.1 (backward compatible w/
PCIe r2.0, & r1.0a/1.1)

− PCI Power Management Spec,
r1.2

• High Performance

− Full line rate on all ports

• Cut-Thru packet latency of less
than 150ns (x16 to x16)

− 2KB Max Payload Size

− Multicast through DMA

• Quality of Service (QoS)

− 8 Trac Classes (TC) supported

• Reliability, Availability,

Serviceability

− visionPAK

− performance PAK

− DPC/eDPC Support

− Read Tracking for surprise
removal

− All ports Hot-Plug capable thru

I2C –SSC isolation on all ports

− SRIS support

− ECRC and Poison bit support

− Port Status bits and GPIO
available

Downstream Port Con­tainment (DPC/eDPC)

Most servers have diculty handling
serious errors, especially when a
PCIe end-point disappears from
the system. DPC/eDPC allows a
downstream link to be disabled
after an uncorrectable error, making
recovery possible in a controlled and
robust manner.

Flexible Topologies

PEX9700 switches eliminate the
topology restrictions of PCIe. The
switch allows other topologies such
as mesh, I/O Expansion Box with
Multiple Hosts, and many others.
And it does this while allowing the
components to remain architecturally
and software compatible with
standard PCIe.

Improved SSC Isolation

The switches oer several
mechanisms for supporting multi­clock domains that include spread
spectrum clocking; eliminating
the need to pass a common clock
across a backplane. In addition to
the standard Avago approach to the
problem, a new PCI-SIG approach
called SRIS (Separate Refclk
Independent SSC Architecture) is
now available.

Applications

Products based on ExpressFabric
technology can help deliver an
outstanding solution for designing
a heterogeneous system with a
requirement for a flexible mix of
processors, storage elements, and
communication devices.

HPC Clusters

HPC clusters are made up of high­performance processing elements
that communicate through high
bandwidth, low latency pathways
in order to execute applications
such as medical imaging, financial
trading, data warehousing, etc.
PEX9700 switches can be used
in switch fabric applications for
HPC clustering. The processing
subsystems can be connected to the
PCIe fabric while running the same
application software. PCIe switch
based clustering eliminates expensive
protocol bridging devices resulting in
lower cost and power. And clustering
systems can be built with I/O sharing
as an additional native capability
when needed.

Software Development
Kit (SDK)

The SDK for the PEX9700 series
includes drivers, source code and
GUI interfaces to aid in configuring
and debugging. Both the
performancePAK™ and visionPAK™
are exclusive to Avago and are
supported by its RDK and SDK, which
are the industry’s most advanced
hardware-and software development
kits.

performancePAK

The performancePAK is a suite of
unique and innovative performance
features that allows Avago Gen 3
switches to be the highest performing
switches in the market today.

visionPAK

The visionPAK is a debug diagnostics
suite of integrated hardware and
software instruments that allows
users to help bring their systems to
market faster.

PEX9700 Series Switch Chips