Intel IXP12xx User Manual

Document Number: 301144-001

IXP12xx ATM OC12/Ethernet IP Router Example Design

Performance and Headroom Analysis

April, 2002

Version 1.0, 4/10/02

Information in this document is provided in connection with Intel products. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted by this document. Except as provided in Intel’s Terms and Conditions of Sale for such products, Intel assumes no liability whatsoever, and Intel disclaims any express or implied warranty, relating to sale and/or use of Intel products including liability or warranties relating to fitness for a particular purpose, merchantability, or infringement of any patent, copyright or other intellectual property right. Intel products are not intended for use in medical, life saving, or life sustaining applications.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel

reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.

The IXP12xx ATM OC12/Ethernet IP Router Example Design may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which have an ordering number and are referenced in this document, or other Intel literature, may be

obtained from:

Page 2 of 17

Version 1.0, 4/10/02

IXP12xx ATM OC12/Ethernet IP Router Example Design

Performance and Headroom Analysis

OVERVIEW

This documents details the performance and headroom analysis done on the IXP12xx ATM OC12 / Ethernet IP Router Example Design. It covers the general performance aspects of the protocols; cycle and instruction budgets; testing under different workloads; and performance measurements in both, simulation and hardware environments.

This document also attempts to analyze the amount of headroom available in this design for customers to add additional features by providing microengine and memory utilization metrics.

Three different configurations are supported:

One ATM OC-12 port & eight 100Mbps Ethernet ports Four ATM OC-3 ports & eight 100Mbps Ethernet ports

Two ATM OC-3 ports & four 100Mbps Ethernet ports

Since in each configuration aggregate Ethernet port bandwidth exceeds aggregate ATM port bandwidth, ATM port bandwidth is the limiting external factor. This example design supports full-duplex, full-bandwidth ATM communication on all available ATM ports.

The design is able to simultaneously transmit and receive any traffic pattern on all available ATM ports at line rate. Line rate means that no idle cells should appear on the ATM links. Furthermore, no ATM PHY FIFO overflows or Ethernet MAC FIFO overflows or underflows should occur.

MEASUREMENT ENVIRONMENT

Simulation and hardware performance testing was performed under the following conditions:

o 232 MHz IXP1240 with an 80 MHz IX Bus

(IXP1200 measurements do not use the hardware-CRC on the IXP1240)

o 133 MHz SDRAM – ‘-75’ speed-grade

(some results for 143 MHz (‘-7E’ speed-grade) are also provided where indicated)

Alternate DRAM Timing

The project ships with two FLASH files for two different DRAM speed grades. atm_ether\tools\flash contains files for 133MHz (-75) and 143 MHz (-7E) DRAM. Most measurements were repeated with both settings to illustrate the sensitivity of the design to DRAM performance. Where not specifically mentioned in this document, the slower 133MHz settings were used.

For use with the IXP1240/1250 with hardware CRC capability Similar to the above configuration (requires the IXP1240/50), except

that it uses four OC-3 ports. For use with the IXP1200 (which does not have hardware CRC capability). Instead, CRC computation is performed by two microengines (thus the reduced data rates).

Page 3 of 17

Version 1.0, 4/10/02

KEY WORKLOADS & APPROACHES TO TESTING THE EXAMPLE DESIGN

Protocol Performance of IP over ATM vs. Ethernet

Figure 1 details the protocol processing required to carry an IP packet over ATM and Ethernet. .

Figure 1 – Protocol Processing

Figures 2 and 3 show that as the size of the IP packet varies so do the efficiencies of ATM and Ethernet. This section details those efficiencies and the resulting performance implications

Single Cell PDU Workload

Single-cell PDUs result from IP packets of size 20 to 32 bytes – for example UDP packets with up to 4 payload bytes (8 bytes of LLC/SNAP plus 8 bytes of AAL5 trailer are included with the IP packet in the 48-byte cell payload). Adding a 4-byte ATM cell header plus 1-byte HEC results in a 53-byte cell. SONET overhead transparently adds about another 2 bytes/cell to the wire-time such that its total cost is 55-bytes in terms of a 155 or 622 Mbps ATM link.

When the same packet is carried over Ethernet, it expands to consume a minimum-sized 64-byte frame. Ethernet then adds at least 960ns of inter-packet gap (12-bytes), plus a preamble (8bytes). The total packet cost is 84-bytes on a 100Mbps Ethernet link.

Page 4 of 17

Version 1.0, 4/10/02

The result is that ATM is significantly more efficient that Ethernet in terms of Mbps for carrying very small PDUs. Every Mbps of single-cell-PDUs on the ATM link requires (84/55) Mbps on the matching Ethernet link(s).

176

160

144

128

112

020406080100120140

IP Pac ke t Le ngth [Bytes]

Et her net F r am e Byt es AAL5 P DU Lengt h

Figure 2 – Frame and PDU Length versus IP Packet Length

1000

900

800

700

600

500

400

300

200

OC-1 2 Input OC-3 Input

100

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500

IP Pac ke t Le ngth [Bytes]

Figure 3 – Expected Ethernet Transmit Bandwidth

Page 5 of 17

Version 1.0, 4/10/02

As shown graphically in Figure 3, 622Mbps of single-cell-PDU input requires 622*(84/55) = 949 Mbps of Ethernet output.

This example design supplies 800Mbps of Ethernet bandwidth (IXP1240 configurations), so under a single cell/PDU workload the design can be expected to transmit Ethernet at line rate, and to discard the excess ATM input. In the reverse direction, if Ethernet data is received at wire rate (even with all 8 ports running at wire-rate), ATM transmit will not be saturated under a single cell/PDU workload.

Multiple Cells/PDU Workload

Following Figure 2 from left to right, it is clear that once a PDU size overflows from one cell into two, Ethernet becomes more efficient in terms of Mbps.

When the packet completely fills two or three cells, ATM is again more efficient, but not by much. For example, two full cells require 622(118/110) = 666 Mbps; and three full cells require 622/(166/159) = 625 Mbps of Ethernet bandwidth for 622Mbps of ATM bandwidth. These numbers are well below the 800Mbps of Ethernet bandwidth available in the example configuration.

Thus for multi-cell/PDU workloads, this design has more Ethernet bandwidth available than ATM bandwidth, and excess Ethernet input will be discarded. In the reverse direction, Ethernet transmit bandwidth will not be exceeded even if all ATM ports receive data at wire-rate (Figure

3). While this design supports any IP packet size between 20 and 1500 bytes, 40 byte packets are expected to be the most common. 40-byte packets corresponds to a 20-byte IP header plus a 20byte TCP header, with no payload. 40-byte IP packets form AAL5 PDUs that consume 2 ATM cells.

The largest PDU supported by the design contains a 1500-byte packet. This packet is carried by a 1518-byte Ethernet frame or by a 32-cell AAL5 PDU.

CYCLE AND INSTRUCTION BUDGETS

Cycle Budgets to support Line Rates

OC-12 line rate is 622Mbps, but SONET overhead reduces it to 599Mbps available to ATM cells. 53 bytes/cell * 8 bits/byte / 599 Mb/sec = 708 ns/cell. So 232MHz * 708 ns/cell = 164

cycles/cell.

OC-3 line rate is 155Mbps, but SONET overhead reduces it to 149Mbps available to ATM cells. 53 bytes/cell * 8 bits/byte / 149 Mb/sec = 2.85 us/cell. So 232MHz * 2.85 us/cell = 660

cycles/cell.

Ethernet has a variable sized frame, and thus a variable per-frame cycle budget. The worst-case is minimum-sized 64-byte frames, thus they are the focus for per-frame calculations here. A 64byte frame actually occupies 84 bytes on the wire. {(12 byte Inter Packet Gap) + (8 byte preamble) + (46 byte IP packet) + (14 byte Ethernet Header) + (4 byte Ethernet FCS) = 84

Page 6 of 17

+ 11 hidden pages