How it Works
Broadcom
Loading...
BCM5720
Programmer's Manual
593 pgs3.53 Mb0
Table of contents
Loading...

Broadcom BCM5720, BCM5717, BCM5718, BCM5719 Programmer's Manual

Download
Page 1
Programmer’s Guide
BCM5718
NetXtreme®/NetLink® BCM5718 Family
5718-PG108-R
5300 California Avenue • Irvine, CA 92617 • Phone: 949-926-5000 • Fax: 949-926-5203 January 29, 2016
Page 2

Revision History

Revision Date Change Description
5718-PG-108-R 01/29/16 Updated:
• “Other Considerations Relating to Producer Ring Setup” on page 90
• “RCB Setup Pseudo Code” on page 91
• “Summary of Register Settings to Support Jumbo Frames” on page 129
• “Receive MTU Size Register (offset: 0x43C)” on page 316
• “BM Hardware Diagnostic 2 Register (offset: 0x4450)” on page 431
5718-PG-107-R 07/17/13 Updated:
• “Send Rings” on page 106
• “Initialization Procedure” on page 140
• Table 49: “GPIO Usage for Power Management for Broadcom Drivers,” on page 192
• Table 101: “Multiple Send Ring Mail Boxes,” on page 357
• “Send BD Ring Host Producer Index Register (offset: 0x5900)” on page 465
• “Send BD Ring NIC Producer Index Register (offset: 0x5980)” on page 466
• Table 121: “GbE Port Internal PHY Register Map,” on page 553
• Table 127: “AUTONEG LINK PARTNER ABILITY,” on page 559
Added:
• Table 124: “02h: PHY_Identifier_MSB_Register,” on page 558
• Table 125: “03h: PHY_Identifier_LSB_Register,” on page 558
Broadcom Corporation
5300 California Avenue
Irvine, CA 92617
© 2016 by Broadcom Corporation
All rights reserved
Printed in the U.S.A.
Broadcom
NetXtreme II
affiliates in the United States, certain other countries and/or the EU. Any other trademarks or trade names
®
, the pulse logo, Connecting everything®, the Connecting everything logo, NetLink®,
®
, NetXtreme®, and NetXtreme® I are among the trademarks of Broadcom Corporation and/or its
mentioned are the property of their respective owners.
Page 3
Revision Date Change Description
5718-PG-106-R 06/25/12 Updated:
• “Base Address Register 1 (offset: 0x10)” on page 275
• “Base Address Register 2 (offset: 0x14)” on page 275
• “Base Address Register 3 (offset: 0x18)” on page 275
• “Base Address Register 4 (offset: 0x1c)” on page 276
• “Mode Control Register (offset: 0x6800)” on page 475
Added:
• Section 8: “IEEE1588,” on page 152
• “RX TIME STAMP LSB REG [Offset 0X06B0]” on page 163
• “RX TIME STAMP MSB REG [Offset 0x06B4]” on page 163
• “RX PTP SEQUENCE ID REG [Offset 0X06B8]” on page 163
• “RX LOCK TIMER LSB REG [Offset 0x06C0]” on page 164
• “RX LOCK TIMER MSB REG [Offset 0x06C4]” on page 164
• “RX PTP CONTROL REG [Offset 0X06C8]” on page 164
• Section 12: “IO Virtualization (IOV),” on page 264
• “Perfect Match Destination Address Registers” on page 465
• “VRQ Filter Set Registers” on page 461
• “VRQ Mapper Registers” on page 462
• “Base Address Register 5 (offset: 0x20)” on page 276
• “Base Address Register 6 (offset: 0x24)” on page 277
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 3
Page 4
Revision Date Change Description
5718-PG-105-R 02/24/12 Updated:
• Table 3: “Family Revision Levels,” on page 48
• Table 5: “Flag Fields for a Ring,” on page 70
• Figure 24: “Ring Control Block,” on page 118
• “Summary of Register Settings to Support Jumbo Frames” on page 126
• “Initialization Procedure” on page 136
• “Reading a PHY Register” on page 186
• “Writing a PHY Register” on page 187
• “Subsystem ID/Vendor ID Register (offset: 0x2C)” on page 251
• “DMA Read/Write Control Register (Offset: 0x6c)” on page 258
• “PCI State Register (offset: 0x70)” on page 259
• “Receive BD Standard Producer Ring Index Register (offset: 0x268­0x26f)” on page 281
• “Transmit MAC Status Register (offset: 0x460)” on page 296
• “Receive MAC Mode Register (offset: 0x468)” on page 297
• “Statistics Registers” on page 319
• “H2B Statistics Registers” on page 320
• “Receive Data and Receive BD Initiator Mode Register (offset: 0x2400)” on page 341
• “Link Speed 10 MB/No Link Power Mode Clock Policy Register (offset: 0x3604)” on page 356
• “Link Speed 100 MB Power Mode Clock Policy Register (offset: 0x3608)” on page 357
• “Link Aware Power Mode Clock Policy Register (offset: 0x3610)” on page 359
• “D0u Clock Policy Register (offset: 0x3614)” on page 360
• “Link Idle Power Mode Clock Policy Register (offset: 0x3618)” on page 360
• “APE CLK Policy Register (offset: 0x361C)” on page 361
• “APE Sleep State Clock Policy Register (offset: 0x3620)” on page 363
• “Clock Speed Override Policy Register (offset: 0x3624)” on page 364
• “Clock Status Register (offset: 0x3630)” on page 367
• “Padring Control Register (offset: 0x3668)” on page 376
• “Receive Coalescing Ticks Register (offset: 0x3C08)” on page 393
• “Send Coalescing Ticks Register (offset: 0x3C0C)” on page 394
• “Receive Max Coalesced BD Count Register (offset: 0x3C10)” on page 395
• “Send Max Coalesced BD Count Register (offset: 0x3C14)” on page 397
• “Status Block Host Address Register (offset: 0x3C38)” on page 400
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 4
Page 5
Revision Date Change Description
• “Status Block Base Address Register (offset: 0x3C44)” on page 426
• “BM Hardware Diagnostic 2 Register (offset: 0x4450)” on page 436
• “LSO Read DMA Mode Register (offset: 0x4800)” on page 438
• “LSO Read DMA Reserved Control Register (offset: 0x4900)” on page 445
• “LSO Read DMA Flow Reserved Control Register (offset: 0x4904)” on page 446
• “LSO/Non-LSO/BD Read DMA Corruption Enable Control Register (offset: 0x4910)” on page 446
• “BD Read DMA Mode Register (Offset: 0x4A00)” on page 449
• “BD READ DMA Reserved Control Register (offset: 0x4A70)” on page 456
• “BD READ DMA Flow Reserved Control Register (offset: 0x4A74)” on page 457
• “BD READ DMA Corruption Enable Control Register (offset: 0x4A78)” on page 457
• “Non_LSO Read DMA Mode Register (offset: 0x4B00)” on page 458
• “Non-LSO Read DMA Reserved Control Register (offset: 0x4B74)” on page 461
• “Non-LSO Read DMA Corruption Enable Control Register (offset: 0X4B7C)” on page 462
• “Write DMA Mode Register (offset: 0x4C00)” on page 464
• “Low Priority Mailboxes” on page 469
• “Interrupt Mailbox 0 Register (offset: 0x5800)” on page 469
• “Other Interrupt Mailbox Register (offset: 0x5808–0x5818)” on page 469
• “General Mailbox Registers 1-8 (offset: 0x5820–0x5824)” on page 469
• “Receive BD Standard Producer Ring Index Register (offset: 0x5868)” on page 470
• “Receive BD Return Ring 0 Consumer Index Register (offset: 0x5880­0x5887)” on page 470
• “Receive BD Return Ring 0 Consumer Index Register (offset: 0x5880­0x5887)” on page 470
• “Send BD Ring Consumer Index Register (offset: 0x5900)” on page 471
• “NVM Write Register (offset: 0x7008)” on page 497
• “NVM Address Register (offset: 0x700C)” on page 497
• “NVM Read Register (offset: 0x7010)” on page 498
• “NVM Config 1 Register (offset: 0x7014)” on page 498
• “NVM Access Register (offset: 0x7024)” on page 502
• “00h: MII_Control_Register” on page 513
• “03h: PHY_Identifier_LSB_Register” on page 515
• “04h: Auto_Negot_Advertisement_Register” on page 515
• “09h: 1000Base_T_Control_Register” on page 518
• “10h: PHY_Extended_Control_Register” on page 522
• “18h: Auxiliary Control Register (Shadow Register Selector = “000”)” on page 526
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 5
Page 6
Revision Date Change Description
• “18h: Miscellaneous Control Register (Shadow Register Selector = “111”)” on page 533
• “1Ch: Cabletron LED Register (Shadow Register Selector = “00h”)” on page 538
• “1Ch: Spare Control 4 Register (Shadow Register Selector = “0bh”)” on page 547
• “1Ch: External SerDes Control Register (Shadow Register Selector = “14h”)” on page 557
• “1Ch: SGMII Slave Register (Shadow Register Selector = “15h”)” on page 559
• “1Ch: Misc 1000-X Control 2 Register (Shadow Register Selector = “16h”)” on page 561
• “1Ch: Misc 1000-X Control Register (Shadow Register Selector = “17h”)” on page 563
• “1Ch: Auto-Detect SGMII/GBIC Register (Shadow Register Selector = “18h”)” on page 564
• “1Ch: Auto-Detect Medium Register (Shadow Register Selector = “1eh”)” on page 572
• “1Ch: Mode Control Register (Shadow Register Selector = “1fh”)” on page 573
Added:
• Table 1: “Register Access Methods,” on page 46
• “Device Reset Procedure” on page 146
• “PHY Loopback Configuration” on page 205
• “PHY Configuration Auto-Negotiation (10/100/1000 Speed with Half and Full Duplex Support)” on page 206
• “MSI-X Capabilities Registers” on page 281
• “PCIe Capabilities Registers” on page 282
• “VRQ Flush Control Register (Offset: 0x2410)” on page 369
• “VRQ Flush Timer Register (offset: 0x2414)” on page 370
• “RDI B2HRX Hardware Debugging Register (offset: 0x2418)” on page 370
• “Receive BD Ring Initiator Local NIC Standard Receive BD Consumer Index (offset: 0x2474)” on page 373
• “B2HRX Byte-count Statistics Count (offset: 0x24D0)” on page 374
• “B2HRX Unicast Statistics Count (offset: 0x24D4)” on page 374
• “B2HRX Multicast Statistics Count (offset: 0x24D8)” on page 374
• “B2HRX Broadcast Statistics Count (offset: 0x24DC))” on page 374
• “B2HRX Drop Packet Count (offset: 0x24E0)” on page 374
• “B2HRX Drop Packet Byte Count (offset: 0x24E4)” on page 374
• “B2HRX APE Byte-count Statistics Count (offset: 0x24E8)” on page 375
• “B2HRX APE Unicast Statistics Count (offset: 0x24EC)” on page 375
• “B2HRX APE Multicast Statistics Count (offset: 0x24F0)” on page 375
• “B2HRX APE Broadcast Statistics Count (offset: 0x24F4)” on page 375
• “B2HRX APE Drop Packet Count (offset: 0x24F8)” on page 375
• “B2HRX APE Drop Packet Byte Count (offset: 0x24FC)” on page 375
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 6
Page 7
Revision Date Change Description
• “Receive Max Coalesced BD Count During Interrupt Register (offset: 0x3C18)” on page 424
• “Send Max Coalesced BD Count During Interrupt Register (offset: 0x3C1C)” on page 425
• “NIC Mini Receive BD Consumer Index (offset: 0x3c58)” on page 428
• “Send BD Ring Producer Index Register (offset: 0x5980)” on page 471
• “DMA Completion Mode Register (Offset: 0x6400)” on page 477
• Figure 58: “Copper PHY Register Mapping Table,” on page 511
• Figure 59: “SerDes PHY Register Map,” on page 512
• “Clause 45 Registers” on page 601
• “SerDes PHY Register Definitions” on page 578
• “PHY 0x18 Shadow 0x1 register read Procedure” on page 527
• Added PHY 0x1C Shadow 0x1 register read Procedure information to “1Ch: Cabletron LED Register (Shadow Register Selector = “00h”)” on page 538
• Added Clause 45 register Dev3 Reg803Eh read Procedure to “Clause 45 Register Dev 3 Reg14h (20d): EEE Capability Register” on page 601
• NIC Ring Addresses information to Memory map tables in Appendix C: “Device Register and Memory Map,” on page 611
Deleted
• Section 11: Host to/from BMC Pass Through
• Appendix D: Appendix
• Top Level MII Registers
5718-PG104-R 06/29/11 Updated:
• Table 27: “Flag Field Description,” on page 113
• Table 31: “Send Buffer Descriptor Flags,” on page 123
• “Clock Control” on page 191
• Table 47: “Ethernet Controller Power Pins,” on page 191
• “Internal Memory” on page 214
• “ISR Flow” on page 230
• Table 82: “Interrupt-Related Registers,” on page 235
• “Status Register (offset: 0x362C)” on page 391
• “Clock Status Register (offset: 0x3630)” on page 393
• “LSO Read DMA Mode Register (offset: 0x4800)” on page 438
• “NVM Write Register (offset: 0x7008)” on page 497
Added:
• “Device Closing Procedure” on page 147
• “TX TIME STAMP LSB REG (offset: 0x5C0)” on page 327
• “TX TIME STAMP MSB REG (offset: 0x5C4)” on page 327
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 7
Page 8
Revision Date Change Description
5718-PG103-R 01/26/11 Updated:
• Added BCM5720 to Section 1: “Introduction,” on page 49.
• Added BCM5720 to “Introduction” on page 49.
• Added Host to BMC to “Transmit MAC Mode Register (offset: 0x45C)” on page 317.
• Added Host to BMC to “Transmit MAC Lengths Register (offset: 0x464)” on page 319.
• Added Host to BMC to “Mode Control Register (offset: 0x6800)” on page 477.
Added
• “HTX2B Perfect Match[1–4] HI Reg (offset: 0x4880, 0x4888, 0x4890, 0x4898)” on page 330.
• “HTX2B Perfect Match[1–4] LO Reg (offset: 0x4884, 0x488C, 0x4894, 0x489C)” on page 330.
• “HTX2B Protocol Filter Reg (offset: 0x6D0)” on page 331.
• “HTX2B Global Filter Reg (address: 0x6D4)” on page 333.
• “H2B Statistics Registers” on page 346.
• “HTX2B Statistics” on page 347
• “B2HRX Statistics” on page 347
• “RMU Registers” on page 504
• “RMU_EGRESS_DA1_MATCH[1-8]_REG (offsets: 0x00B0, 0x00B8, 0x00C0, 0x00C8 … 0xE8)” on page 504
• “RMU_EGRESS_DA2_MATCH[1-8]_REG (Offsets 0x00B4, 0x00BC, 0x00C4, 0xCC …0xEC)” on page 504
• “RMU_EGRESS_STATUS_REG (Offset 0x0000)” on page 504
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 8
Page 9
Revision Date Change Description
5718-PG102-R 12/16/10 Updated:
• Added BCM5719 to “Introduction” on page 39.
• Added BCM5719 to “Related Documents” on page 39.
• Added BCM5719 to Table 1: “BCM5718 Family Product Features,” on page 40.
• Removed PHY core column and added BCM5717 B0, BCM5718 BO, and BCM5719 to Table 2: “Family Revision Levels,” on page 42.
• Updated note in “Revision Levels” on page 42.
• Added Memory Arbiter to Figure 1: “Individual Port Functional Block Diagram,” on page 45.
• Added BCM5719 to “Overview of Features” on page 46.
• Added note about BCM5719 to Figure 2: “High-Level System Functional Block Diagram,” on page 47.
• Added max ring sizes to “Ring Control Block” on page 99.
• Added BCM5719 toTable 6: “Defined Flags for Send Buffer Descriptors,” on page 67.
• Updated Host Ring Size to Table 7: “Receive Return Rings,” on page 70.
• Corrected typo in Figure 26: “Send Driver Interface,” on page 119.
• Corrected typo in Figure 27: “Receive Producer Interface,” on page 120.
• Corrected typo in Figure 28: “Receive Return Interface,” on page 121.
• Updated Step 36 in “Initialization Procedure” on page 137.
• Added BCM5719 to “Description” on page 168.
• Corrected typo in “PCI Classcode and Revision ID Register (offset: 0x08)—Function 0” on page 255
• Corrected typos in “Power Management Control/Status Register (offset: 0x4C) — Function 0” on page 261
• Added note to Enable Endian Byte Swap in “Miscellaneous Host Control Register (offset: 0x68)” on page 264.
• Updated all Indirection Table register descriptions in “RSS Registers” on page 276.
• Added BCM5717 and BCM5718 values to “CPMU Control Register (offset: 0x3600)” on page 346.
• Added BCM5719 to “Link Aware Power Mode Clock Policy Register (offset: 0x3610)” on page 349.
• Added BCM5719 to “APE CLK Policy Register (offset: 0x361C)” on page 352.
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 9
Page 10
Revision Date Change Description
Updated (continued):
• Added BCM5718 to “Clock Speed Override Policy Register (offset: 0x3624) for BCM5718” on page 354
• Added BCM5718 to “Clock Status Register (offset: 0x3630)” on page 358
• Added Reserved for BCM5719 to “PCIE Status Register (offset: 0x3634)” on page 359
• Added BCM5719 to “GPHY Control/Status Register (offset: 0x3638)” on page 360
• Updated introduction to “PCIE Idle Detection De-Bounce Control Register (offset: 0x364C)” on page 362
• Corrected typo and added BCM5719 to “DLL Lock Timer Register (offset: 0x3654)” on page 364
• Updated Chip ID default value in “CHIP ID Register (offset: 0x3658)” on page 365
• Added BCM5719 to “Padring Control Register (offset: 0x3668)” on page 367
• Added BCM5719 to “Reserved (offset: 0x366C)” on page 368
• Added BCM5719 to “Reserved (offset: 0x367C)” on page 373
• Added BCM5719 to “Read DMA Mode Register (offset: 0x4800)” on page 398
• “LSO Read DMA Corruption Enable Control Register (offset: 0x4910)” on page 413
• Added BCM5719 to “Write DMA Mode Register (offset: 0x4C00)” on page 443
• Added BCM5719 to “MSI Mode Register (offset: 0x6000)” on page 454
Added:
• “Receive BD Standard Producer Ring Index (High Priority Mailbox) Register (offset: 0x268-0x26f)” on page 276
• “TX Time Stamp LSB Reg (offset: 0x5C0)” on page 282
• “TX Time Stamp MSB Reg (offset: 0x5C4)” on page 283
• “RX Time Stamp LSB Reg (offset 0x06B0)” on page 308
• “RX Time Stamp MSB Reg (offset 0x06B4)” on page 308
• “RX PTP Sequence ID Reg (offset 0x06B8)” on page 308
• “RX Lock Timer LSB Reg (offset 0x6C0)” on page 309
• “RX Lock Timer MSB Reg (offset 0x06C4)” on page 309
• “RX PTP Control Reg (offset: 0x6C8)” on page 310
• “Clock Speed Override Policy Register (offset: 0x3624)” on page 355
• “Clock Status Register (offset: 0x3630)” on page 358
• “Global Mutex Request Register (offset: 0x36F0)” on page 381
• “Global Mutex Grant Register (offset: 0x36F4)” on page 381
• “Temperature Monitor Control Register (offset: 0x36FC)” on page 382
• “BCM5719 Registers” on page 469
Removed:
• “Reserved (offset: 0x378C)”
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 10
Page 11
Revision Date Change Description
5718-PG101-R 11/12/10 Updated:
• Table 1: “BCM5718 Family Product Features,” on page 40
• IP cksum description in “Receive Buffer Descriptors” on page 70
• “Extended RX Buffer Descriptor (BD)” on page 110
• Table title for Table 34: “Jumbo Producer Ring Host Address Low Register (offset: 0x2444),” on page 123
• Default value and description in Table 36: “Jumbo Producer Ring NIC Address Register (offset: 0x244C),” on page 123
• NIC ring address values in Table 45: “NIC Ring Addresses,” on page 126
• PCI version in “Description” on page 148
• “EMAC Status Register (offset: 0x404)” on page 279
• “DMA Flag Register for TCP Segmentation (offset: 0xCEC)” on page 321
• “Jumbo Producer Ring NIC Address Register (offset: 0x244C)” on page 337
• “Receive Producer Length/Flags Register (offset: 0x2458)” on page 337
• “Receive Producer Ring NIC Address Register (offset: 0x245C)” on page 338
• “GPHY Strap Register (offset: 0x3664)” on page 366
• “Read DMA Mode Register (offset: 0x4800)” on page 398
• “BCM5718 Family MII Bus PHY Addressing” on page 496
Added:
• Section 8: “Device Control,” on page 137
• Registers 0x00 to 0x3c and 0x48 to 0x64 to “PCI Configuration Registers” on page 254
• Section 14: “Transceiver Registers,” on page 496
Removed:
• References to BCM5724 throughout
• Column from Table 1: “BCM5718 Family Product Features,” on page 40
• Register control mode from “MDI Register Access” on page 187
• MDI Control Register (offset: 0x6844)
5718-PG100-R 04/13/10 Initial release
Revision HistoryBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 11
Page 12
Table of ContentsBCM5718 Programmer’s Guide

Table of Contents

About This Document................................................................................................................................44
Purpose and Audience .......................................................................................................................... 44
Acronyms and Abbreviations................................................................................................................. 44
Document Conventions ......................................................................................................................... 44
References ............................................................................................................................................ 45
Technical Support...................................................................................................................................... 46
Section 1: Introduction .....................................................................................................47
Product Features........................................................................................................................................ 47
Revision Levels .......................................................................................................................................... 49
Programming the Ethernet Controllers.................................................................................................... 50
Section 2: Hardware Architecture....................................................................................51
Theory of Operation................................................................................................................................... 51
Overview of Features ................................................................................................................................. 52
Receive Data Path ...................................................................................................................................... 54
RX Engine ............................................................................................................................................. 54
RX FIFO ................................................................................................................................................ 54
Rules Checker....................................................................................................................................... 55
RX List Initiator ...................................................................................................................................... 55
Transmit Data Path..................................................................................................................................... 56
TX MAC................................................................................................................................................. 56
TX FIFO................................................................................................................................................. 56
DMA Read.................................................................................................................................................... 57
Read Engine.......................................................................................................................................... 57
Read FIFO............................................................................................................................................. 57
Buffer Manager...................................................................................................................................... 58
DMA Write ................................................................................................................................................... 58
Write Engine.......................................................................................................................................... 58
Write FIFO............................................................................................................................................. 58
Buffer Manager...................................................................................................................................... 59
LED Control................................................................................................................................................. 59
Memory Arbiter........................................................................................................................................... 59
Host Coalescing ......................................................................................................................................... 60
Host Coalescing Engine ........................................................................................................................ 60
MSI FIFO............................................................................................................................................... 61
Status Block .......................................................................................................................................... 61
10BT/100BTx/1000BASE-T Transceiver ................................................................................................... 62
Broadcom
January 29, 2016 • 5718-PG108-R Page 12
®
Page 13
Table of ContentsBCM5718 Programmer’s Guide
Auto-Negotiation.................................................................................................................................... 62
Automatic MDI Crossover ..................................................................................................................... 62
PHY Control................................................................................................................................................ 62
MII Block................................................................................................................................................ 62
GMII Block............................................................................................................................................. 64
MDIO Register Interface........................................................................................................................ 66
Management Data Clock................................................................................................................ 66
Management Data Input/Output..................................................................................................... 66
Management Data Interrupt ........................................................................................................... 66
Management Register Block .......................................................................................................... 66
Section 3: NVRAM Configuration.....................................................................................67
Overview...................................................................................................................................................... 67
Self-Boot...................................................................................................................................................... 68
Section 4: Common Data Structures...............................................................................69
Theory of Operation................................................................................................................................... 69
Descriptor Rings......................................................................................................................................... 69
Producer and Consumer Indices........................................................................................................... 70
Ring Control Blocks............................................................................................................................... 71
Send Ring Control Blocks .............................................................................................................. 71
Receive Ring Control Blocks.......................................................................................................... 72
Send Rings............................................................................................................................................ 73
Send Buffer Descriptors ................................................................................................................. 75
Standard (Not Large Segment Offload) ................................................................................. 75
Large Segment Offload (LSO) Send BD................................................................................ 76
Receive Rings ....................................................................................................................................... 77
Receive Producer Ring .................................................................................................................. 77
Receive Return Rings .................................................................................................................... 78
Receive Buffer Descriptors ............................................................................................................ 78
Additional Ring Information for the BCM5718 Family .................................................................... 81
Status Block................................................................................................................................................ 82
Status Block Format .............................................................................................................................. 82
INTx/MSI — Legacy Mode Status Block Format ............................................................................ 83
Single-Vector or INTx — RSS Mode Status Block Format ............................................................. 84
Multivector RSS Mode Status Block Format .................................................................................. 85
Status Block and INT MailBox Addresses...................................................................................... 86
Section 5: Receive Data Flow...........................................................................................88
Introduction................................................................................................................................................. 88
Receive Producer Ring.............................................................................................................................. 90
Broadcom
January 29, 2016 • 5718-PG108-R Page 13
®
Page 14
Table of ContentsBCM5718 Programmer’s Guide
Setup of Producer Rings Using RCBs................................................................................................... 90
Receive Producer Ring RCB—Register Offset 0x2450–0x245f .................................................... 90
Other Considerations Relating to Producer Ring Setup......................................................... 90
RCB Setup Pseudo Code .............................................................................................................. 91
Receive Buffer Descriptors.................................................................................................................... 91
Management of Rx Producer Rings with Mailbox Registers and Status Block ..................................... 92
Status Block ................................................................................................................................... 92
Mailbox........................................................................................................................................... 92
Receive BD Producer Ring Producer Index........................................................................... 92
Receive Return Rings ................................................................................................................................93
Management of Return Rings with Mailbox Registers and Status Block .............................................. 94
Host Buffer Allocation............................................................................................................................ 94
Receive Rules Setup and Frame Classification ....................................................................................95
Receive Rules Configuration Register ........................................................................................... 95
Receive List Placement Rules Array..............................................................................................96
Class of Service Example .............................................................................................................. 97
Checksum Calculation............................................................................................................................... 98
VLAN Tag Strip........................................................................................................................................... 98
RX Data Flow Diagram ............................................................................................................................. 100
Receive Side Scaling................................................................................................................................101
Overview ............................................................................................................................................. 101
Functional Description......................................................................................................................... 101
RSS Parameters ................................................................................................................................. 102
Hash Function .............................................................................................................................. 102
Hash Type.................................................................................................................................... 102
Hash Mask ................................................................................................................................... 102
Indirection Table........................................................................................................................... 103
Secret Hash Key .......................................................................................................................... 103
RSS Initialization ................................................................................................................................. 103
RSS Rx Packet Flow ........................................................................................................................... 104
Section 6: Transmit Data Flow .......................................................................................105
Introduction............................................................................................................................................... 105
Send Rings................................................................................................................................................ 105
Ring Control Block............................................................................................................................... 107
Host-Based Send Ring........................................................................................................................ 108
Checksum Offload.................................................................................................................................... 109
Large Segment Offload............................................................................................................................ 110
QuickStart............................................................................................................................................ 110
LSO-Related Hardware Control Bits ................................................................................................... 111
Broadcom
January 29, 2016 • 5718-PG108-R Page 14
®
Page 15
Table of ContentsBCM5718 Programmer’s Guide
Send Buffer Descriptor ........................................................................................................................ 112
Host Address................................................................................................................................ 112
Length[15:0] ................................................................................................................................. 112
VLAN Tag[15:0]............................................................................................................................ 112
HdrLen[7:0] .................................................................................................................................. 112
MSS[13:0] .................................................................................................................................... 113
Flags ............................................................................................................................................ 113
LSO Limitations ................................................................................................................................... 114
Additional LSO Notes .......................................................................................................................... 114
Example TCP-segmentation-related (LSO) register values ......................................................... 115
Jumbo Frames.......................................................................................................................................... 116
Affected Data Structures ..................................................................................................................... 117
Extended RX Buffer Descriptor (BD)............................................................................................ 117
Receive Jumbo Producer Ring .................................................................................................... 120
Ring Control Blocks...................................................................................................................... 121
Receive Return Ring(s)................................................................................................................ 122
Send Buffer Descriptor ................................................................................................................. 122
Status Block ................................................................................................................................. 124
Misc BD Memory.......................................................................................................................... 125
Device Driver Interface........................................................................................................................ 125
Send Interface.............................................................................................................................. 125
Receive Interface ......................................................................................................................... 126
Large Segment Offload (LSO/TSO) ............................................................................................. 128
Summary of Register Settings to Support Jumbo Frames.................................................................. 129
Scatter/Gather........................................................................................................................................... 130
VLAN Tag Insertion.................................................................................................................................. 131
TX Data Flow Diagram.............................................................................................................................. 131
Reset.......................................................................................................................................................... 134
MAC Address Setup/Configuration........................................................................................................ 135
Packet Filtering......................................................................................................................................... 135
Multicast Hash Table Setup/Configuration .......................................................................................... 135
Ethernet CRC Calculation ................................................................................................................... 136
Generating CRC.................................................................................................................................. 136
Checking CRC..................................................................................................................................... 136
Initializing the MAC Hash Registers .................................................................................................... 136
Promiscuous Mode Setup/Configuration............................................................................................. 138
Broadcast Setup/Configuration ........................................................................................................... 138
Section 7: Device Control............................................................................................... 139
Initialization Procedure............................................................................................................................ 139
Broadcom
January 29, 2016 • 5718-PG108-R Page 15
®
Page 16
Table of ContentsBCM5718 Programmer’s Guide
Device Reset Procedure .......................................................................................................................... 146
Device Closing Procedure....................................................................................................................... 147
™
Energy Efficient Ethernet
...................................................................................................................... 148
Section 8: IEEE1588........................................................................................................152
IEEE1588 Time Sync Introduction .......................................................................................................... 152
NetXtreme Time Sync Assist................................................................................................................... 152
Coexistence......................................................................................................................................... 152
PTP Link Delay Measurement............................................................................................................. 153
PTP Time Synchronization Messaging ............................................................................................... 153
Hardware Description .............................................................................................................................. 154
EAV Reference Clock/Counter ............................................................................................................ 155
EAV Reference Corrector ........................................................................................................................ 156
Time Watchdogs ................................................................................................................................. 156
Divided EAV Reference Clock Output................................................................................................. 156
Transmit Time Stamping Service............................................................................................................ 157
Receive Time Stamp and Sequence ID Registers................................................................................. 158
Time Sync Registers................................................................................................................................160
GRC MODE REG [0x6800] ................................................................................................................. 160
EAV REF COUNT CAPTURE LSB REG [Offset 0x6900] ................................................................... 160
EAV REF COUNT CAPTURE MSB REG [Offset 0x6904] .................................................................. 160
EAV REF CLOCK CONTROL REG [Offset 0x6908]........................................................................... 161
EAV REF-COUNT SNAP-SHOT LSB[0] REG [Offset 0X6910] .......................................................... 162
EAV REF-COUNT SNAP-SHOT MSB[0] REG [Offset 0X6914] ......................................................... 162
EAV REF CORRECTOR REG [Offset 0x6928]................................................................................... 162
TX TIME STAMP LSB REG [Offset 0x05C0] ...................................................................................... 162
TX TIME STAMP MSB REG [Offset 0X05C4]..................................................................................... 163
RX TIME STAMP LSB REG [Offset 0X06B0] ..................................................................................... 163
RX TIME STAMP MSB REG [Offset 0x06B4] ..................................................................................... 163
RX PTP SEQUENCE ID REG [Offset 0X06B8] .................................................................................. 163
RX LOCK TIMER LSB REG [Offset 0x06C0]...................................................................................... 164
RX LOCK TIMER MSB REG [Offset 0x06C4]..................................................................................... 164
RX PTP CONTROL REG [Offset 0X06C8] ......................................................................................... 164
TX TIME WATCHDOG LSB[0] REG [Offset 0x6918].......................................................................... 165
TX TIME WATCHDOG MSB[0] REG [Offset 0x691C] ........................................................................ 165
TX TIME WATCHDOG LSB[1] REG [Offset 0x6920].......................................................................... 166
TX TIME WATCHDOG MSB[1] REG [Offset 0x6924]......................................................................... 166
EAV REF-COUNT SNAP-SHOT LSB[1] REG [Offset 0X6930] .......................................................... 166
EAV REF-COUNT SNAP-SHOT MSB[1] REG [Offset 0X6934] ......................................................... 167
Broadcom
January 29, 2016 • 5718-PG108-R Page 16
®
Page 17
Table of ContentsBCM5718 Programmer’s Guide
Section 9: PCI ..................................................................................................................168
Configuration Space ................................................................................................................................168
Description .......................................................................................................................................... 168
Functional Overview............................................................................................................................ 171
PCI Configuration Space Registers ............................................................................................. 171
PCI Required Header Region ......................................................................................................171
Indirect Mode ............................................................................................................................... 172
Indirect Register Access .............................................................................................................. 173
Indirect Memory Access............................................................................................................... 175
UNDI Mailbox Access .................................................................................................................. 177
Standard Mode............................................................................................................................. 179
Memory Mapped I/O Registers ........................................................................................................... 184
PCI Command Register ............................................................................................................... 184
PCI State Register ....................................................................................................................... 184
PCI Base Address Register ......................................................................................................... 184
Bus Interface............................................................................................................................................. 186
Description .......................................................................................................................................... 186
Operational Characteristics................................................................................................................. 187
Read/Write DMA Engines ............................................................................................................ 187
Expansion ROM........................................................................................................................................ 187
Description .......................................................................................................................................... 187
Operational Characteristics................................................................................................................. 187
BIOS.................................................................................................................................................... 187
Preboot Execution Environment...................................................................................................188
Power Management.................................................................................................................................. 188
Description .......................................................................................................................................... 188
Operational Characteristics................................................................................................................. 189
Device State D0 (Uninitialized) .................................................................................................... 189
Device State D0 (Active) .............................................................................................................. 190
Device State D3 (Hot) .................................................................................................................. 190
Device State D3 (Cold) ................................................................................................................ 190
Wake on LAN ...................................................................................................................................... 190
GPIO ................................................................................................................................................... 191
Power Supply in D3 State ................................................................................................................... 191
Clock Control....................................................................................................................................... 191
Device ACPI Transitions ..................................................................................................................... 192
Disable Device Through BIOS ............................................................................................................ 192
Endian Control (Byte and Word Swapping)........................................................................................... 193
Background ......................................................................................................................................... 193
Broadcom
January 29, 2016 • 5718-PG108-R Page 17
®
Page 18
Table of ContentsBCM5718 Programmer’s Guide
Architecture ......................................................................................................................................... 194
Enable Endian Word Swap and Enable Endian Byte Swap Bits......................................................... 194
Word Swap Data and Byte Swap Data Bits ........................................................................................ 197
Word Swap Data = 0, and Byte Swap Data = 0 ........................................................................... 197
Word Swap Data = 0, and Byte Swap Data = 1 ........................................................................... 198
Word Swap Data = 1, and Byte Swap Data = 0 ........................................................................... 198
Word Swap Data = 1, and Byte Swap Data = 1 ........................................................................... 199
Word Swap Non-Frame Data and Byte Swap Non-Frame Data Bits .................................................. 200
Word Swap Non-Frame Data = 0 and Byte Swap Non-Frame Data = 0...................................... 201
Word Swap Non-Frame Data = 1 and Byte Swap Non-Frame Data = 0...................................... 201
Word Swap Non-Frame Data = 0 and Byte Swap Non-Frame Data = 1...................................... 201
Word Swap Non-Frame Data = 1 and Byte Swap Non-Frame Data = 1...................................... 202
Section 10: Ethernet Link Configuration.......................................................................203
Overview.................................................................................................................................................... 203
GMII/MII...................................................................................................................................................... 203
Configuring the Ethernet Controller for GMII and MII Modes .............................................................. 203
Link Status Change Indications........................................................................................................... 204
Configuring the GMII/MII PHY............................................................................................................. 204
Reading a PHY Register.............................................................................................................. 204
Writing a PHY Register ................................................................................................................ 204
PHY Loopback Configuration ....................................................................................................... 205
External PHY Loopback....................................................................................................... 205
Internal PHY Loopback ........................................................................................................ 205
PHY Configuration Auto-Negotiation (10/100/1000 Speed with Half and Full Duplex Support) .. 206
MDI Register Access................................................................................................................................210
Operational Characteristics................................................................................................................. 210
Access Method.................................................................................................................................... 211
Auto-Access Method .................................................................................................................... 211
Wake on LAN Mode/Low-Power.............................................................................................................. 212
Description .......................................................................................................................................... 212
Functional Overview............................................................................................................................ 213
Operational Characteristics................................................................................................................. 214
Internal Memory ........................................................................................................................... 214
WOL Pattern Configuration Register............................................................................................ 214
WOL Streams............................................................................................................................... 215
Pattern Data Structure ................................................................................................................. 217
Firmware Mailbox......................................................................................................................... 218
PHY Auto-Negotiation .................................................................................................................. 219
Power Management ..................................................................................................................... 219
Broadcom
January 29, 2016 • 5718-PG108-R Page 18
®
Page 19
Table of ContentsBCM5718 Programmer’s Guide
Integrated MACs .......................................................................................................................... 220
WOL Data Flow Diagram .................................................................................................................... 221
Flow Control.............................................................................................................................................. 223
Description .......................................................................................................................................... 223
Operational Characteristics................................................................................................................. 223
Transmit MAC .............................................................................................................................. 223
Receive MAC ............................................................................................................................... 224
Statistics Block ............................................................................................................................. 225
PHY Auto-Negotiation .................................................................................................................. 226
Integrated MACs .......................................................................................................................... 226
Flow Control Initialization Pseudocode ............................................................................................... 227
Section 11: Interrupt Processing ...................................................................................229
NetXtreme Legacy Interrupt Model......................................................................................................... 229
ISR Flow.............................................................................................................................................. 230
Legacy Status TAGGING Mode................................................................................................... 231
Basic Driver Interrupt Processing Flow................................................................................................. 232
Flowchart for Servicing an Interrupt .................................................................................................... 232
Interrupt Procedure ............................................................................................................................. 233
Host Coalescing ....................................................................................................................................... 234
Description .......................................................................................................................................... 234
Operational Characteristics................................................................................................................. 234
Registers ............................................................................................................................................. 235
MSI............................................................................................................................................................. 236
Traditional Interrupt Scheme............................................................................................................... 236
Message Signaled Interrupt ................................................................................................................ 237
PCI Configuration Registers ................................................................................................................ 238
MSI Address................................................................................................................................. 238
MSI Data ...................................................................................................................................... 238
Host Coalescing Engine ...................................................................................................................... 239
Firmware ............................................................................................................................................. 239
MSI-X.......................................................................................................................................................... 240
MSI-X Plumbing........................................................................................................................................ 244
Replication of Status Blocks and INT Mailboxes................................................................................. 244
Single-Vector RSS Mode Status Block Format............................................................................ 246
Single-Vector IOV Mode Status Block Format ............................................................................. 247
Multivector RSS Mode Status Block Format ................................................................................ 248
Multivector IOV Mode Status Block Format ................................................................................. 249
MSI-X Capability Structure .................................................................................................................. 250
MSI-X Data Structures ........................................................................................................................ 251
Broadcom
January 29, 2016 • 5718-PG108-R Page 19
®
Page 20
Table of ContentsBCM5718 Programmer’s Guide
MSI-X Cognizant Host Coalescing ...................................................................................................... 253
Legacy Host Coalescing Parameters........................................................................................... 253
Receive Coalescing Ticks Register (Offset: 0x3c08)........................................................... 253
Send Coalescing Ticks Register (Offset: 0x3c0c)................................................................ 254
Receive Max Coalesced Bd Count Register (Offset: 0x3c10) ............................................. 254
Send Max Coalesced BD Count Register (Offset: 0x3c14) ................................................. 254
Receive Max Coalesced BD Count During Interrupt Register (Offset 0x3c18).................... 255
Send Max Coalesced BD Count During Interrupt Register (Offset 0x3c1c)......................... 255
BCM5718 Family Host Coalescing Parameter Sets .................................................................... 255
MSI-X One Shot Mode ................................................................................................................. 258
Coalesce Now or Forced Update ................................................................................................. 258
Misc Coalescing Controls............................................................................................................. 258
Broadcom Tagged Status Mode (0x68[9])........................................................................... 258
Clear Interrupt, Mask Interrupt, Mask Mode (0x68[0], 0x68[1], 0x68[8]).............................. 259
Clear Ticks On Rx Bd Events Mode (0x3c00[9]).................................................................. 259
No Interrupt On Force Update (0x3c00[11])......................................................................... 259
No Interrupt On DMAD Force (0x3c00[12]).......................................................................... 259
Do Not Interrupt On Receives (0x6800[14])......................................................................... 259
End of Receive Stream Interrupt.................................................................................................. 260
Host Coalescing Mode Register (Offset 0x3c00)................................................................. 260
End Stream Debounce Register (Offset 0x3cd4)................................................................. 260
Other Configuration Controls.................................................................................................................. 262
Broadcom Mask Mode ........................................................................................................................ 262
Broadcom Tagged Status Mode.......................................................................................................... 262
Clear Ticks on BD Events Mode ......................................................................................................... 262
No Interrupt on Force Update.............................................................................................................. 262
No Interrupt on DMAD Force............................................................................................................... 262
Section 12: IO Virtualization (IOV) ................................................................................. 263
Data Structure and Register Changes for IOV....................................................................................... 264
Mail Box Register Changes................................................................................................................. 264
Receive Mail Box Register Changes................................................................................................... 264
Send Mail Box Register Changes ....................................................................................................... 264
Ring Control Block Changes ............................................................................................................... 264
VRQ Statistics ..................................................................................................................................... 264
MSI-X Vectors Changes...................................................................................................................... 265
Register Changes................................................................................................................................ 265
IOV – Receive Side................................................................................................................................... 266
IOV – Transmit Side.................................................................................................................................. 267
Broadcom
January 29, 2016 • 5718-PG108-R Page 20
®
Page 21
Table of ContentsBCM5718 Programmer’s Guide
Section 13: Ethernet Controller Register Definitions................................................... 269
BCM5718 Family Register MAP............................................................................................................... 269
PCI Configuration Registers.................................................................................................................... 271
Device ID and Vendor ID Register (offset: 0x00) ................................................................................ 271
Status and Command Register (offset: 0x04) ..................................................................................... 271
PCI Classcode and Revision ID Register (offset: 0x08) ...................................................................... 273
BIST, Header Type, Latency Timer, Cache Line Size Register (offset: 0x0C).................................... 273
Base Address Register 1 (offset: 0x10)............................................................................................... 274
Base Address Register 2 (offset: 0x14)............................................................................................... 274
Base Address Register 3 (offset: 0x18)............................................................................................... 274
Base Address Register 4 (offset: 0x1c)............................................................................................... 275
Base Address Register 5 (offset: 0x20)............................................................................................... 275
Base Address Register 6 (offset: 0x24)............................................................................................... 275
Cardbus CIS Pointer Register (offset: 0x28) ....................................................................................... 276
Subsystem ID/Vendor ID Register (offset: 0x2C) ................................................................................ 277
Expansion ROM Base Address Register (offset: 0x30) ...................................................................... 277
Capabilities Pointer Register (offset: 0x34) ......................................................................................... 277
Interrupt Register (offset: 0x3C) .......................................................................................................... 278
INT Mailbox Register (offset: 0x40–0x44) ........................................................................................... 278
Power Management Capability Register (offset: 0x48) ....................................................................... 279
Power Management Control/Status Register (offset: 0x4C) ............................................................... 279
MSI Capability Header (offset: 0x58) .................................................................................................. 281
MSI Lower Address Register (offset: 0x5C) ........................................................................................ 282
MSI Upper Address Register (offset: 0x60)......................................................................................... 282
MSI Data Register (offset: 0x64) ......................................................................................................... 282
Miscellaneous Host Control Register (offset: 0x68) ............................................................................ 282
DMA Read/Write Control Register (Offset: 0x6C) ............................................................................... 283
PCI State Register (offset: 0x70)......................................................................................................... 285
Reset Counters Initial Values Register (offset: 0x74).......................................................................... 286
Register Base Register (offset: 0x78) ................................................................................................. 286
Memory Base Register (offset: 0x7C) ................................................................................................. 286
Register Data Register (offset: 0x80) .................................................................................................. 286
Memory Data Register (offset: 0x84) .................................................................................................. 287
UNDI Receive Return Ring Consumer Index Register (offset: 0x88–0x8C) ....................................... 287
UNDI Send BD Producer Index Mailbox Register (offset: 0x90–0x94) ............................................... 287
UNDI Receive BD Standard Producer Ring Producer Index Mailbox Register (offset: 0x98–0x9C) .. 287
MSI-X Capabilities Registers............................................................................................................... 288
MSI-X Capability Header Register (offset: 0xA0)......................................................................... 288
MSIX_TBL_OFF_BIR – 0xa4 ....................................................................................................... 288
Broadcom
January 29, 2016 • 5718-PG108-R Page 21
®
Page 22
Table of ContentsBCM5718 Programmer’s Guide
MSIX_PBA_BIR_OFF – 0xa8 ...................................................................................................... 288
PCIe Capabilities Registers................................................................................................................. 289
PCIE_CAPABILITY – 0xac ..........................................................................................................289
DEVICE_CAPABILITY – 0xb0 ..................................................................................................... 290
DEVICE_STATUS_CONTROL – 0xb4 ........................................................................................ 291
LINK_CAPABILITY – 0xb8........................................................................................................... 292
LINK_STATUS_CONTROL – 0xbc.............................................................................................. 294
SLOT_CAPABILITY – 0xc0 ......................................................................................................... 296
SLOT_CONTROL_STATUS – 0xc4 ............................................................................................ 296
ROOT_CAP_CONTROL – 0xc8 .................................................................................................. 296
ROOT_STATUS – 0xcc ............................................................................................................... 296
DEVICE_CAPABILITY_2 – 0xd0 ................................................................................................. 297
DEVICE_STATUS_CONTROL2 – 0xd4 ...................................................................................... 297
LINK_CAPABILITY_2 – 0xd8....................................................................................................... 298
LINK_STATUS_CONTROL_2 – 0xdc.......................................................................................... 298
SLOT_CAPABILITY_2 – 0xe0 ..................................................................................................... 300
SLOT_STATUS_CONTROL_2 – 0xe4 ........................................................................................ 300
Product ASIC ID (offset: 0xF4)............................................................................................................ 300
Advanced Error Reporting Enhanced Capability Header (offset: 0x100) ............................................ 301
Uncorrectable Error Status Register (offset: 0x104) ........................................................................... 301
Uncorrectable Error Mask Register (offset: 0x108) ............................................................................. 302
Uncorrectable Error Severity Register (offset: 0x10C) ........................................................................ 303
Correctable Error Status Register (offset: 0x110) ...............................................................................304
Correctable Error Mask Register (offset: 0x114).................................................................................304
Advanced Error Capabilities and Control Register (offset: 0x118)...................................................... 305
Header Log Register (offset: 0x11C)................................................................................................... 305
Header Log Register (offset: 0x120) ................................................................................................... 305
Header Log Register (offset: 0x124) ................................................................................................... 306
Header Log Register (offset: 0x128) ................................................................................................... 306
Interrupt mail box (High Priority Mailbox) Register
(offset: 0x200 – 0x21c).................................................................................................................... 306
General mail box (High Priority Mailbox)
Register (offset: 0x220–0x25c) ....................................................................................................... 306
Reload Statistics mail box (High Priority Mailbox)
Register (offset: 0x260–0x264) ....................................................................................................... 306
High Priority Mailbox Registers.............................................................................................................. 307
Receive BD Standard Producer Ring Index
Register (offset: 0x268-0x26F)........................................................................................................ 307
Receive BD Jumbo Producer Ring Index Register (offset: 0x270) .............................................. 307
Broadcom
January 29, 2016 • 5718-PG108-R Page 22
®
Page 23
Table of ContentsBCM5718 Programmer’s Guide
Receive BD Return Ring 0 Consumer Index
Register (offset: 0x280–0x287) ....................................................................................................... 307
Receive BD Return Ring 1 Consumer Index
Register (offset: 0x288–0x28F) ....................................................................................................... 308
Receive BD Return Ring 2 Consumer Index
Register (offset: 0x290–0x297) ....................................................................................................... 308
Receive BD Return Ring 3 Consumer Index
Register (offset: 0x298–0x29F) ....................................................................................................... 308
Send BD Ring Host Producer Index Register (offset: 0x300–0x307).................................................. 308
RX Mail Box Registers for VRQ............................................................................................................... 308
Ethernet MAC (EMAC) Registers ............................................................................................................ 309
EMAC Mode Register (offset: 0x400).................................................................................................. 310
EMAC Status Register (offset: 0x404) ................................................................................................ 311
EMAC Event Enable Register (offset: 0x408) ..................................................................................... 312
LED Control Register (offset: 0x40C).................................................................................................. 313
EMAC MAC Addresses 0 High Register (offset: 0x410) ..................................................................... 314
EMAC MAC Addresses 0 Low Register (offset: 0x414) ...................................................................... 314
EMAC MAC Addresses 1 High Register (offset: 0x418) ..................................................................... 315
EMAC MAC Addresses 1 Low Register (offset: 0x41C) ..................................................................... 315
EMAC MAC Addresses 2 High Register (offset: 0x420) ..................................................................... 315
EMAC MAC Addresses 2 Low Register (offset: 0x424) ...................................................................... 315
EMAC MAC Addresses 3 High Register (offset: 0x428) ..................................................................... 315
EMAC MAC Addresses 3 Low Register (offset: 0x42C) ..................................................................... 315
WOL Pattern Pointer Register (offset: 0x430)..................................................................................... 316
WOL Pattern Configuration Register (offset: 0x434)........................................................................... 316
Ethernet Transmit Random Backoff Register (offset: 0x438) .............................................................. 316
Receive MTU Size Register (offset: 0x43C)........................................................................................ 316
Gigabit PCS Test Register (offset: 0x440) .......................................................................................... 317
Transmit 1000BASE-X Auto-Negotiation Register (offset: 0x444) ...................................................... 317
Receive 1000BASE-X Auto-Negotiation Register (offset: 0x448) ....................................................... 317
MII Communication Register (offset: 0x44C) ......................................................................................317
MII Status Register (offset: 0x450)...................................................................................................... 318
MII Mode Register (offset: 0x454) ....................................................................................................... 318
Autopolling Status Register (offset: 0x458) ......................................................................................... 319
Transmit MAC Mode Register (offset: 0x45C) .................................................................................... 319
Transmit MAC Status Register (offset: 0x460)....................................................................................321
Transmit MAC Lengths Register (offset: 0x464) ................................................................................. 321
Receive MAC Mode Register (offset: 0x468) ...................................................................................... 322
Receive MAC Status Register (offset: 0x46C) .................................................................................... 324
MAC Hash Register 0 (offset: 0x470).................................................................................................. 324
Broadcom
January 29, 2016 • 5718-PG108-R Page 23
®
Page 24
Table of ContentsBCM5718 Programmer’s Guide
MAC Hash Register 1 (offset: 0x474).................................................................................................. 324
MAC Hash Register 2 (offset: 0x478).................................................................................................. 324
MAC Hash Register 3 (offset: 0x47C) ................................................................................................. 324
Receive Rules Control Registers (offset: 0x480 + 8*N) ...................................................................... 325
Receive Rules Value/Mask Registers (offset: 0x484 + 8*N) ............................................................... 326
Receive Rules Configuration Register (offset: 0x500) ........................................................................ 326
Low Watermark Maximum Receive Frame Register (offset: 0x504)................................................... 327
APE_PERFECT_MATCH[1–4]_HIGH_REG (Offsets 0x540, 0x548, 0x550, 0x558).......................... 327
APE_PERFECT_MATCH[1–4]_LOW_REG (Offsets 0x544, 0x54C, 0x554, 0x55C) ......................... 327
SGMII Control Register (offset: 0x5B0)............................................................................................... 327
SGMII Status Register (offset: 0x5B4) ................................................................................................ 328
HTX2B Perfect Match[1 – 4] HI Reg (offset: 0x4880, 0x4888, 0x4890, 0x4898) ................................. 328
HTX2B Perfect Match[1 –4] LO Reg (offset: 0x4884, 0x488C, 0x4894, 0x489C) ............................... 328
HTX2B Protocol Filter Reg (offset: 0x6D0) ......................................................................................... 329
HTX2B Global Filter Reg (address: 0x6D4) ........................................................................................331
RSS Registers........................................................................................................................................... 331
Indirection Table Register 0 (offset: 0x630) ........................................................................................ 331
Indirection Table Register 2 (offset: 0x634) ........................................................................................ 331
Indirection Table Register 3 (offset: 0x638) ........................................................................................ 332
Indirection Table Register 4 (offset: 0x63C)........................................................................................ 332
Indirection Table Register 5 (offset: 0x640) ........................................................................................ 333
Indirection Table Register 6 (offset: 0x644) ........................................................................................ 333
Indirection Table Register 8 (offset: 0x648) ........................................................................................ 333
Indirection Table Register 8 (offset: 0x64C)........................................................................................ 334
Indirection Table Register 9 (offset: 0x650) ........................................................................................ 334
Indirection Table Register 10 (offset: 0x654) ...................................................................................... 334
Indirection Table Register 11 (offset: 0x658) ...................................................................................... 335
Indirection Table Register 12 (offset: 0x65C)...................................................................................... 335
Indirection Table Register 12 (offset: 0x660) ...................................................................................... 335
Indirection Table Register 13 (offset: 0x664) ...................................................................................... 336
Indirection Table Register 14 (offset: 0x668) ...................................................................................... 336
Indirection Table Register 15 (offset: 0x66C)...................................................................................... 336
Hash Key Register 0 (offset: 0x670) ................................................................................................... 337
Hash Key Registers 1–8 (offset: 0x674–0x693) .................................................................................. 337
Hash Key Register 9 (offset: 0x694) ................................................................................................... 337
Receive MAC Programmable IPv6 Extension Header Register (offset: 0x6A0) ................................. 338
Statistics Registers .................................................................................................................................. 339
Transmit MAC Static Counters............................................................................................................ 339
ifHCOutOctets (offset: 0x800) ...................................................................................................... 339
Broadcom
January 29, 2016 • 5718-PG108-R Page 24
®
Page 25
Table of ContentsBCM5718 Programmer’s Guide
etherStatsCollisions (offset: 0x808) ............................................................................................. 339
outXonSent (offset: 0x80C).......................................................................................................... 339
outXoffSent (offset: 0x810) .......................................................................................................... 339
dot3StatsInternalMacTransmitErrors (offset: 0x818) ................................................................... 339
dot3StatsSingleCollisionFrames (offset: 0x81C).......................................................................... 339
dot3StatsMultipleCollisionFrames (offset: 0x820) ........................................................................ 339
dot3StatsDeferredTransmissions (offset: 0x824)......................................................................... 339
dot3StatsExcessiveTransmissions (offset: 0x82C) ...................................................................... 340
dot3StatsLateCollisions (offset: 0x830)........................................................................................ 340
iHCOutUcastPkts (offset: 0x86C).................................................................................................340
iHCOutMulticastPkts (offset: 0x870) ............................................................................................ 340
iHCOutBroadcastPkts (offset: 0x874) .......................................................................................... 340
ifCRSERRORS (offset: 0x878) .................................................................................................... 340
iOUTDISCARDS (offset: 0x87C).................................................................................................. 340
H2B Statistics Registers...................................................................................................................... 340
HTX2B Statistics .......................................................................................................................... 341
B2HRX Statistics.......................................................................................................................... 341
Receive MAC Static Counters............................................................................................................. 341
ifHCInOctets (offset: 0x880) ......................................................................................................... 341
ifHCINOctets_bad (offset: 0x884) ................................................................................................341
etherStatsFragments (offset: 0x888)............................................................................................ 341
ifHCInUcastPkts (offset: 0x88C) .................................................................................................. 342
ifHCInMulticastPkts (offset: 0x890) .............................................................................................. 342
ifHCInBroadcastPkts (offset: 0x894) ............................................................................................ 342
dot3StatsFCSErrors (offset: 0x898) ............................................................................................. 342
dot3StatsAlignmentErrors (offset: 0x89C).................................................................................... 342
xonPauseFrameReceived (offset: 0x8A0) ................................................................................... 342
xoffPauseFrameReceived (offset: 0x8A4) ................................................................................... 342
macControlFramesRecevied (offset: 0x8A8) ............................................................................... 342
xoffStateEntered (offset: 0x8AC) ................................................................................................. 342
dot3StatsFramesTooLongs (offset: 0x8B0) ................................................................................. 342
etherStatsJabbers (offset: 0x8B4)................................................................................................ 343
etherStatsUndersizePkts (offset: 0x8B8) ..................................................................................... 343
Ifnomorerxbd:0x224C................................................................................................................... 343
Ifindiscard:0x2250 ........................................................................................................................ 343
Ifinerror:0x2254 ............................................................................................................................ 343
APE_NETWORK_STATS_REGS (Offsets 0x900–0x9BC)................................................................. 344
Send Data Initiator Registers .................................................................................................................. 345
Send Data Initiator Mode Register (offset: 0xC00).............................................................................. 345
Broadcom
January 29, 2016 • 5718-PG108-R Page 25
®
Page 26
Table of ContentsBCM5718 Programmer’s Guide
Send Data Initiator Status Register (offset: 0xC04) ............................................................................ 345
Send Data Initiator Statistics Control Register (offset: 0xC08)............................................................ 345
Send Data Initiator Statistics Mask Register (offset: 0xC0C) .............................................................. 346
Send Data Initiator Statistics Increment Mask Register (offset: 0xC10).............................................. 346
Local Statistics Register (offset: 0xC80–0xCDF) ................................................................................ 346
TCP Segmentation Control Registers.................................................................................................... 347
Lower Host Address Register for TCP Segmentation (offset: 0xCE0) ................................................ 347
Upper Host Address Register for TCP Segmentation (offset: 0xCE4) ................................................ 347
Length/Offset Register for TCP Segmentation (offset: 0xCE8) ........................................................... 347
DMA Flag Register for TCP Segmentation (offset: 0xCEC) ................................................................ 348
VLAN Tag Register for TCP Segmentation (offset: 0xCF0) ................................................................ 349
Pre-DMA Command Exchange Register for TCP Segmentation (offset: 0xCF4) ............................... 349
Send Data Completion Control Registers.............................................................................................. 350
Send Data Completion Mode Register (offset: 0x1000) ...................................................................... 350
Pre-DMA Command Exchange Register for TCP Segmentation (offset: 0x1008) .............................. 350
Send BD Selector Control Registers...................................................................................................... 351
Send BD Ring Selector Mode Register (offset: 0x1400) ..................................................................... 351
Send BD Ring Selector Status Register (offset: 0x1404) .................................................................... 351
Send BD Ring Selector Hardware Diagnostics Register (offset: 0x1408)........................................... 351
Send BD Ring Selector Local NIC Send BD Consumer Index Register (offset: 0x1440–0x147C) ..... 352
Send BD Initiator Control Registers....................................................................................................... 353
Send BD Initiator Mode Register (offset: 0x1800)............................................................................... 353
Send BD Initiator Status Register (offset: 0x1804)..............................................................................353
Send BD Diagnostic Initiator Local NIC BD N Producer Index Registers (offset: 0x1808–0x1844).... 354
Send BD Fetch Threshold Register (offset: 0x1850)........................................................................... 355
Send Mail Box Registers..................................................................................................................... 355
Send BD Completion Control Registers................................................................................................. 356
Send BD Completion Mode Register (offset: 0x1C00)........................................................................ 356
Receive List Placement Registers.......................................................................................................... 357
Receive List Placement Mode Register (offset: 0x2000) .................................................................... 357
Receive List Placement Status Register (offset: 0x2004) ................................................................... 357
Receive Selector Non-Empty Bits Register (offset: 0x200C) .............................................................. 358
Receive List Placement Configuration Register (offset: 0x2010) ........................................................ 358
Receive List Placement Statistics Control Register (offset: 0x2014) .................................................. 359
Receive List Placement Statistics Enable Mask Register (offset: 0x2018) ......................................... 359
Receive List Placement Statistics Increment Mask Register (offset: 0x201C) .................................... 360
Receive Selector List Head & Tail Pointers (offset: 0x2100)............................................................... 360
Receive Selector List 1 Count Registers (Offset: 0x2108).................................................................. 360
Receive Data and Receive BD Initiator Control Registers.................................................................... 362
Broadcom
January 29, 2016 • 5718-PG108-R Page 26
®
Page 27
Table of ContentsBCM5718 Programmer’s Guide
Receive Data and Receive BD Initiator Mode Register (offset: 0x2400)............................................. 362
Receive Data and Receive BD Initiator Status Register (offset: 0x2404) ........................................... 363
VRQ Status Register (offset: 0x240C) ................................................................................................ 364
VRQ Flush Control Register (Offset: 0x2410) .....................................................................................364
VRQ Flush Timer Register (offset: 0x2414) ........................................................................................ 365
RDI B2HRX Hardware Debugging Register (offset: 0x2418) .............................................................. 365
Jumbo Producer Ring Host Address High Register (offset: 0x2440) .................................................. 365
Jumbo Producer Ring Host Address Low Register (offset: 0x2444) ................................................... 366
Jumbo Producer Length/Flags Register (offset: 0x2448).................................................................... 366
Jumbo Producer Ring NIC Address Register (offset: 0x244C) ........................................................... 366
Standard Receive BD Ring RCB Registers......................................................................................... 366
Receive Producer Ring Host Address High Register (offset: 0x2450)......................................... 366
Receive Producer Ring Host Address Low Register (offset: 0x2454).......................................... 367
Receive Producer Length/Flags Register (offset: 0x2458) .......................................................... 367
Receive Producer Ring NIC Address Register (offset: 0x245C) .................................................. 367
Receive Diagnostic Data and Receive BD Ring Initiator Local NIC Jumbo Receive BD Consumer Index
(offset: 0x2470) ............................................................................................................................... 367
Receive BD Ring Initiator Local NIC Standard Receive BD Consumer Index (offset: 0x2474) .......... 368
Receive Data and Receive BD Initiator Hardware Diagnostic Register (offset: 0x24C0).................... 368
B2HRX Byte-count Statistics Count (offset: 0x24D0).......................................................................... 368
B2HRX Unicast Statistics Count (offset: 0x24D4)............................................................................... 368
B2HRX Multicast Statistics Count (offset: 0x24D8)............................................................................. 368
B2HRX Broadcast Statistics Count (offset: 0x24DC) .......................................................................... 368
B2HRX Drop Packet Count (offset: 0x24E0)....................................................................................... 369
B2HRX Drop Packet Byte Count (offset: 0x24E4) .............................................................................. 369
B2HRX APE Byte-count Statistics Count (offset: 0x24E8).................................................................. 369
B2HRX APE Unicast Statistics Count (offset: 0x24EC) ...................................................................... 369
B2HRX APE Multicast Statistics Count (offset: 0x24F0) ..................................................................... 369
B2HRX APE Broadcast Statistics Count (offset: 0x24F4) ................................................................... 369
B2HRX APE Drop Packet Count (offset: 0x24F8)............................................................................... 370
B2HRX APE Drop Packet Byte Count (offset: 0x24FC) ...................................................................... 370
Receive Data Completion Control Registers......................................................................................... 371
Receive Data Completion Mode Register (offset: 0x2800) ................................................................. 371
Receive BD Initiator Control Registers .................................................................................................. 372
Receive BD Initiator Mode Register (offset: 0x2C00).......................................................................... 372
Receive BD Initiator Status Register (offset: 0x2C04) ........................................................................ 372
Receive BD Initiator Local NIC Jumbo Receive BD Producer Index (offset: 0x2C08) ........................ 372
Receive BD Initiator Local NIC Receive BD Producer Index Register (offset: 0x2C0C–0x2C13) ...... 373
Standard Receive BD Producer Ring Replenish Threshold Register (offset: 0x2C18) ....................... 373
Broadcom
January 29, 2016 • 5718-PG108-R Page 27
®
Page 28
Table of ContentsBCM5718 Programmer’s Guide
Jumbo Receive BD Producer Ring Replenish Threshold Register (offset: 0x2C1C) .......................... 373
Standard Replenish LWM Register (offset 0x2D00) ........................................................................... 373
Jumbo Replenish LWM Register (offset 0x2D04) ............................................................................... 374
BD Fetch Limit Register (Offset 0x2D08)..................................................................................... 375
Receive BD Completion Control Registers............................................................................................ 375
Receive BD Completion Mode Register (offset: 0x3000) .................................................................... 375
Receive BD Completion Status Register (offset: 0x3004)................................................................... 375
NIC Jumbo Receive BD Producer Index Register (offset: 0x3008)..................................................... 376
NIC Standard Receive BD Producer Index Register (offset: 0x300C) ................................................ 376
Central Power Management Unit (CPMU) Registers............................................................................. 376
CPMU Control Register (offset: 0x3600)............................................................................................. 376
Link Speed 10 MB/No Link Power Mode Clock Policy Register (offset: 0x3604)................................ 378
Link Speed 100 MB Power Mode Clock Policy Register (offset: 0x3608)........................................... 379
Link Speed 1000 MB Power Mode Clock Policy Register (offset: 0x360C) ........................................ 380
Link Aware Power Mode Clock Policy Register (offset: 0x3610) ........................................................ 381
D0u Clock Policy Register (offset: 0x3614)......................................................................................... 382
Link Idle Power Mode Clock Policy Register (offset: 0x3618)............................................................. 382
APE CLK Policy Register (offset: 0x361C).......................................................................................... 383
APE Sleep State Clock Policy Register (offset: 0x3620)..................................................................... 385
Clock Speed Override Policy Register (offset: 0x3624) ...................................................................... 386
Clock Override Enable Register (offset: 0x3628) ................................................................................ 386
Status Register (offset: 0x362C) ......................................................................................................... 387
Clock Status Register (offset: 0x3630)................................................................................................ 389
Clock Status Register (offset: 0x3630)................................................................................................ 389
GPHY Control/Status Register (offset: 0x3638) .................................................................................. 391
RAM Control Register (offset: 0x363C)............................................................................................... 392
Core Idle Detection De-Bounce Control Register (offset: 0x3648)...................................................... 393
PCIE Idle Detection De-Bounce Control Register (offset: 0x364C) .................................................... 394
Energy Detection De-Bounce Timer (offset: 0x3650).......................................................................... 394
DLL Lock Timer Register (offset: 0x3654)........................................................................................... 396
CHIP ID Register (offset: 0x3658) ....................................................................................................... 396
Mutex Request Register (offset: 0x365C) ........................................................................................... 397
Mutex Grant Register (offset: 0x3660) ................................................................................................ 397
GPHY Strap Register (offset: 0x3664) ................................................................................................ 397
Padring Control Register (offset: 0x3668) ........................................................................................... 398
Flash Clock Policy Register (offset: 0x366C) ...................................................................................... 399
Link Idle Control Register (offset: 0x3670) .......................................................................................... 401
Link Idle Status Register (offset: 0x3674) ........................................................................................... 404
Top Level Miscellaneous Control 1 Register (offset: 0x367C) ............................................................ 405
Broadcom
January 29, 2016 • 5718-PG108-R Page 28
®
Page 29
Table of ContentsBCM5718 Programmer’s Guide
Miscellaneous Control Register (offset: 0x36AC)................................................................................ 406
EEE Mode Register (offset: 0x36B0) .................................................................................................. 407
EEE Debounce Timer 1 Control Register (offset: 0x36B4) ................................................................. 407
EEE Debounce Timer 2 Control Register (offset: 0x36B8) ................................................................. 408
EEE Link Idle Control Register (offset: 0x36BC) .................................................................................408
EEE Link Idle Status Register (offset: 0x36C0)................................................................................... 409
EEE Statistic Counter 1 Register (offset: 0x36C4).............................................................................. 409
EEE Statistic Counter 2 Register (offset: 0x36C8).............................................................................. 409
EEE Statistics Counter 3 Register (offset: 0x36CC) ........................................................................... 409
EEE Control Register (offset: 0x36D0)................................................................................................ 410
Current Measurement Control Register (offset: 0x36D4) .................................................................... 410
Current Measurement Upper 32-bit Read Register (offset: 0x36D8) .................................................. 411
Current Measurement Lower 32-bit Read Register (offset: 0x36DC) ................................................. 411
Global Mutex Request Register (offset: 0x36F0) ................................................................................ 411
Global Mutex Grant Register (offset: 0x36F4)..................................................................................... 412
Temperature Monitor Control Register (offset: 0x36FC) ..................................................................... 412
Host Coalescing Control Registers ........................................................................................................ 413
Host Coalescing Mode Register (offset: 0x3C00) ............................................................................... 413
Host Coalescing Status Register (offset: 0x3C04) .............................................................................. 414
Receive Coalescing Ticks Register (offset: 0x3C08) .......................................................................... 414
Send Coalescing Ticks Register (offset: 0x3C0C) .............................................................................. 415
Receive Max Coalesced BD Count Register (offset: 0x3C10) ............................................................ 416
Send Max Coalesced BD Count Register (offset: 0x3C14)................................................................. 418
Receive Max Coalesced BD Count During Interrupt Register (offset: 0x3C18) .................................. 419
Send Max Coalesced BD Count During Interrupt Register (offset: 0x3C1C) ...................................... 419
HC Parameter Set Reset Register (Offset: 0x3C28).................................................................... 421
Status Block Host Address Register (offset: 0x3C38).................................................................. 421
Status Block Base Address Register (offset: 0x3C44) ........................................................................ 422
Flow Attention Register (offset: 0x3C48)............................................................................................. 422
NIC Jumbo Receive BD Consumer Index Register (offset: 0x3C50–0x3C58) ............................ 423
NIC Diag Receive Return Ring BD 0 Index Register (offset: 0x3C80)......................................... 423
NIC Jumbo Receive BD Consumer Index Register (offset: 0x3C50) .................................................. 423
NIC Standard Receive BD Consumer Index Register (offset: 0x3C54) .............................................. 424
NIC Mini Receive BD Consumer Index (offset: 0x3c58) ..................................................................... 424
NIC Diagnostic Return Ring 0 Producer Index Register (offset: 0x3C80)........................................... 424
NIC Diagnostic Return Ring 1 Producer Index Register (offset: 0x3C84)........................................... 424
NIC Diagnostic Return Ring 2 Producer Index Register (offset: 0x3C88)........................................... 425
NIC Diagnostic Return Ring 3 Producer Index Register (offset: 0x3C8C) .......................................... 425
NIC Diagnostic Send BD Consumer Index Register (offset: 0x3CC0) ................................................ 425
Broadcom
January 29, 2016 • 5718-PG108-R Page 29
®
Page 30
Table of ContentsBCM5718 Programmer’s Guide
Memory Arbiter Control Registers.......................................................................................................... 426
Memory Arbiter Mode Register (offset: 0x4000) ................................................................................. 426
Memory Arbiter Status Register (offset: 0x4004) ................................................................................ 427
Memory Arbiter Trap Address Low Register (offset: 0x4008) ............................................................. 428
Memory Arbiter Trap Address High Register (offset: 0x400C)............................................................ 428
Buffer Manager Registers........................................................................................................................ 428
Buffer Manager Mode Register (offset: 0x4400) ................................................................................. 428
Buffer Manager Status Register (offset: 0x4404)................................................................................ 429
MBUF Pool Base Address Register (offset: 0x4408) .......................................................................... 429
MBUF Pool Length Register (offset: 0x440C) ..................................................................................... 430
Read DMA MBUF Low Watermark Register (offset: 0x4410) ............................................................. 430
MAC RX MBUF Low Watermark Register (offset: 0x4414)................................................................. 430
Read DMA MBUF High Watermark Register (offset: 0x4418) ............................................................ 430
RX RISC MBUF Cluster Allocation Request Register (offset: 0x441C) .............................................. 430
RX RISC MBUF Allocation Response Register (offset: 0x4420) ........................................................ 431
BM Hardware Diagnostic 1 Register (offset: 0x444C)......................................................................... 431
BM Hardware Diagnostic 2 Register (offset: 0x4450) ......................................................................... 431
BM Hardware Diagnostic 3 Register (offset: 0x4454) ......................................................................... 431
Receive Flow Threshold Register (offset: 0x4458) ............................................................................. 432
RDMA Registers ....................................................................................................................................... 433
LSO Read DMA Mode Register (offset: 0x4800) ................................................................................ 433
LSO Read DMA Status Register (offset: 0x4804) ............................................................................... 435
LSO Read DMA Programmable IPv6 Extension Header Register (offset: 0x4808)............................ 435
LSO Read DMA Reserved Control Register (offset: 0x4900) ............................................................. 436
LSO Read DMA Flow Reserved Control Register (offset: 0x4904)..................................................... 436
LSO/Non-LSO/BD Read DMA Corruption Enable Control Register (offset: 0x4910).......................... 437
BD Read DMA Mode Register (offset: 0x4A00) .................................................................................. 440
BD READ DMA Status Register (offset: 0x4A04) ............................................................................... 441
BD READ DMA Reserved Control Register (offset: 0x4A70).............................................................. 442
BD READ DMA Flow Reserved Control Register (offset: 0x4A74) ..................................................... 442
BD READ DMA Corruption Enable Control Register (offset: 0x4A78) ................................................ 443
Non_LSO Read DMA Mode Register (offset: 0x4B00) ....................................................................... 443
Non-LSO Read DMA Status Register (offset: 0x4B04)....................................................................... 445
Non-LSO Read DMA Programmable IPv6 Extension Header Register (offset: 0x4B08).................... 446
Host Address for the DMA Read Channel 0 (Offset: 0x4B28) ............................................................ 446
Host Address for the DMA Read Channel 1 (offset: 0x4B30) ............................................................. 446
Host Address for the DMA Read Channel 2 (offset: 0x4B38) ............................................................. 446
Host Address for the DMA Read Channel 3 (offset: 0x4B40) ............................................................. 447
Non-LSO Read DMA Reserved Control Register (offset: 0x4B74) ..................................................... 447
Broadcom
January 29, 2016 • 5718-PG108-R Page 30
®
Page 31
Table of ContentsBCM5718 Programmer’s Guide
Non-LSO Read DMA Flow Reserved Control Register (offset: 0x4B78) ............................................ 447
Non-LSO Read DMA Corruption Enable Control Register
(offset: 0x4B7C) .............................................................................................................................. 448
Write DMA Registers................................................................................................................................449
Write DMA Mode Register (offset: 0x4C00) ........................................................................................449
Write DMA Status Register (offset: 0x4C04)....................................................................................... 450
RX-CPU Registers .................................................................................................................................... 451
RX RISC Mode Register (offset: 0x5000) ........................................................................................... 451
RX RISC Status Register (offset: 0x5004) .......................................................................................... 452
RX RISC Program Counter (offset: 0x501C)....................................................................................... 453
RX RISC Hardware Breakpoint Register (offset: 0x5034)................................................................... 453
VRQ Statistics........................................................................................................................................... 454
VRQ Filter Set Registers.......................................................................................................................... 455
VRQ Mapper Registers............................................................................................................................. 456
VRQ Enable Register (Offset 0x560) ........................................................................................... 458
RX Mail Box Registers for VRQ ...................................................................................................458
Perfect Match Destination Address Registers....................................................................................... 459
VRQ_PERFECT_MATCH[4–23]_HIGH_REG (Offsets: 0x5690, 0x5698, 0x56A0 … 0x5728) .......... 459
VRQ_PERFECT_MATCH[4 – 23]_LOW_REG (Offsets: 0x5694, 0x569C, 0x56A4 … 0x572C) ........ 460
Low Priority Mailboxes............................................................................................................................ 460
Interrupt Mailbox 0 Register (offset: 0x5800) ...................................................................................... 460
Other Interrupt Mailbox Register (offset: 0x5808–0x5818) ................................................................. 460
General Mailbox Registers 1-8 (offset: 0x5820–0x5824) .................................................................... 461
Receive BD Standard Producer Ring Index Register (offset: 0x5868)................................................ 461
Receive BD Jumbo Producer Ring Index Register (offset: 0x5870-5877) .......................................... 461
Receive BD Return Ring 0 Consumer Index Register
(offset: 0x5880-0x5887) .................................................................................................................. 461
Receive BD Return Ring 1 Consumer Index Register
(offset: 0x5888-0x588F) .................................................................................................................. 461
Receive BD Return Ring 2 Consumer Index Register
(offset: 0x5890-0x5897) .................................................................................................................. 462
Receive BD Return Ring 3 Consumer Index Register
(offset: 0x5898-0x589F) .................................................................................................................. 462
Send BD Ring Host Producer Index Register (offset: 0x5900) ........................................................... 462
Send BD Ring NIC Producer Index Register (offset: 0x5980)............................................................. 463
Flow Through Queues.............................................................................................................................. 463
FTQ Reset Register (offset: 0x5C00) .................................................................................................. 464
MAC TX FIFO Enqueue Register (offset: 0x5CB8)............................................................................. 465
RXMBUF Cluster Free Enqueue Register (offset: 0x5CC8) ............................................................... 465
RDIQ FTQ Write/Peak Register (offset: 0x5CFC)............................................................................... 465
Broadcom
January 29, 2016 • 5718-PG108-R Page 31
®
Page 32
Table of ContentsBCM5718 Programmer’s Guide
Message Signaled Interrupt Registers................................................................................................... 466
MSI Mode Register (offset: 0x6000).................................................................................................... 466
MSI Status Register (offset: 0x6004) .................................................................................................. 467
DMA Completion Registers..................................................................................................................... 468
DMA Completion Mode Register (offset: 0x6400) ............................................................................... 468
GRC Registers.......................................................................................................................................... 468
Mode Control Register (offset: 0x6800) .............................................................................................. 468
Miscellaneous Configuration Register (offset: 0x6804)....................................................................... 470
Miscellaneous Local Control Register (offset: 0x6808) ....................................................................... 471
Timer Register (offset: 0x680C) .......................................................................................................... 473
RX-CPU Event Register (offset: 0x6810) ............................................................................................ 473
RX-CPU Timer Reference Register (offset: 0x6814) .......................................................................... 474
RX-CPU Semaphore Register (offset: 0x6818)................................................................................... 474
Serial EEPROM Address Register .......................................................................................................... 475
Serial EEPROM Delay Register (offset: 0x6848) ................................................................................ 475
RX CPU Event Enable Register (offset: 0x684C) ............................................................................... 475
Miscellaneous Control Registers............................................................................................................ 477
Miscellaneous Control Register (offset: 0x6890).................................................................................477
Fast Boot Program Counter Register (offset: 0x6894) ........................................................................ 477
Power Management Debug Register (offset: 0x68A4) ........................................................................ 478
5755ME Miscellaneous Control Register (offset: 0x68B0).................................................................. 479
Memory TM control1 (offset: 0x68E0)................................................................................................. 479
Memory TM control 2 (offset: 0x68E4) ................................................................................................ 479
Mem TM control 3(offset: 0x68E8) ...................................................................................................... 480
Expansion ROM Address Register (offset: 0x68EC)........................................................................... 480
BCM5719/BCM5720 Registers ........................................................................................................... 480
Mem TM Control 4 (offset: 0x68F8) ............................................................................................. 480
TPH Hint Register (Offset: 0x68FC)............................................................................................. 481
EAV Ref Count Capture LSB Reg (offset: 0x6900)...................................................................... 482
EAV Ref Count Capture MSB Reg (offset: 0x6904)..................................................................... 482
EAV Ref Clock Control Reg (offset: 0x6908) ............................................................................... 482
EAV Ref Count Snapshot LSB[0] Reg (offset 0x6910) ................................................................ 483
EAV Ref Count Snapshot MSB[0] Reg (offset: 0x6914) .............................................................. 484
TX Time Watchdog LSB[0] Reg (offset: 0x6918) ......................................................................... 484
TX Time Watchdog MSB[0] Reg (offset: 0x691C) ....................................................................... 484
TX Time Watchdog LSB[1] Reg (offset: 0x6920) ......................................................................... 484
TX Time Watchdog MSB[1] Reg (offset: 0x6924) ........................................................................ 485
EAV Ref Corrector Reg [Offset 0x6928) ...................................................................................... 485
EAV Ref Count Snapshot LSB[1] Reg (Offset 0x6930) ............................................................... 485
Broadcom
January 29, 2016 • 5718-PG108-R Page 32
®
Page 33
Table of ContentsBCM5718 Programmer’s Guide
EAV Ref Count Snapshot MSB[1] Reg [Offset 0x6934] ............................................................... 486
Non-Volatile Memory (NVM) Interface Registers................................................................................... 487
NVM Command Register (0x7000) ..................................................................................................... 487
NVM Write Register (offset: 0x7008)................................................................................................... 488
NVM Address Register (offset: 0x700C) ............................................................................................. 488
NVM Read Register (offset: 0x7010) .................................................................................................. 488
NVM Config 1 Register (offset: 0x7014).............................................................................................. 489
NVM Config 2 Register (offset: 0x7018).............................................................................................. 490
NVM Config 3 Register (offset: 0x701C) ............................................................................................. 491
Software Arbitration Register (offset: 0x7020) .................................................................................... 491
NVM Access Register (offset: 0x7024) ............................................................................................... 493
NVM Write1 Register (offset: 0x7028)................................................................................................. 493
Arbitration Watchdog Timer Register (offset: 0x702C)........................................................................ 494
NVM Auto-Sense Status Register (offset: 0x7038) ............................................................................. 494
Section 14: Transceiver Registers.................................................................................495
Purpose ..................................................................................................................................................... 495
BCM5718 Family MII Bus PHY Addressing............................................................................................ 495
Register Field Access Type..................................................................................................................... 496
Transceiver Register Map........................................................................................................................ 496
00h–0Fh 10/100/1000T Register Map Detailed Description.................................................................. 500
00h: MII_Control_Register .................................................................................................................. 500
01h: MII_Status_Register.................................................................................................................... 501
02h: PHY_Identifier_MSB_Register .................................................................................................... 502
03h: PHY_Identifier_LSB_Register ..................................................................................................... 502
04h: Auto_Negot_Advertisement_Register ......................................................................................... 502
05h: Auto_Negot_Link_Partner_Ability_Base_Pg_Register ............................................................... 503
06h: Auto_Negot_Expansion_Register ............................................................................................... 503
07h: Auto_Negot_Next_Page_Transmit_Register (Software Controlled Next Pages)........................ 504
08h: Auto_Negot_Link_Partner_Ability_Nxt_Pg_Register .................................................................. 504
09h: 1000Base_T_Control_Register ................................................................................................... 505
0Ah: 1000Base_T_Status_Register .................................................................................................... 506
0Eh: BroadReach LRE Access Register .............................................................................................506
0Fh: IEEE_Extended_Status_Register ............................................................................................... 506
10h–1Fh Register Map Detailed Description......................................................................................... 508
10h: PHY_Extended_Control_Register............................................................................................... 508
11h: PHY_Extended_Status_Register (copper side only)................................................................... 509
12h: Receive_Error_Counter_Register ............................................................................................... 510
13h: False_Carrier_Sense_Counter_Register .................................................................................... 511
14h: Local_Remote_Receiver_NOT_OK_Counters_Register ............................................................ 511
Broadcom
January 29, 2016 • 5718-PG108-R Page 33
®
Page 34
Table of ContentsBCM5718 Programmer’s Guide
18h: Auxiliary Control Register (Shadow Register Selector = “000”) .................................................. 511
18h: 10BASE-T Register (Shadow Register Selector = “001”) ........................................................... 513
18h: Power/MII Control Register (Shadow Register Selector = “010”)................................................ 515
18h: IP Phone Register (Shadow Register Selector = “011”).............................................................. 515
18h: Misc Test Register 1 (Shadow Register Selector = “100”) .......................................................... 516
18h: Misc Test Register 2 (Shadow Register Selector = “101”) .......................................................... 517
18h: Manual IP Phone Seed Register (Shadow Register Selector = “110”) ....................................... 519
18h: Miscellaneous Control Register (Shadow Register Selector = “111”) ......................................... 519
19h: Auxiliary Status Summary (Copper Side Only) ........................................................................... 520
1Ah: Interrupt Status Register (Copper Side Only) ............................................................................. 521
1Bh: Interrupt Mask Register............................................................................................................... 522
1Ch: Cabletron LED Register (Shadow Register Selector = “00h”) .................................................... 523
1Ch: DLL Selection Register (Shadow Register Selector = “01h”) ..................................................... 524
1Ch: Spare Control 1 Register (Shadow Register Selector = “02h”)................................................... 524
1Ch: Clock Alignment Control Register (Shadow Register Selector = “03h”) ..................................... 525
1Ch: Spare Control 2 Register (Shadow Register Selector = “04h”)................................................... 526
1Ch: Spare Control 3 Register (Shadow Register Selector = “05h”)................................................... 527
1Ch: TDR Control 1 Register (Shadow Register Selector = “06h” ...................................................... 528
1Ch: TDR Control 2 Register (Shadow Register Selector = “07h”) ..................................................... 528
1Ch: LED Status Register (Shadow Register Selector = “08h”).......................................................... 529
1Ch: Led Control Register (Shadow Register Selector = “09h”) ......................................................... 529
1Ch: SGMII Slave Register (Shadow Register Selector = “15h”)........................................................ 530
1Ch: Misc 1000-X Control 2 Register (Shadow Register Selector = “16h”) ........................................ 532
1Ch: Misc 1000-X Control Register (Shadow Register Selector = “17h”) ........................................... 533
1Ch: Auto-Detect SGMII/GBIC Register (Shadow Register Selector = “18h”) .................................... 535
1Ch: Test 1000-X Register (Shadow Register Selector = “19h”) ........................................................ 536
1Ch: Autoneg 1000-X Debug Register (Shadow Register Selector = “1ah”) ...................................... 537
1Ch: Auxiliary 1000-X Control Register (Shadow Register Selector = “1bh”) ..................................... 538
1Ch: Auxiliary 1000-X Status Register (Shadow Register Selector = “1ch”)....................................... 539
1Ch: Misc 1000-X Status Register (Shadow Register Selector = “1dh”)............................................. 541
1Ch: Auto-Detect Medium Register (Shadow Register Selector = “1eh”) ........................................... 542
1Ch: Mode Control Register (Shadow Register Selector = “1fh”) ....................................................... 543
1Dh: Master/Slave Seed Register (Bit 15 = 0) ....................................................................................543
1Dh: HCD Status Register (Bit 15 = 1)................................................................................................ 544
1Eh: Test1_Register............................................................................................................................ 545
1Fh: Test2_Register............................................................................................................................ 546
SerDes PHY Register Definitions............................................................................................................ 547
Register Map ....................................................................................................................................... 548
MII Control........................................................................................................................................... 550
Broadcom
January 29, 2016 • 5718-PG108-R Page 34
®
Page 35
Table of ContentsBCM5718 Programmer’s Guide
MII Status ............................................................................................................................................ 551
AUTONEGADV ................................................................................................................................... 552
AUTONEG Link Partner Ability............................................................................................................ 553
AUTONEGEXPANSION...................................................................................................................... 554
EXTENDEDSTATUS........................................................................................................................... 554
1000XCONTROL1 .............................................................................................................................. 555
1000XCONTROL2 .............................................................................................................................. 556
1000XCONTROL3 .............................................................................................................................. 558
1000XSTATUS1.................................................................................................................................. 559
1000XSTATUS2.................................................................................................................................. 560
1000XSTATUS3.................................................................................................................................. 561
FXCONTROL1 .................................................................................................................................... 562
FXCONTROL2 .................................................................................................................................... 563
FXCONTROL3 .................................................................................................................................... 564
FXSTATUS1........................................................................................................................................ 564
ANALOG_TX1..................................................................................................................................... 565
ANALOG_TX2..................................................................................................................................... 566
ANALOG_TXAMP ............................................................................................................................... 566
ANALOG_RX1 .................................................................................................................................... 568
ANALOG_RX2 .................................................................................................................................... 569
ANALOG_PLL ..................................................................................................................................... 569
GE_PRBS_CONTROL........................................................................................................................ 570
GE_PRBS_STATUS ........................................................................................................................... 570
Clause 45 Registers................................................................................................................................. 571
Clause 45 Register Dev 3 Reg14h (20d): EEE Capability Register .................................................... 571
Clause 45 Register Dev 7 Reg3ch (60d): EEE Advertisement Register ............................................. 572
1000BASE-T EEE ........................................................................................................................ 572
100BASE-TX EEE........................................................................................................................ 572
Clause 45 Register Dev 7 Reg803Eh (32830d): EEE Resolution Status............................................ 572
EEE 1000BASE-T Resolution ...................................................................................................... 573
EEE 100BASE-TX Resolution...................................................................................................... 573
Clause 45 Register Dev 7 Reg803dh (32817d): EEE Control Register .............................................. 573
LPI Feature Enable ...................................................................................................................... 573
Appendix A: Flow Control ..............................................................................................574
Notes.......................................................................................................................................................... 574
Flow Control Scenario ............................................................................................................................. 574
File Transfer ........................................................................................................................................ 575
Speed Mismatch.................................................................................................................................. 575
Switch Buffers Run Low ...................................................................................................................... 576
Broadcom
January 29, 2016 • 5718-PG108-R Page 35
®
Page 36
Table of ContentsBCM5718 Programmer’s Guide
Switch Backpressure........................................................................................................................... 577
Switch Flow Control............................................................................................................................. 577
File Transfer Complete........................................................................................................................ 578
Pause Control Frame ............................................................................................................................... 578
Appendix B: Terminology...............................................................................................579
Appendix C: Device Register and Memory Map...........................................................580
BCM5717 / BCM5718 Memory Map......................................................................................................... 580
BCM5717 / BCM5718 Register Map......................................................................................................... 583
BCM5719 Memory Map ............................................................................................................................ 585
BCM5719 Register Map............................................................................................................................ 587
BCM5720 Memory Map ............................................................................................................................ 589
BCM5720 Register Map............................................................................................................................ 590
Broadcom
January 29, 2016 • 5718-PG108-R Page 36
®
Page 37
List of FiguresBCM5718 Programmer’s Guide

List of Figures

Figure 1: Individual Port Functional Block Diagram ......................................................................................... 51
Figure 2: High-Level System Functional Block Diagram.................................................................................. 53
Figure 3: Receive Data Path ............................................................................................................................ 54
Figure 4: Transmit Data Path ........................................................................................................................... 56
Figure 5: DMA Read Engine ............................................................................................................................ 57
Figure 6: DMA Write Engine ............................................................................................................................ 58
Figure 7: Host Coalescing Engine ................................................................................................................... 60
Figure 8: Media Independent Interface ............................................................................................................ 63
Figure 9: GMII Block ........................................................................................................................................ 65
Figure 10: MDI Register Interface .................................................................................................................... 66
Figure 11: Generic Ring Diagram .................................................................................................................... 70
Figure 12: Transmit Ring Data Structure Architecture Diagram ...................................................................... 74
Figure 13: Receive Return Ring Memory Architecture Diagram ...................................................................... 77
Figure 14: Receive Buffer Descriptor Cycle..................................................................................................... 89
Figure 15: Receive Producer Ring RCB Setup ................................................................................................ 91
Figure 16: Class of Service Example............................................................................................................... 98
Figure 17: Overview Diagram of RX Flow...................................................................................................... 100
Figure 18: RSS Receive Processing Sequence ............................................................................................ 102
Figure 19: Relationships Between All Components of a Send Ring .............................................................. 106
Figure 20: Max_Len Field in Ring Control Block............................................................................................ 107
Figure 21: Relationship Between Send Buffer Descriptors............................................................................ 108
Figure 22: Send Buffer Descriptor ................................................................................................................. 112
Figure 23: Extended RX Buffer Descriptor..................................................................................................... 118
Figure 24: Ring Control Block ........................................................................................................................ 121
Figure 25: Send Buffer Descriptor ................................................................................................................. 123
Figure 26: Send Driver Interface .................................................................................................................... 126
Figure 27: Receive Producer Interface .......................................................................................................... 127
Figure 28: Receive Return Interface .............................................................................................................. 128
Figure 29: Scatter Gather of Frame Fragments ............................................................................................. 130
Figure 30: Transmit Data Flow....................................................................................................................... 132
Figure 31: Basic Driver Flow to Send a Packet ............................................................................................. 133
Figure 32: Local Contexts .............................................................................................................................. 170
Figure 33: Header Type Register 0xE ........................................................................................................... 171
Figure 34: Register Indirect Access ............................................................................................................... 174
Figure 35: Indirect Memory Access ............................................................................................................... 176
Broadcom®
January 29, 2016 • 5718-PG108-R Page 37
Page 38
List of FiguresBCM5718 Programmer’s Guide
Figure 36: Low-Priority Mailbox Access for Indirect Mode ............................................................................. 178
Figure 37: Standard Memory Mapped I/O Mode ........................................................................................... 179
Figure 38: Memory Window Base Address Register ..................................................................................... 180
Figure 39: Standard Mode Memory Window ................................................................................................. 181
Figure 40: Techniques for Accessing Ethernet Controller Local Memory ...................................................... 182
Figure 41: PCI Command Register................................................................................................................ 184
Figure 42: PCI Base Address Register .......................................................................................................... 185
Figure 43: PCI Base Address Register Bits Read in Standard Mode ............................................................ 185
Figure 44: Read and Write Channels of DMA Engine ................................................................................... 186
Figure 45: Power State Transition Diagram................................................................................................... 189
Figure 46: WOL Functional Block Diagram.................................................................................................... 213
Figure 47: Comparing Ethernet Frames Against Available Patterns (10/100 Ethernet WOL) ....................... 216
Figure 48: Unused Rows and Rules Must Be Initialized with Zeros .............................................................. 217
Figure 49: Basic Driver Interrupt Service Routine Flow ................................................................................. 232
Figure 50: Traditional Interrupt Scheme ........................................................................................................ 236
Figure 51: Message-Signaled Interrupt Scheme ........................................................................................... 237
Figure 52: MSI Data FIeld .............................................................................................................................. 238
Figure 53: IOV Receive Flow ......................................................................................................................... 267
Figure 54: Copper PHY Register Mapping Table .......................................................................................... 499
Figure 55: SerDes PHY Register Map ........................................................................................................... 549
Figure 56: File Transfer Scenario: FTP Session Begins ................................................................................ 575
Figure 57: File Transfer Scenario: Speed Mismatch...................................................................................... 575
Figure 58: File Transfer Scenario: Speed Buffers Run Low .......................................................................... 576
Figure 59: File Transfer Scenario: Switch Backpressure ............................................................................... 577
Figure 60: File Transfer Scenario: Switch Flow Control................................................................................. 577
Figure 61: File Transfer Scenario: File Transfer Complete ............................................................................ 578
Figure 62: Pause Control Frame.................................................................................................................... 578
Broadcom®
January 29, 2016 • 5718-PG108-R Page 38
Page 39
List of TablesBCM5718 Programmer’s Guide

List of Tables

Table 1: Register Access Methods .................................................................................................................. 44
Table 2: BCM5718 Family Product Features................................................................................................... 47
Table 3: Family Revision Levels ...................................................................................................................... 49
Table 4: Ring Control Block Format ................................................................................................................. 71
Table 5: Flag Fields for a Ring ......................................................................................................................... 71
Table 6: Send RCBs for Multiple Rings ........................................................................................................... 72
Table 7: High Priority Mail Box Registers for VRQ Rings ................................................................................ 72
Table 8: Send Buffer Descriptors Format ........................................................................................................ 75
Table 9: Defined Flags for Send Buffer Descriptors ........................................................................................ 75
Table 10: Receive Return Rings ...................................................................................................................... 78
Table 11: Receive Descriptors Format ............................................................................................................ 78
Table 12: Defined Flags for Receive Buffers ................................................................................................... 79
Table 13: Defined Error Flags for Receive Buffers .......................................................................................... 80
Table 14: Status Block Format (MSI-X Single-Vector or INTx —RSS Mode).................................................. 83
Table 15: Status Block Format (MSI-X Single-Vector or INTx —RSS Mode).................................................. 84
Table 16: Status Block [0] Format (MSI-X Multivector RSS Mode) ................................................................. 85
Table 17: Status Blocks [1 thru 4] Formats (MSI-X Multivector RSS Mode).................................................... 85
Table 18: Status Block Host Addresses and INT MailBox Addresses ............................................................. 86
Table 19: Status Word Flags ........................................................................................................................... 86
Table 20: Mailbox Registers ............................................................................................................................ 92
Table 21: Receive Rules Configuration Register ............................................................................................. 95
Table 22: Receive BD Rules Control Register ................................................................................................. 96
Table 23: Receive List Placement Rules Array (memory offset 0x480–0x4ff) ................................................. 96
Table 24: Receive BD Rules Value/Mask Register.......................................................................................... 97
Table 25: Frame Format with 802.1Q VLAN Tag Inserted .............................................................................. 99
Table 26: Send Data Initiator Mode Register (Offset: 0xC00) ....................................................................... 111
Table 27: ISO Send Data Initiator Mode Register (Offset: 0xD00) ................................................................ 111
Table 28: Read DMA Mode Register (offset: 0x4800) ................................................................................... 111
Table 29: ISO Read DMA Mode Register (Offset: 0x4A00)........................................................................... 111
Table 30: Flag Field Description .................................................................................................................... 113
Table 31: Receive BD Error Flags ................................................................................................................. 119
Table 32: Receive BD Flags .......................................................................................................................... 120
Table 33: Receive BD Flags .......................................................................................................................... 122
Table 34: Send Buffer Descriptor Flags ......................................................................................................... 123
Table 35: Status Block ................................................................................................................................... 124
Broadcom®
January 29, 2016 • 5718-PG108-R Page 39
Page 40
List of TablesBCM5718 Programmer’s Guide
Table 36: Mac Address Registers .................................................................................................................. 135
Table 37: Multicast Hash Table Registers ..................................................................................................... 137
Table 38: Recommended BCM57XX Ethernet Controller Memory Pool Watermark Settings ....................... 140
Table 39: Recommended BCM57XX Ethernet controller Low Watermark Maximum Receive Frames Settings.
140
Table 40: Recommended BCM57XX Ethernet Controller Host Coalescing Tick Counter Settings ............... 143
Table 41: Recommended BCM57XX Ethernet Controller Host Coalescing Frame Counter Settings ........... 143
Table 42: Recommended BCM57XX Ethernet Controller Max Coalesced Frames During Interrupt Counter
Settings......................................................................................................................................... 143
Table 43: PTP Link Delay Measure Roles ..................................................................................................... 153
Table 44: PTP Time Synchronization Messaging Roles ................................................................................ 154
Table 45: Send Ring SBD Flags .................................................................................................................... 157
Table 46: Receive Return Ring RBD Flags ................................................................................................... 158
Table 47: Device Specific Registers .............................................................................................................. 172
Table 48: PCI Address Map Standard View .................................................................................................. 180
Table 49: GPIO Usage for Power Management for Broadcom Drivers ......................................................... 191
Table 50: Ethernet Controller Power Pins...................................................................................................... 191
Table 51: Endian Example ............................................................................................................................. 193
Table 52: Storage of Big-Endian Data ........................................................................................................... 193
Table 53: Storage of Little-Endian Data......................................................................................................... 193
Table 54: Default Translation (No Swapping) on 64-Bit PCI.......................................................................... 194
Table 55: Default Translation (No Swapping) on 32-bit PCI .......................................................................... 195
Table 56: RCB (Big Endian 32-Bit Format).................................................................................................... 195
Table 57: Internal Byte Ordering.......................................................................................PCI Byte Ordering195
Table 58: Byte Swap Enable Translation on 32-Bit PCI (No Word Swap)..................................................... 196
Table 59: Big-Endian Internal Packet Data Format ....................................................................................... 197
Table 60: 64-Bit PCI Bus (WSD = 0, BSD = 0) .............................................................................................. 197
Table 61: 32-Bit PCI Bus (WSD = 0, BSD = 0) .............................................................................................. 198
Table 62: 64-Bit PCI Bus (WSD = 0, BSD = 1) .............................................................................................. 198
Table 63: 32-Bit PCI Bus (WSD = 0, BSD = 1) .............................................................................................. 198
Table 64: 64-Bit PCI Bus (WSD = 1, BSD = 0) .............................................................................................. 198
Table 65: 32-Bit PCI Bus (WSD = 1, BSD = 0) .............................................................................................. 198
Table 66: 64-Bit PCI Bus (WSD = 1, BSD = 1) .............................................................................................. 199
Table 67: 32-Bit PCI Bus (WSD = 1, BSD = 1) .............................................................................................. 199
Table 68: Send Buffer Descriptor (Big-Endian 64-Bit format) ........................................................................ 200
Table 69: Send Buffer Descriptor (Big-Endian 32-Bit format) ........................................................................ 200
Table 70: Send Buffer Descriptor (Little-Endian 32-Bit format) with No Swapping ........................................ 201
Broadcom®
January 29, 2016 • 5718-PG108-R Page 40
Page 41
List of TablesBCM5718 Programmer’s Guide
Table 71: Send Buffer Descriptor (Little-Endian 32-Bit format) with Word Swapping .................................... 201
Table 72: Send Buffer Descriptor (Big-Endian 32-bit format) with Byte Swapping ........................................ 201
Table 73: Send Buffer Descriptor (Big-Endian 32-bit format) with Word and Byte Swapping ....................... 202
Table 74: Required Memory Regions for WOL Pattern ................................................................................. 214
Table 75: 10/100 Mbps Mode Frame Patterns Memory ................................................................................ 217
Table 76: Frame Control Field for 10/100 Mbps Mode .................................................................................. 217
Table 77: Example of Splitting 10/100 Mbps Frame Data in Pattern Data Structure ..................................... 218
Table 78: Firmware Mailbox Initialization ....................................................................................................... 219
Table 79: Recommended Settings for PHY Auto-Negotiation ....................................................................... 219
Table 80: WOL Mode Clock Inputs ................................................................................................................ 219
Table 81: Magic Packet Detection Logic Enable ........................................................................................... 220
Table 82: Integrated MAC WOL Mode Control Registers ..............................................................................220
Table 83: Transmit MAC Watermark Recommendation ................................................................................ 224
Table 84: Pause Quanta ................................................................................................................................ 224
Table 85: Keep_Pause Recommended Value ............................................................................................... 224
Table 86: Statistic Block................................................................................................................................. 225
Table 87: Integrated MAC Flow Control Registers ........................................................................................ 226
Table 88: NetXtreme Legacy Status Block Format ........................................................................................ 229
Table 89: Interrupt-Related Registers ............................................................................................................ 235
Table 90: MSI-X Vector Mode Selection........................................................................................................ 241
Table 91: MIS-X Status-Block and Mail Box Addresses ................................................................................ 244
Table 92: Status Block Format (MSI-X Single-Vector RSS Mode) ................................................................ 247
Table 93: Status Block format (MSI-X Single-Vector IOV Mode)................................................................... 247
Table 94: Status Block [0] Format (MSI-X Multivector RSS Mode]................................................................ 248
Table 95: Status Block [1  N  4] Formats (MSI-X Multivector RSS Mode) ................................................. 249
Table 96: Status Block [0] Format (MSI-X Multivector IOV Mode)................................................................. 249
Table 97: Status Block [1  N  16] Format (MSI-X Multivector IOV Mode) ................................................. 250
Table 98: MSI-X Capability Structure............................................................................................................. 250
Table 99: MSI-X Table and PBA Structures in BCM5718 Family .................................................................. 252
Table 100: MSI-X Host Coalescing Parameters ............................................................................................ 256
Table 101: BCM5718 Family Register Map ................................................................................................... 269
Table 102: Receive BD Jumbo Producer Ring Index Register (offset: 0x270) .............................................. 307
Table 103: High Priority Mail Box Registers for VRQ Rings .......................................................................... 308
Table 104: Send BD Diagnostic Initiator ........................................................................................................ 354
Table 105: Multiple Send Ring Mail Boxes .................................................................................................... 355
Table 106: BD Fetch Limit Register (Offset 0x2D08)..................................................................................... 375
Table 107: HC Parameter Set Reset Register (Offset: 0x3C28) ................................................................... 421
Broadcom®
January 29, 2016 • 5718-PG108-R Page 41
Page 42
List of TablesBCM5718 Programmer’s Guide
Table 108: NIC Receive BD Consumer Index Register (offset: 0x3C50 – 0x3C58) ...................................... 423
Table 109: NIC Diag Receive Return Ring BD 0 Index Register (offset: 0x3C80) ........................................ 423
Table 110: Generic VRQ Statistics (Offset 0x0BFF – 0x0A00) ..................................................................... 454
Table 111: Default & Drop VRQ Statistics (Offset 0x09F7 – 0x09D0)........................................................... 455
Table 112: First Half VRQ Mapper Entry Register ......................................................................................... 456
Table 113: Second Half VRQ Mapper Entry Register.................................................................................... 456
Table 114: VRQ Mapper Register List ........................................................................................................... 457
Table 115: VRQ Enable Register (Offset 0x560) ........................................................................................... 458
Table 116: High Priority Mail Box Registers for VRQ Rings .......................................................................... 458
Table 117: VRQ_PERFECT_MATCH[4 – 23]_HIGH_REG (Offsets: 0x5690, 0x5698, 0x56A0 … 0x5728) . 460
Table 118: VRQ_PERFECT_MATCH[4 – 23]_LOW_REG (Offsets: 0x5694, 0x569C, 0x56A4 … 0x572C) 460
Table 119: BCM5717 ..................................................................................................................................... 495
Table 120: BCM5718 ..................................................................................................................................... 495
Table 121: BCM5719 ..................................................................................................................................... 495
Table 122: BCM5720 ..................................................................................................................................... 495
Table 123: 02h: PHY_Identifier_MSB_Register............................................................................................. 502
Table 124: 03h: PHY_Identifier_LSB_Register.............................................................................................. 502
Table 125: GbE Port Internal PHY Register Map .......................................................................................... 548
Table 126: MII Control ................................................................................................................................... 550
Table 127: MII Status ..................................................................................................................................... 551
Table 128: 02h: PHY_Identifier_MSB_Register............................................................................................. 552
Table 129: 03h: PHY_Identifier_LSB_Register.............................................................................................. 552
Table 130: AUTONEGADV ............................................................................................................................ 552
Table 131: AUTONEG LINK PARTNER ABILITY..........................................................................................553
Table 132: AUTONEGEXPANSION .............................................................................................................. 554
Table 133: EXTENDEDSTATUS ................................................................................................................... 554
Table 134: 1000XCONTROL1 ....................................................................................................................... 555
Table 135: 1000XCONTROL2 ....................................................................................................................... 556
Table 136: 1000XCONTROL3 ....................................................................................................................... 558
Table 137: 1000XSTATUS1 .......................................................................................................................... 559
Table 138: 1000XSTATUS2 .......................................................................................................................... 561
Table 139: 1000XSTATUS3 .......................................................................................................................... 561
Table 140: FXCONTROL1 ............................................................................................................................. 562
Table 141: FXCONTROL2 ............................................................................................................................. 563
Table 142: FXCONTROL3 ............................................................................................................................. 564
Table 143: FXSTATUS1 ................................................................................................................................ 564
Table 144: ANALOG_TX1 ............................................................................................................................. 565
Broadcom®
January 29, 2016 • 5718-PG108-R Page 42
Page 43
List of TablesBCM5718 Programmer’s Guide
Table 145: ANALOG_TX2 ............................................................................................................................. 566
Table 146: ANALOG_TXAMP........................................................................................................................ 566
Table 147: ANALOG_RX1 ............................................................................................................................. 568
Table 148: ANALOG_RX2 ............................................................................................................................. 569
Table 149: ANALOG_PLL.............................................................................................................................. 569
Table 150: GE_prbs_status ........................................................................................................................... 570
Table 151: GE_prbs_status ........................................................................................................................... 570
Table 152: Clause 45 Register Dev 3 Reg14h: EEE Capability Register ...................................................... 572
Table 153: Clause 45 Register Dev 7 Reg3Ch: EEE Advertisement Register .............................................. 572
Table 154: Clause 45 Register Dev 7 Reg803Eh: EEE Resolution Status.................................................... 572
Table 155: Clause 45 Register Dev 7 Reg803dh: EEE Control Register ...................................................... 573
Table 156: Terminology ................................................................................................................................. 579
Table 157: BCM5717 / BCM5718 Memory Map ............................................................................................ 580
Table 158: BCM5717 / BCM5718 Register Map............................................................................................ 583
Table 159: BCM5719 Memory Map ............................................................................................................... 585
Table 160: BCM5719 Register Map............................................................................................................... 587
Table 161: BCM5720 Memory Map ............................................................................................................... 589
Table 162: BCM5720 Register Map............................................................................................................... 590
Broadcom®
January 29, 2016 • 5718-PG108-R Page 43
Page 44
BCM5718 Programmer’s Guide

About This Document

Purpose and Audience

This document covers the Broadcom® BCM5718 family of NetXtreme®/NetLink® Ethernet controllers. This family of controllers includes the following devices:
• BCM5717
• BCM5718
• BCM5719
• BCM5720
The document focuses on the registers, control blocks, and software interfaces necessary for host software programming. It is intended to complement the data sheet for the appropriate member of the NetXtreme/NetLink Ethernet controller family. The errata documentation (see “Revision Levels” on page 49) complements this document.

Acronyms and Abbreviations

In most cases, acronyms and abbreviations are defined on first use.
For a comprehensive list of acronyms and other terms used in Broadcom documents, go to:
http://www.broadcom.com/press/glossary.php.

Document Conventions

The following conventions may be used in this document:
Convention Description
Bold User input and actions: for example, type exit, click OK, press Alt+C
Monospace
< > Placeholders for required elements: enter your <username> or wl <command> [ ] Indicates optional command-line parameters: wl [-l]
Convention Description
Code: #include <iostream> HTML: <td rowspan = 3> Command line commands and parameters: wl [-l] <command>
Indicates bit and byte ranges (inclusive): [0:3] or [7:0]
Table 1: Register Access Methods
A/C Clear contents after read
RO Read-Only access, Write has no effect
RW Full Read and Write access
WO Write-Only access, Read returns garbage
Broadcom®
January 29, 2016 • 5718-PG108-R Page 44
Page 45
BCM5718 Programmer’s Guide
Table 1: Register Access Methods (Cont.)
Convention Description
W1C Write a value 1 to clear the bit.
W1S Write a value 1 to set the bit
WZO Write 0 only to avoid side effects, Read returns garbage
RW1C Readable / Write a 1’b1 value to clear the bit.
RW1S Readable / Write a 1’b1 to set the bit
RW/S Readable / Writeable but is not affected by any CPU reset or AHB reset.

References

The references in this section may be used in conjunction with this document.
Note: Broadcom provides customer access to technical documentation and software through its
Customer Support Portal (CSP) and Downloads & Support site (see Technical Support).
For Broadcom documents, replace the “xx” in the document number with the largest number available in the repository to ensure that you have the most current version of the document.
Document (or Item) Name Number Source Broadcom Items
[1] BCM57XX NetXtreme
®
Programmer’s Guide:
57XX-PG1XX-R Broadcom CSP
Programming details for the BCM5700, BCM5701, BCM5702, BCM5703, BCM5704, BCM5705, BCM5721, BCM5751, BCM5752, BCM5714, BCM5715, and BCM57XX devices
[2] x2 PCI Express Dual-Port Gigabit Ethernet
5717-DS0X-R Broadcom CSP
Controller
[3] x2 PCI Express Dual-Port Gigabit Ethernet
5718-DS0X-R Broadcom CSP
Controller
[4] x4 PCI Express
®
Quad-Port Dual-Media Gigabit
5719-DS0X-R Broadcom CSP
Ethernet Controller
[5] x2 PCI Express
Dual-Port Dual-Media Gigabit
5720-DS0X-R Broadcom CSP
®
Ethernet Controller
[6] x2 PCI Express
Dual-Port Gigabit Ethernet
5717-ES10X-R Broadcom CSP
®
Controller Errata (A0 and B0) [7] BCM5718 Revision A0, B0 5718-ES10X-R Broadcom CSP [8] BCM5719 A0 Errata 5719-ES10X-R Broadcom CSP [9] BCM5720 Errata 5720-ES10X-R Broadcom CSP [10] Self Boot Option 5754X_5787X-AN10X-R Broadcom CSP [11] NetXtreme/NetLink Software Self-Boot NVRAM NetXtreme-AN40X-R Broadcom CSP [12] NetXtreme [13] NetXtreme/NetLink NVRAM Configuration
®
/NetLink® NVRAM Access NetXtreme-AN50X-R Broadcom CSP
NetXtreme-AN60X-R Broadcom CSP
Options
Broadcom®
January 29, 2016 • 5718-PG108-R Page 45
Page 46
BCM5718 Programmer’s Guide
Document (or Item) Name (Cont.) Number Source
[14] NetXtreme/NetLink Shared Memory
Communication
Other Items
[15] NC-SI (Network Controller–Sideband Interface)
Specification
NetXtreme-AN80X-R Broadcom CSP
– (http://www.dmtf.org/home)

Technical Support

Broadcom provides customer access to a wide range of information, including technical documentation, schematic diagrams, product bill of materials, PCB layout information, and software updates through its customer support portal (https://support.broadcom.com support representative.
In addition, Broadcom provides other product support through its Downloads & Support site (http://www.broadcom.com/support/
).
). For a CSP account, contact your Sales or Engineering
Broadcom®
January 29, 2016 • 5718-PG108-R Page 46
Page 47
IntroductionBCM5718 Programmer’s Guide

Section 1: Introduction

The NetXtreme and NetLink family of Media Access Controller (MAC) devices are highly-integrated, single-chip Gigabit Ethernet LAN controller solutions for high-performance network applications. These devices integrate the following major functions to provide a single-chip solution for Gigabit LAN-on-motherboard (LOM) and network interface card (NIC) applications.
• Triple-speed IEEE 802.3-compliant MAC functionality
• Triple-speed IEEE 802.3-compliant Ethernet PHY transceiver
• PCI Express (PCIe) bus interface
• On-chip packet buffer memory
• On-chip RISC processor for custom frame processing

Product Features

Table 2: BCM5718 Family Product Features
BCM5717 Dual-Port
Feature Data Management
VLAN tag support (IEEE 802.1Q) Yes Yes Yes Yes
Layer 2 priority encoding (IEEE 802.1p) Yes Yes Yes Yes
Link aggregation (IEEE 802.3ad) Yes Yes Yes Yes
Full-duplex flow control (IEEE 802.3x) Yes Yes Yes Yes
Programmable rules checker for advanced packet filtering and classification
Frame/packet buffer memory 32 KB RX,
TCP checksum offload (hardware based) on Tx/Rx over IPv4/IPv6
UDP checksum offload (hardware based) on Tx/Rx over IPv4/IPv6
IP checksum offload on Tx/Rx over IPv4/IPv6
Hardware TCP segmentation offload over IPv4/IPv6
Jumbo frame support Yes Yes Yes Yes
Receive-side scaling (RSS) Yes Yes Yes Yes
UDP Receive-side scaling (UDP RSS) No No Yes Yes
Transmit-side scaling (TSS) Yes Yes Yes Yes
Copper
YesYesYesYes
29 KB TX
YesYesYesYes
YesYesYesYes
YesYesYesYes
YesYesYesYes
BCM5718 Dual-Port Copper/SerDes
32 KB RX, 29 KB TX
BCM5719 Quad-Port Copper/SerDes
40 KB RX, 29 KB TX
BCM5720 Dual-Port Copper/SerDes
40 KB RX, 29 KB TX
Broadcom®
January 29, 2016 • 5718-PG108-R Page 47
Page 48
Table 2: BCM5718 Family Product Features (Cont.)
Product FeaturesBCM5718 Programmer’s Guide
Feature
BCM5717 Dual-Port Copper
BCM5718 Dual-Port Copper/SerDes
BCM5719 Quad-Port Copper/SerDes
BCM5720 Dual-Port Copper/SerDes
Multiple receive descriptor queues Yes Yes Yes Yes
IOV support (I/O Virtualization) for:
•VMWare
•Microsoft
®
NetQueue
®
Virtual Machine Queue
No Yes Yes Yes
(VMQ)
MSI Yes Yes Yes Yes
MSI-X Yes (5 Vectors) Yes (17 Vectors) Yes (17 Vectors) Yes (17 Vectors)
Function Level Reset Yes Yes Yes Yes
Scatter/gather bus mastering
YesYesYesYes
architecture
Statistics for SNMP MIB II, Ethernet like
Yes Yes Yes Ye s MIB, Ethernet MIB (IEEE 802.3z, Clause 30)
ASF v2.0 support No No No No
iSC S I Boo t Ye s Ye s Ye s Ye s
Teami n g Yes Yes Yes Ye s
Host Bus Interfaces
PCIe 2.0 x2 Yes Yes Yes Yes
PCIe 2.1 x4 No No Yes No
LAN Interfaces
Integrated 10/100/1000 transceiver Yes Yes Yes Yes
Internal MII/GMII Interface Yes Yes Yes Yes
Other Bus Interfaces
NC-SI (version 1.0.0a) Yes Yes Yes Yes
Host 2 BMC No No No Yes
SMBus 2.0 interface Yes Yes Yes Yes
Interface to Flash memory Yes Yes Yes Yes
Interface to Serial EEPROM Yes Yes Yes Yes
Flash Autoconfig Support Yes Yes Yes Yes
Self-Test
Test modes (BIST, SCAN, etc) Yes Yes Yes Yes
JTAG support Yes Yes Yes Yes
Technology
High-performance, low-overhead
Yes Yes Yes Ye s software/hardware interface
High-speed on-chip RISC processors
YesYesYesYes (one per port)
Broadcom®
January 29, 2016 • 5718-PG108-R Page 48
Page 49
Table 2: BCM5718 Family Product Features (Cont.)
Revision LevelsBCM5718 Programmer’s Guide
Feature
High-speed on-chip Application
BCM5717
Dual-Port
Copper
YesYesYesYes
BCM5718 Dual-Port Copper/SerDes
BCM5719 Quad-Port Copper/SerDes
BCM5720 Dual-Port Copper/SerDes
Processor Engine (APE)
Wake-on-LAN (WOL) Yes Yes Yes Yes
Energy Efficient Ethernet (EEE) No Yes Yes Yes
Ethernet Audio Video (EAV) No No No No
Secure Digital (SD) Card Reader No No No No
IEEE 1588 / IEEE 802.1AS Timestamp
No No Yes Yes Support
On-chip temperature monitor No No Yes Yes
Integrated Trusted Platform Module
No No No No (TPM) Security Engine
Process voltage 1.2V 1.2V 1.2V 1.2V
CMOS linewidth 65 nm 65 nm 65 nm 65 nm

Revision Levels

See Table 3 for the revision levels of the Ethernet controllers covered by this document. Host software can use the PCI Revision ID and Chip ID information in the PCI configuration registers to determine the revision level of the Ethernet controller on the board, and then load the appropriate workaround described in the errata sheets.
The Broadcom PCI vendor ID is 0x14E4. Ta bl e 3 shows the default values of PCI device IDs. These values may be modified by firmware in accordance with the manufacturing information supplied in NVRAM (see “NVRAM
Configuration” on page 67 for more details).
Table 3: Family Revision Levels
Family Member Device ID
a
Revisio n Level
PCI Revision
b
ID
Chip ID
c
Product ASIC
d
ID
Errata Sheet
e
BCM5717 0x1655 A0 0x00 0xF000xxxx 0X5717000 5717-ES1xx-R
BCM5717 0x1655 B0 0x10 0xF100xxxx 0X5717100 5717-ES1xx-R
BCM5718 0x1656 A0 0x00 0xF000xxxx 0X5717000 5718-ES1xx-R
BCM5718 0x1656 B0 0x10 0xF100xxxx 0X5717100 5718-ES1xx-R
BCM5719 0x1657 A0 0x00 0xF000xxxx 0X5719000 5719-ES1xx-R
BCM5719 0x1657 A1 0x01 0XF100XXXX 0X5719100 5719-ES1XX-R
BCM5720 0X165F A0 0x00 0xF000xxxx 0X5720000 5720-ES1xx-R
a. See Device ID and Vendor ID Register (Offset: 0x00) —Function 0”, per PCI specification. b. See “PCI Classcode and Revision ID Register (offset: 0x8)—Function 0” as per the PCI specification. The
hardware default value of this register is 0x00. The boot code firmware programs this register with the value as given in the table.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 49
Page 50
Programming the Ethernet ControllersBCM5718 Programmer’s Guide
c. See “Miscellaneous Host Control Register (offset: 0x68)” on page 282. The lower 16 bits are don’t cares for
determining chip id. d. See “Product ASIC ID (offset: 0xF4)” on page 300 or determining ASIC ID. e. See the appropriate errata documentation for the errata information and resolutions.
Note: If you are using silicon revision A0 of the BCM5718, you must load a “patch” image
(ap5718.012) into the boot code NVRAM in addition to the usual legacy boot code image common to all NetXtreme/NetLink controllers. This extra patch image, which masquerades as “Management Firmware”, works around an issue with A0 silicon in which the device may fail to load reliably and execute the primary boot code image. This issue is fixed in BCM5718 B0 and is not an issue in BCM5719 or BCM5720.

Programming the Ethernet Controllers

See Table 3 on page 49 for the revision levels of the Ethernet controllers. Host software can use the PCI Revision ID and Chip ID information in the PCI configuration registers to determine the revision level of the Ethernet controller on the board, and then load the appropriate workarounds described in the errata sheets.
Choice of host access mode determines the mailboxes:
• Host standard mode uses the high-priority mailboxes (see “Mailbox Registers” on page 92).
• Indirect mode uses the low-priority mailboxes (see “Mailbox Registers” on page 92).
The reference documents for Ethernet controller software development include this manual and the errata documentation (see “Revision Levels” on page 49) that provide the necessary information for writing a host-
®
based device driver. The Broadcom Linux
driver (a.k.a. “tg3”) is also a very good reference source for writing
your own driver.
The programming model for the NetXtreme/NetLink Ethernet controllers does not depend on OS or processor
®
instruction sets. Programmers using Motorola
68000, Intel® x86, or DEC Alpha host instruction sets can
leverage this document to aid in device driver development. Concepts provided in this document are also
®
applicable to device drivers native to any operating system (i.e., DOS, UNIX
, Microsoft®, or Novell®).
Broadcom®
January 29, 2016 • 5718-PG108-R Page 50
Page 51
Hardware ArchitectureBCM5718 Programmer’s Guide
Receive
MAC
Rx
FIFO
Transmit
MAC
Tx
FIFO
Statistics
Rule
Check
Memory
Arbiter
Tx Frame Buffer
Memory
Send BD RING
RISC
Processor
Boot ROM
Frame Buffer
Manager
Queue
Memory
Read DMA
Read
FIFO
Write
DMA
Write
FIFO
Registers
PCIe Bus
Ring Controllers
Host Coalescing
Queue Management
Receive
GMII
Transmit
GMII
LED Control
PLL
LED Signals
125-MHz Clock
Receive BD RING
DMA Descriptor
Config
EEPROM Control
NVRAM
Interface
Physical Layer
Transceiver
PCIe
Rx Frame Buffer
Memory/RISC Scratch
Pad Memory
Applications
Processing
Engine
(APE runs
firmware such
as NC-SI)

Section 2: Hardware Architecture

Theory of Operation

Figure 1 shows the major functional blocks and interfaces of the Ethernet controllers covered in this document.
Only a single port is illustrated in Figure 1. The dual-port controllers in this family of controllers essentially replicate a second instance of the major areas of functionality shown in the diagram below. The dual-port controllers have only a single PCIe and NVRAM interfaces.
There are two packet flows: MAC-transmit and receive. The device’s DMA engine bus-masters packets from host memory to device local storage, and vice-versa. The host bus interface is compliant with PCIe standards. The RX MAC moves packets from the integrated PHY into device internal memory. All incoming packets are checked against a set of QOS rules and then categorized. When a packet is transmitted, the TX MAC moves data from device internal memory to the PHY. Both flows operate independently of each other in full-duplex mode. An on-chip RISC processor is provided for running value-added firmware that can be used for custom frame processing. The on-chip RISC operates independently of all the architectural blocks; essentially, RISC is available for the auxiliary processing of data streams.
Figure 1: Individual Port Functional Block Diagram
Broadcom®
January 29, 2016 • 5718-PG108-R Page 51
Page 52
Overview of FeaturesBCM5718 Programmer’s Guide

Overview of Features

The BCM5718 family of controllers represents the third generation of Broadcom NetXtreme multi-port Gigabit Ethernet controllers. This family is the successor to the BCM5714/BCM5715 family. The BCM5717, BCM5718, and BCM5720 feature two independent 1 Gb Ethernet ports on the network side. A host computer can communicate with the controller over a single PCIe link. However, the two network controller ports appear as two independent PCIe functions to the host operating system (four functions in the quad-port BCM5719). In essence, the chip consists of a single PCIe interface controller that offers multiple function-level interfaces, and each function-level interface is further attached to an independent DMA engine which in turn feeds an independent Ethernet Media Access Controller (EMAC).
Attached to the other end of each EMAC is the respective 802.3 Ethernet physical media interface. The controller essentially consists of multiple DMA+EMAC logic instances, multiple Ethernet physical interfaces, and a single instance of a PCIe core that is shared by the DMA blocks.
The BCM5718 is available as four SKUs, BCM5717, BCM5718, BCM5719 and BCM5720. These are also referred to as the Dual-Copper SKU and the Dual-Media SKU. For the BCM5718, BCM5719, and BCM5720 part, any of the network ports may be independently configured as a copper-based (1000BASE-T) or as SerDes­based (1000BASE-X/SGMII) media interfaces. The choice of media interface on each port is configurable via a power-on strapping option.
The BCM5717 part is permanently bonded as a copper (1000BASE-T) only device.
For the Dual-Media SKU (BCM5718 and BCM5719), whenever a port is configured as a SerDes medium, there are two protocol choices: 1000BASE-X or SGMII. This choice can be made by an Auto-Detect feature or by explicit software programming.
The software driver for this device is capable of loading or unloading each network port independently. The DMA+EMAC associated with each port is also able to acquire different ACPI power states irrespective of the other; however, the power state of the PCIe link may not necessarily follow that of either one or both ports. This is a significant behavioral difference of a dual-port controllers compared to single-port. The BCM5718 family hardware and firmware is cognizant of this effect and adds the necessary intelligence to handle it. This functionality remains transparent to device driver software.
On top of the basic dual-port network controller functionality, the BCM5718, BCM5719, and BCM5720 also support an advanced feature known as IO Virtualization (IOV).
Network Controller Sideband Interface (NC-SI) pass-through functionality is the Server Management solution offered by the BCM5718 family of controllers. NC-SI pass-through is a part of the Server Management infrastructure. Such technology offers server platform management via a BMC (baseboard management controller) chip. A server platform and, in turn, a BMC is typically administered remotely over the network, but BMCs are not equipped with direct network connectivity. Here, a network controller chip comes into the picture— a pass-through-capable NIC chip offers a sideband packet interface to the BMC. Over this interface the BMC can send and receive Ethernet packets to and from the network. In essence, the network adapter functionality of the network controller chip is shared by the host computer system and the BMC chip. NC-SI pass-through protocol is an industry standard (DMTF) for a side band interface between the BMC and network controller. The physical and L2 layers of NC-SI are upwardly compliant to the respective sections of the IEEE 802.3 specification, thus allowing the exchange of Ethernet packets between the network port and BMC without any protocol transformation whatsoever.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 52
Page 53
Overview of FeaturesBCM5718 Programmer’s Guide
Dual Function
PCIe
DMA 0 DMA 1
EMAC 0 EMAC 1
GHPY 0 Serdes 0 GHPY 1 Serdes 1
BMC
Host
Chipset
Mgmt
Engine
NC-SI
Pass - through
PCIe Link
BCM5718 Family
Port 0
Port 1
NVRAM
Device
RMII
x2 Gen1 /
x1 Gen2
In an environment of several network servers, typically no server is dedicated to running only the management console. Host to BMC pass-through functionality identifies host to local BMC-bound packets and routes them internally to the chip. Similarly, BMC to local host-bound packets are also identified and routed internally to the NIC chip. Host to BMC pass-through functionality is offered by the BCM5720 controller.
In addition to IEEE 802.3 standard size Ethernet frames, the BCM5718 family also supports jumbo frames of sizes up to 9622 bytes.
The BCM5718 family of controllers replaces the traditional PNP-based linear regulator with a more efficient switching regulator. This regulator steps down system supplied 3.3V rail to 1.2V, which it supplies to the chip’s core.
Figure 2: High-Level System Functional Block Diagram
Broadcom®
January 29, 2016 • 5718-PG108-R Page 53
Note: BCM5719 has the same general architecture, but four ports (ports 0, 1, 2, 3) and a quad-function
PCIe interface.
Page 54

Receive Data Path

RX
Engine
Rules Checker
Rx
FIFO
Frame Buffers
Empty BD
NIC Standard RX Producer Ring
Host RX
Return
Ring
List
Initiator
Rx Return BD
Rx Return BD
Rx Return BD
Rx Return BD
Rx Return BD
Hos t St an d a rd R X P ro d u c er
Ring
DMA
DMA
Full BD
Empty BD
NIC Jumbo RX Producer Ring
DMA
Host Jumbo RX Produce r Ring

RX Engine

The receive engine (see Figure 3) activates whenever a packet arrives from the PHY.
Figure 3: Receive Data Path
Receive Data PathBCM5718 Programmer’s Guide
The receive engine performs the following four functions:
• Moves the data from the PHY to an internal FIFO
• Moves the data from the FIFO to NIC internal memory
• Classifies the frame and checks it for rules matches
• Performs the offloaded checksum calculations

RX FIFO

The RX FIFO provides elasticity while data is read from PHY transceiver and written into internal memory. There are no programmable settings for the RX FIFO. This FIFO’s operation is completely transparent to host software.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 54
Page 55
Receive Data PathBCM5718 Programmer’s Guide

Rules Checker

The rules checker examines frames. After a frame has been examined, the appropriate classification bits are set in the buffer descriptor. The rules checker is part of the RX data path and the frames are classified during data movement to NIC memory. The following frame positions may be established by the rules checker:
• IP Header Start Pointer
• TCP/UDP Header Start Pointer
• Data Start Pointer

RX List Initiator

The RX List Initiator function activates whenever the receive producer index for any of receive buffer descriptor (BD) rings is written. This value is located in one of the receive BD producer mailboxes. The host software writes to the producer mailbox and causes the RX Initiator function to enqueue an internal data structure/request, which initiates the DMA of one or more new BDs to the NIC. The actual DMAs generated depend on the comparison of the value of the received BD host producer index mailbox, the NIC copy of the received BD consumer index, and the local copy of the received BD producer index.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 55
Page 56
Transmit Data PathBCM5718 Programmer’s Guide
TX
MAC
Consumer
Index
Update
Tx
FIFO
Send BD
NIC Send Ring Cache
Select Send BDs
(SBDs) from Send
Ring
Host Send Producer Rings
DMA
DMA
TX Data
TX Data
TX Data
TX Data
TX Data
TX Data
TX Data
TX Data
Buffer0 Buffer1 Buffer2 Buffer3 Buffer4 Buffer5

Transmit Data Path

TX MAC

The Read DMA engine moves packets from host memory into internal NIC memory (see Figure 4). When the entire packet is available, the transmit MAC is activated.
Figure 4: Transmit Data Path
The transmit MAC is responsible for the following functions:
• Moving data from NIC internal memory into TX FIFO
• Moving data from TX FIFO to PHY
• Checksum substitutions (not calculation)
• Updating statistics

TX FIFO

The TX FIFO provides elasticity while data is moved from device internal memory to PHY. There are no programmable settings for the TX FIFO. This FIFO’s operation is completely transparent to host software.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 56
Page 57
DMA ReadBCM5718 Programmer’s Guide
text
text
DMA
BD Packet#1
Host Send Buffer
Descriptors
Packet Data #1
Host Send Buffer
Memory
Buffer M anager
BD Packet#1
Frame Clas sify &
Checksum Calculat ion
text
Packet Data #1
Frame Header #1
NIC BD Memory
NIC Buffer
Memory
Tx
FIFO
Frame
Mod
TX
MAC
Statistics
TX
PCS
TX
RMII
TX
GMII
TX IO
6416

Read FIFO

DMA Read

Read Engine

The DMA read engine (see Figure 5) activates whenever a host read is initiated by the send or receive data paths.
Figure 5: DMA Read Engine
The DMA read engine dequeues an internal data structure/request and performs the following functions:
• DMAs the data from the host memory to an internal Read DMA FIFO
• Moves the data from the Read DMA FIFO to NIC internal memory
• Classifies the frame
• Performs checksum calculations
• Copies the VLAN tag field from the DMA descriptor to the frame header
Read FIFO
The read FIFO provides elasticity during data movement from host memory to device local memory. The memory arbiter is a gatekeeper for multiple internal blocks; several portions of the architecture may simultaneously request internal memory. The PCI read FIFO provides a small buffer for the data read from host memory while the Read DMA engine requests internal memory via the memory arbiter. The data is moved out of the read DMA FIFO into device local memory once a memory data path is available. The FIFO isolates the PCI clock domain from the device clock domain. This reduces latency internally and externally on the PCI bus. The PCIe Read DMA FIFO holds 1024 bytes. The operation of the read DMA FIFO is transparent to host software. The Read DMA engine makes sure there is enough space in internal Tx Packet Buffer Memory before initiating a DMA request for transfer of Tx packet data from host memory to device internal packet memory.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 57
Page 58
DMA WriteBCM5718 Programmer’s Guide
text
RX
PCS
RX
RMII
RX
GMII
RX IO
RX
MAC
Frame
Mod
WOL
Filter
Rx
FIFO
Power
Management
Frame Header #1
Packet Data #1
Frame
Cracker
Checksum
Calculation
Rules
Checker
Statistics
BD Packet #1
NIC
BD Memory
NIC
BufferMemory

Buffer Manager

DMA
Packet Data #1
BD Packet #1
Host Receive Buffer
Descriptor Ring
Host Receive Buffer
Memory
Write
FIFO
Buffer Manager
The buffer manager maintains pools of internal memory used by transmit and receive engines. The buffer manager has logic blocks for allocation, free, control, and initialization of internal memory pools. The DMA read engine requests internal memory for BDs and frame data. Figure 5 on page 57 shows the transmit data path using the DMA Read Engine. The read DMA engine also fetches Rx BDs for the receive data path.

DMA Write

Write Engine

The DMA write engine, as shown in Figure 6, activates when a host write is initiated by the send or receive data paths.
Figure 6: DMA Write Engine
The DMA write engine dequeues an internal request and performs the following functions:
• Gathers the data from device internal memory into the write DMA FIFO
• DMAs the data to the host memory from the write FIFO
• Performs byte and word swapping
• Interrupts the host using a line or message signaled interrupt

Write FIFO

The write FIFO provides elasticity during data movement from device memory to the host memory. The write FIFO absorbs small delays created by PCIe bus arbitration. The NetXtreme family uses the write FIFO to buffer data, so internal memory arbitration is efficient. Additionally, the FIFO isolates the PCI clock domain from the device’s clock domain. This reduces latency on the PCI bus during the write operation (wait states are not inserted while data is fetched from internal memory). The operation of the write DMA FIFO is transparent to host software.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 58
Page 59
LED ControlBCM5718 Programmer’s Guide

Buffer Manager

The buffer manager maintains pools of internal memory used in transmit and receive functions. The buffer manager has logic blocks for allocation, free, control, and initialization of internal memory pools. The receive MAC requests NIC Rx Mbuf memory so inbound frames can be buffered. The read DMA engine requests the device Tx Mbuf memory for buffering the packets from host memory before they are sent out on the wire. The DMA write engine requests a small amount of internal memory for DMA and interrupt operations. The usage of this internal memory is transparent to host software, and does not affect device/system performance.

LED Control

Refer to section “LED Control” in the applicable data sheet.

Memory Arbiter

The Memory Arbiter (MA) is a gatekeeper for internal memory access. The MA is responsible for decoding the internal memory addresses that correspond to Ethernet controller data structures and control maps. If a functional block faults or traps during access to internal memory, the MA handles the failing condition and reports the error in a status register. In addition to architectural blocks, the MA provides a gateway for the RISC processor to access local memory. The RISC has an MA interface that pipelines up to three access requests. The MA negotiates local memory access, so all portions of the architecture are provided with fair access to memory resources. The MA prevents starvation and bounds access latency. Host software may enable/disable/ reset the MA, and there are no tunable parameters.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 59
Page 60
Host CoalescingBCM5718 Programmer’s Guide
...
DMA
Write
Engine
Status Block
PCIe
Interface
Buffer
Manager
Host
Coalescing
Engine
MSI Mailbox
I/O
Driver
Host
Interrupt
Controller
IRQ
Write
FIFO
Tick
CounterBDCounter
Status
Memory
Host software may configure
line IRQ or MSI
MSI
FIFO

Host Coalescing

Host Coalesci ng Engine

The Host Coalescing Engine is responsible for pacing the rate at which the NIC updates the send and receive ring indices located in host memory space. The completion of a NIC update is reflected through an interrupt on the Ethernet controller INTA separately, all updates occur at once. This is because all of the ring indices are in one status block, and any host update updates all ring indices simultaneously. The Host Coalescing Engine triggers based on a tick and/or a frame counter.
pin or a Message Signaled Interrupt (MSI). Although update criteria are calculated
Figure 7: Host Coalescing Engine
Broadcom®
January 29, 2016 • 5718-PG108-R Page 60
Page 61
Host CoalescingBCM5718 Programmer’s Guide
A host update occurs whenever one of the following criteria is met:
• The number of BDs consumed for frames received, without updating receive indices on the host, is equal to
or has exceeded the threshold set in the Receive_Max_Coalesced_BD register (see “Receive Max
Coalesced Bd Count Register (Offset: 0x3c10)” on page 254).
• The number of BDs consumed for transmitting frames, without updating the send indices, on the host is equal to or has exceeded the threshold set in the Send_Max_Coalesced_BD register (see “Send Max
Coalesced BD Count Register (Offset: 0x3c14)” on page 254). Updates can occur when the number of BDs
(not frames) meets the thresholds set in the various coalescing registers (see Section 11: “Interrupt
Processing,” on page 229 for more information).
• The receive coalescing timer has expired, and new frames have been received on any of the receive rings, and a host update has not occurred. The receive coalescing timer is then reset to the value in the
Receive_Coalescing_Ticks register (see “Send Coalescing Ticks Register (Offset: 0x3c0c)” on page 254).
• The send coalescing timer has expired, and new frames have been consumed from any send ring, and a host update has not occurred. The send coalescing timer is then reset to the value in the Send_Coalescing_Ticks register.

MSI FIFO

This FIFO is eight entries deep and four bits wide. This FIFO is used to send MSIs via the PCI interface. The host coalescing engine uses this FIFO to enqueue requests for the generation of MSI. There are no configurable options for this FIFO and this FIFOs operation is completely transparent to host software.

Status Block

This data structure contains consumer and producer indices/values. Host software reads this control block, to assess hardware updates in the send and receive rings. Two copies of the status block exist. The local copy is DMAed to host memory by the DMA write engine. Host software does not want to generate PCI transactions to read ring status; rather quicker memory bus transactions are desired. The host coalescing engine enqueues a request to the DMA write engine, so host software gets a refreshed copy of status. The status block is refreshed before a line IRQ or MSI is generated. See “Status Block Format” on page 82 for a complete discussion of the status block.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 61
Page 62
10BT/100BTx/1000BASE-T TransceiverBCM5718 Programmer’s Guide

10BT/100BTx/1000BASE-T Transceiver

Auto-Negotiation

The Ethernet controller devices negotiate their mode of operation over the twisted-pair link using the auto­negotiation mechanism defined in the IEEE 802.3u and IEEE 802.3ab specifications. Auto-negotiation can be enabled or disabled by hardware or software control. When the auto-negotiation function is enabled, the Ethernet controllers automatically choose the mode of operation by advertising its abilities and comparing them with those received from its link partner. The Ethernet controllers can be configured to advertise 1000BASE-T full-duplex and/or half-duplex, 100BASE-TX full-duplex and/or half-duplex, and 10BASE-T full-duplex and/or half-duplex. The transceiver negotiates with its link partner and chooses the highest operating speed and duplex that are common between them. Auto-negotiation can be disabled for testing or for forcing 100BASE-TX or 10BASE-T operation, but is always required for normal 1000BASE-T operation.

Automatic MDI Crossover

During auto-negotiation, one end of the link must perform an MDI crossover so that each transceiver’s transmitter is connected to the other receiver. The Ethernet controllers can perform an automatic MDI crossover when the Disable Automatic MDI Crossover bit in the PHY Extended Control register is disabled, thus eliminating the need for crossover cables or cross-wired (MDIX) ports. During auto-negotiation, the Ethernet controllers normally transmit on TRD{0} and receive on TRD{1}. When connected to another device that does not perform the MDI crossover, the Ethernet controller automatically switches its transmitter to TRD{1} and its receiver to TRD{0} to communicate with the remote device. If two devices that both have MDI crossover capability are connected, an algorithm determines which end performs the crossover function. During 1000BASE-T operation, the Ethernet controllers swap the transmit symbols on pairs 0 and 1 and pairs 2 and 3 if auto-negotiation completes in the MDI crossover state. The 1000BASE-T receiver automatically detects pair swaps on the receive inputs and aligns the symbols properly within the decoder.

PHY Control

The NetXtreme/NetLink Ethernet controller supports the following physical layer interfaces:
• The MII is used in conjunction with 10/100 Mbps copper Ethernet transceivers.
• GMII supports 1000 Mbps copper Ethernet transceivers.

MII Block

The MII interconnects the MAC and PHY sublayers (as shown in Figure 8 on page 63).
Broadcom®
January 29, 2016 • 5718-PG108-R Page 62
Page 63
Figure 8: Media Independent Interface
RX
I/O
RXD /4
RX_CLK1
RX_ER
RX_DV
TX
I/O
TXD /4
MII_TXCLK
TX_ER TX_EN
Media
Status
I/O
COL CRS
LNKRDY
RX Media
Access
Mgmnt
RX
MAC
Rx Data
Decapsulation
TX Media Access
Mgmnt
TX
MAC
Tx Data
Encapsulation
MAC Sublayer
Physical Layer
RX
I/O
Symbol
Decoder
TX
I/O
Symbol
Encoder
4-bit Data Path
2.5 MHz at 10 Mbps
25 MHz at 100 Mbps
4-bit Data Path
LED
Control
LED
I/O
2.5 MHz at 10 Mbps
25 MHz at 100 Mbps
MII
PHY ControlBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 63
Page 64
PHY ControlBCM5718 Programmer’s Guide
The specifics of MII may be located in section 22 of the IEEE 802.3 specification. RXD[3:0] are the receive data signals; TXD[3:0] are the transmit data signals. MII operates at both 10-Mbps and 100-Mbps wire-speeds. (Gigabit Ethernet uses the GMII standard.) When MAC and PHY are configured for 10 Mbps operation, the RX_CLK1 and MII_TXCLK clocks run at 2.5 MHz. Both RX_CLK1 and MII_TXCLK are sourced by the PHY. 100 Mbps wire speed requires RX_CLK1 and MII_TXCLK to provide a 25 MHz reference clock. Receive Data Valid (RX_DV) is asserted when valid frame data is received; at any point during data reception, the PHY may assert Receive Error (RX_ER) to indicate a receive error. The MAC will record this error in the statistics block. The MAC may discard a bad RX frame—exception being sniffer/promiscuous modes (see Allow_Bad_Frames bit in MAC mode register). The Transmit Enable (TX_EN) signal is asserted when the MAC presents the PHY with a valid frame for transmission. The MAC may assert TX_ER to indicate the remaining portion of frame is bad. The PHY will insert Bad Code symbols into the remaining portion of the frame. A detected collision in half-duplex mode may be such a scenario where TX_ER is asserted. The PHY will assert COL when a collision is detected. The COL signal is routed to both the RX and TX MACs. The transmit MAC will back off transmission and the RX MAC will throw away partial frames.
The 10 Mbps physical layer uses Differential Manchester encoding on the wire. Manchester encoding uses two encoding levels: 0 and 1. 100 Mbps Ethernet requires MLT-3 waveshaping on the transmission media. MLT-3 uses three encoding levels: – 1, 0, and 1. Both physical signaling protocols are transparent to the MAC sublayer and are digitized by the PHY. The PHY encodes/decodes analog waveforms at its lower edge while the PHY presents digital data at its upper edge (MII).

GMII Block

The GMII is full-duplex (see Figure 9 on page 65); the send and receive data paths operate independently.
The transmit signals TXD[7:0] create a eight-bit wide data path. The TXD[7:0] signals are synchronized to the reference clockTX_CLK0. The TX_CLK0 clock runs at 125 MHz and is sourced by the MAC sublayer. Transmit Error (TX_ER) is asserted by the MAC sublayer. The PHY will transmit a bad code until TX_ER is deasserted by the MAC. TX_ER is driven synchronously with TX_CLK0. The Transmit Enable (TX_EN) indicates that valid data is presented on the TXD lines. The TXD[7:0] data is framed on the rising edge of TX_EN.
The receive data path is also eight bits wide. RXD[7:0] are sourced by the PHY. When valid data is presented to the MAC sublayer, the PHY will also assert Receive Data Valid (RX_DV). The rising edge of RX_DV indicates the beginning of a frame sequence. The PHY drives the reference clock RX_CLK1, so the MAC sublayer can synchronize data sampling on RXD[7:0]. The PHY may assert RX_ER to indicate frame data is invalid; the MAC sublayer must consider frames in progress incomplete.
When the MAC operates in half-duplex mode, a switch or node may transmit a jamming pattern. The PHY will drive the Collision (COL) signal so the MAC may back off transmission and throw away partially received packet(s). The COL signal will also cause the TX MAC to stop the transmission of a packet. The COL signal is not driven for full-duplex operation since collisions are undefined. The PHY will drive Carrier Sense (CRS) as a response to traffic being sent/received. The MAC sublayer can monitor traffic and subsequently drive traffic LEDs.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 64
Page 65
PHY ControlBCM5718 Programmer’s Guide
RX
I/O
RXD /8
RX_CLK1
RX_ER
RX_DV
TX
I/O
TXD /8
TX_CLK0
TX_ER
TX_EN
Media Status
I/O
COL
CRS
LNKRDY
RX Media
Access
Mgmnt
RX
MAC
Rx Data
Decapsulation
TX Media
Access
Mgmnt
TX
MAC
Tx Data
Encapsulation
MAC Sublayer
Physical Layer
RX I/O
Symbol
Decoder
LED
Control
TX
I/O
Symbol
Encoder
125-MHz Ref Clock 8-bit Data Path
LED
I/O
8-bit Data Path125-MHz Ref Clock
GMII
Pulse Amplitude Modulated Symbol (PAM5) encoding is leveraged for Gigabit Ethernet wire transmissions. PAM5 uses five encoding levels: –2, – 1, 0, 1, and 2. Four symbols are transmitted in parallel on the four twisted­wire pairs. The four symbols create a code group (an eight-bit octet). The process of creating the code-group is called 4D-PAM5. Essentially, eight data bits are represented by four symbols. Table 40-1 in the IEEE 802.3ab specification shows the data bit to symbol mapping. The code group representation is also referred to as a quartet of quinary symbols {TA, TB, TC, TD}. The modulation rate on the wire is measured at 125 Mbaud. The resultant bandwidth is calculated by multiplying 125 MHz by eight bits, for 1000 Mbps wire speed.
Figure 9: GMII Block
Broadcom®
January 29, 2016 • 5718-PG108-R Page 65
Page 66

MDIO Register Interface

MAC Sublayer
Mgmnt
I/O
Mgmnt
Control
(MII & GMII)
Physical Layer
Mgmnt
I/O
Mgmnt
Control
MDC
MDIO
MDINT
MDI
Register
block
Figure 10 shows the MDI register interface.
Figure 10: MDI Register Interface
PHY ControlBCM5718 Programmer’s Guide
Management Data Clock
The Management Data Clock (MDC) is driven by the MAC sublayer. The PHY will sink this signal to synchronize data transfer on the MDIO signal—MDC is a reference clock. This clock is not functionally associated to either RX_CLK or TX_CLK. The minimum period for this clock is 400 ns with high and low times having 160 ns duration.
Management Data Inpu t/Output
The Management Data Input/Output (MDIO) signal passes control and status data, between the MAC and PHY sublayers. MDIO is a bidirectional signal, meaning both the PHY and MAC may transfer data. The MAC typically transfers control information and polls status; whereas, the PHY transfers status back to the MAC, using MDIO.
Management Data Interrupt
The integrated Broadcom PHY may be programmed to generate interrupts. A PHY status change initiates a Management Data Interrupt (MDINT). A MDI mask register allows host software to selectively enable/disable status types, which cause MDINT notification. The PHY will assert INTR Reading the status register will clear the interrupt.
until software clears the interrupt.
Management Register Block
The layout and configuration of MDI register block is device dependent. The MDI register block is the control/ status access point, which host software may read/write. The IEEE 802.3 specification defines a basic register block for MII and GMII; the basic register set contains control and status registers. GMII also exposes an extended register set, used in 1000 Mbps configuration/status. The fundamental point is to understand that the MDC and MDIO signals are used to access the MDI register block.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 66
Page 67
NVRAM ConfigurationBCM5718 Programmer’s Guide

Section 3: NVRAM Configuration

Overview

Broadcom NetXtreme and NetLink controllers require the use of an external non-volatile memory (NVRAM) device (Flash or SEEPROM), which contains a boot code program that the controller's on-chip CPU core loads and executes upon release from reset. This external NVRAM device also contains many configuration items that direct the behavior of the controller, enable/disable various management and/or value-add firmware components, etc.
All configuration settings are default-configured in the official release binary image files provided in Broadcom's CD software releases. However, the settings chosen as default by Broadcom may not be what best suits a particular OEM's application, so some settings may need to be changed by the OEM.
The BCM5718 family supports the following boot code styles:
• “Legacy”—Boot code plus configuration options fully contained in an external 8k byte NVRAM device
• “Self-boot”—Uses a fixed internal ROM'd copy of the boot code program (requires only a very small external NVRAM) for the purpose of housing OEM-programmable configuration items (refer to Application Note NetXtreme-AN60x-R “NetXtreme/NetLink NVRAM Configuration Options”).
Note: If using self-boot, the design must use a very small (approximately 256 byte) external NVRAM
to hold configuration items and self-boot code patches.
Refer to Broadcom Application Note NetXtreme-AN40X-R (NetXtreme Application Note) for additional detail regarding self-boot NVRAM structure.
Details relating to the legacy style NVRAM organization can be found in NetXtreme/NetLink NVRAM Access
Broadcom application note (NetXtreme-AN50X-R). Some of the topics addressed by this application note include the following:
• Programming NVRAM (sample C code, x86 assembly)
• NVRAM map
• Configuration settings
• Boot code
• Multiple boot agent (MBA), PXE, etc.
Note: NVRAM CRC-32: There are multiple distinct regions contained within the NVRAM map. Each
of these regions has its own CRC-32 checksum value associated with it. If any data element contained within a region is modified, then that region's CRC-32 value must also be updated. Details relating to
calculating the CRC-32 can be found in Calculating CRC32 Checksums for Broadcom NetLink,
NetXtreme, and NetXtreme II
AN20X-R).
®
Controllers Broadcom application note (NetXtreme_NetXtremeII-
®
/NetLink™ Software Self-Boot NVRAM
Broadcom®
January 29, 2016 • 5718-PG108-R Page 67
Page 68
Self-BootBCM5718 Programmer’s Guide

Self-Boot

Some NetLink controllers offer a capability known as self-boot. Self-boot allows the controller to use a very small, low-cost, external NVRAM device that contains only a very condensed amount of configuration information, along with any small boot code patches that may be necessary to optimize the functionality of a particular controller.
Details relating to self-boot can be found in Self Boot Option (5754X_5787X-AN10X-R) and NetXtreme/NetLink Software Self-Boot NVRAM (NetXtreme-AN40X-R) Broadcom application notes.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 68
Page 69
Common Data StructuresBCM5718 Programmer’s Guide

Section 4: Common Data Structures

Theory of Operation

Several device data structures are common to the receive, transmit, and interrupt processing routines. These data structures are hardware-related and are used by device drivers to read and update state information.

Descriptor Rings

In order to send and receive packets, the host and the controller use a series of shared buffer descriptor (BD) rings to communicate information back and forth. Each ring is composed of an array of buffer descriptors that reside in host memory. These buffer descriptors point to either send or receive packet data buffers. The largest amount of data that a single buffer may contain is 65535 (64K-1) bytes (The length field in BD is 16 bits). Multiple descriptors can be used per packet in order to achieve scatter-gather DMA capabilities.
Note: The maximum number of Send BDs for a single packet is (0.75)*(ring size).
There are three main types of descriptor rings:
• Send Rings
• Receive Producer Rings
• Receive Return Rings
The TX/RX ring base requires an 8 byte alignment. The receive buffer address (recorded in SBD/RBD) cannot cross 4G.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 69
Page 70
Descriptor RingsBCM5718 Programmer’s Guide
The drawing shows a generic host descriptor ring (could be either a send ring or a receive ring), and demonstrates how the consumer and producer indices are used to determine which descriptors in the ring are vali d at an y given moment in ti m e.
1st
Cons
Prod
The delta between the producer and
consumer indices is indicated by the
shaded areas. These shaded
descriptors are considered to be valid
(non-empty) and thus need to be
processed.

Producer and Consumer Indices

The Producer Index and the Consumer Index control which descriptors are valid for a given ring. Each ring will have its own separate Producer and Consumer Indices. When incremented, the Producer Index can be used to add elements to the ring. Conversely, when incremented, the Consumer Index is used to remove elements from the ring. The difference between the Producer and Consumer Indices mark which descriptors are currently valid in the ring (see Figure 11). When the Producer and Consumer Index are equal, the ring is empty. When the producer is one behind the consumer, the ring is considered to be full.
Figure 11: Generic Ring Diagram
Broadcom®
January 29, 2016 • 5718-PG108-R Page 70
Page 71
Descriptor RingsBCM5718 Programmer’s Guide

Ring Control Blocks

Each ring (send or receive) has a Ring Control Block (RCB) associated with it. Each RCB has the format shown in Ta b le 4 .
Table 4: Ring Control Block Format
Offset (bytes) 31 16 15 0
0x00 Host Ring Address
0x04
0x08 Max_len Flags
0x0c Reserved
The fields are defined as follows:
• The Host Ring Address field contains the 64-bit host address of the first element in the ring. Basically, this field tells the controller precisely where in host memory the ring is located. This field only applies to rings that are located in host memory. The Host_Ring_Address field contains the 64-bit address, in big-endian ordering, of the first Send BD in host memory.
• The Flags field contains bits flags that contain control information about a given ring. Ta bl e 5 shows the two flags that are defined.
Table 5: Flag Fields for a Ring
Bits Name Description
0 Reserved Should be set to 0.
1 RCB_FLAG_RING_DISABLED Indicates that the ring is not in use.
15:2 STD_MAX_PACKET_SIZE Indicates maximum frame size for the ring.
• The Max_len field has a different meaning for different types of rings.
– This field indicates the number elements in the ring.
– The valid values for this field are 32, 64, 128, 256, 512, 1024, 2048, and 4096.
• Max ring sizes supported in the BCM5718 Family are shown below.
– Rx Return: 4096
– Rx Producer: 2048
– Rx Producer Jumbo: 1024
– Tx Producer: 512
• The NIC Ring Address field contains the address where the BD cache is located in the internal NIC address space. This address is only valid for Receive Producer Rings. The Send Rings and Receive Return Rings do not require this field to be populated. The location within the NIC address map for Receive Producer Ring is provided in “PCI” on page 168.
Send Ring Control Blocks
The format of the Send RCB remains unchanged, only 15 more are added as shown in the Table below.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 71
Page 72
Table 6: Send RCBs for Multiple Rings
Send Ring RCB
Send Ring# Register Name
Host Address High 0x100 Legacy Mode, Legacy / 1
2
3
4
5
…. ……… ……
16 Host Address High 0x1F0
** These are Memory Mapped registers
Host Address Low 0x104
Max Length/Flag 0x108
NIC Address 0x10C
Host Address High 0x110
Host Address Low 0x114
Max Length/Flag 0x118
NIC Address 0x11C
Host Address Low 0x1F4
Max Length/Flag 0x1F8
NIC Address 0x1FC
Register Address (**)
Descriptor RingsBCM5718 Programmer’s Guide
Usable in
RSS Mode & IOV Mode
RSS Mode & IOV Mode
IOV Mode Only
Note: Address range [0x100–0x10F] is the legacy single RCB address and is also used by RCB1.
Receive Ring Control Blocks
Ta bl e 7 lists all of the receive RCB Register Addresses.
Table 7: High Priority Mail Box Registers for VRQ Rings
Standard Ring RCB
VRQ # Register Name
Host Address High 0x2450 0x2440 0x200
0 (Default)
1
Host Address Low 0x2454 0x2444 0x204
Max Length/Flag 0x2458 0x2448 0x208
NIC Address 0x245C 0x244C 0x20C
Host Address High 0x2510 0x2500 0x210
Host Address Low 0x2514 0x2504 0x214
Max Length/Flag 0x2518 0x2508 0x218
NIC Address 0x251C 0x250C 0x21C
Register Address
Jumbo Ring RCB Register Address
Return Ring RCB Register Address (**)
Broadcom®
January 29, 2016 • 5718-PG108-R Page 72
Page 73
Table 7: High Priority Mail Box Registers for VRQ Rings (Cont.)
Descriptor RingsBCM5718 Programmer’s Guide
Standard Ring RCB
VRQ # Register Name
Host Address High 0x2530 0x2520 0x220
2
3
4 Host Address High 0x2570 0x2560 0x240
..... ..... ..... ..... .....
16
Host Address Low 0x2534 0x2524 0x224
Max Length/Flag 0x2538 0x2528 0x228
NIC Address 0x253C 0x252C 0x22C
Host Address High 0x2550 0x2540 0x230
Host Address Low 0x2554 0x2544 0x234
Max Length/Flag 0x2558 0x2548 0x238
NIC Address 0x255C 0x254C 0x23C
Host Address High 0x26F0 0x26E0 0x300
Host Address Low 0x26F4 0x26E4 0x304
Max Length/Flag 0x26F8 0x26E8 0x308
NIC Address 0x26FC 0x26EC 0x30C
Note: [0x2450 .. 0x245C] and [0x2440 .. 0x244C] are legacy RCB addresses and are being assigned
to VRQ Ring# 0.
Register Address
Jumbo Ring RCB Register Address
Return Ring RCB Register Address (**)
Note: The Return Ring RCBs are Memory Mapped. Memory Address [0x200 .. 0x23C] are legacy
Return Ring RCB addresses and are being assigned to VRQ Return Ring# 0 through Ring# 3

Send Rings

The controller devices covered in this document support only one host based Send Ring.
The Send Ring Producer Index is incremented by host software to add descriptors to the Send Ring (see
Figure 12 on page 74). By adding descriptors to the ring, the device is instructed to transmit packets that are
composed of the buffers pointed to by the descriptors. A single transmit packet may be composed of multiple buffers that are pointed to by multiple send descriptors. The maximum number of send descriptors for a single packet is (0.75)*(ring size).
Broadcom®
January 29, 2016 • 5718-PG108-R Page 73
Page 74
Figure 12: Transmit Ring Data Structure Architecture Diagram
Host Memory
1-(64K-1) Bytes
Host Buffer
Send Host BD
Ring Control Block
1st
Host Ring Address
max_len
NIC Ring Address
flags
Host Send Ring #1
Cons
Prod
RCB
Status Word
unused
RX Prod #1
RX std cons
RX Prod #2
Unused
Status Block
Unused Unused
TX Cons
RX Prod #4
Mailbox Registers
TX Host Ring Prod
Status Block (80 bytes)
The Status block resides in the NIC memory space and is periodic ally DMA'd to the host when the TX/RX coalescing timers expire, or wh en the RX/TX max coalesced frames thresholds are met. Sof twa re can examine the TX consumer indices in the status block to determine which packets have been sent by the hardware.
The mailbox registers reside on-chip starting at offs e t 0x 300. Each mailbox register is 64 bits wide. Writing the lower 32 bits triggers an event in the HW. SW updates the TX Host Ring producer index to indicate that there are buffer descriptors ready for the HW to process.
Host Address
length
rsvd for firmware
Send Buffer Descriptor
flags
VLAN tag
Data Structures in the host
Data Structures kept on-chip
Transmit Ring Data Scr ucture is located in the host (as shown below), and the de v ice will keep a local (not shown) copy of the rings.
RX Prod #3
Descriptor RingsBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 74
Page 75
Descriptor RingsBCM5718 Programmer’s Guide
Send Buffer Descriptors
Standard (Not La rge Segment Offload)
The format of an individual send buffer descriptor is shown in Ta bl e 8 .
Table 8: Send Buffe r Descriptors Format
Offset (Bytes) 31 16 15 0
0x00 Host Address [63:0]
0x04
0x08 Length [15:0] Flags [15:0]
0x0c Reserved VLAN Tag
The fields are defined as follows:
• The Host Address field contains the 64-bit host address of the buffer that the descriptor points to. A length
of 0 indicates that the descriptor does not have a buffer associated with it.
• The Flags field contains bits flags that contain control information for the device for transmitting the
packets. The defined flags are listed in Ta b le 9 .
Table 9: Defined Flags for Send Buffer Descriptors
Bits Name Description
a
0
TCP_UDP_CKSUM
If set to 1, the controller replaces the TCP/UDP checksum field of TCP/UDP packets with the hardware calculated TCP/UDP checksum for the packet associated with this descriptor.
1 IP_CKSUM If set to 1, the controller replaces the IPv4 checksum field of TCP/UDP packets
over IPv4 with the hardware calculated IPv4 checksum for the packet associated with this descriptor. This bit should only be set in the descriptor that points to the buffer containing the IPv4 header. It is assumed that the IPv4 header is contained in a single buffer.
2 PACKET_END If set to 1, the packet ends with the data in the buffer pointed to by this descriptor.
3 Jumbo Frame The driver must set this bit to 1 if the MTU length of the Send Frame is > 1500B.
The MTU length is the Ethernet payload length and excludes Header length (and Trailer length).
All BDs belonging to a Send Packet must configure this bit identically.
4 HDRLEN[2] The length of the Ether+IP+TCP Headers to be replicated in each segment
arising out of a Large TCP Segment (LSO).
5 Capture Time Stamp (BCM5719/5720 only) If this bit is 1, this frame’s launch time shall be captured
in the TX Time-Stamp Register.
6
VLAN_TAG
a
If set to 1, the device inserts an IEEE 802.1Q VLAN tag into the packet. The 16­bit TCI (Tag Control Information) field of four byte VLAN tag comes from the VLAN Tag field in the descriptor.
7 COAL_NOW If set to 1, the device immediately updates the Send Consumer Index after the
buffer associated with this descriptor has been transferred via DMA to NIC memory from host memory. An interrupt may or may not be generated according to the state of the interrupt avoidance mechanisms. If this bit is set to 0, then the Consumer Index is only updated as soon as one of the host interrupt coalescing conditions has been met.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 75
Page 76
Descriptor RingsBCM5718 Programmer’s Guide
Table 9: Defined Flags for Send Buffer Descriptors (Cont.)
Bits Name Description
8 CPU_PRE_DMA If set to 1, the controller’s internal CPU is required to act upon the packet before
the packet is given to the internal Send Data Initiator state machine. Normally
this bit should be set to 0.
9
CPU_POST_DMA
a
If set to 1, the controller’s internal CPU is required to act upon the packet before
the packet is given to the internal Send Data Completion state machine.
Normally this bit should be set to 0.
10 HDRLEN[3] The length of the Ether+IP+TCP Headers (combined) to be replicated in each
frame arising out of a Large TCP Segment (LSO). Maximum Header Length is 256B.
11 HDRLEN[4] –
12 HDRLEN[5] –
13 HDRLEN[6] –
14 HDRLEN[7] –
15
DON’T_GEN_CRC
a. Indicates that this bit should be set in all descriptors for a given packet if the desired capability is to be
enabled for that packet.
a
If set to 1, the controller will not append an Ethernet CRC to the end of the frame.
Note: The UDP checksum engine does not span IP fragmented frames.
• The Length field specifies the length of the data buffer. The lengths for the buffers associated with a given
packet will add up to the length of the packet.
Note: The Ethernet controller does not validate the value of the Length field and may generate an
error on the PCI bus if the Length field has a value of 0. The host driver must ensure that the Length field is nonzero before enqueueing the BD onto the Send Ring.
• The VLAN Tag field is only valid when the VLAN_TAG bit of Flags field is set. This VLAN Tag field contains
the 16-bit VLAN tag that is to be inserted into an IEEE 802.1Q (and IEEE 802.3ac)-compliant packet by the controller. If VLAN tag insertion is desired, this field (and the flag) should be set in the first descriptor for that packet (i.e., the descriptor that points to the buffer that contains the Ethernet header).
Large Segment Offload (LSO) Send BD
See “Large Segment Offload” on page 110.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 76
Page 77
Descriptor RingsBCM5718 Programmer’s Guide
Host Memory
1-(64K-1) Byt es
Host Buffer
RX BD
Host Address
index type
ip chksum
error flag
reserved opaque
Receive Buffer Descriptor
len
flags
tcp_udp_chsum
vlan tag
Ring Control Block
1st
Host Ring Address
max_len
NIC Ring Address
flags
Receive Ring #1
Receive Ring #4
1st
Prod
Cons
RCB #1
RCB #4
Status Word
unused
RX Prod #1
RX std cons
RX Prod #2
Unused
Status Block
Unused Unused
TX Cons
RX Prod #4
Con
Prod
Mailbox Registers
RX Cons #1 RX Cons #2
RX Cons #4
Note: The receive return rings are stored in host memory.
Status Block (80 bytes)
Status block resides in the NIC memory space and is periodically DMA'd to the host based on the host coalescing Timer.
The NIC is the Producer of the receive return ring. It increments the internal producer index to add elements to the ring.
RX Prod #3
RX Cons #4

Receive Rings

The Ethernet controllers support two types of Receive Descriptor Rings: Producer Rings and Return Rings (see
Figure 13). Unlike previous NetXtreme controllers, the BCM5718 family adds a second Receive Producer ring
dedicated to jumbo frame reception. Descriptors in the Producer Rings point to free buffers in the host. When the controller receives a packet and consumes a receive buffer, the controller will modify and write back the descriptor for the consumed buffer into the given Receive Return Ring. Basically the Producer Rings contain descriptors that point to buffers that the controller is free to use, whereas the Return Rings contain descriptors that the device has used and await processing from host software.
Figure 13: Receive Return Ring Memory Architecture Diagram
Receive Producer Ring
The receive producer ring resides in the host and points to empty host receive buffers that will later be filled with received packet data. The controller will internally cache a copy of the producer ring. When the host software driver has a free host receive packet buffer available for incoming packets, it will fill out a receive buffer descriptor and have that descriptor point to the available buffer. Host software will then update the producer index for that receive producer ring to indicate to the controller that there is a newly available receive buffer. After the controller fetches and caches (e.g., consumes) this receive producer descriptor, the controller will update the consumer index of the receive producer ring.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 77
Page 78
Descriptor RingsBCM5718 Programmer’s Guide
Receive Return Rings
When the NIC receives a packet, it will DMA that packet to a host receive packet data buffer pointed to by the available receive buffer descriptor (see Section 5: “Receive Data Flow,” on page 88). Earlier the NIC will have received ownership of that data buffer via an update of the producer index of receive producer ring. After the controller does the packet data write DMA, it will DMA a corresponding buffer descriptor into the appropriate receive return ring. The buffer descriptor that is returned in the receive return ring will be slightly modified from the original buffer descriptor that the controller fetched out of the receive producer ring. After the controller has completed the DMA of the receive return ring descriptor, the controller will update its internal copy of the producer index for that particular receive return ring. That new value for that receive return ring producer index will be included in the next status block update that is made to the host. The updated value of receive return ring producer index in status block will be used by host software in determining whether new packets have been received.
Table 10: Receive Return Rings
Description BCM5718 Family
Number of Rings 4
Buffer Descriptor Size (bytes) 32
Host Ring Size (# of Buffer Descriptors) Can be configured for 32 or 64 or 128 or 256 or 512 or
1024 or 208 or 4096
NIC Cache Size (# of Buffer Descriptors) 0
Receive Buffer Descriptors
The format of Standard Receive Buffer Descriptors (in both producer ring and return rings) is shown in Table 11.
Table 11: Receive Descriptors Format
Offset (bytes) 3116 150
0x00 Host Address
0x04
0x08 Index Length
0x0c Type Flags
0x10 IP_Cksum TCP_UDP_Cksum
0x14 Error_Flags VLAN tag
0x18 RSS Hash
0x1C Opaque
The fields are defined as follows:
• Host Address—Contains the 64-bit host address of the buffer that the descriptor points to. A length of 0
indicates that the descriptor does not have a buffer associated with it.
• Length— Specifies the length of the data buffer. For Producer Rings this value is set by the host software to
correspond to the size of the buffer that is available for a receive packet. Once a packet has been received, the controller will modify this length field to correspond to the length of the packet received. A value of 0 indicates that there is no valid data in the buffer.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 78
Page 79
Descriptor RingsBCM5718 Programmer’s Guide
• Index— Is set by host software in the descriptors in the producer rings. When the controller uses a given
buffer descriptor, it will opaquely pass the Index field for that buffer descriptor through to the corresponding descriptor in the return ring. This index field of the BD in Return Ring is then used by the host software to associate the BD in Return Ring with the BD in Producer Ring that points to the given receive buffer.
• Flags— Contains bits flags that contain control information about a given descriptor. The defined flags are
listed in Table 13 on page 80.
Table 12: Defined Flags for Receive Buffers
Bits Name Description
15 IP Version Indicates whether the received IP packet is an IPv6 or IPv4 packet. This bit
will be 1 for IPv6 packet and 0 for IPv4 packet.
14 TCP_UDP_IS_TCP In producer rings this bit should be set to 0. In return rings this bit will be set
to 1 by the controller if the controller calculated that the incoming packet was a TCP packet. If the packet is UDP or a non IP frame, then this bit should be set to 0.
13 TCP_UDP_CHECKSUM In producer rings this bit should be set to 0. In return rings this bit will be set
to 1 by the controller if the controller calculated that the TCP or UDP checksum in the corresponding incoming packet was correct.
12 IP_CHECKSUM In producer rings this bit should be set to 0. In return rings this bit will be set
to 1 by the controller if the controller calculated that the IP checksum in the corresponding incoming packet was correct.
11 Re s e r v ed –
10 FRAME_HAS_ERROR If set to 1 in a return ring, it indicates that the controller detected an error.
The specific type of error is specified in the Error_Flag field of the receive return descriptor.
9:7 RSS Hash Type Indicates the hash type used in RSS hash calculation for a received packet.
Available hash types are:
• 0 2_TUPLE_IPV4
• 1 4_TUPLE_IPV4
• 2 2_TUPLE_IPV6
• 3 4_TUPLE_IPV6
•4 Reserved
•5 Reserved
•6 Reserved
•7 Reserved
See “Receive MAC Mode Register (offset: 0x468)” on page 322 for additional information about enabling the different RSS hash types.
6 VLAN_TAG* If set to 1 in a return ring, it indicates that the packet associated with this
buffer contained a four-byte IEEE 802.1Q VLAN tag. The 16 VLAN ID is stripped from the packet and located in the VLAN tag field in the descriptor.
5 Reserved Should be set to 0.
4 Reserved Should be set to 0.
3 RSS_Hash Valid If set to 1, indicates host that the RSS_Hash in Receive BD of return ring is
valid.
2 PACKET_END If set to 1, the packet ends with the data in the buffer pointed to by this
descriptor.
1:0 Reserved Should be set to 0.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 79
Page 80
Descriptor RingsBCM5718 Programmer’s Guide
• Type— Used internally by the controller. In producer rings it should be set to 0, and in return rings it should
be ignored by host software.
• TCP_UDP_Cksum — Holds the TCP/UDP checksum that the controller calculated for all data following the
IP header given the length defined in the IP header. If the Receive No Pseudo-header Checksum bit is set (see “Mode Control Register (offset: 0x6800)” on page 468) to 1, then the pseudo-header checksum value is not added to this value. Otherwise, the TCP_UDP_Cksum field includes the pseudo-header in the controller’s calculation of the TCP or UDP checksum. If the packet is not a TCP or UDP packet, this field has no meaning. Host software should zero this value in the producer ring descriptors. If the host is capable of TCP or UDP checksum off load, then host software may examine this field in the return rings to determine if the TCP or UDP checksum was correct.
• IP_Cksum — Host software should zero this value in the producer ring descriptors. For Receive Return
Ring descriptors, if using IP checksum offload, the host driver software should rely on the Flags bit IP_CHECKSUM (Flags bit 12) to determine if the IP checksum in the received packet is correct. This field used to contain the actual IP checksum value but that is not true for the BCM5718 family of controllers. Only the Flags bit IP_CHECKSUM should be relied on by host driver software as is done by Broadcom drivers.
• VLAN— Only valid when the VLAN_TAG bit is set. This field contains the 16-bit VLAN ID that was extracted
from an incoming packet that had an IEEE 802.1Q (and IEEE 802.3ac) -compliant header.
• Error_Flags — Contains bits flags that contain error information about an incoming packet that the
descriptor is associated with. The bits in this field are only valid if the FRAME_HAS_ERROR bit is set in the Flags field in the descriptor. The defined error flags are listed in Ta bl e 1 3.
Table 13: Defined Error Flags for Receive Buffers
Bits Name Description
31:9 Reserved Should be set to 0.
8 GIANT_PKT_RCVD If set to 1, the received packet was longer than the maximum packet length
value set in the Receive MTU Size register (see “Receive MTU Size Register
(offset: 0x43C)” on page 316). The data in the received packet was truncated at
the length specified in the Receive MTU Size register.
7 TRUNC_NO_RES If set to 1, the received packet was truncated because the controller did not
have the resources to receive a packet of this length.
6 LEN_LESS_64 If set to 1, the received packet was less than the required 64 bytes in length.
5 MAC_ABORT If set to 1, the MAC aborted due to an unspecified internal error while receiving
this packet. The packet could be corrupted.
4 ODD_NIBBLE_RX_MIIIf set to 1, the received packet contained an odd number of nibbles. Thus,
packet data could be corrupt.
3 PHY_DECODE_ERR If set to 1, while receiving this packet the device encountered an unspecified
frame decoding error. This packet could be corrupted. This bit is set for valid packets that are received with a dribble nibble. True
alignment errors will be dropped by that MAC and never show up to the driver.
2 LINK_LOST If set to 1, link was lost while receiving this frame. Therefore, this packet is
incomplete.
1 COLL_DETECT If set to 1, a collision was encountered while receiving this packet.
0 BAD_CRC If set to 1, the received packet has a bad Ethernet CRC.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 80
Page 81
Descriptor RingsBCM5718 Programmer’s Guide
• When the RSS Hash Valid flag bit is 1, the RSS Hash field holds the 32-bit RSS hash value calculated for a
packet. This field should be ignored when the RSS Hash Valid flag bit is zero.
• The Opaque field is reserved for the host software driver. Any data placed in this field in a producer ring
descriptor will be passed through unchanged to the corresponding return ring descriptor.
Additional Ring Information for the BCM5718 Family
Rings in the BCM5718 Family are more complicated than in previous NetXtreme controllers. Before considering ring structure details, first choose the mode of operation:
• Legacy
•RSS
• RSS+TSS
•IOV
Note: RSS (and/or TSS) and IOV are mutually exclusive.
Once the mode of operation is decided, determine if jumbo frames are to be used.
Setting ring sizes involves setting some ring sizes (such as the Rx Producer Rings) in registers and setting other ring sizes (such as the Rx Return Rings) in memory locations (for example, the upper 16 bits of the 32 bit value at memory offset 0x208 for Rx Return Ring number 0, rather than in a register). See Appendix C: “Device
Register and Memory Map,” on page 580 for more information. A Ring Control Block (RCB) consists of four 32
bit words:
• host address high
• host address low
• len/flags
• NIC ring address
For example, the RX Return Ring size is set in the high 16 bits of 32bit value in device memory offset 0x208.
If using RSS or IOV, there are either 4 or 17 Rx Return and Producer rings. The 17th Rx ring is a default throw­away ring for any Rx traffic that does not map to rings 0-3 (RSS) or 0-15 (IOV). Driver software only pulls valid Rx BDs from 16 Rx Ret Rings (rings 0-15) in IOV mode.
For Rx Producer Rings, there are only 1 or 2 in non-IOV mode (the second one dedicated to the driver to supply jumbo Rx BDs to the chip, if using jumbo frames). In RSS mode, there are up to 4. In IOV mode, there are up to 16 (17, including the default ring).
The Tx path is simpler. Tx is 1 or 4 or 16 rings (legacy, TSS, or IOV) for giving Send BDs to the chip. Jumbo BDs go onto the same Send ring as non-jumbos. There are no separate jumbo-dedicated Send ring(s) like there are for Rx producer rings.
The max sizes of the various rings is as follow:
• Rx Ret: 4096
• Rx Prod: 2048
Broadcom®
January 29, 2016 • 5718-PG108-R Page 81
Page 82
Status BlockBCM5718 Programmer’s Guide
• Rx Prod Jumbo: 1024
• Tx Prod: 512
As a maximum use example, there can be an IOV implementation with the following total max ring size/count arrangement:
• 16 Send rings (size=512 each ring)
• 16 Rx Prod rings (size=2048 each ring)
• 16 Rx Prod jumbo rings (size=1024 each ring)
• 17 Rx Ret rings (size=4096 each ring)

Status Block

The Status Block is another shared memory data structure that is located in host memory. The Status Block is 32 bytes in length. Host software will need to allocate 32 bytes of non-paged memory space for the Status Block and set the Status Block Host Address register to point to the host memory physical address reserved for this structure.
The controller will update the Status Block to host memory prior to a host coalescing interrupt or MSI. The frequency of these Status Block updates is determined by the host coalescing logic (see “Host Coalescing
Engine” on page 60). Using the software configurable coalescing parameters, the device driver can optimize the
frequency of status block updates for a particular application or operating system.
The Status Block contains some of the Producer and Consumer indices for the rings described in “Descriptor
Rings” on page 69. These Producer and Consumer indices allow host software to know what the current status
of the controller is regarding its processing of the various send and receive rings. From information in the status block a software driver can determine:
• Whether the Status Block has been recently updated (via a bit in the status word).
• Whether the Link State has changed (via a bit in the status word).
• Whether the controller has recently received a packet and deposited that packet into host memory for a
given ring (via the Receive Return Ring Producer Indices).
• Which host receive descriptors that controller has fetched, and it will consume when future packets are
received (via the Receive Producer Ring Consumer Indices).
• Whether the controller has recently completed a transmit descriptor buffer DMA for a given ring (via the
Send Ring Consumer Indices).

Status Block Format

Note: Reference registers 0x3C50–0x3CC0 for debug access to the various ring indices.
Each MSI-X vector is associated with a status-block structure. A status block is DMAed to the host memory immediately prior to raising a legacy style interrupt (INTx, MSI) or MSI-X interrupt. Status block formats vary depending on RSS and the MSI-X vector number.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 82
Page 83
Status BlockBCM5718 Programmer’s Guide
The various status block formats are shown in this section.
INTx/MSI—Legacy Mode Status Block Format
INTx and MSI use this status-block format in non-RSS mode.
Table 14: Status Block Format (MSI-X Single-Vector or INTx—RSS Mode)
31
1
Offset
0x00 Status Word
0x04 [31:8] Reserved 0x0 [7:0]Status Tag
0x08 Receive Standard Producer Ring Consumer
0x0C Reserved 0x0 Reserved 0x0
0x10 Send BD Consumer Index Receive Return Ring Producer Index
0x14 Reserved 0x0 Receive Jumbo Producer Ring Consumer Index
6
Index
15 0
Reserved 0x0
Status-Block [0] Status Word Format (single-vector RSS):
• Bit [0]: Update bit
• Bit [1]: Link status change
• Bit [2]: Error/attention
• Bits [31:3]: Reserved 0x0
Broadcom®
January 29, 2016 • 5718-PG108-R Page 83
Page 84
Status BlockBCM5718 Programmer’s Guide
Single-Vector or INTx— RSS Mode Status Block Format
In the single-vector RSS mode, the status-block format used by Vector#0 is shown below.
INTx and MSI also use this status-block format.
Table 15: Status Block Format (MSI-X Single-Vector or INTx—RSS Mode)
31
1
Offset
0x00 Status Word
0x04 [31:8] Reserved 0x0 [7:0]Status Tag
0x08 Receive Standard Producer Ring Consumer
0x0C Receive Return Ring 2 Producer Index Receive Return Ring 3 Producer Index
0x10 Send BD Consumer Index Receive Return Ring 0 Producer Index
0x14 Reserved 0x0 Receive Jumbo Producer Ring Consumer Index
Status-Block [0] Status Word Format (single-vector RSS):
• Bit [0]: Update bit
• Bit [1]: Link status change
• Bit [2]: Error/attention
• Bits [31:3]: Reserved — always 0x0
6
Index
15 0
Receive Return Ring 1 Producer Index
Broadcom®
January 29, 2016 • 5718-PG108-R Page 84
Page 85
Status BlockBCM5718 Programmer’s Guide
Multivector RSS Mode Status Block Format
There are five slightly different status-block formats used by the multivector RSS mode. Each of these formats associate with their respective vector numbers as shown in the tables below.
Table 16: Status Block [0] Format (MSI-X Multivector RSS Mode)
31
1
Offset
0x00 Status Word
0x04 [31:8] Reserved 0x0 [7:0]Status Tag
0x08 Receive Standard Producer Ring Consumer
0x0C Reserved 0x0 Reserved 0x0
0x10 Send BD Consumer Index Reserved 0x0
0x14 Reserved 0x0 Receive Jumbo Producer Ring Consumer Index
Status-Block [0] Status Word Format (multivector RSS):
• Bit [0]: Update bit
• Bit [1]: Link status change
• Bit [2]: Error/attention
• Bits [3]: Reserved — always 0
• Bits [4]: Reserved — always 0
• Bits [5]: Reserved — always 0
• Bits [31:6]: Reserved 0x0
6
Index
15 0
Reserved 0x0
Table 17: Status Blocks [1 thru 4] Formats (MSI-X Multivector RSS Mode)
31
1
Offset
0x00 Status Word {Valid for all Status Blocks}
0x04 [31:8] Reserved 0x0 [7:0] Status Tag[n]
0x08 Reserved 0x0 Receive Return Ring 1 Producer Index
0x0C Receive Return Ring 2 Producer Index
0x10 Reserved 0x0 Receive Return Ring 0 Producer Index
0x14 Reserved 0x0 Reserved 0x0
Status-Block [1 – 4] Status Word Format (multivector RSS):
Broadcom®
January 29, 2016 • 5718-PG108-R Page 85
6
Valid only for Status Block3 else RSVD 0x0
15 0
{independent for each status blocks}
Valid only for Status Block2 else RSVD 0x0
Receive Return Ring 3 Producer Index Valid only for Status Block4 else RSVD 0x0
Valid only for Status Block1 else RSVD 0x0
Page 86
Status BlockBCM5718 Programmer’s Guide
• Bit [0]: Update bit
• Bit [31:1]: Reserved 0x0
Status Block and INT MailBox Address es
Each status block may be placed in an independent host memory address (64-bit). Each vector may be acknowledged via associated INT MailBoxes.
Table 18: Status Block Host Addresses and INT MailBox Addresses
RSS Mode
Status Blo ck Ho st Status Block #
Legacy 0x3C3C, 0x3C38 0x200 All Legacy status block
0 x3C3C, 0x3C38 0x200 Link-Status change
1 0x3D00, 0x3D04 0x208 Rx Return Ring 0
2 0x3D08, 0x3D0C 0x210 Rx Return Ring 1
3 0x3D10, 0x3D14 0x218 Rx Return Ring 2
4 0x3D18, 0x3D1C 0x220 Rx Return Ring 3
5 0x3D20, 0x3D24 N/A N/A Used only in MSI-X multivector
Address Register
(64-Bit)
INT MailBox Register Address
Indication Items
Error/Attention SBD Ring 1 Cons Index Std RBD Cons Index Jmb RBD Cons Index
Prod Index
Prod Index
Prod Index
Prod Index
Comments
Used by INTx or MSI
Used in all MSI-X modes for Vector#0
Used only in MSI-X multivector RSS mode or multivector EAV mode for Vector#1–Vector#4
EAV mode for Vector#5
The Status word field contains bit flags that contain error information about the status of the controller. The defined flags are listed in Table 19.
Table 19: Status Word Flags
Bits Name Description
0 Updated This bit is always set to 1 each time the status block is updated in the host via
DMA. It is expected that host software clear this bit in the status block each time it examines the status block. This provides the host driver with a mechanism to determine whether the status block has been updated since the last time the driver looked at the status block.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 86
Page 87
Status BlockBCM5718 Programmer’s Guide
Table 19: Status Word Flags (Cont.)
Bits Name Description
1 Link State Changed Indicates that link status has changed. This method of determining link change
status provides a small performance increase over doing a PIO read of the Ethernet MAC Status register (see “EMAC Status Register (offset: 0x404)” on
page 311. See “Wake on LAN Mode/Low-Power” on page 212 for a description
of the PHY setup required when link state changes.
2 Error When this bit is asserted by the chip, the following conditions may have
occurred. Bit 2 of the status word is the OR of:
• All bits in Flow Attention register (0x3c48) (see “Flow Attention Register
(offset: 0x3C48)” on page 422.
• MAC_ATTN—Events from the MAC block (see “EMAC Status Register
(offset: 0x404)” on page 311.
• DMA_EVENT—Events from the following blocks:
– MSI (see “MSI Status Register (offset: 0x6004)” on page 467.
– DMA_RD (see “LSO Read DMA Status Register (offset: 0x4804)” on
page 435.
– DMA_WR (see “Write DMA Status Register (offset: 0x4C04)” on
page 450.
• RXCP_ATTN—Events from RX RISC (see “RX RISC Status Register (offset:
0x5004)” on page 452.
The Status Block format for these devices is as follows:
• Status Tag—Contains an unique eight-bit tag value in bits 7:0 when the Status Tagged Status mode bit of the Miscellaneous Host Control register (see “Miscellaneous Host Control Register (offset: 0x68)” on
page 282) is set to 1. This Status Tag can be returned to the Mailbox 0 register at location 31:24
(see“Interrupt Mailbox 0 Register (offset: 0x5800)” on page 460) by host driver. When the remaining Mailbox 0 register bits 23:0 are written as 0, the tag field of the Mailbox 0 register is compared with the tag field of the last status block to be DMAed to host. If the tag returned is not equivalent to the tag of the last status block DMAed to the host, the interrupt state is entered.
• Receive Producer Ring Consumer Index —Contains the controller’s current Consumer Index value for the Receive Producer Ring. This field indicates how many receive descriptors are in the receive producer ring that the controller has consumed. For more information regarding this ring, see “Receive Producer Ring” on
page 77.
• Receive Return Rings 0–3 Producer Indices —Contain controller’s current Producer Index value for the each of the Receive Return Rings. When the controller receives a packet and writes that packet data into host memory via DMA, it will increment the Producer Index for the corresponding Receive Return ring. When a Producer Index is incremented, it is a signal to software that a newly arrived receive packet is ready to be processed.
• Send Ring Consumer Index—Contains controller’s current Consumer Index value for the Send Ring. When the controller completes the read DMA of the host buffer associated with a send BD, the controller will update the Send Ring Consumer Index. This provides the host software with an indication that the controller has buffered this send data and, therefore, the host software may free the buffer that was just consumed by the device.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 87
Page 88
Receive Data FlowBCM5718 Programmer’s Guide

Section 5: Receive Data Flow

Introduction

The RX MAC pulls BDs from RX producer rings. The RX BD specifies the location(s) in host memory where packet data may be written. Figure 14 on page 89 shows the receive buffer descriptor cycle.
All ingress Ethernet frames are classified by the RX rules engine. A class ID is associated to each frame based on QOS rules setup in the RX MAC (see “Receive Rules Setup and Frame Classification” on page 95). The Receive List Placement and Receive List Initiator portions of the MAC architecture move BDs to the RX return rings; the class ID associated to the packet is examined to route the BD to a specific RX return ring.
Once the packet is queued to the RX return ring, the device driver will wait for indication of the same through the status block update and interrupt from the host coalescing engine. The host coalescing engine will update the status block and generate a line interrupt or MSI (see “Host Coalescing” on page 234 for further details regarding interrupts) when a specified host coalescence criteria is met. Once the interrupt is generated, the host device driver will service the interrupt. The ISR will determine if new BDs have been completed on the RX Return Rings. Next, the device driver will indicate to the network protocol that the completed RX packets are available. The network protocol will consume the packets and return physical buffers to the network driver at a later point.
The BDs may then be reused for new RX frames. The device driver must return the BD to an RX producer ring. For this purpose, the driver should fill out either the opaque field or index field of the Rx BD when inserting/ initializing the BD in an RX Producer ring. When the BD is returned by the device through Return Ring, the opaque or index data field of the BD will be used by the driver to identify the BD in Producer Ring that corresponds to the Returned BD in Return Ring. The device driver will then reinitialize the identified BD in Producer Ring with a new allocated buffer and replenish the Receive Producer Ring with this BD.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 88
Page 89
IntroductionBCM5718 Programmer’s Guide
RX
Return
Ring 1
RX
Return
Ring 2
RX
Return
Ring 3
RX
Return
Ring 4
DMA Read Engine
Local
Memory
List
Initiator
DMA Write Engi ne
Interrupt
Service
Routine
RX Indicate Available
Rx MAC
RX Return Packet
Protocol Interface
(i.e. TCP/IP)
Device DriverMAC
Host
Coalescing
Engine
RX
Jumbo
Producer
Ring
RX
Standard
Producer
Ring
Figure 14: Receive Buffer Descriptor Cycle
Broadcom®
January 29, 2016 • 5718-PG108-R Page 89
Page 90
Receive Producer RingBCM5718 Programmer’s Guide

Receive Producer Ring

A Receive Producer Ring is an array containing a series of Receive Buffer Descriptors (BD). The Receive Producer Ring is host-based and a portion of the available buffer descriptors are cached in Ethernet controller internal memory.
A receive producer ring contains a series of buffer descriptors which in turn contain information of host memory locations to where packets are placed by the Ethernet controller at reception.

Setup of Producer Rings Using RCBs

A Ring Control Block (RCB) is used by the host software to set up the shared rings in host memory. In the context of producer ring, the Receive Producer Ring RCB is a set of registers (or internal device memory offsets) used to define the Receive Producer Ring. The host software must initialize the Receive Producer Ring RCB.
Receive Producer Ring RCB—Register Offset 0x24 50–0 x2 45 f
Other Considerations Relating to Producer Ring Setup
Other registers that affect the producer rings must be initialized by the host software. These registers include the Receive BD Ring Replenish Threshold register, the Receive MTU register, and the Accept Oversized bit (bit
5) in the Receive MAC Mode register.
• Receive BD Producer Ring Replenish Threshold registers:
– “Standard Receive BD Producer Ring Replenish Threshold Register (offset: 0x2C18)” on page 373
– “Receive Data Completion Control Registers” on page 371.
These registers are used for setting the number of BDs that the Ethernet controller can use up before requesting that more BDs be DMAed from a producer ring. In other words, the device does not initiate a DMA for fetching the Rx BDs until the number of BDs made available to the device by the host is at least the value programmed in this register.
• Receive MTU register (“Receive MTU Size Register (offset: 0x43C)” on page 316). This 32-bit register is set to a value that is the maximum size of a packet that the Ethernet controller receives. Any packets above this size is labeled as an oversized packet. The value for this register is typically set to 1522, which is the Standard Producer Ring RCB typical setting. If Jumbo frames are supported, the MTU would be set to the maximum Jumbo frame size.
• Receive MAC Mode register (“Receive MAC Mode Register (offset: 0x468)” on page 322). If the Accept Oversized bit (bit 5) of this register is set, the Ethernet controller accepts packets (of size up to 64 KB) larger than the size specified in the MTU.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 90
Page 91
Receive Producer RingBCM5718 Programmer’s Guide
Offset
0x00 0x04 0x08 0x0c
31 16
Host Ring Address
Max_Len Flags
NIC Ring Address
15 0
Std RingBD 1
512
511
BD 2 BD 3
Standard Producer Ring RCB
Standard Producer Ring
RCB Setup Pseudo Code
An example of setting up receive producer ring RCB:
Content of Pointer_to_RX_PRODUCER_RING_RCB + 0x00 = Host address of standard receive producer ring high 32. Content of Pointer_to_RX_PRODUCER_RING_RCB + 0x04 = Host address of standard receive producer ring low 32. Content of Pointer_to_RX_PRODUCER_RING_RCB + 0x0a = No flags. Content of Pointer_to_RX_PRODUCER_RING_RCB + 0x08 = Max packet size of 1522. Content of Pointer_to_RX_PRODUCER_RING_RCB + 0x0c = Internal Memory address for device copy of ring.
Figure 15 shows the standard ring RCB for the setup of a host-based standard producer ring.
Receive Buffer Descriptors (BDs) begin on the Receive Producer Ring. The host device driver will populate the receive producer ring with a specified number of BDs supported by the receive producer ring (see “Receive MTU
Size Register (offset: 0x43C)” on page 316). When a packet is received, the RX MAC moves the packet data
into internal memory. The Receive MTU Size register (see “Receive MTU Size Register (offset: 0x43C)” on
page 316) specifies the largest packet accepted by the RX MAC; packets larger than the Receive MTU are
marked oversized and are discarded.
Figure 15: Receive Producer Ring RCB Setup

Receive Buffer Descriptors

The Receive Buffer Descriptor is a data structure in host memory. It is the basic element that makes up each receive producer and receive return ring. The format of receive buffer descriptors can be seen in Table 11 on
page 78. A receive buffer descriptor has a 64-bit memory address and may be in any memory alignment and
may point to any byte boundary. For performance and CPU efficiency reasons, it is recommended that memory be cache-aligned. The cache line size value is important for the controller to determine when to use the PCI memory write and invalidate command. There are no requirements for memory alignment or cache line integrity for the Ethernet controller.
Unlike send buffer descriptors, the receive buffer descriptors cannot be chained to support multiple fragments.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 91
Page 92
Receive Producer RingBCM5718 Programmer’s Guide

Management of Rx Producer Rings with Mailbox Registers and Status Block

Status Block
The host software manages the producer rings through the Mailbox registers and by using the status block. It does this by writing to the Mail Box registers when a BD is available to DMA to the Ethernet controller and reading the status block to see how many BDs have been consumed by the Ethernet controller. The status block can be seen in “Status Block” on page 82.
The status block is controlled and updated by the Ethernet controller. The status block in host memory is constantly updated through a DMA copy by the Ethernet controller from an internal status block. The updates occur at specific intervals and host coalescence conditions that are specified by host software during initialization of the Ethernet controller. The registers for setting the intervals and conditions are in the Host Coalescing Control registers (see “Host Coalescing” on page 234) starting at memory offset 0x3c00. The Ethernet controller DMAs an updated status block to the 32-bit address that is set by the host software in the Host Coalescing Control registers, 0x3c38.
Among other status, the status block displays the last 16-bit value, BD index that was DMAed to the Ethernet controller from receive producer ring. The Ethernet controller updates these indices as the recipient or consumer of the BD from the producer rings.
Mailbox
The host software is responsible for writing to the Mailbox registers (see Table 20) when a BD is available from the producer rings for use by the Ethernet controller. Host software should use the high-priority mailbox region from 0x200–0x3FF for host standard and the low-priority mailbox region from 0x5800–0x59FF for indirect register access mode.
The Mailbox registers (starting at memory offset 0x200 for host standard and offset 0x5800 for indirect mode) contain the following receive producer index register.
Receive BD Producer Ring Producer Index
• Host standard: memory offset 0x268–0x26F
• Indirect mode: memory offset 0x5868–0x586F
Table 20: Mailbox Registers
Offset (High-Priority Mailboxes for
Host Standard Mode)
0x200–0x207 0x5800–0x5807 Interrupt Mailbox 0 RW
0x208–0x20F 0x5808–0x580F Interrupt Mailbox 1 RW
0x268–0x26F 0x5868–0x586F Receive BD Standard Producer Ring Producer Index RW
0x280–0x287 0x5880–0x5887 Receive BD Return Ring 1 Consumer Index RW
Offset (Low-Priority Mailboxes for Indirect Mode)
Register Access
Broadcom®
January 29, 2016 • 5718-PG108-R Page 92
Page 93
Receive Return RingsBCM5718 Programmer’s Guide
Table 20: Mailbox Registers (Cont.)
Offset (High-Priority Mailboxes for
Host Standard Mode)
0x288–0x28F 0x5888–0x588F Receive BD Return Ring 2 Consumer Index RW
0x290–0x297 0x5890–0x5897 Receive BD Return Ring Consumer Index RW
The Receive Producer Ring Producer Index register contains the index value of the next buffer descriptor from the producer ring that is available for DMA to the Ethernet controller from the host. When the host software updates the Receive Producer Ring Producer Index, the Ethernet controller is automatically signaled that a new BD is waiting for DMA. At initialization time, these values must be initialized to zero. These indices are 64-bit wide.
Offset (Low-Priority Mailboxes for Indirect Mode)
Register Access

Receive Return Rings

Receive Return Rings (RR) are host-based memory blocks that are used by host software to keep track of the where the Ethernet controller is putting the received packets related receive buffer descriptors. Unlike the producer rings, the return rings reside only in host memory. The Ethernet controller uses the BDs in the NIC memory that are previously copied from the producer rings to use when packets are received from the LAN. It places the BDs that correspond to received packets in the return rings.
Return rings are the exact opposite of producer rings, except that they are not categorized by the maximum length receive packets supported. They are actually categorized by priority or class of received packet. The highest priority return ring is ring 1, and the lowest priority is the last ring (Return Ring 2–Return Ring 4 depending on how many rings are set up by the host software).
The Receive Return Ring RCBs are used to set up return rings in much the same way the Receive Producer Ring RCB is used to set up the receive producer ring. These RCBs for the return rings are set in the Miscellaneous memory region (SSRAM) at offset 0x200 (this region should not be confused with the register space in the chip). The RCB max_len field is used to indicate the number of buffer descriptor entries in a return ring. If an invalid value is set, the Ethernet controller indicates an attention error in the Flow Attention register.
Figure 13 on page 77 shows receive return rings.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 93
Page 94
Receive Return RingsBCM5718 Programmer’s Guide

Management of Return Rings with Mailbox Registers and Status Block

The return rings are managed by the host using the Mailbox registers and status block.
When a packet is received from the LAN, the Ethernet controller DMAs the packet to a location in the host, and then DMAs the related BD to a return ring. As the producer of this packet to the host, the Ethernet controller updates the status block producer indices for the related return ring (i.e., return ring 1 to return ring 4 that was DMAed the BD received packet). These return ring indices can then be read by the host software to determine the last BD index value of a particular ring that has information of the last received packet.
As the consumer of the received packet, the host software must update the return ring consumer indices in Mailbox registers Receive BD Return Ring 1 Consumer Index (memory offset 0x280–0x287 for host standard and 0x5880–0x5887 for indirect mode) through Receive BD Return Ring 4 Consumer Index.

Host Buffer Allocation

The allocation of memory in the host is dependent on the operating system in which the controller is being used. There are two crucial items:
• The use of non-cached and physically contiguous memory is best for adapter performance.
• Physical memory mapping is required for the controller’s internal copies of logical host memory.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 94
Page 95
Receive Return RingsBCM5718 Programmer’s Guide

Receive Rules Setup and Frame Cla ssification

The Ethernet controller has a feature that allows for the classification of receive packets based on a set of rules. The rules are determined by the host software and then input into the Ethernet controller.
A packet can be accepted or rejected based on the rules initialized into two rules register areas. The packets can also be classified into groups of packets of higher to lower priority using the rules registers. This occurs when the packet is directed to a specific return ring. Return rings 1–4 have an inherent priority associated with them. The priority is from lowest ring number to highest ring number; return ring 1 being the highest priority ring and return ring 4 being the lowest. The implementation of priority class is based on how many rings the host software has initialized and made available to the Ethernet controller. As packets arrive, the Ethernet controller may classify each packet based on the rules. When the host services the receive packet, it can service the lower numbered rings first.
A rule can be changed by first disabling it by setting 0 into Enable bit (bit 31) in Receive BD Rules Control register (see Table 22 on page 96). Wait about 20 receive clocks (rx_clock) and then reenable it when it is programmed with a new rule. Otherwise, changing the rules dynamically during runtime may cause the rule checker to output erroneous results because the rule checker is a pipelined design and uses the various fields of the rules at different clock cycles.
Receive Rules Confi guration Register
The Receive Rules Configuration register (memory offset 0x500–0x503, see Ta b le 2 1) uses bits 3:7 to specify the ring where a received packet should be placed into if no rules are met, or if the rules have not been set up. A value of 0 means the received packet will be discarded. A value of 1–16 specifies a corresponding ring. This ring should be initialized to at least a value of 1 if the rules are not being used to ensure that all received packets will be DMAed to return ring 1.
Table 21: Receive Rules Configuration Register
Bits Field Access
31:8 Reserved RO
7:3 Specifies the default class (ring) if no rules are matched RW
2:0 Reserved RO
Broadcom®
January 29, 2016 • 5718-PG108-R Page 95
Page 96
Receive Return RingsBCM5718 Programmer’s Guide
Receive List Placement Rules Array
The Receive List Placement Rules Array (memory offset 0x480–0x4ff) is made up of 16 combined element registers. The combined element is actually two 32-bit registers called the Receive BD Rules Control register (see Tab le 2 2) and the Receive BD Rules Value/Mask register (see Table 24 on page 97). The element can be looked at as a single 64-bit entity with a Control part and Value/Mask part since they use a single element. Bit 26 of the control part determines how the value/mask part is used. The Receive BD Rules Value/Mask register can be used as either a 32-bit left-justified Value or a 16-bit Mask followed by a 16-bit Value.
Note: Receive rules cannot be used to match VLAN headers because the VLAN tag is stripped from
the Ethernet frame before the rule checker runs.
Table 22: Receive BD Rules Control Register
Table 23: Receive List Placement Rules Array (memory offset 0x480–0x4ff)
Bit Name RW Description Default
31 E RW Enable. Enabled if set to 1 –
30 & RW And With Next. This rule and next must both be true to match. The class
fields must be the same. A disabled next rule is considered true. Processor activation bits are specified in the first rule in a series.
29 P1 RW If the rule matches, the processor is activated in the queue descriptor for
the Receive List Placement state machine.
28 P2 RW If the rule matches, the processor is activated in the queue descriptor for
the Receive Data and Receive BD Initiator state machine.
27 P3 RW If the rule matches, the processor is activated in the queue descriptor for
the Receive Data Completion state machine.
26 M RW Mask If set, specifies that the value/mask field is split into a 16-bit value
and 16-bit mask instead of a 32-bit value.
25 D RW Discard Frame if it matches the rule. –
24 Map RW Map Use the masked value and map it to the class. –
23:18 Reserved RW Must be set to zero. 0
17:16 Op RW Comparison Operator specifies how to determine the match:
• 00 = Equal
• 01 = Not Equal
• 10 = Greater than
• 11 = Less Than
–
–
–
–
–
–
Broadcom®
January 29, 2016 • 5718-PG108-R Page 96
Page 97
Receive Return RingsBCM5718 Programmer’s Guide
Table 23: Receive List Placement Rules Array (memory offset 0x480–0x4ff) (Cont.)
Bit Name RW Description Default
15:13 Header RW Header Type specifies which header the offset is for:
• 000: Start of Frame (always valid)
• 001: Start of IP Header (if present)
• 010: Start of TCP Header (if present)
• 011: Start of UDP Header (if present)
• 100: Start of Data (always valid, context sensitive)
• 101–111: Reserved
12:8 Class RW The class this frame is placed into if the rule matches. 0:4, where 0 means
discard. The number of valid classes is the Number of Active Queues divided by the Number of Interrupt Distribution Groups. Ring 1 has the highest priority and Ring 4 has the lowest priority.
7:0 Offset RW Number of bytes offset specified by the header type. –
Table 24: Receive BD Rules Value/Mask Register
–
–
Bit Name RW Description Default
31:16 Mask – – –
15:0 Value – – –
Class of Service Example
If either Start of IP Header, Start of TCP Header, or Start of UDP Header is specified, and the frame has no IP, TCP, or UDP header, respectively, there is no frame match. The full set of rules provides a fairly rich selection and filtering criteria.
Example: If you wanted to set a Class of Service (CoS) of 2 based on the eighth byte in the data portion of
an encapsulated IPX frame using Ethernet Type 2 having a value greater than 6, you could set up the rules shown in Figure 16 on page 98.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 97
Page 98
Figure 16: Class of Service Example
Rule 1: Control = 0xc400020C Where: Enable + And with next (chain with next rule)
Mask -Value/Mask is split into two 16-bit values
Class -Return Ring 2 Offset -12 bytes from start of frame
Mask/Value = 0xffff 8137 Where: Mask – 0xffff
Value - IPX
Rule 2: Control
= 0x84028207
Where: Enable
Mask – Value/Mask split into two 16-bit values Comparison Operator –Greater Than Header Type – Start of Data
Offset – 7 bytes from start of data
Mask/Value =0xff00 0600 Where: Mask – 0xff00
Value – 0600
Header Type – Start of Data
Comparison Operator –Equal
Class -Return Ring 2
Checksum CalculationBCM5718 Programmer’s Guide

Checksum Calculation

Whether the host software NOS supports checksum offload or not, the Ethernet controller automatically calculates the IP, TCP, and UDP of received packets as described in RFC 791, RFC 793, and RFC 768, respectively.
Which protocol checksum value is produced can be determined by reading the status flag field in the Receive Return Ring. The valid flag values in the status flag field are IP_CHECKSUM and TCP_UDP_CHECKSUM. When a valid checksum is produced, the values of the checksums are found in the corresponding receive buffer descriptor register. These values should be 0xFFFF for a valid checksum or any other value if the checksum was incorrectly calculated. Assert the Receive No Pseudo-header Checksum bit of the Mode Control register (see “Mode Control Register (offset: 0x6800)” on page 468) to not to include Pseudo-header in TCP/UDP checksums.

VLAN Tag Strip

Receiving VLAN-tagged (IEEE 802.1q-compliant) packets are automatically supported by the Ethernet controller. There is no register or setting required to receive packets that are VLAN-tagged. The VLAN tag is automatically stripped from the IEEE 802.1q-compliant packet at reception and then placed in a receive buffer descriptor’s two byte VLAN tag field. The flag field has the BD_FLAGS_VLAN_TAG bit set when a valid VLAN packet is received. After the packet has been serviced by the host software, these fields should be zeroed out.
In the Receive MAC Mode register (offset 0x468–0x46b), the Keep VLAN Tag Diag Mode bit (bit 10) can be set to force the Ethernet controller to not strip the VLAN tag from the packet. This is only for diagnostic purposes.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 98
Page 99
Ta bl e 2 5 shows the frame format with IEEE 802.1Q VLAN tag inserted.
Table 25: Frame Format with 802.1Q VLAN Tag Inserted
Offset Description
0:5 MAC destination address
6:11 MAC source address
12:13 Tag Protocol ID (TPID)—0x8100
14:15 Tag Control Information (TCI):
• Bit 15:13—IEEE 802.1P priority
• Bit 12—CFI bit
• Bit 11:0—VLAN ID
16:17 The original EtherType
18:1517 Payload
VLAN Tag StripBCM5718 Programmer’s Guide
Broadcom®
January 29, 2016 • 5718-PG108-R Page 99
Page 100
RX Data Flow DiagramBCM5718 Programmer’s Guide
MailBox Registers
status word
rcv std cons
unused
unused
unused unused
Status Block
Standard and Jumbo Producer Rings
In Host Memory
BD n
Buffer Descriptor points to free RX buffer in host
TX cons #1 RX prod #1
Receive Return Rings in
host memory
BD n
Used Buffer Descriptor points to host
memory where packet was copied
1
2
3
4
6
5
Host Memory
Network
Rcv BD Std Producer Ring Index
BCM570X
Family

RX Data Flow Diagram

The receive data flow can be summarized in Figure 17. The Receive Producer Ring, Receive Buffer Descriptors, Receive Return Rings, Mailbox registers, and status block registers are the main areas of the receive data flow.
Figure 17: Overview Diagram of RX Flow
The RX flow sequence is as follows:
1. The host software updates a Receive Producer Ring Index in the Mailbox registers.
2. A receive BD or series of BDs with the corresponding index is DMAed to the Ethernet controller from the
host-based Receive Producer Ring.
3. The Ethernet controller updates the Receive Consumer Index in the Host Block register and stores copy of
the BD.
4. A valid Ethernet packet is received from the network into the device.
5. The Ethernet packet is DMAed to host memory using a BD previously DMAed from a Receive Producer
Ring.
Broadcom®
January 29, 2016 • 5718-PG108-R Page 100
Loading...
Buy Points How it Works FAQ Contact Us Questions and Suggestions Privacy Policy Cookie Policy Terms of Use