GitHub repository: https://github.com/corundum/corundum
Documentation: https://docs.corundum.io/
GitHub wiki: https://github.com/corundum/corundum/wiki
Google group: https://groups.google.com/d/forum/corundum-nic
Zulip: https://corundum.zulipchat.com/
Corundum is an open-source, high-performance FPGA-based NIC and platform for in-network compute. Features include a high performance datapath, 10G/25G/100G Ethernet, PCI express gen 3, a custom, high performance, tightly-integrated PCIe DMA engine, many (1000+) transmit, receive, completion, and event queues, scatter/gather DMA, MSI interrupts, multiple interfaces, multiple ports per interface, per-port transmit scheduling including high precision TDMA, flow hashing, RSS, checksum offloading, and native IEEE 1588 PTP timestamping. A Linux driver is included that integrates with the Linux networking stack. Development and debugging is facilitated by an extensive simulation framework that covers the entire system from a simulation model of the driver and PCI express interface on one side to the Ethernet interfaces on the other side.
Corundum has several unique architectural features. First, transmit, receive, completion, and event queue states are stored efficiently in block RAM or ultra RAM, enabling support for thousands of individually-controllable queues. These queues are associated with interfaces, and each interface can have multiple ports, each with its own independent scheduler. This enables extremely fine-grained control over packet transmission. Coupled with PTP time synchronization, this enables high precision TDMA.
Corundum also provides an application section for implementing custom logic. The application section has a dedicated PCIe BAR for control and a number of interfaces that provide access to the core datapath and DMA infrastructure.
Corundum currently supports devices from both Xilinx and Intel, on boards from several different manufacturers. Designs are included for the following FPGA boards:
For operation at 10G and 25G, Corundum uses the open source 10G/25G MAC and PHY modules from the verilog-ethernet repository, no extra licenses are required. However, it is possible to use other MAC and/or PHY modules.
Operation at 100G on Xilinx UltraScale+ devices currently requires using the Xilinx CMAC core with RS-FEC enabled, which is covered by the free CMAC license.
For detailed documentation, see https://docs.corundum.io/
Block diagram of the Corundum NIC. PCIe HIP: PCIe hard IP core; AXIL M: AXI lite master; DMA IF: DMA interface; AXI M: AXI master; PHC: PTP hardware clock; TXQ: transmit queue manager; TXCQ: transmit completion queue manager; RXQ: receive queue manager; RXCQ: receive completion queue manager; EQ: event queue manager; MAC + PHY: Ethernet media access controller (MAC) and physical interface layer (PHY).
cmac_pad
moduleFrame pad module for 512 bit 100G CMAC TX interface. Zero pads transmit frames to minimum 64 bytes.
cpl_op_mux
moduleCompletion operation multiplexer module. Merges completion write operations from different sources to enable sharing a single cpl_write
module instance.
cpl_queue_manager
moduleCompletion queue manager module. Stores device to host queue state in block RAM or ultra RAM.
cpl_write
moduleCompletion write module. Responsible for enqueuing completion and event records into the completion queue managers and writing records into host memory via DMA.
desc_fetch
moduleDescriptor fetch module. Responsible for dequeuing descriptors from the queue managers and reading descriptors from host memory via DMA.
desc_op_mux
moduleDescriptor operation multiplexer module. Merges descriptor fetch operations from different sources to enable sharing a single desc_fetch
module instance.
event_mux
moduleEvent mux module. Enables multiple event sources to feed the same event queue.
mqnic_core
moduleCore module. Contains the interfaces, asynchronous FIFOs, PTP subsystem, statistics collection subsystem, and application block.
mqnic_core_pcie
moduleCore module for a PCIe host interface. Wraps mqnic_core
along with generic PCIe interface components, including DMA engine and AXI lite masters.
mqnic_core_pcie_us
moduleCore module for a PCIe host interface on Xilinx 7-series, UltraScale, and UltraScale+. Wraps mqnic_core_pcie
along with FPGA-specific interface logic.
mqnic_interface
moduleInterface module. Contains the event queues, interface queues, and ports.
mqnic_port
modulePort module. Contains the transmit and receive datapath components, including transmit and receive engines and checksum and hash offloading.
mqnic_ptp
modulePTP subsystem. Contains one mqnic_ptp_clock
instance and a parametrizable number of mqnic_ptp_perout
instances.
mqnic_ptp_clock
modulePTP clock module. Contains an instance of ptp_clock
with a register interface.
mqnic_ptp_perout
modulePTP period output module. Contains an instance of ptp_perout
with a register interface.
mqnic_tx_scheduler_block_rr
moduleTransmit scheduler block with round-robin transmit scheduler and register interface.
mqnic_tx_scheduler_block_rr_tdma
moduleTransmit scheduler block with round-robin transmit scheduler, TDMA scheduler, TDMA scheduler controller, and register interface.
queue_manager
moduleQueue manager module. Stores host to device queue state in block RAM or ultra RAM.
rx_checksum
moduleReceive checksum computation module. Computes 16 bit checksum of Ethernet frame payload to aid in IP checksum offloading.
rx_engine
moduleReceive engine. Manages receive datapath operations including descriptor dequeue and fetch via DMA, packet reception, data writeback via DMA, and completion enqueue and writeback via DMA. Handles PTP timestamps for inclusion in completion records.
rx_hash
moduleReceive hash computation module. Extracts IP addresses and ports from packet headers and computes 32 bit Toeplitz flow hash.
stats_collect
moduleStatistics collector module. Parametrizable number of increment inputs, single AXI stream output for accumulated counts.
stats_counter
moduleStatistics counter module. Receives increments over AXI stream and accumulates them in block RAM, which is accessible via AXI lite.
stats_dma_if_pcie
moduleCollects DMA-related statistics for dma_if_pcie
module, including operation latency.
stats_dma_if_latency
moduleDMA latency measurement module.
stats_pcie_if
moduleCollects TLP-level statistics for the generic PCIe interface.
stats_pcie_tlp
moduleExtracts TLP-level statistics for the generic PCIe interface (single channel).
tdma_ber_ch
moduleTDMA bit error ratio (BER) test channel module. Controls PRBS logic in Ethernet PHY and accumulates bit errors. Can be configured to bin error counts by TDMA timeslot.
tdma_ber
moduleTDMA bit error ratio (BER) test module. Wrapper for a tdma_scheduler and multiple instances of tdma_ber_ch
.
tdma_scheduler
moduleTDMA scheduler module. Generates TDMA timeslot index and timing signals from PTP time.
tx_checksum
moduleTransmit checksum computation and insertion module. Computes 16 bit checksum of frame data with specified start offset, then inserts computed checksum at the specified position.
tx_engine
moduleTransmit engine. Manages transmit datapath operations including descriptor dequeue and fetch via DMA, packet data fetch via DMA, packet transmission, and completion enqueue and writeback via DMA. Handles PTP timestamps for inclusion in completion records.
tx_scheduler_ctrl_tdma
moduleTDMA transmit scheduler control module. Controls queues in a transmit scheduler based on PTP time, via a tdma_scheduler
instance.
tx_scheduler_rr
moduleRound-robin transmit scheduler. Determines which queues from which to send packets.
cmac_pad.v : Pad frames to 64 bytes for CMAC TX
cpl_op_mux.v : Completion operation mux
cpl_queue_manager.v : Completion queue manager
cpl_write.v : Completion write module
desc_fetch.v : Descriptor fetch module
desc_op_mux.v : Descriptor operation mux
event_mux.v : Event mux
event_queue.v : Event queue
mqnic_core.v : Core logic
mqnic_core_pcie.v : Core logic for PCIe
mqnic_core_pcie_us.v : Core logic for PCIe (UltraScale)
mqnic_interface.v : Interface
mqnic_port.v : Port
mqnic_ptp.v : PTP subsystem
mqnic_ptp_clock.v : PTP clock wrapper
mqnic_ptp_perout.v : PTP period output wrapper
mqnic_tx_scheduler_block_rr.v : Scheduler block (round-robin)
mqnic_tx_scheduler_block_rr_tdma.v : Scheduler block (round-robin TDMA)
queue_manager.v : Queue manager
rx_checksum.v : Receive checksum offload
rx_engine.v : Receive engine
rx_hash.v : Receive hashing module
stats_collect.v : Statistics collector
stats_counter.v : Statistics counter
stats_dma_if_pcie.v : DMA interface statistics
stats_dma_latency.v : DMA latency measurement
stats_pcie_if.v : PCIe interface statistics
stats_pcie_tlp.v : PCIe TLP statistics
tdma_ber_ch.v : TDMA BER channel
tdma_ber.v : TDMA BER
tdma_scheduler.v : TDMA scheduler
tx_checksum.v : Transmit checksum offload
tx_engine.v : Transmit engine
tx_scheduler_ctrl_tdma.v : TDMA transmit scheduler controller
tx_scheduler_rr.v : Round robin transmit scheduler
Running the included testbenches requires cocotb, cocotbext-axi, cocotbext-eth, cocotbext-pcie, scapy, and Icarus Verilog. The testbenches can be run with pytest directly (requires cocotb-test), pytest via tox, or via cocotb makefiles.
A. Forencich, A. C. Snoeren, G. Porter, G. Papen, Corundum: An Open-Source 100-Gbps NIC, in FCCM'20. (FCCM Paper, FCCM Presentation)
J. A. Forencich, System-Level Considerations for Optical Switching in Data Center Networks. (Thesis)
If you use Corundum in your project, please cite one of the following papers and/or link to the project on GitHub:
@inproceedings{forencich2020fccm,
author = {Alex Forencich and Alex C. Snoeren and George Porter and George Papen},
title = {Corundum: An Open-Source {100-Gbps} {NIC}},
booktitle = {28th IEEE International Symposium on Field-Programmable Custom Computing Machines},
year = {2020},
}
@phdthesis{forencich2020thesis,
author = {John Alexander Forencich},
title = {System-Level Considerations for Optical Switching in Data Center Networks},
school = {UC San Diego},
year = {2020},
url = {https://escholarship.org/uc/item/3mc9070t},
}
Corundum internally uses the following libraries:
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。