Modern data centres supporting AI/ML, distributed storage, HPC, and cloud-native workloads require extreme throughput, ultra-low latency, and predictable performance, but as Ethernet scales to 400G and 800G, traditional host-centric networking struggles due to CPU overhead from protocol handling, queue management, and completion processing, increasing tail latency and resource contention. RDMA addresses this by enabling zero-copy, kernel-bypass data transfers that remove the CPU from the critical data path while preserving application-level work requests, and iWave Fibre SmartNICs extend this approach by offloading RDMA and storage data paths entirely into hardware, enabling autonomous, wire-speed operation with deterministic latency and minimal CPU involvement.
Understanding the Data Centre Landscape
As Ethernet speeds scale to 400G and 800G, the inefficiencies of software-based networking stacks become impossible to ignore:
- Multiple memory copies across kernel and user space
- Interrupt-driven packet processing
- CPU contention between application workloads and I/O handling
- Unpredictable tail latencies under load
While RDMA fundamentally changes data movement between servers, end-to-end system performance is still constrained if protocol processing and storage translation remain software-driven within the infrastructure.
Compared to traditional NICs, iWave Fibre SmartNICs take acceleration further by moving RDMA transport processing, DMA orchestration, and storage data paths into programmable hardware, enabling:
- True zero-copy data transfers
- Line-rate throughput at 400G
- Predictable latency independent of host CPU load
400GbE and 800GbE SmartNIC with FPGA-Based RDMA and NVMe-oF Offload
At 400GbE and 800GbE speeds, network bandwidth is no longer the constraint CPU-centric protocol handling is. Interrupts, software queues, and kernel mediation introduce latency, jitter, and throughput collapse under load.
The iWave Fibre SmartNIC eliminates this bottleneck by implementing a full RDMA Core and NVMe-oF Host/Target Controller directly inside the Agilex™ 7 FPGA, enabling a fully hardware-managed, CPU-free data path.
Key Enablers:
- 400GbE and 800GbE interfaces via F-Tile transceivers with hardened Ethernet PHY and MAC blocks for deterministic, line-rate packet processing.
- PCIe Gen5 x16 connectivity via Intel® R-Tile, providing sufficient host and fabric bandwidth to sustain wire-speed data movement without backpressure.
The FPGA-resident RDMA Core fully terminates RoCE traffic in hardware, implementing transport-level functions such as connection context management, reliable delivery, congestion handling, queue pair (QP) state machines, DMA scheduling, and work completion generation without host CPU or kernel involvement. Memory registration, doorbell handling, and data placement are executed directly by the FPGA, enabling true kernel bypass and zero-copy transfers at line rate.
Figure 1: Traditional NIC data path with CPU-mediated memory copies.
Figure 1 illustrates data movement using a traditional NIC. Data originating in the application memory space (host DDR) is first copied by the CPU into an intermediate networking buffer in host memory, and then copied again into the NIC for transmission. On the receiving client, the process is reversed: the NIC transfers the data into a CPU-managed networking buffer, after which the CPU copies it into the application memory space. This multi-stage data movement results in additional memory copies, increased CPU involvement, and higher end-to-end latency.
Figure 2: RDMA NIC data path with direct application memory access.
Figure 2 shows data movement using iWave’s RDMA-enabled NIC. Here, the RDMA NIC directly accesses application memory by registering a region of host DDR during initialization. This enables direct data transfers between application memory and the NIC, eliminating CPU-mediated memory copies. On the receiving client, the RDMA-enabled NIC writes incoming data directly into the application memory space without CPU intervention. This architecture significantly reduces CPU overhead, memory bandwidth consumption, and overall latency.
RDMA payloads are then handed off internally to a hardware NVMe-oF Host/Target Controller instantiated in the same FPGA fabric. This controller performs NVMe command decoding, submission and completion queue management, namespace mapping, and PCIe transaction generation toward NVMe SSDs or downstream PCIe switch fabrics. The entire NVMe command lifecycle is handled deterministically in hardware, eliminating software queues, interrupts, and polling loops.
By tightly coupling the RDMA Core and NVMe-oF Controller within the FPGA, the SmartNIC creates a single, continuous data path from Ethernet to NVMe. Network packets are transformed directly into NVMe operations with no intermediate buffering in host memory and no dependency on a host OS or embedded CPU. This hardware-only pipeline removes OS jitter, CPU scheduling effects, and tail-latency spikes, delivering predictable latency and sustained throughput at 400GbE and 800GbE.
CPU-Free JBoF
While RDMA solves transport inefficiencies on the wire, the architecture of the storage enclosure itself has remained a bottleneck. Traditional Just-a-Bunch-of-Flash (JBoF) systems act as dense storage shelves but suffer from a critical design flaw: they rely on embedded x86 CPUs to manage data flow.
iWave, in collaboration with Altera, has introduced a CPU-Free JBoF architecture that fundamentally alters this paradigm. By removing the general-purpose processor from the storage shelf, the architecture shifts all control and data plane logic to the SmartNIC, creating a system that is leaner, faster, and more secure.
In a standard JBoF implementation, the data path is interrupted by software:
- The CPU Tax: Inside the JBoF, the NIC forwards requests to an embedded CPU, which is responsible for NVMe-oF target processing, translating network packets into PCIe commands, and managing SSD routing.
- Latency Penalty: This software intervention introduces unavoidable latency bottlenecks and creates a resource contention point as SSD parallelism increases. It also mandates a full OS stack, requiring constant patching, driver maintenance, and cooling overhead.
The iWave architecture eliminates the embedded CPU entirely. The Agilex™ 7-based iWave Fibre SmartNIC becomes the central control engine and data mover, connecting directly to the PCIe switches and NVMe drives.
- Direct Data Path: The SmartNIC handles NVMe-oF Target functionality and NVMe-MI (Management Interface) operations entirely in hardware. Data moves from Ethernet ports to SSDs without ever touching a host OS or external RAM buffer.
- Hardware Logic: Powered by an OFED RDMA Core and a dedicated NVMe-oF Host Controller, the SmartNIC autonomously manages NVMe command decoding, queue management, and SSD health monitoring.
Enabling Next-Gen Workloads
This architectural shift is specifically designed for environments where average performance is insufficient:
- AI/ML Training: With the CPU bottleneck removed, the JBoF ensures training pipelines remain fully saturated with I/O, preventing GPU stalls. The deterministic nature of hardware-managed queues is critical for large-scale model training.
- Edge Deployments: By removing the host CPU and OS, the system becomes ideal for remote edge nodes or telco POPs where maintenance windows are scarce and power is at a premium.
- Real-Time Analytics and Streaming: Line-rate, zero-copy ingestion at 400G/800G allows continuous processing of high-velocity data streams while hardware-managed storage paths prevent backpressure during burst conditions.
- High-Frequency Trading (HFT): Deterministic, CPU-independent data paths minimize latency variance and tail events, enabling consistent order execution and market data capture at line rate.
For more information contact us at mktg@iwave-global.com and visit www.iwave-global.com.
