An NVMe SSD front end,
built in 7 days.
Directed by one designer, Melten's design-agent system built the Host-facing front end of an NVMe SSD controller on an AMD Versal FPGA in 7 days.1 Ubuntu's standard NVMe driver runs it like any SSD. Over PCIe Gen4 x8, it moved 13.9 GB/s each way, 88–89% of the link's raw rate, and 3.3 million 4 KiB I/Os per second.2
- Host NVMe queuesStorage back end
- An SSD controller's front end turns the Host's NVMe commands into PCIe data transfers, completions, and interrupts. This one is hardware on one AMD Versal FPGA, with a DRAM disk standing in for the flash back end.
- 7 days
- From specification to Gen4 x8 board results1
- 13.9 GB/s
- Sequential reads and writes, each, through PCIe Gen4 x82
- 3.3M IOPS
- 4 KiB random reads (3.27M) and writes (3.30M)2
- 8 / 8
- Data-integrity checks passed, including after controller reset, reboot, and power-on5
Why it matters
Specification in. Working NVMe drive out.
Directed by one designer, agents produced the architecture, RTL, firmware, verification, FPGA builds, board bring-up, and performance tuning.
Standard to the Host.
Ubuntu's own NVMe driver finds the drive, creates eight I/O queues, and runs I/O. BIOS boot, shutdown, reboot, and controller reset work too.
Near the link limit.
13.9 GB/s each way and 3.3M random IOPS. Packet headers take most of what remains of the Gen4 x8 link's raw rate.
Built for a flash back end.
Every I/O step runs in hardware. A DRAM disk stands in for the FTL and NAND, behind a back-end interface designed for them.
# lspci -nnvv (excerpt)
01:00.0 Non-Volatile memory controller [0108]: Melten NVMe DRAM Disk [fb38:eb80] (rev 01) (prog-if 02 [NVM Express])
Capabilities: [60] MSI-X: Enable+ Count=16 Masked-
MaxPayload 256 bytes, MaxReadReq 512 bytes
LnkCap: Port #0, Speed 16GT/s, Width x8, ASPM not supported
LnkSta: Speed 16GT/s, Width x8
Kernel driver in use: nvmenvme driver. Lines are copied from the run's log; other lines are omitted.Millions of commands a second,
all in hardware.
An NVMe front end must follow the specification closely enough for an unmodified Host driver, and keep enough commands and data in flight to fill a Gen4 x8 link whose Host-memory reads take on the order of a microsecond.
| Requirement | What it takes | How this front end does it |
|---|---|---|
| Host compatibility | Standard driver, full lifecycle: NVMe registers, Admin and I/O queues, PRP data pointers, MSI and MSI-X, plus BIOS boot, shutdown, reboot, and controller reset. | Registers and queues in hardware; Admin commands and controller lifecycle in Arm Cortex-R5F firmware; all checked with Ubuntu's driver on the board. |
| Command rate | A command every 306 ns: at 3.3M IOPS, fetch, decode, data pointers, transfer, completion, and interrupt must overlap. | A hardware pipeline with no processor in the I/O path. Each measured command-path stage serves at least 15M commands/s in simulation.6 |
| Bandwidth | Microsecond-scale Host reads: filling 15.75 GB/s takes many transfers in flight at once. | A 512-bit datapath at 250 MHz (16 GB/s), with 16 transfer slots and 12 outstanding Host reads, sized from latencies measured on the board. |
| Vendor IP | A vendor path that stalled: with AMD's CPM4 PCIe AXI bridge, Host-memory reads cross the on-chip network (NoC). On Gen4 x4, NVMe writes, where the device reads Host memory, reached only 1.9 GB/s, while NVMe reads reached 6.9 GB/s. | Agents wrote Melten's own PCIe AXI Bridge on the CPM4's native packet interface. It bypasses the on-chip network for Host-memory access and puts packet packing and buffering under their control. |
Overcoming IP bottlenecks with the Melten PCIe AXI bridge.
With AMD's CPM4 PCIe AXI bridge, the on-chip network's 2 KiB read reorder buffer was a likely bottleneck, given the long Host-read latency; a trial that gave NVMe writes a second network port nearly doubled them. Agents specified a replacement bridge and finished validating it with NVMe traffic on the board 21 hours later. With outstanding requests tuned to board-measured Host-read latency, the final Gen4 x8 image reached 13.94 GB/s for NVMe writes, the Host-memory-read direction, and 13.90 GB/s for NVMe reads.4
One PCIe link,
every I/O in hardware.
The Host posts commands to queues in its own memory and rings a doorbell. The front end fetches each command, resolves its data pointers, moves the data between Host memory and on-board DRAM buffers, writes the completion entry, and raises the interrupt, all in hardware. Firmware on one Arm Cortex-R5F handles Admin commands; a second runs the DRAM back end.
Written by agents: the NVMe front end (51 SystemVerilog files, about 10,100 lines), the PCIe AXI Bridge (35 files, about 7,600 lines), firmware for both cores (about 1,400 lines of C, plus shared utilities), FPGA integration, and board test tooling.
From AMD: the CPM4 PCIe controller, the Versal on-chip network and DDR4 memory controller, and the Arm Cortex-R5F cores.
Near the link limit,
reads and writes.
fio on the Ubuntu Host in performance mode: data crosses PCIe to and from on-board DRAM buffers, and the back end skips only the DRAM-disk copy.2
PCIe Gen4 x8 raw rate vs. measured through the front end
- Sequential read
- Link raw · 15.75 GB/s13.90 GB/s (88%)
- Sequential write
- Link raw · 15.75 GB/s13.94 GB/s (89%)
- 4 KiB random read
- Link raw · 15.75 GB/s3.27M IOPS (85%)
- 4 KiB random write
- Link raw · 15.75 GB/s3.30M IOPS (86%)
The link carries at most 15.75 GB/s each way after encoding. Most of the rest is packet overhead: at the 256-byte payload size the Host set, headers and framing take 8.6% of the raw rate.3
3.3 million commands a second, at 96% of large-block bandwidth.
Each 4 KiB random read is a full NVMe command: queue fetch, data pointers, transfer, completion, and interrupt. At 3.27M per second, one finishes every 306 ns on average, and the link still carries 13.4 GB/s.
Every data check passed.
On the same FPGA image in normal storage mode, the Host wrote and verified the whole 1 GiB namespace, then checked it again after a controller reset, a reboot, and a shutdown and power-on.5
- Write/read comparisons: CRC32C, SHA-256, and byte checks
- 5 / 5
- Data intact after reset, reboot, and power-on
- 3 / 3
- Namespace verified end to end
- 1 GiB
Performance mode. Throughput runs skip only the DRAM-disk copy; Host data still crosses PCIe into on-board DRAM buffers.
Scope. Graceful Host transitions only; no power-loss or error-recovery claim.
Specification to 13.9 GB/s,
in 7 days.
Agents carried the work from requirements to board tuning, one campaign at a time.
Function first, then speed.
The first image ran Ubuntu I/O with a simple data mover on Day 1.7. Later campaigns raised the clock to 250 MHz, the link to x8, and throughput to 13.9 GB/s, rechecking data correctness each time.
Measured latency, not assumptions.
On-chip logic analyzers measured Host- and DRAM-read latency on the board. Agents replayed those delays against the RTL in simulation to size queues and outstanding requests.
Bottlenecks measured, then removed.
Board trials, including a second write engine on its own network port, showed AMD's CPM4 PCIe AXI bridge limiting the Host-memory read path. Agents replaced the bridge rather than accept the limit.
- Day 0SpecificationNVMe requirements and architecture.
- Day 1.7Ubuntu ran I/OGen4 x4 with the standard driver; data checks passed.
- Day 2.5250 MHz on the boardTiming closed; AMD's CPM4 PCIe AXI bridge held NVMe writes to about 1.9 GB/s on x4.
- Day 3.2Boot, shutdown, rebootController lifecycle with the Host's BIOS and Ubuntu.
- Day 3.6Gen4 x8 link512-bit datapath on a Gen4 x8 link.
- Day 4.6Melten bridge on the boardNVMe traffic through the new PCIe AXI Bridge.
- Day 5.1Timing-closed x8 imageNew bridge at 250 MHz; lifecycle and data checks passed.
- Day 713.9 GB/s, 3.3M IOPSTuned final image; all data checks passed.
Day counts run from the first repository commit; no time is excluded.1
Engineering details.
The front-end specification, final board measurements, and internal service rates, without interrupting the main story.
Open the specification and full results
Front-end specification
- Hardware
- AMD Versal VMK180 evaluation board (XCVM1802), PCIe Gen4 x8 to an Ubuntu 26.04 Host.
- NVMe
- NVMe 1.4c subset: one 1 GiB namespace with 4 KiB blocks; up to eight I/O queue pairs of 256 entries; PRP lists up to 256 KiB; volatile write cache with FUA and Flush.
- Admin commands
- Identify, Create/Delete I/O queues, Get/Set Features (Number of Queues), Get Features (Volatile Write Cache), and Get Log Page (SMART, zero-filled).
- Interrupts
- MSI and MSI-X, 16 vectors each; legacy INTx is not supported.
- Datapath
- 512-bit AXI at 250 MHz; 16 transfer slots, 12 outstanding Host reads, 6 outstanding DRAM reads, 8 write credits.
- PCIe
- AMD CPM4 in native-stream mode with Melten's PCIe AXI Bridge; 512-byte payload capability, 256-byte payloads and 512-byte read requests as set by the Host.
- Processors
- Two Arm Cortex-R5F cores in split mode: one for Admin commands and controller lifecycle, one for the DRAM back end.
- Implementation
- Timing closed at 250 MHz (setup 0.000 ns, hold +0.006 ns); 153,948 LUTs, 126,276 flip-flops, 130 block RAMs; no DSPs or UltraRAM.
- Verification
- Verilator module and integration regressions with the production firmware, AMD interconnect simulation, on-board logic-analyzer captures, and Host-side fio and data checks.
Final board measurements
| Workload | Throughput | IOPS | Share of raw link rate |
|---|---|---|---|
| Sequential read | 13.90 GB/s12.94 GiB/s | 53.0K | 88.2% |
| Sequential write | 13.94 GB/s12.99 GiB/s | 53.0K | 88.5% |
| 4 KiB random read | 13.38 GB/s12.46 GiB/s | 3.27M | 85.0% |
| 4 KiB random write | 13.53 GB/s12.60 GiB/s | 3.30M | 85.9% |
One 60-second run per workload on the final image. Three further 10-second runs of each workload, on the same image and Host settings, agreed within 0.2%. GB/s is decimal (109 bytes per second); GiB/s is binary.2
Internal service rates (RTL simulation)
| Module and case | Service interval | Rate |
|---|---|---|
| Host-to-buffer data, aligned 4 KiB | 64 cycles | 14.901 GiB/s (16.000 GB/s) |
| Buffer-to-Host data, aligned 4 KiB | 64 cycles | 14.901 GiB/s (16.000 GB/s) |
| Either direction, unaligned 4 KiB | 65 cycles | 14.672 GiB/s (15.754 GB/s) |
| Command fetch, immediate responses | 3 cycles/command | 83.333M commands/s |
| Command fetch, 256-cycle response latency | 16.649 cycles/command; peak 16 outstanding | 15.016M commands/s |
| PRP processing, 4 KiB command | 9 cycles/command | 27.778M commands/s |
| Command decode through context lookup | 6 cycles/command | 41.667M commands/s |
| Completion entry through context lookup | 9 cycles/command | 27.778M commands/s |
Actual RTL with finite resources and checked data, against peers that respond without delay, at 250 MHz. These cover the payload and command-path modules named here, not every stage of the controller.6
- Elapsed time runs from the first commit in the design repository to the commit that recorded the final board results: 7.2 days, with no time excluded. The work ran as a sequence of campaigns: a functional Gen4 x4 drive, 250 MHz timing, Host lifecycle, Gen4 x8, the PCIe AXI Bridge, x8 timing closure with the new bridge, and final tuning.
- Measured with fio 3.41 on an Ubuntu 26.04.1 Host (Linux 7.0, Intel Core i5-11400): io_uring with direct I/O, fixed buffers, and registered files. Sequential cases use 256 KiB transfers, one job, and queue depth 16; random cases use 4 KiB transfers, eight jobs, and queue depth 16 per job. Performance mode replaces the back-end firmware with a hardware responder that skips the DRAM-disk copy; commands, PRP processing, data transfers between the Host and on-board DRAM buffers, completions, and MSI-X interrupts all run normally. The Host's IOMMU (VT-d) was disabled in BIOS and its CPU governor set to performance.
- One direction of a PCIe Gen4 x8 link carries at most 15.75 GB/s (16 GT/s per lane, eight lanes, 128b/130b encoding). The Host set a 256-byte maximum payload. A 256-byte Memory Write packet, which carries NVMe read data to the Host, adds a 16-byte header, a 4-byte LCRC, and a 4-byte framing token: 8.6% overhead, for a ceiling of 14.40 GB/s before link-layer acknowledgment and flow-control packets. Sequential reads reached 96.5% of that ceiling.
- Agents measured Host-memory read latency of ~1.7 μs directly on the physical board. At this latency, NVMe writes through AMD's CPM4 PCIe AXI bridge and NoC achieved only 1.9 GB/s on Gen4 x4.
- Normal storage mode runs the back-end firmware that copies data to and from the DRAM disk. With the whole 1 GiB namespace overwritten, the checks were: sequential and random 4 KiB writes with CRC32C read-back; Flush and FUA write/read byte comparisons; eight concurrent mixed readers and writers; a full per-chunk SHA-256 comparison; and the same pattern read back after an NVMe controller reset, a Host reboot, and a Host shutdown and power-on, without reprogramming the FPGA. One Host power-on needed a retry; the cause of the first failure is undetermined, and the data matched afterward. The checks do not establish behavior under power loss, abrupt reset during I/O, or error recovery.
- Service rates come from the actual RTL in simulation, with finite transfer slots and credits and with data and ordering checked, against peers that respond without delay. A 512-bit path at 250 MHz moves at most 16 GB/s. With injected peer latency, the rates fall, as shown in the project's reports. Stages not listed, such as queue arbitration, tag allocation, and the interrupt path, were verified for correctness on the board but not measured in isolation.