A PCIe Gen4 switch,
built in 17 days.
Directed by one designer, Melten's design-agent system built a three-port PCIe Gen4 switch on an AMD Versal FPGA in 17 days.1 A standard Ubuntu Host recognizes it as a PCIe switch, and two off-the-shelf NVMe SSDs ran through it at 87–99% of their rated speeds.3
- 1 Host link2 NVMe SSDs
- A PCIe switch fans one Host connection out to several devices. This one runs on a single AMD Versal FPGA, with three PCIe Gen4 x4 ports.
- 17 days
- From specification to a working switch serving two SSDs, excluding a wait for hardware1
- 87–99%
- Of each SSD's rated speed, measured through the switch3
- 11.2 GB/s
- Combined, with one SSD reading while the other writes
- 2.4M IOPS
- Combined 4 KiB random I/O, with one SSD writing while the other reads2
Why it matters
Specification in. Working switch out.
Directed by one designer, agents produced the detailed architecture, RTL, firmware, verification, FPGA builds, and lab bring-up.
Standard to the Host.
Ubuntu sees an ordinary PCIe switch and uses its own PCIe and NVMe drivers for the switch and the SSDs behind it.
Near rated SSD speed.
Alone, each SSD ran at 87–99% of its vendor rating through the switch. Together, the two moved 11.2 GB/s.
Scalable by design.
Configurable Port Adapter IP blocks connect through a packet-based NoC, providing a modular architecture for expanding the port count.
Correct for the Host,
fast for the SSDs.
A PCIe switch must look exactly like standard PCIe bridges, route every transaction to the right port, and keep pace with Gen4 links, here through three vendor PCIe blocks on one FPGA.
| Requirement | What it takes | How this switch does it |
|---|---|---|
| Host compatibility | Standard bridges: the Host must discover and configure the switch like any PCIe switch, with its own drivers. | The PCIe blocks and firmware on the Arm Cortex-R5F answer the Host's configuration requests for every port. |
| Packet routing | Right port, every time: memory requests and completions between the Host and two SSDs, at Gen4 rates. | Routing tables in each port's hardware; firmware stays off the data path. |
| On-chip network | Interleaved packets: the Versal on-chip network can interleave packets from different senders. | Every port reassembles packets by sender at one data beat per clock. |
| Vendor IP | Three PCIe blocks, two kinds: CPM4 upstream and PL PCIe4 downstream, with startup behavior the documentation leaves open. | Agents ran focused experiments on the generated AMD PCIe models and set the simulation's assumptions from the results. |
A bug simulation never saw, fixed on the board.
The simulated CPU only issued aligned register writes. On the board, a single-byte write was rejected and the Host stopped enumerating the switch. Agents captured the failing write with on-chip logic analyzers, repaired the logic, and added tests that fail on the old design.
Configurable ports,
a scalable switch.
Each port uses a configurable Port Adapter IP block, connected to the others through the Versal AXI-Stream NoC. This packet interconnect provides a modular path to adding ports. The VMK180 prototype uses three ports to fit the board's available PCIe connections: one upstream port for the Host and two downstream ports for SSDs. Packets move in hardware; firmware on the Arm Cortex-R5F handles configuration.
Written by agents: everything that makes it a switch. That covers the Port Adapters that convert and route each port's packets, packet reassembly, CPU queues and registers, configuration firmware, and FPGA integration.
From AMD: the three PCIe controllers, which run the links and hold each port's configuration registers, the on-chip network, and the Arm Cortex-R5F.
CQ (Completer Request) / RC (Requester Completion): received requests / completions, from PCIe IP to Fabric.
RQ (Requester Request) / CC (Completer Completion): outgoing requests / completions, from Fabric to PCIe IP.
Near rated speed,
through the switch.
fio on the Ubuntu Host.2 Each SSD first ran alone, then both together.
Each SSD alone: vendor rating vs. through the switch
- Sequential read
- Rated · 7.00 / 7.45 GB/s6.92 / 6.94 GB/s
- Sequential write
- Rated · 6.50 / 6.90 GB/s6.10 / 6.25 GB/s
- 4 KiB random read
- Rated · 1.40M / 1.20M IOPS1.22M / 1.19M IOPS
- 4 KiB random write
- Rated · 1.30M / 1.55M IOPS1.28M / 1.37M IOPS
Each cell shows the SK hynix P41 on DSP0 / the Samsung 990 PRO on DSP1. Ratings are the vendors' maximums, measured without a switch under their own conditions.3
11.2 GB/s through one Gen4 x4 upstream link.
A PCIe link carries data both ways at once, up to 7.88 GB/s each way on Gen4 x4. Two reading SSDs send all their data toward the Host, sharing one direction: 6.94 GB/s in total. With one reading and one writing, data flows both ways: 11.23 GB/s combined.3
Data integrity. A separate verification session covered sequential and random I/O with each SSD alone and both together. Reads checked CRC32C on every 4 KiB block; after each write workload, the tested region was read back in full. No data errors were found.4
Specification to working switch,
in 17 days.
Agents carried the work from detailed design to lab debugging.
Firmware before hardware.
The switch's real C firmware ran against the RTL in simulation, so firmware and hardware mismatches surfaced before the first FPGA build.
Vendor IP, tested before trusted.
Where documentation left gaps, agents ran focused experiments on the generated AMD PCIe models before relying on their behavior.
The lab in the loop.
Agents programmed the FPGA, power-cycled the Host, ran workloads, and checked that every deployed image matched its build.
- Day 0SpecificationSpecification, architecture overview, and vendor-IP findings.
- Day 3.8Firmware drove the RTLControl firmware ran against the switch RTL in co-simulation.
- Day 8.2Three ports in simulationConfiguration and memory traffic on both downstream branches.
- Day 10.4Full FPGA designThree PCIe blocks, the R5F, and the on-chip network built together.
- Day 12.4Host enumerated the switchAfter the on-board repair, Ubuntu listed the switch ports.
- Day 16First SSD workloadsfio through the switch to one SSD.
- Day 17Two SSDs through the switchBoth SSDs at once; all data checks passed.
Day counts exclude about seven days spent waiting for the second FMC PCIe module.1
Engineering details.
The switch specification, the rating comparison, and all twelve throughput cases, without interrupting the main story.
Open the specification and full results
Switch specification
- Hardware
- AMD Versal VMK180 evaluation board (XCVM1802), with two FMC PCIe modules for the downstream SSD links.
- Ports
- USP0 on the CPM4 PCIe block; DSP0 and DSP1 on PL PCIe4 blocks; each PCIe Gen4 x4.
- Port Adapter IP
- Configurable port identity, upstream/downstream role, PCIe block interface, and fabric parameters; one reusable block per port.
- Packet fabric
- Versal AXI-Stream NoC with four 256-bit endpoints; PCIe user interfaces at 250 MHz.
- Control
- Arm Cortex-R5F running polling firmware from TCM: Configuration service, port control, and routing-table updates.
- Written by agents
- 46 SystemVerilog source files (about 5,900 lines), about 2,200 lines of C firmware, and the FPGA block-design scripts and constraints.
- AMD IP
- Versal CIPS (Arm Cortex-R5F and the CPM4 PCIe controller), two PL PCIe4 controllers, AXI4-Stream NoC, SmartConnect, and ILA debug cores.
- Verification
- Verilator block regressions, AMD PCIe IP simulation with production firmware through DPI-C, and on-board ILA captures.
Each SSD alone vs. its rating
| SSD and workload | Vendor rating | Through the switch | Share of rating |
|---|---|---|---|
| SK hynix P41 · sequential read | 7.00 GB/s | 6.92 GB/s | 98.8% |
| SK hynix P41 · sequential write | 6.50 GB/s | 6.10 GB/s | 93.8% |
| SK hynix P41 · 4 KiB random read | 1.40M IOPS | 1.22M IOPS | 86.9% |
| SK hynix P41 · 4 KiB random write | 1.30M IOPS | 1.28M IOPS | 98.3% |
| Samsung 990 PRO · sequential read | 7.45 GB/s | 6.94 GB/s | 93.1% |
| Samsung 990 PRO · sequential write | 6.90 GB/s | 6.25 GB/s | 90.5% |
| Samsung 990 PRO · 4 KiB random read | 1.20M IOPS | 1.19M IOPS | 98.9% |
| Samsung 990 PRO · 4 KiB random write | 1.55M IOPS | 1.37M IOPS | 88.1% |
All throughput cases
| Case | SK hynix P41 on DSP0 | Samsung 990 PRO on DSP1 | Combined |
|---|---|---|---|
| Alone · sequential read | 6.92 GB/s | 6.94 GB/s | — |
| Alone · sequential write | 6.10 GB/s | 6.25 GB/s | — |
| Alone · 4 KiB random read | 4.98 GB/s1.22M IOPS | 4.86 GB/s1.19M IOPS | — |
| Alone · 4 KiB random write | 5.23 GB/s1.28M IOPS | 5.59 GB/s1.37M IOPS | — |
| Together · sequential read | 3.59 GB/s | 3.35 GB/s | 6.94 GB/s |
| Together · sequential write | 2.45 GB/s | 4.39 GB/s | 6.83 GB/s |
| Together · 4 KiB random read | 3.47 GB/s0.85M IOPS | 3.24 GB/s0.79M IOPS | 6.72 GB/s |
| Together · 4 KiB random write | 3.40 GB/s0.83M IOPS | 3.14 GB/s0.77M IOPS | 6.54 GB/s |
| Mixed · sequential, DSP0 writes | 4.35 GB/swrite | 6.35 GB/sread | 10.70 GB/s |
| Mixed · sequential, DSP1 writes | 5.95 GB/sread | 5.29 GB/swrite | 11.23 GB/s |
| Mixed · 4 KiB random, DSP0 writes | 5.47 GB/swrite · 1.34M IOPS | 4.51 GB/sread · 1.10M IOPS | 9.98 GB/s |
| Mixed · 4 KiB random, DSP1 writes | 4.77 GB/sread · 1.17M IOPS | 4.78 GB/swrite · 1.17M IOPS | 9.56 GB/s |
GB/s is decimal (109 bytes per second). Each SSD ran alone; then both ran together, either doing the same operation or with one reading while the other writes. Combined values add the two SSDs' concurrent rates.2
- Elapsed time runs from the specification commit in the design repository to the completion of the two-SSD tests: 24.2 calendar days. About seven of those days were spent waiting for the second FMC PCIe module; no hardware work ran, and only documentation was updated. Excluding that wait leaves about 17 days.
- Measured with fio 3.41 on an Ubuntu Host: libaio, direct I/O, and ten seconds per case. Sequential cases use 256 KiB transfers, one job per SSD, and queue depth 32. Random cases use 4 KiB transfers, eight jobs per SSD in separate ranges, and queue depth 16. One SSD alone: single runs. Two SSDs: means of three runs. Ten-second runs characterize this setup; they are not steady-state SSD qualification.
- Vendor ratings are maximums for the 1 TB models: SK hynix Platinum P41, 7,000 / 6,500 MB/s sequential read / write and 1,400K / 1,300K random read / write IOPS; Samsung 990 PRO, 7,450 / 6,900 MB/s and 1,200K / 1,550K IOPS. Vendors measure with the SSD attached directly to a host, with their own tools and queue depths. The same SSDs were not measured without the switch, so these shares are a reference, not a measured switch overhead. One direction of a PCIe Gen4 x4 link carries at most 7.88 GB/s (16 GT/s per lane, 128b/130b encoding) before packet overhead.
- The CRC session used fio's buffer refill and CRC32C verification at 4 KiB intervals. The checker rejected a deliberately corrupted test file before SSD writes began. The 56 checked steps, all passed, are the 32 timed workloads, a full CRC readback after each of the 22 that wrote data, and an initial seeding pass with its readback. Write rates in this session include the refill and CRC cost, for example about 3.0 GB/s for sequential writes to one SSD. The checks do not establish full-drive integrity, the freshness of a stale but self-consistent record, or power-loss durability.