Computing is about communicating. Some would also say about networking. Digital independence tags along on the wave of "Recommendations and Roadmap for European Sovereignty in open source HW, SW and RISC-V Technologies (2021)", calling for the development of critical open source IP blocks, such as PCIE Root Complex (RC). This is the first step in that direction. And, if you are looking for an opensource, soft PCIE EndPoint (EP) core, please check our other repo.

This project opens Artix7 PCIE Gen2 RC IP blocks for use outside of proprietary tool flows. While still reliant on Xilinx Series7 Hard Macros (HMs), it surrounds them with open-source soft logic for PIO accesses — The RTL and, even more importantly, the layered software Driver with Demo App.

All that with full HW/SW opensource co-sim. Augmented with a rock-solid openBackplane in the basement of our hardware solution, the geek community thus gets all it takes for building their own, end-to-end openCompute systems.

The project‘s immediate goal, now accomplished, was to empower the makers with ability to drive PCIE-based peripherals from their own soft RISC-V SOCs.

Given that the PCIE End-Point (EP) with DMA is already available in opensource, the opensource PCIE peripherals do exist for Artix7. Except that they are always, without exception, controlled by the proprietary RC on the motherboard side, typically in the form of RaspberryPi ASIC, or x86 PC. This project changes that status quo.

Our long-term goal is to set the stage for the development of full opensource PCIE stack, gradually phasing out Xilinx HMs from the solution. That’s a long, ambitious track, esp. when it comes to mixed-signal SerDes and high-quality PLLs. We therefore anticipate a series of follow on projects that would build on the foundations hereby set.

This first phase implemented an open source PCIE Root Complex (RC) for Artix7 FPGA, utilizing Xilinx Series7 PCIE HM and GTP IP blocks, along with their low-jitter PLL.

References

• PCIE Primer by Simon Southwell ✔

---

HW Architecture

The project relies on a modular hardware ecosystem that combines our custom-designed openPCIE Backplane with SQRL Acorn CLE-215+ FPGA modules and various PCIE adapters to create a flexible testing platform.

---

openBackplane PCB

Please, refer to 1.pcb for additional detail, and to 1.pcb/KiCad-library for the self-contained KiCad library. Plain-text versions of the PDF and slide documents are in 0.doc/1.pcb.

FPGA Carrier

The hardware platform for this project is the SQRL Acorn CLE-215+, a versatile FPGA development board. Although originally designed as a crypto-accelerator, its powerful Artix-7 FPGA and modular design make it an excellent choice for general-purpose PCIE development.

The system consists of two main components:

• M.2 FPGA Module (Acorn CLE-215+): This is the core of the system, a compact board in an M.2 form factor. It houses the Xilinx Artix-7 XC7A200T FPGA and is designed to be plugged into a standard M.2 M-key slot.

• PCIE Adapter Board (Acorn Baseboard Mini): A carrier board that holds the M.2 FPGA module. Its primary function is to adapt the M.2 interface to a standard PCIE x4 edge connector, allowing the entire assembly to be installed and tested in a regular PC motherboard slot.

The fully assembled Acorn CLE-215+ development board, ready for use in a PCIE slot.

It is important to note that the Acorn CLE-215+ is functionally identical to the more widely known NiteFury board, with the primary difference being the amount of onboard memory. The Acorn model features 1 GB of DDR3 RAM, while the standard NiteFury has 512 MB. Therefore, the NiteFury schematic serves as a direct and accurate reference for the board's hardware layout.

The central component of the SQRL Acorn CLE-215+ system is the Xilinx Artix-7 XC7A200T-FBG484 chip. This FPGA is crucial for implementing the PCIE Root Complex - and, on the other card, the Endpoint - possessing a range of features that make it highly suitable for this purpose.

The key specifications are summarized below:

References:

• Acorn-CLE215+ QuickStart
• NiteFury
• fpgas.online-test-designs PCIE ENUMERATION by RPI, for Acorn-CLE215+ EP

---

RTL Architecture

The opensource Root Complex RTL is a chain of blocks joined by SystemVerilog interfaces: the RISC-V SoC hands TLPs over AXI-Stream to the PCIE stack, which converts them for the PCIE_2_1 hard block and drives the GTP transceiver over PIPE.

Details are in the RTL README. Per design: common RTL, EP, RC-direct, RC-switched. The AMD/Vivado-IP reference designs are in 2.amd-rtl-with-Vivado-build (EP, RC-direct, RC-switched).

References:

• Basic PCIE EP for LiteFury
• Regymm PCIE
• LiteX PCIE EP
• PCIE EP DMA - Wupper
• Xilinx UG477 - 7Series Integrated Block PCIe
• Xilinx DS821 - 7series_PCIE Datasheet
• Xapp1052 - BusMaster DMA for EP

---

SW Architecture

The software stack is designed to run bare-metal on the soft RISC-V SoC embedded within the FPGA. In a standard PC, the Operating System (Linux/Windows) manages PCIE enumeration automatically in the background, hiding the complexity from the developer. In this project, our baremetal open-source driver takes full control, manually performing every step of the enumeration process to act as the PCIE Host.

The architecture adopts a clean, well-defined, layered approach:

1. Application Layer (The "User" Logic):

   • The final stage of the program that executes the high-level task.
   • It performs Memory Write operations to send a data payload to the Endpoint and uses Memory Read to verify the integrity of the data path.

2. PCIE Driver (Enumeration & Setup):

   • Responsible for the initialization sequence required to perform enumeration and establish a functional connection (link).
   • It manually performs device discovery, sizes the BARs by readback, assigns aligned memory addresses, checks the completion status of every request, and configures the Command Register to enable the device for communication.

3. HAL (Hardware Abstraction Layer):

   • Low-level helper functions that interact with the hardware by reading and writing data to specific memory addresses.

Note: This structure allows developers to treat PCIE devices just like any other local peripheral, abstracting away the complexities of the physical link.

Details are in the SW README.

---

Simulation Architecture

Sim Test Bench

The openPCIE test bench aims to have a flexible approach to simulation which allows a common test environment to be used whilst selecting between alternative CPU components, one of which uses the VProc virtual processor co-simulation element. This allows simulations to be fully HDL, with the RISC-V RTL core that goes into the bitstream (eduBOS5 by default, or PicoRV32), or to co-simulate software using the virtual processor, with a significant speed up in simulation times.

The test bench has the following features:

• A VProc virtual processor based soc_cpu.VPROC.picorv32 component
  • Selectable between this or the RTL core (eduBOS5 by default, or PicoRV32), with make CPU=rtl|vproc|iss
  • Can run natively compiled test code
  • Can run the application compiled natively with the auto-generated co-sim HAL
  • Can run RISC-V compiled code using the rv32 RISC-V ISS model
  • All three have been built and measured; each reaches the same result over the same PCIE link
• The pcieVHost VIP is used to drive the logic's PCIE link
• Uses a C sparse memory model
  • An HDL component instantiated in logic gives logic access to this memory
  • An API is provided to VProc running code for direct access from the pcieVHost software, which implements this sparse memory C model.

More details on the architecture and usage of the openPCIE test bench can be found in the README.md in the 5.sim directory.

The co-simulation can also be followed at TLP level in the WaveCrux waveform viewer, with an open-source PCIE TLP decoder plugin and a ready-made session -- here the firmware sizing BAR0 by readback and placing it at 0x80000000 (more):

One level down, Ferrite's PCIe PIPE and Data Link Layer decoders show the same link on the wire: link training, flow-control credits, and every TLP with its sequence number and Ack (more). With all three layers in one view, you can also see how long a TLP spends inside the hard macro: about 0.4 µs each way (more).

Co-sim HAL

The openPCIE control and status register hardware abstraction layer (HAL) software is auto-generated, as is the CSR RTL, using peakrdl. A single SystemRDL specification, 4.build/csr_build/csr.rdl, describes the TLP transmit/receive window the RISC-V firmware drives, and everything else is derived from it: the register block RTL (csr.sv, csr_pkg.sv), which soc_csr.sv bridges to the picorv32 memory bus, and the software header. For co-simulation purposes an additional layer is auto-generated from the same SystemRDL specification using systemrdl-compiler that accompanies the peakrdl tools. This produces two header files that define a common API to the application layer for both the RISC-V platform and the VProc based co-simulation verification environment. The details of the HAL generation can be found in the README.md in the 4.build/ directory.

More details of the test bench, the pcievhost component and its usage can be found in the 5.sim/README.md file. See also the pcievhost model, its LTSSM support code, and the superseded DUT stubs.

References

• pcieVHost
• WaveCrux waveform viewer, decoder plugin interface
• wavecrux-decoders: Ferrite's open-source PCIE PIPE and Data Link Layer decoders for WaveCrux

---

Implementation Workflow

The openPCIE implementation workflow is organized into several stages, covering the complete process from setting up the development environment to verifying the design on physical hardware.

Prerequisites

Before building the project, install and configure the required development tools. These include an FPGA toolchain such as AMD Vivado or openXC7, a bare-metal RISC-V cross-compilation toolchain, Python, and the project-specific dependencies listed in the corresponding directories.

HW/SW Co-Sim

Before deploying the design to physical hardware, run the available simulation environment to verify the interaction between the PCIE hardware design and the software components.

The co-simulation flow is used to test essential operations such as device initialization, register access, PCIE link behavior, and data exchange between the Root Complex (RC) and Endpoint (EP). This stage helps identify hardware and software integration issues before synthesis and hardware deployment.

SW Build

Compile the bare-metal RISC-V driver and test application using the configured cross-compilation toolchain, see 4.build/sw_build.

The software is responsible for initializing the required hardware components, configuring the PCIE communication flow, and transferring test payloads between the RC and EP. The generated executable can be used during simulation or deployed together with the FPGA design for hardware verification.

HW Synth and Bitstream Generation

The FPGA design can be synthesized and implemented using either the commercial AMD Vivado toolchain or the open-source openXC7 toolchain. Depending on the selected flow, the process includes design elaboration, synthesis, placement, routing, timing analysis, and final bitstream generation.

The resulting .bit file contains the complete hardware configuration for the selected FPGA board. The appropriate board-specific constraints, build configuration, and target design must be selected depending on whether the board operates as the Root Complex (RC) or Endpoint (EP).

Detailed build instructions for both the Vivado and openXC7 flows are provided in the corresponding project directories.

Flashing and Deployment

Program the RC and EP FPGA boards with their corresponding bitstreams. After both designs have been loaded, start the software application and allow the boards to initialize the PCIE connection.

The deployed system can then be verified by checking the PCIE link-status indicators and comparing the transmitted payload with the data displayed or received on the Endpoint.

Detailed commands, configuration options, supported boards, and stage-specific instructions are provided in the corresponding project directories.

---

Bringup, Debug, Bench Testing and Demo

FPGA Board Setup

Properly programming and operating the Artix-7 FPGA on the SQRL board requires some soldering and work on open desktop motherboard.

1. Custom JTAG Cable

The JTAG connector on the Acorn CLE-215+ is non-standard and not directly compatible with the standard 14-pin connector on the Xilinx Platform Cable. A custom adapter cable is therefore required.

The connector on the board is a Molex Pico-Lock 1.50mm pitch male header. This is not a standard 2.54mm or 2.00mm header, so standard DuPont-style cables will not fit.

To simplify making the cable, we highly recommend purchasing a pre-assembled cable with the correct female connector.

• Recommended Part: Molex 0369200603 on Digi-Key

This cable has the correct female connector on both ends. The easiest method is to cut the cable in half, which gives you two connector cables with open ends. You can then splice one of these cable ends onto the wires of your Xilinx programmer cable, matching the signals according to the following wiring diagram.

JTAG Connection Guide: Physical Pinout and Wiring Diagram.

2. External 12V Power Supply

The board cannot be programmed or operated solely from the PCIE/M.2 slot power. It requires an external 12V supply to function correctly, especially when complex designs and high-speed transceivers are active. Power is provided via a standard 6-pin PCIE power connector from an ATX power supply.

External 12V power connection.

3. Final Assembly

The complete system, including the custom cabling, is mounted in a test PC chassis for verification.

The complete FPGA system mounted in a PCIE slot.

4. JTAG Connectivity Test

After the hardware was prepared, the connection was verified using the Vivado Hardware Manager. As shown below, the tool successfully detected the JTAG programmer and identified the xc7a200t_0 FPGA chip. This confirms that the physical connections are correct and the board is ready for programming.

Successful device detection in Vivado Hardware Manager.

Test Jig

• PCB: openPCIE Backplane (see 1.pcb) which provides slots for the RC, EP, and the Switch.
• FPGA Boards: Two SQRL Acorn CLE-215+ (Artix-7) boards.
• JTAG Programmers: Two Xilinx Platform Cable USB units are used, each equipped with a custom adapter cable.

Before interacting over PCIE, here is the complete bench validation setup used for the Direct (Point-to-Point) connection scenario. It features the openPCIE Backplane powered by an external 12V supply, populated with two Acorn CLE-215+ FPGA modules (one configured as RC, the other as EP).

Dual Xilinx Platform Cable USB units are connected via custom adapters to allow simultaneous debugging and bitstream loading from two separate host workstations.

The complete hardware validation environment for the Direct connection test.

The "Dual-PC" JTAG Approach

We used a two-PC setup to streamline development:

• PC 1: Connected to the Root Complex (RC) FPGA via JTAG.
• PC 2: Connected to the EndPoint (EP) FPGA via JTAG.

Why this setup?

• Speed and Efficiency: This setup avoids the repetitive task of manually swapping the JTAG cable between boards and eliminates the time-consuming process of flashing onboard memory for every test iteration.
• Simultaneous Debugging: This allows us to run two instances of Vivado Hardware Manager (ILA - Integrated Logic Analyzer) at the same time. We can trigger on the RC and EP simultaneously to view the transaction from both sides of the link.

Test Procedure

To bring up the system and verify the PCIE link, follow these steps:

• Hardware Assembly: Insert the FPGA cards into their designated slots on the openPCIE Backplane (RC and EP) and connect the external 12V power supply.
• Bitstream Programming: Program both FPGAs using Vivado Hardware Manager. PC 1 is used to program the Root Complex, while PC 2 programs the EndPoint.
• Manual Reset: Once both devices are programmed, press the manual reset button on the backplane.
• Enumeration: Upon releasing the reset button, the Root Complex initiates the enumeration process and establishes the link with the EndPoint.
• Re-Initialization: Every subsequent press of the reset button triggers a full re-initialization of the PCIE connection, allowing for repeated testing and debugging without the need to re-program the FPGAs.

Objectives

The goal of this test is to verify that the Root Complex can successfully enumerate the link and perform both Memory Write and Memory Read transactions.

1. Write: The RC sends a data payload to the EP.
2. Transfer: The EP receives the TLP and writes it into its internal Block RAM (BRAM).
3. Read: The RC requests to read the data back from the EP memory.

Bench Validation Steps

1. Visual Verification (LEDs) Visual feedback is provided via the 4 user LEDs on both FPGA boards:

• RC Board: The LEDs indicate the Link Status, confirming that the physical connection is established and the devices are ready to communicate.
• EP Board: The LEDs display the data received by the EP.

2. Internal Signal Monitoring (ILA) Using the Vivado Integrated Logic Analyzer (ILA) on both PCs enables detailed monitoring of internal signals to see exactly what data was sent and received, its precise timing, and the low-level transaction details.

Bench Validation Results

The implementation was verified on physical hardware by checking both PCIE link establishment and successful payload transfer between the Root Complex (RC) and Endpoint (EP).

On the RC board, LED A3 indicates the PCIE link status. When this LED is illuminated, the PCIE link has been successfully established. We can see that the LED A3 is active, confirming that the link between the RC and EP is up.

On the EP board, the LEDs display the received data payload. The RC transmitted the decimal value 6, represented in binary as 0110. The LED pattern on the EP matches this value, confirming that the payload was transferred and received correctly.

These results confirm that PCIE link initialization and RC-to-EP data transfer operate successfully on physical hardware.

Primary References

• PCIE Utils
• Debug PCIE issues using 'lspci' and 'setpci'
• Using busybox (devmem) for register access

Secondary References

PCIE Protocol Analyzers

• PCIE Sniffing
• Stark 75T Card
• ngscopeclient
• PCI Leech
• PCI Leech/ZDMA
• LiteX PCIE Screamer
• LiteX PCIE Analyzer
• Wireshark PCIe Dissector
• PCIe Tool Hunt
• PCIe network simulator
• An interesting PCIE tidbit: Peer-to-Peer communication. Also see this
• NetTLP - An invasive method for intercepting PCIE TLPs
• PCIe on STM32MP257 with ngscopeclient
• opensource PCIE Controller

---

Project Status

PART 1. Mini PCIE Backplane PCB

Almost all consumer PCIE installations have the RC chip soldered down on the motherboard, typically embodied in the CPU or "North Bridge" ASIC, where PCIE connectors are used solely for the EP cards. Similarly, all FPGA boards on the market are designed for EP applications. As such, they expect clock, reset and a few other signals from the infrastructure. It is only the professional and military-grade electronics that may have both RC and EP functions on add-on cards, with a backplane or mid-plane connecting them (see VPX chassis, or VITA 46.4).

This dev activity is about creating the minimal PCIE infrastructure necessary for using a plethora of ready-made FPGA EP cards as a Root Complex. This infrastructure takes the physical form of a mini backplane that provides the necessary PCIE context similarly to what a typical motherboard would give, but without a soldered-down RC chip that would be conflicting with our own FPGA RC node.

Such approach is less work and less risk than to design our own PCIE motherboard, with a large FPGA on it. But, it is also a task that we did not appreciate from the get-go. In a bit of a surprise, half-way through planning, we've realized that a suitable, ready-made backplane was not available on the market. This initial disappointment then turned into excitement knowing that this new outcome would make the project even more attractive / more valuable for the community... esp. when Envox.eu has agreed to step in and help. They will take on the PCIE backplane PCB development activity.

• ✅ Create requirements document.
• ✅ Select components. Schematic and PCB layout design.
• ✅ Review and iterate design to ensure robust operation at 5 GT/s (PCIE Gen2), using openEMS for simulation of high-speed traces.
• ✅ Manufacture prototype. Debug and bringup, using AMD-proprietary on-chip IBERT IP core to assess Signal Integrity.
• ✅ Produce second batch that includes all improvements. Distribute it, and release design files with full documentation.

PART 2. Project setup and preparatory activities

• ✅ Procure FPGA development boards and PCIE accessories.
• ✅ Put together a prototype system. Bring it up using proprietary RTL IP, proprietary SW Driver, TestApp and Vivado toolchain.

PART 3. Initial HW/SW implementation

• ✅ HW development of opensource RTL that mimics the functionality of PCIE RC proprietary solution.
• ✅ SW development of opensource driver for the PCIE RC HW function. It runs bare-metal on the RISC-V SOC, without Linux - see SW Architecture.
• ✅ Design SOC based on RISC-V CPU with PCIE RC as its main peripheral.

PART 4. HW/SW co-simulation using full PCIE EP model

This dev activity is significantly beefed up compared to our original plan, which was to use a much simpler PCIE EP BFM, and non-SOC sim framework. While that would have reduced the time and effort spent on the sim, prompted by NLnet astute questions, we're happy to announce that wyvernSemi is now also onboard!

Their VProc can be used not only to faithfully model the RISC-V CPU and SW interactions with HW, but it also comes with an implementation of the PCIE model. The PCIE model has some EP capabilities with a configurable configuration space, which can be paired in sim with our RC RTL design. Moreover, the existence of both RC and EP models paves the way for future plug-and-play, pick-and-choose opensource sims of the entire PCIE subsystem.

With the full end-to-end simulation thus in place, we hope that the need for hardware debugging, using ChipScope, expensive test equipment and PCIE protocol analyzers would be alleviated.

• ✅ Extension of the existing PCIE RC model for some additional configurability of the EP capabilities.
• ✅ Testbench development and build up. Execution and debug of sim testcases.
• ✅ Documentation of EP model, TB and sim environment, with objectives to make it all simple enough to pickup, adapt and deploy in other projects.

PART 5. Integration, testing and iterative design refinements

• ✅ One-by-one replace proprietary design elements of the PART 2 prototype system with our opensource versions (except for Vivado and TestApp). Test it along the way, fixing problems as they occur.

PART 6. Prepare Demo and port it to openXC7

• ✅ Develop our opensource PIO TestApp software and representative Demo.
• ✅ Build design with openXC7, reporting issues and working with developers to fix them, possibly also trying ScalePNR flow.

Given that PCIE is an advanced, high-speed design, in the light of nextpnr-xilinx and openXC7 shortcomings when it comes to timing constraints and timing closure, we expect to run into showstoppers, and have therefore planned to experiment with WIP ScalePNR. In turned out that, with some corrections, the nextpnr-xilinx was good enough -- The bitstream built with it brings the link up at Gen2 speeds on hardware, the same as Vivado does. Getting there took six fixes - one latent bug in our RTL and five workarounds for openXC7 defects - all documented in 4.build/hw_build.openXC7, along with the issues we filed against openXC7.

The design has also been ported to, and tested with openXC7 1.0.0 edition, which was a major step forward and essentially what "ScalePNR" idea eventually evolved into. It brought to the forefront Yosys0.69 with integrated Slang, nextpnr-himbaechel (instead of nextpnr-xilinx) and fpga-as. With that, the earlier SV2V standalone prep step for our SystemVerilog design sources proved to be unnecessary, which simplified the build process. The SV2V option is however still retained for cross-tool checking.

---

Academia Outreach

openPCIE has served as the basis for a number of Master's theses at the Faculty of Electrical Engineering, University of Sarajevo (ETF Sarajevo). Among them are the theses of Anes Vrce and Aladin Brdar, the latter shown here presenting his work on the openPCIE platform, tapping into its hardware, software and simulation facets.

---

Acknowledgements

We are grateful to NLnet Foundation for their sponsorship of this development activity.

This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101092990.

The wyvernSemi's wisdom and contribution made a great deal of difference -- Thank you, we are honored to have you on the project.

The Envox, our next-door buddy, is responsible for the birth of our backplane, which we like to call BB (not to be mistaken for their gorgeous blue beauty BB3)

KiCad turns electrical idea into a manufacturable board. openXC7, with Yosys at its core, turns RTL into a loadable FPGA bitfile. PeakRDL turns our SystemRDL file into CSR RTL+headers+RegMap docs. WaveCrux greatly facilitates waveform decoding and viewing. But, that's not everything that made this project possible -- Thank you all for these wonderful opensource tools ✨.

Crowd Supply Campaign

• Info Page, Intro Video
• Hackster
• Elecrow

Community Outreach:

• 2026-10-20, Hacker News
• 2026-10-20, Mastodon
• 2026-10-19, daily.dev
• 2026-10-10
• 2026-10-05
• 2026-09-15
• 2026-06-23
• 2026-01-26
• 2025-12-19
• 2025-08-17
• 2025-07-22
• 2025-04-03

Community Involvement

Feel like helping opensource PCIE community grow better and larger?! See Contributions.

Don't feel shy to reach out and ask for help ===> .

---

Future outlook

Our next step is openPCIE3, a 24-lane Gen3 backplane with Optical Copper Link (OCuLink). It is built around the Diodes PI7C9X3G1224GP switch (PCIE 3.1, 24 lanes, 12 ports), and links one or two PCs, each over an x4 OCuLink cable, to four x4 EP connectors (two slots, two M.2).

Its main use is as a PC I/O extender, adding compute, storage or I/O cards, e.g. for opensource AI servers. The PC drives REFCLK and reset over the cable, so the board needs no local clock or reset generator. Design input is welcome in #9 and #13.

---

GenAI

Our repo may contain elements of work facilitated by the modern GenAI tools , used under strict guidance and tight supervision of the blood-and-flesh developers, and solely as suggestions.

In the end, it is us, the human devs, who are fully responsible for any shortfalls you may uncover while tapping into repo resources.

---

End-of-Document