FPGA Setup
The following installation steps are FPGA-specific and should be run on all run farm
machines that install an FPGA. You will need sudo access to set up the FPGA.
Unlike the XDMA-based boards, the Xilinx Alveo V80 does not use the XDMA driver: the FireSim driver talks to the FPGA through the Versal CPM’s PCIe endpoint, by memory-mapping its BAR. The XDMA and XVSEC drivers from Local FPGA System Setup are not needed on a machine that only has V80s.
Installing the FPGA
Section titled “Installing the FPGA”Poweroff your machine.
Install the V80 and its auxiliary power cable following AMD’s Alveo V80 Installation Guide (UG1617).
Attach a JTAG cable between the FPGA and the host machine. FireSim programs the FPGA over JTAG before each simulation.
Boot the machine.
Writing the FireSim image to flash
Section titled “Writing the FireSim image to flash”FireSim needs each V80 to boot a FireSim image from its flash, so that the FPGA shows up
as a FireSim FPGA (10ee:903f) and can be reprogrammed without rebooting. V80s ship
with AMD’s AVED example design in flash, whose management firmware (AMC) can write the
flash over PCIe. We use it, through AMD’s AVED Management Interface (AMI) tool, to save
a copy of the factory image and write the FireSim image.
First, check what each V80 is running:
lspci -nn -d 10ee:A FireSim image (
Memory controller [0580]: ... [10ee:903f]): the V80 is already set up for FireSim. Skip to the last step below to check it.AMD’s factory image (e.g.,
Processing accelerators [1200]: ... [10ee:50b4]): follow all of the steps below.Anything else, or no entry for the V80 (e.g., blank or corrupted flash, another design, or a flash write that was interrupted): first restore AMD’s factory image over JTAG with AMD’s Updating FPT Image in Flash instructions, which use the Vivado Hardware Manager and files from AMD’s AVED deployment package. If the V80 cannot be programmed that way, first switch it to JTAG boot mode as described in AMD’s AVED JTAG boot recovery instructions. Then cold-boot and follow all of the steps below.
Build and load AMI by following AMD’s AVED documentation. The AMI version must match the AMC version running on the card (same major and minor version), so pick the AVED release accordingly. Then check that AMI can talk to the card:
sudo ami_tool overviewEach V80 should be listed with an AMC version and the state
READY. Note each V80’s PCIe address (theBDFcolumn, e.g.,01:00.0) for the next steps. If AMI reportsNO_AMC, check the kernel log (sudo dmesg) for a version mismatch. Older AMI releases may need small fixes to build on recent Linux kernels.Obtain an existing bitstream tar file for your FPGA by opening the
bitstream_tarURL listed underxilinx_alveo_v80_firesim_rocket_singlecore_no_nicin the following file:${CY_DIR}/sims/firesim-staging/sample_config_hwdb.yaml.Download/extract the
.tar.gzfile to a known location. Inside, you will findxilinx_alveo_v80/firesim.pdi. Note its full path for the next steps.The flash is split into two partitions; the V80 boots from partition 0. Save a copy of the factory image, which is in partition 0, to partition 1 (replace
01:00.0with your V80’s PCIe address):sudo ami_tool cfgmem_copy -d 01:00.0 -i 0 -p 1Newer AMI releases name the partitions
primary:0andprimary:1instead; seeami_tool cfgmem_copy -h.Write the FireSim image to partition 0:
sudo ami_tool cfgmem_program -d 01:00.0 -i /path/to/xilinx_alveo_v80/firesim.pdi -p 0 -qThis can take over 10 minutes. Do not interrupt it or power off the machine until AMI reports that the image was programmed successfully.
-qskips booting the new image right away; we cold-boot instead.Repeat steps 4 and 5 for each V80, then unload AMI:
sudo rmmod amiPower off your machine fully, then cold-boot it.
Once the machine has booted, run the following to ensure that your FPGA is set up properly:
lspci -vvv -d 10ee:903fEach V80 should now show up as a
Memory controller(PCIe class0580) with two memory regions (512K and 32M). There should be one entry for each V80 you’ve added to the Run Farm Machine.
Partition 1 now holds a copy of the factory image. To go back to the factory image, use the same AMD instructions as for restoring it above.
Note
Remember to keep the JTAG cable connected at all times when running FireSim simulations (it is used to program the FPGA).
Machines with other FireSim FPGAs
Section titled “Machines with other FireSim FPGAs”All FireSim Xilinx FPGA images show up on PCIe as 10ee:903f. FireSim tells V80s
apart from XDMA-based boards (e.g., Alveo U250) by their PCIe class: V80s report
0580, and the other boards report a different class. Each FireSim run uses one board
type per Run Farm Machine, so if a machine has both V80s and XDMA-based FPGAs, use a
separate config_runtime.yaml for each board type, and give each board type its own
FPGA database on that machine with the override_fpga_db host option (see
deploy/run-farm-recipes/externally_provisioned.yaml).

