Running Exact Mode Simulations

In this section, we provide a step-by-step tutorial on how to partition a RocketTile out from the SoC and run exact-mode simulations on EC1 F1. Similar steps can be applied to perform locally partitioned FPGA simulations. This assumes that you have read the FireAxe running fast-mode simulations.

1. Building Partitioned Sims: Setting up FireAxe Target configs

We will be reusing the FireAxe target configurations from fast-mode.

//////////////////////////////////////////////////////////////////////////////
// - F1 partition a RocketTile out
// - Connect FPGAs using PCIe peer to peer communication scheme
//
// FPGA 0 (RocketTile) ----------- FPGA 1 (SoC subsystem)
//////////////////////////////////////////////////////////////////////////////
class RocketTileF1Config
    extends Config(
      new WithPCIM ++ // Use PCIM (PCIe peer to peer) communication scheme
        // WithPartitionGlobalInfo takes in a Seq of Seq
        // The inner Seq specifies the list of modules that should be grouped together in a single partition.
        // The outer Seq specifies the list of partition groups.
        // Each partition group is mapped onto a separate FPGA.
        new WithPartitionGlobalInfo(
          Seq(
            Seq("RocketTile")
          )
        ) ++
        new BaseF1Config
    )

class RocketTileF1PCIMBase
    extends Config(
      new WithPartitionBase ++ // Base partition (SoC subsystem)
        new RocketTileF1Config
    )

class RocketTileF1PCIMPartition0
    extends Config(
      new WithPartitionIndex(0) ++ // Partition 0 containing the partitioned RocketTile
        new RocketTileF1Config
    )

2. Building Partitioned Sims: config_build_recipes.yaml

We can specify the config_build_recipes.yaml at this point. One thing to note is that we added the ExactMode_ in the PLATFORM_CONFIG field. This indicates to the FireAxe compiler to perform additional steps while partitioning so that the target behavior can be simulated in a cycle-exact manner.

################################################################################################
# Exact-mode : pull out a RocketTile out from your SoC
################################################################################################
f1_firesim_rocket_soc_exact:
  PLATFORM: f1
  TARGET_PROJECT: firesim
  DESIGN: FireSim
  TARGET_CONFIG: FireSimRocketConfig
  PLATFORM_CONFIG: ExactMode_RocketTileF1PCIMBase
  deploy_quintuplet: null
  platform_config_args:
    fpga_frequency: 90
    build_strategy: TIMING
  post_build_hook: null
  metasim_customruntimeconfig: null
  bit_builder_recipe: bit-builder-recipes/f1.yaml

f1_firesim_rocket_tile_exact:
  PLATFORM: f1
  TARGET_PROJECT: firesim
  DESIGN: FireSim
  TARGET_CONFIG: FireSimRocketConfig
  PLATFORM_CONFIG: ExactMode_RocketTileF1PCIMPartition0
  deploy_quintuplet: null
  platform_config_args:
    fpga_frequency: 90
    build_strategy: TIMING
  post_build_hook: null
  metasim_customruntimeconfig: null
  bit_builder_recipe: bit-builder-recipes/f1.yaml

3. Running Partitioned Simulations: user_topology.py

Again, we have to specify the deploy/runtools/user_topology.py to run FireAxe simulations.

    def fireaxe_rocket_exactmode_config(self) -> None:
        hwdb_entries = {
            0 : "f1_firesim_rocket_tile_exact",
            1 : "f1_firesim_rocket_soc_exact"
        }
        slotid_to_pidx = [0, 1]
        edges = [
            # DOC include start: fireaxe_rocket_exactmode_config edge 0
            FireAxeEdge(FireAxeNodeBridgePair(0, 0), FireAxeNodeBridgePair(1, 0)),
            # DOC include end: fireaxe_rocket_exactmode_config edge 0
            # DOC include start: fireaxe_rocket_exactmode_config edge 1
            FireAxeEdge(FireAxeNodeBridgePair(0, 1), FireAxeNodeBridgePair(1, 1))
            # DOC include end: fireaxe_rocket_exactmode_config edge 1
        ]
        # DOC include start: fireaxe_rocket_exactmode_config mode
        mode = PartitionMode.EXACT_MODE
        # DOC include end: fireaxe_rocket_exactmode_config mode
        self.fireaxe_topology_config(hwdb_entries, edges, slotid_to_pidx, mode)

We should go over a couple of changes that are made compared to the fast-mode configuration.

First of all the FireAxe topology specified by edges has changed. This is because in the exact-mode, the compiler has to generate multiple communication channels (or edges) in between the partitions in order to model combinational logic correctly.

The partitioning topology now looks like this:

The upper edge connecting partition 0’s bridge 0 to partition 1’s bridge 0 can be described like this:

            FireAxeEdge(FireAxeNodeBridgePair(0, 0), FireAxeNodeBridgePair(1, 0)),

The lower edge connecting partition 0’s bridge 1 to partition 1’s bridge 1 can be described like this:

            FireAxeEdge(FireAxeNodeBridgePair(0, 1), FireAxeNodeBridgePair(1, 1))

We also changed the partition mode to EXACT_MODE:

        mode = PartitionMode.EXACT_MODE

4. Running Partitioned Simulations: config_runtime.yaml

Now we can update config_runtime.yaml to run FireAxe simulations.

target_config:
    topology: fireaxe_rocket_exactmode_config