# Targets

FireSim generates SoC models by transforming RTL emitted by a Chisel generator, such as the Rocket SoC generator. Subject to conditions outlined in Restrictions on Target RTL, if it can be generated by Chisel, it can be simulated in FireSim.

## Restrictions on Target RTL

Current limitations in Golden Gate place the following restrictions on the (FIR)RTL that can be transformed and thus used in FireSim:

1. The top-level module must have no inputs or outputs. Input stimulus and output capture must be implemented using target RTL or target-to-host Bridges.

2. All target clocks must be generated by a single RationalClockBridge.

3. Black boxes must be “clock-gateable” by replacing its input clock with a gated equivalent which will be used to stall simulation time in that module.

1. As a consequence, target clock-gating cannot be implemented using black-box primitives, and must instead be modeled by adding clock-enables to all state elements of the gated clock domain (i.e., by adding an enable or feedback mux on registers to conditionally block updates, and by gating write-enables on memories).

4. Asynchronous reset must only be implemented using Rocket Chip’s black-box async reset. These are replaced with synchronously reset registers using a FIRRTL transformation.

### Including Verilog IP

FireSim now supports target designs that incorporate Verilog IP using the standard BlackBox interface from Chisel. For an example of how to add Verilog IP to a target system based on Rocket Chip, see the Incorporating Verilog Blocks section of the Chipyard documentation.

1. For the transform to work, the Chisel Blackbox that wraps the Verilog IP must have input clocks that can safely be clock-gated.

2. The compiler that produces the decoupled simulator (“FAME Transform”) automatically recognizes such blackboxes inside the target design.

3. The compiler automatically gates each clock of the Verilog IP to ensure that it deterministically advances in lockstep with the rest of the simulator.

4. This allows any Verilog module, subject to the constraint above, to be instantiated anywhere in the target design using the standard Chisel Blackbox interface.

### Multiple Clock Domains

FireSim can support simulating targets that have multiple clock domains. As stated above, all clocks must be generated using a single RationalClockBridge. For most users the default FireSim test harness in Chipyard will suffice, if you need to define a custom test harness instantiate the RationalClockBridge like so:

  // Here we request three target clocks (the base clock is implicit). All
// clocks beyond the base clock are specified using the RationalClock case
// class which gives the clock domain's name, and its clock multiplier and
// divisor relative to the base clock.
val clockBridge = RationalClockBridge(RationalClock("HalfRate", 1, 2),
RationalClock("ThirdRate", 1, 3))

// The clock bridge has a single output: a Vec[Clock] of the requested clocks
// in the order they were specified, which we are now free to use through our
// Chisel design.  While not necessary, here we unassign the Vec to give them
val Seq(fullRate, halfRate, thirdRate) = clockBridge.io.clocks.toSeq


Further documentation can be found in the source (sim/midas/src/main/scala/midas/widgets/ClockBridge.scala).

#### The Base Clock

By convention, target time is specified in cycles of the base clock, which is defined to be the clock of the RationalClockBridge whose clock ratio (multiplier / divisor) is one. While we suggest making the base clock the fastest clock in your system, as in any microprocessor-based system it will likely correspond to your core clock frequency, this is not a constraint.

#### Limitations:

• The number of target clocks FireSim can simulate is bounded by the number of BUFGCE resources available on the host FPGA, as these are used to independently clock-gate each target clock.

• As its name suggests, the RationalClockBridge can only generate target clocks that are rationally related. Specifically, all requested frequencies must be expressable in the form:

$f_{i} = \frac{f_{lcm}}{k_{i}}$
Where,
• $$f_{i}$$ is the desired frequency of the $$i^{th}$$ clock

• $$f_{lcm}$$, is the least-common multiple of all requested frequencies

• $$k_{i}$$ is a 16-bit unsigned integer

An arbitrary frequency can be modeled using a sufficiently precise rational multiple. Golden Gate will raise a compile-time error if it cannot support a desired frequency.

• Each bridge module must reside entirely within a single clock domain. The Bridge’s target interface must contain a single input clock, and all inputs and outputs of the bridge module must be latched and launched, respectively, by registers in the same clock domain.

## Target-Side FPGA Constraints

FireSim provides utilities to generate Xilinx Design Constraints (XDC) from string snippets in target’s Chisel. Golden Gate collects these annotations and emits separate xdc files for synthesis and implementation. See FPGA Build Files for a complete listing of output files used in FPGA compilation.

### RAM Inference Hints

Vivado generally does a reasonable job inferring embedded memories from FireSim-generated RTL, though there are some cases in which it must be coaxed. For example:

• Due to insufficient BRAM resources, you may wish to use URAM for a memory that’d infer as BRAM.

• If Vivado can’t find pipeline registers to absorb into a URAM or none exist in the target, you may get an warning like:

[Synth 8-6057] Memory: "<memory>" defined in module: "<module>" implemented as Ultra-Ram
has no pipeline registers. It is recommended to use pipeline registers to achieve high
performance.


Since Golden Gate modifies the module hierarchy extensively, it’s highly desirable to annotate these memories in the Chisel source so that their hints may move with the memory instance. This is a more robust alternative to relying on wildcard / glob matches from a static XDC specification.

Chisel memories can be annotated in situ like so:

    import midas.targetutils.xdc._
RAMStyleHint(mem, RAMStyles.ULTRA)
// Alternatively: RAMStyleHint(mem, RAMStyles.BRAM)


Alternatively, you can “dot-in” (traverse public members of a Scala class hierarchy) to annotate a memory in a submodule. Here’s an example:

    val modA = Module(new SyncReadMemModule(None))
RAMStyleHint(modA.mem, RAMStyles.ULTRA)
RAMStyleHint(modB.mem, RAMStyles.BRAM)


These annotations can be deployed anywhere: in the target, in bridge modules, and in internal FireSim RTL. The resulting constraints should appear the synthesis xdc file emitted by Golden Gate. For more information see the ScalaDoc for RAMStyleHint or read the source located at: sim/midas/targetutils/src/main/scala/midas/xdc/RAMStyleHint.scala.

## Provided Target Designs

### Target Generator Organization

FireSim provides multiple projects, each for a different type of target. Each project has its own chisel generator that invokes Golden Gate, its own driver sources, and a makefrag that plugs into the Make-based build system that resides in sim/. These projects are:

1. firesim (Default): rocket chip-based targets. These include targets with either BOOM or rocket pipelines, and should be your starting point if you’re building an SoC with the Rocket Chip generator.

2. midasexamples: the Golden Gate example designs. Located at sim/src/main/scala/midasexamples, these are a set of simple chisel circuits like GCD, that demonstrate how to use Golden Gate. These are useful test cases for bringing up new Golden Gate features.

3. fasedtests: designs to do integration testing of FASED memory-system timing models.

Projects have the following directory structure:

sim/
├-Makefile # Top-level makefile for projects where FireSim is the top-level repo
├-Makefrag # Target-agnostic makefrag, with recipes to generate drivers and RTL simulators
├-src/main/scala/{target-project}/
│                └─Makefrag # Defines target-specific make variables and recipes.
├-src/main/cc/{target-project}/
│             ├─{driver-csrcs}.cc # The target's simulation driver, and sofware-model sources
└-src/main/makefrag/{target-project}/
├─Generator.scala # Contains the main class that generates target RTL and calls Golden Gate
└─{other-scala-sources}.scala


### Specifying A Target Instance

To generate a specific instance of a target, the build system leverages four Make variables:

1. TARGET_PROJECT: this points the Makefile (sim/Makefile) at the right target-specific Makefrag, which defines the generation and metasimulation software recipes. The makefrag for the default target project is defined at sim/src/main/makefrag/firesim.

2. DESIGN: the name of the top-level Chisel module to generate (a Scala class name). These are defined in FireChip Chipyard generator.

3. TARGET_CONFIG: specifies a Config instance that is consumed by the target design’s generator. For the default firesim target project, predefined configs are described in in the FireChip Chipyard generator.

4. PLATFORM_CONFIG: specifies a Config instance that is consumed by Golden Gate and specifies compiler-level and host-land parameters, such as whether to enable assertion synthesis, or multi-ported RAM optimizations. Common platform configs are described in firesim-lib/sim/src/main/scala/configs/CompilerConfigs.scala).

TARGET_CONFIG and PLATFORM_CONFIG are strings that are used to construct a Config instance (derives from RocketChip’s parameterization system, Config, see freechips.rocketchip.config). These strings are of the form “{…_}{<Class Name>_}<Class Name>”. Only the final, base class name is compulsory: class names that are prepended with “_” are used to create a compound Config instance.

// Specify by setting TARGET_CONFIG=Base
class Base extends Config((site, here, up) => {...})
class Override1 extends Config((site, here, up) => {...})
class Override2 extends Config((site, here, up) => {...})
// Specify by setting TARGET_CONFIG=Compound
class Compound extends Config(new Override2 ++ new Override1 ++ new Base)
// OR by setting TARGET_CONFIG=Override2_Override1_Base
// Can specify undefined classes this way. ex: TARGET_CONFIG=Override2_Base


With this scheme, you don’t need to define a Config class for every instance you wish to generate. We use this scheme to specify FPGA frequencies (eg. “BaseF1Config_F90MHz”) in manager build recipes, but it’s also very useful for doing sweeping over a parameterization space.

Note that the precedence of Configs decreases from left to right in a string. Appending a config to an existing one will only have an effect if it sets a field not already set in higher precendence Configs. For example, “BaseF1Config_F90MHz” is equivalent to “BaseF1Config_F90MHz_F80MHz” as DesiredHostFrequency resolves to 90 MHz, but “F90MHz_BaseF1Config” is distinct from “F80MHz_F90MHz_BaseF1Config” in that DesiredHostFrequency resolves to 90 and 80 MHz respectively.

How a particular Config resolves it’s Field s can be unintuitive for complex compound Config s. One precise way to check a config is doing what you expect is to open the scala REPL, instantiate an instance of the desired Config, and inspect its fields.

$make sbt # Launch into SBT's shell with extra FireSim arguments sbt:firechip> console # Launch the REPL scala> val inst = (new firesim.firesim.FireSimRocketChipConfig).toInstance # Make an instance inst: freechips.rocketchip.config.Config = FireSimRocketChipConfig scala> import freechips.rocketchip.subsystem._ # Get some important Fields import freechips.rocketchip.subsystem.RocketTilesKey scala> inst(RocketTilesKey).size # Query number of cores res2: Int = 1 scala> inst(RocketTilesKey).head.dcache.get.nWays # Query L1 D$ associativity

res3: Int = 4


## Rocket Chip Generator-based SoCs (firesim project)

Using the Make variables listed above, we give examples of generating different targets using the default Rocket Chip-based target project.

### Rocket-based SoCs

Three design classes use Rocket scalar in-order pipelines.

Single core, Rocket pipeline (default)

make TARGET_CONFIG=FireSimRocketConfig


Single-core, Rocket pipeline, with network interface

make TARGET_CONFIG=WithNIC_FireSimRocketChipConfig


make TARGET_CONFIG=FireSimQuadRocketConfig


### BOOM-based SoCs

The BOOM (Berkeley Out-of-Order Machine) superscalar out-of-order pipelines can also be used with the same design classes that the Rocket pipelines use. Only the TARGET_CONFIG needs to be changed, as shown below:

Single-core BOOM

make TARGET_CONFIG=FireSimLargeBoomConfig


Single-core BOOM, with network interface

make TARGET_CONFIG=WithNIC_FireSimBoomConfig


### Generating A Different FASED Memory-Timing Model Instance

Golden Gate’s memory-timing model generator, FASED, can elaborate a space of different DRAM model instances: we give some typical ones here. These targets use the Makefile-defined defaults of DESIGN=FireSim PLATFORM_CONFIG=BaseF1Config.

make TARGET_CONFIG=DDR3FRFCFS_FireSimRocketConfig


As above, but with a 4 MiB (maximum simulatable capacity) last-level-cache model

make TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimRocketConfig


FASED timing-model configurations are passed to the FASED Bridges in your Target’s FIRRTL, and so must be prepended to TARGET_CONFIG.

## Midas Examples (midasexamples project)

This project can generate a handful of toy target-designs (set with the make variable DESIGN). Each of these designs has their own chisel source file and serves to demostrate the features of Golden Gate.

Some notable examples are:

1. GCD: the “Hello World!” of hardware.

2. WireInterconnect: demonstrates how combinational paths can be modeled with Golden Gate.

3. PrintfModule: demonstrates synthesizable printfs

4. AssertModule: demonstrates synthesizable assertions

To generate a target, set the make variable TARGET_PROJECT=midasexamples. so that the right project makefrag is sourced.

### Examples

To generate the GCD midasexample:

make DESIGN=GCD TARGET_PROJECT=midasexamples


## FASED Tests (fasedtests project)

This project generates target designs capable of driving considerably more bandwidth to an AXI4-memory slave than current FireSim targets. These are used to do integration and stress testing of FASED instances.

### Examples

Generate a synthesizable AXI4Fuzzer (based off of Rocket Chip’s TL fuzzer), driving a DDR3 FR-FCFS-based FASED instance.

make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=FRFCFSConfig


As above, now configured to drive 10 million transactions through the instance.

make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=NT10e7_FRFCFSConfig