# 4. Manager Configuration Files¶

This page contains a centralized reference for all of the configuration options in config_runtime.ini, config_build.ini, config_build_recipes.ini, and config_hwdb.ini.

## 4.1. config_runtime.ini¶

Here is a sample of this configuration file:

# RUNTIME configuration for the FireSim Simulation Manager
# documentation of all of these params.

[runfarm]
runfarmtag=mainrunfarm

f1_16xlarges=1
m4_16xlarges=0
f1_4xlarges=0
f1_2xlarges=0

runinstancemarket=ondemand
spotinterruptionbehavior=terminate
spotmaxprice=ondemand

[targetconfig]
#Set topology=no_net_config to run without a network simulation
topology=example_8config
no_net_num_nodes=2
switchinglatency=10
netbandwidth=200
profileinterval=-1

# This references a section from config_hwconfigs.ini
# In homogeneous configurations, use this to set the hardware config deployed
# for all simulators

[tracing]
enable=no

# Trace output formats. Only enabled if "enable" is set to "yes" above
# 0 = human readable; 1 = binary (compressed raw data); 2 = flamegraph (stack
# unwinding -> Flame Graph)
output_format=0

# Trigger selector.
# 0 = no trigger; 1 = cycle count trigger; 2 = program counter trigger; 3 =
# instruction trigger
selector=1
start=0
end=-1

[autocounter]

terminateoncompletion=no
suffixtag=

[hostdebug]
# When enabled (=yes), Zeros-out FPGA-attached DRAM before simulations
# begin (takes 2-5 minutes).
# In general, this is not required to produce deterministic simulations on
# target machines running linux. Enable if you observe simulation non-determinism.
zerooutdram=no

[synthprint]
# Start and end cycles for outputting synthesized prints.
# They are given in terms of the base clock and will be converted
# for each clock domain.
start=0
end=-1
# When enabled (=yes), prefix print output with the target cycle at which the print was triggered
cycleprefix=yes


Below, we outline each section and parameter in detail.

### 4.1.1. [runfarm]¶

The [runfarm] options below allow you to specify the number, types, and other characteristics of instances in your FireSim Run Farm, so that the manager can automatically launch them, run workloads on them, and terminate them.

#### 4.1.1.1. runfarmtag¶

Use runfarmtag to differentiate between different Run Farms in FireSim. Having multiple config_runtime.ini files with different runfarmtag values allows you to run many experiments at once from the same manager instance.

The instances launched by the launchrunfarm command will be tagged with this value. All later operations done by the manager rely on this tag, so you should not change it unless you are done with your current Run Farm.

Per AWS restrictions, this tag can be no longer than 255 characters.

#### 4.1.1.2. f1_16xlarges, m4_16xlarges, f1_4xlarges, f1_2xlarges¶

Set these three values respectively based on the number and types of instances you need. While we could automate this setting, we choose not to, so that users are never surprised by how many instances they are running.

Note that these values are ONLY used to launch instances. After launch, the manager will query the AWS API to find the instances of each type that have the runfarmtag set above assigned to them.

#### 4.1.1.3. runinstancemarket¶

You can specify either spot or ondemand here, to use one of those markets on AWS.

#### 4.1.1.4. spotinterruptionbehavior¶

When runinstancemarket=spot, this value determines what happens to an instance if it receives the interruption signal from AWS. You can specify either hibernate, stop, or terminate.

#### 4.1.1.5. spotmaxprice¶

When runinstancemarket=spot, this value determines the max price you are willing to pay per instance, in dollars. You can also set it to ondemand to set your max to the on-demand price for the instance.

### 4.1.2. [targetconfig]¶

The [targetconfig] options below allow you to specify the high-level configuration of the target you are simulating. You can change these parameters after launching a Run Farm (assuming you have the correct number of instances), but in many cases you will need to re-run the infrasetup command to make sure the correct simulation infrastructure is available on your instances.

#### 4.1.2.1. topology¶

This field dictates the network topology of the simulated system. Some examples:

no_net_config: This runs N (see no_net_num_nodes below) independent simulations, without a network simulation. You can currently only use this option if you build one of the NoNIC hardware configs of FireSim.

example_8config: This requires a single f1.16xlarge, which will simulate 1 ToR switch attached to 8 simulated servers.

example_16config: This requires two f1.16xlarge instances and one m4.16xlarge instance, which will simulate 2 ToR switches, each attached to 8 simulated servers, with the two ToR switches connected by a root switch.

example_64config: This requires eight f1.16xlarge instances and one m4.16xlarge instance, which will simulate 8 ToR switches, each attached to 8 simulated servers (for a total of 64 nodes), with the eight ToR switches connected by a root switch.

Additional configurations are available in deploy/runtools/user_topology.py and more can be added there. See the Manager Network Topology Definitions (user_topology.py) section for more info.

#### 4.1.2.2. no_net_num_nodes¶

This determines the number of simulated nodes when you are using topology=no_net_config.

#### 4.1.2.3. linklatency¶

In a networked simulation, this allows you to specify the link latency of the simulated network in CYCLES. For example, 6405 cycles is roughly 2 microseconds at 3.2 GHz. A current limitation is that this value (in cycles) must be a multiple of 7. Furthermore, you must not exceed the buffer size specified in the NIC’s simulation widget.

#### 4.1.2.4. switchinglatency¶

In a networked simulation, this specifies the minimum port-to-port switching latency of the switch models, in CYCLES.

#### 4.1.2.5. netbandwidth¶

In a networked simulation, this specifies the maximum output bandwidth that a NIC is allowed to produce as an integer in Gbit/s. Currently, this must be a number between 1 and 200, allowing you to model NICs between 1 and 200 Gbit/s.

#### 4.1.2.6. profileinterval¶

The simulation driver periodically samples performance counters in FASED timing model instances and dumps the result to a file on the host. profileinterval defines the number of target cycles between samples; setting this field to -1 disables polling.

#### 4.1.2.7. defaulthwconfig¶

This sets the server configuration launched by default in the above topologies. Heterogeneous configurations can be achieved by manually specifying different names within the topology itself, but all the example_Nconfig configurations are homogeneous and use this value for all nodes.

You should set this to one of the hardware configurations you have defined already in config_hwdb.ini. You should set this to the NAME (section title) of the hardware configuration from config_hwdb.ini, NOT the actual agfi itself (NOT something like agfi-XYZ...).

### 4.1.3. [tracing]¶

This section manages TracerV-based tracing at simulation runtime. For more details, see the Capturing RISC-V Instruction Traces with TracerV page for more details.

#### 4.1.3.1. enable¶

This turns tracing on, when set to yes and off when set to no. See the Enabling Tracing at Runtime.

#### 4.1.3.2. output_format¶

This sets the output format for TracerV tracing. See the Selecting a Trace Output Format section.

#### 4.1.3.3. selector, start, and end¶

These configure triggering for TracerV. See the Setting a TracerV Trigger section.

### 4.1.4. [autocounter]¶

This section configures AutoCounter. See the AutoCounter: Profiling with Out-of-Band Performance Counter Collection page for more details.

#### 4.1.4.1. readrate¶

This sets the rate at which AutoCounters are read. See the AutoCounter Runtime Parameters section for more details.

### 4.1.5. [workload]¶

This section defines the software that will run on the simulated system.

#### 4.1.5.1. workloadname¶

This selects a workload to run across the set of simulated nodes. A workload consists of a series of jobs that need to be run on simulated nodes (one job per node).

Workload definitions are located in firesim/deploy/workloads/*.json.

linux-uniform.json: This runs the default FireSim Linux distro on as many nodes as you specify when setting the [targetconfig] parameters.

spec17-intrate.json: This runs SPECint 2017’s rate benchmarks. In this type of workload, you should launch EXACTLY the correct number of nodes necessary to run the benchmark. If you specify fewer nodes, the manager will warn that not all jobs were assigned to a simulation. If you specify too many simulations and not enough jobs, the manager will not launch the jobs.

Others can be found in the aforementioned directory. For a description of the JSON format, see Defining Custom Workloads.

#### 4.1.5.2. terminateoncompletion¶

Set this to no if you want your Run Farm to keep running once the workload has completed. Set this to yes if you want your Run Farm to be TERMINATED after the workload has completed and results have been copied off.

#### 4.1.5.3. suffixtag¶

This allows you to append a string to a workload’s output directory name, useful for differentiating between successive runs of the same workload, without renaming the entire workload. For example, specifying suffixtag=test-v1 with a workload named super-application will result in a workload results directory named results-workload/DATE--TIME-super-application-test-v1/.

### 4.1.6. [hostdebug]¶

#### 4.1.6.1. zerooutdram¶

Set this to yes to zero-out FPGA-attached DRAM before simulation begins. This process takes 2-5 minutes. In general, this is not required to produce deterministic simulations on target machines running linux, but should be enabled if you observe simulation non-determinism.

## 4.2. config_build.ini¶

Here is a sample of this configuration file:

# BUILDTIME/AGFI management configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.

[afibuild]

buildinstancemarket=ondemand
spotinterruptionbehavior=terminate
spotmaxprice=ondemand
postbuildhook=

[builds]
# this section references builds defined in config_build_recipes.ini
# if you add a build here, it will be built when you run buildafi
firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3
firesim-boom-singlecore-nic-l2-llc4mb-ddr3

firesim-supernode-rocket-singlecore-nic-l2-lbp
#firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3-ramopts

# SHA3 configs for tutorial
# firesim-singlecore-sha3-no-nic-l2-llc4mb-ddr3
# firesim-singlecore-sha3-print-no-nic-l2-llc4mb-ddr3

[agfistoshare]
firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3
firesim-boom-singlecore-nic-l2-llc4mb-ddr3

firesim-supernode-rocket-singlecore-nic-l2-lbp
#firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3-ramopts

# SHA3 configs for tutorial
# firesim-singlecore-sha3-no-nic-l2-llc4mb-ddr3
# firesim-singlecore-sha3-print-no-nic-l2-llc4mb-ddr3

[sharewithaccounts]
# To share with a specific user:
somebodysname=123456789012
# To share publicly:
#public=public


Below, we outline each section and parameter in detail.

### 4.2.1. [afibuild]¶

This exposes options for AWS resources used in the process of building FireSim AGFIs (FPGA Images).

#### 4.2.1.1. s3bucketname¶

This is used behind the scenes in the AGFI creation process. You will only ever need to access this bucket manually if there is a failure in AGFI creation in Amazon’s backend.

Naming rules: this must be all lowercase and you should stick to letters and numbers.

The first time you try to run a build, the FireSim manager will try to create the bucket you name here. If the name is unavailable, it will complain and you will need to change this name. Once you choose a working name, you should never need to change it.

In general, firesim-yournamehere is a good choice.

#### 4.2.1.2. buildinstancemarket¶

You can specify either spot or ondemand here, to use one of those markets on AWS.

#### 4.2.1.3. spotinterruptionbehavior¶

When buildinstancemarket=spot, this value determines what happens to an instance if it receives the interruption signal from AWS. You can specify either hibernate, stop, or terminate.

#### 4.2.1.4. spotmaxprice¶

When buildinstancemarket=spot, this value determines the max price you are willing to pay per instance, in dollars. You can also set it to ondemand to set your max to the on-demand price for the instance.

#### 4.2.1.5. postbuildhook¶

(Optional) Provide an a script to run on the results copied back from a _single_ build instance. Upon completion of each design’s build, the manager invokes this script and passing the absolute path to that instance’s build-results directory as it’s first argument.

### 4.2.2. [builds]¶

In this section, you can list as many build entries as you want to run for a particular call to the buildafi command (see config_build_recipes.ini below for how to define a build entry). For example, if we want to run the builds named [awesome-firesim-config] and [quad-core-awesome-firesim-config], we would write:

[builds]
awesome-firesim-config


### 4.2.3. [agfistoshare]¶

This is used by the shareagfi command to share the specified agfis with the users specified in the next ([sharewithaccounts]) section. In this section, you should specify the section title (i.e. the name you made up) for a hardware configuration in config_hwdb.ini. For example, to share the hardware config:

[firesim-rocket-quadcore-nic-l2-llc4mb-ddr3]
# this is a comment that describes my favorite configuration!
agfi=agfi-0a6449b5894e96e53
deploytripletoverride=None
customruntimeconfig=None


you would use:

[agfistoshare]


### 4.2.4. [sharewithaccounts]¶

A list of AWS account IDs that you want to share the AGFIs listed in [agfistoshare] with when calling the manager’s shareagfi command. You should specify names in the form usersname=AWSACCTID. The left-hand-side is just for human readability, only the actual account IDs listed here matter. If you specify public=public here, the AGFIs are shared publicly, regardless of any other entires that are present.

## 4.3. config_build_recipes.ini¶

Here is a sample of this configuration file:

# Build-time design configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.

# this file contains sections that describe hardware designs that /can/ be built.
# edit config_build.ini to actually "turn on" a config to be built when you run
# buildafi

# Note: For large designs (ones that would fill a EC2.2xlarge/Xilinx VU9P)
# Vivado uses in excess of 32 GiB. Keep this in mind when selecting a
# non-default instancetype.

DESIGN=FireSim
PLATFORM_CONFIG=F90MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

DESIGN=FireSim
PLATFORM_CONFIG=F90MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# Single-core, BOOM-based recipes
[firesim-boom-singlecore-nic-l2-llc4mb-ddr3]
DESIGN=FireSim
TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimLargeBoomConfig
PLATFORM_CONFIG=F65MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

[firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3]
DESIGN=FireSim
TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimLargeBoomConfig
PLATFORM_CONFIG=F70MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# Single-core, Ariane-based recipes
[firesim-ariane-singlecore-no-nic-l2-llc4mb-ddr3]
DESIGN=FireSim
TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimArianeConfig
PLATFORM_CONFIG=F90MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# Single-core, Rocket-based recipes with Gemmini
[firesim-rocket-singlecore-gemmini-no-nic-l2-llc4mb-ddr3]
DESIGN=FireSim
TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimGemminiRocketConfig
PLATFORM_CONFIG=F110MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# RAM Optimizations enabled by adding _MCRams PLATFORM_CONFIG string
[firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3-ramopts]
DESIGN=FireSim
TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimLargeBoomConfig
PLATFORM_CONFIG=MCRams_F90MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# Supernode configurations -- multiple instances of an SoC in a single simulator
[firesim-supernode-rocket-singlecore-nic-l2-lbp]
DESIGN=FireSim
TARGET_CONFIG=WithNIC_SupernodeFireSimRocketConfig
PLATFORM_CONFIG=F85MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# Multiclock Temporary Example:ncept Quad-core, Rocket-based recipes
DESIGN=FireSimMulticlockPOC
PLATFORM_CONFIG=F90MHz_BaseF1Config
instancetype=z1d.2xlarge
deploytriplet=None

# MIDAS Examples -- BUILD SUPPORT ONLY; Can't launch driver correctly on runfarm
[midasexamples-gcd]
TARGET_PROJECT=midasexamples
DESIGN=GCD
TARGET_CONFIG=NoConfig
PLATFORM_CONFIG=DefaultF1Config
instancetype=z1d.2xlarge
deploytriplet=None


Below, we outline each section and parameter in detail.

### 4.3.1. Build definition sections, e.g. [awesome-firesim-config]¶

In this file, you can specify as many build definition sections as you want, each with a header like [awesome-firesim-config] (i.e. a nice, short name you made up). Such a section must contain the following fields:

#### 4.3.1.1. DESIGN¶

This specifies the basic target design that will be built. Unless you are defining a custom system, this should be set to FireSim. We describe this in greater detail in Generating Different Targets).

#### 4.3.1.2. TARGET_CONFIG¶

This specifies the hardware configuration of the target being simulated. Some examples include FireSimRocketConfig and FireSimQuadRocketConfig. We describe this in greater detail in Generating Different Targets).

#### 4.3.1.3. PLATFORM_CONFIG¶

This specifies hardware parameters of the simulation environment - for example, selecting between a Latency-Bandwidth Pipe or DDR3 memory models. These are defined in firesim/sim/src/main/scala/firesim/SimConfigs.scala. We specify the host FPGA frequency in the PLATFORM_CONFIG by appending a frequency Config with an underscore (ex. BaseF1Config_F160MHz). We describe this in greater detail in Generating Different Targets).

#### 4.3.1.4. instancetype¶

This defines the type of instance that the build will run on. Generally, running on a z1d.2xlarge is sufficient. In our experience, using more powerful instances than this provides little gain.

#### 4.3.1.5. deploytriplet¶

This allows you to override the deploytriplet stored with the AGFI. Otherwise, the DESIGN/TARGET_CONFIG/PLATFORM_CONFIG you specify above will be used. See the AGFI Tagging section for more details. Most likely, you should leave this set to None. This is usually only used if you have proprietary RTL that you bake into an FPGA image, but don’t want to share with users of the simulator.

#### 4.3.1.6. TARGET_PROJECT(Optional)¶

This specifies the target project in which the target is defined (this is described in greater detail here). If TARGET_PROJECT is undefined the manager will default to firesim. Setting TARGET_PROJECT is required for building the MIDAS examples (TARGET_PROJECT=midasexamples) with the manager, or for building a user-provided target project.

## 4.4. config_hwdb.ini¶

Here is a sample of this configuration file:

# Hardware config database for FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.

# Hardware configs represent a combination of an agfi, a deploytriplet override
# (if needed), and a custom runtime config (if needed)

# The AGFIs provided below are public and available to all users.
# Only AGFIs for the latest release of FireSim are guaranteed to be available.
# If you are using an older version of FireSim, you will need to generate your
# own images.

[firesim-boom-singlecore-nic-l2-llc4mb-ddr3]
agfi=agfi-02c64d9441726e956
deploytripletoverride=None
customruntimeconfig=None

[firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3]
agfi=agfi-01057aa9ca7f38969
deploytripletoverride=None
customruntimeconfig=None

agfi=agfi-009e7a5f7c315a6ec
deploytripletoverride=None
customruntimeconfig=None

agfi=agfi-0e6e54cd1923411cf
deploytripletoverride=None
customruntimeconfig=None

deploytripletoverride=None
customruntimeconfig=None

[firesim-supernode-rocket-singlecore-nic-l2-lbp]
agfi=agfi-0d814ea4b81eb484e
deploytripletoverride=None
customruntimeconfig=None


This file tracks hardware configurations that you can deploy as simulated nodes in FireSim. Each such configuration contains a name for easy reference in higher-level configurations, defined in the section header, an agfi, which represents the FPGA image, a custom runtime config, if one is needed, and a deploy triplet override if one is necessary.

When you build a new AGFI, you should put the default version of it in this file so that it can be referenced from your other configuration files.

The following is an example section from this file - you can add as many of these as necessary:

[firesim-rocket-quadcore-nic-l2-llc4mb-ddr3]
# this is a comment that describes my favorite configuration!
agfi=agfi-0a6449b5894e96e53
deploytripletoverride=None
customruntimeconfig=None


### 4.4.1. [NAME_GOES_HERE]¶

In this example, firesim-rocket-quadcore-nic-l2-llc4mb-ddr3 is the name that will be used to reference this hardware design in other configuration locations. The following items describe this hardware configuration:

#### 4.4.1.1. agfi¶

This represents the AGFI (FPGA Image) used by this hardware configuration.

#### 4.4.1.2. deploytripletoverride¶

This is an advanced feature - under normal conditions, you should leave this set to None, so that the manager uses the configuration triplet that is automatically stored with the AGFI at build time. Advanced users can set this to a different value to build and use a different driver when deploying simulations. Since the driver depends on logic now hardwired into the FPGA bitstream, drivers cannot generally be changed without requiring FPGA recompilation.

#### 4.4.1.3. customruntimeconfig¶

This is an advanced feature - under normal conditions, you can use the default parameters generated automatically by the simulator by setting this field to None. If you want to customize runtime parameters for certain parts of the simulation (e.g. the DRAM model’s runtime parameters), you can place a custom config file in sim/custom-runtime-configs/. Then, set this field to the relative name of the config. For example, sim/custom-runtime-configs/GREATCONFIG.conf becomes customruntimeconfig=GREATCONFIG.conf.

### 4.4.2. Add more hardware config sections, like [NAME_GOES_HERE_2]¶

You can add as many of these entries to config_hwdb.ini as you want, following the format discussed above (i.e. you provide agfi, deploytripletoverride, or customruntimeconfig).