4. Manager Configuration Files¶
This page contains a centralized reference for all of the configuration options
in config_runtime.ini
, config_build.ini
, config_build_recipes.ini
,
and config_hwdb.ini
.
4.1. config_runtime.ini
¶
Here is a sample of this configuration file:
# RUNTIME configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.
[runfarm]
runfarmtag=mainrunfarm
f1_16xlarges=1
m4_16xlarges=0
f1_2xlarges=0
runinstancemarket=ondemand
spotinterruptionbehavior=terminate
spotmaxprice=ondemand
[targetconfig]
topology=example_8config
no_net_num_nodes=2
linklatency=6405
switchinglatency=10
netbandwidth=200
# This references a section from config_hwconfigs.ini
# In homogeneous configurations, use this to set the hardware config deployed
# for all simulators
defaulthwconfig=firesim-quadcore-nic-ddr3-llc4mb
[workload]
workloadname=linux-uniform.json
terminateoncompletion=no
Below, we outline each section and parameter in detail.
4.1.1. [runfarm]
¶
The [runfarm]
options below allow you to specify the number, types, and
other characteristics of instances in your FireSim Run Farm, so that the
manager can automatically launch them, run workloads on them, and terminate
them.
4.1.1.1. runfarmtag
¶
Use runfarmtag
to differentiate between different Run Farms in FireSim.
Having multiple config_runtime.ini
files with different runfarmtag
values allows you to run many experiments at once from the same manager instance.
The instances launched by the launchrunfarm
command will be tagged with
this value. All later operations done by the manager rely on this tag, so
you should not change it unless you are done with your current Run Farm.
Per AWS restrictions, this tag can be no longer than 255 characters.
4.1.1.2. f1_16xlarges
, m4_16xlarges
, f1_2xlarges
¶
Set these three values respectively based on the number and types of instances you need. While we could automate this setting, we choose not to, so that users are never surprised by how many instances they are running.
Note that these values are ONLY used to launch instances. After launch, the
manager will query the AWS API to find the instances of each type that have the
runfarmtag
set above assigned to them.
4.1.1.3. runinstancemarket
¶
You can specify either spot
or ondemand
here, to use one of those
markets on AWS.
4.1.1.4. spotinterruptionbehavior
¶
When runinstancemarket=spot
, this value determines what happens to an instance
if it receives the interruption signal from AWS. You can specify either
hibernate
, stop
, or terminate
.
4.1.1.5. spotmaxprice
¶
When runinstancemarket=spot
, this value determines the max price you are
willing to pay per instance, in dollars. You can also set it to ondemand
to set your max to the on-demand price for the instance.
4.1.2. [targetconfig]
¶
The [targetconfig]
options below allow you to specify the high-level
configuration of the target you are simulating. You can change these parameters
after launching a Run Farm (assuming you have the correct number of instances),
but in many cases you will need to re-run the infrasetup
command to make
sure the correct simulation infrastructure is available on your instances.
4.1.2.1. topology
¶
This field dictates the network topology of the simulated system. Some examples:
no_net_config
: This runs N (see no_net_num_nodes
below) independent
simulations, without a network simulation. You can currently only use this
option if you build one of the NoNIC hardware configs of FireSim.
example_8config
: This requires a single f1.16xlarge
, which will
simulate 1 ToR switch attached to 8 simulated servers.
example_16config
: This requires two f1.16xlarge
instances and one
m4.16xlarge
instance, which will
simulate 2 ToR switches, each attached to 8 simulated servers, with the two
ToR switches connected by a root switch.
example_64config
: This requires eight f1.16xlarge
instances and one
m4.16xlarge
instance, which will simulate 8 ToR switches, each attached to
8 simulated servers (for a total of 64 nodes), with the eight ToR switches
connected by a root switch.
Additional configurations are available in deploy/runtools/user_topology.py
and more can be added there. See the Manager Network Topology Definitions (user_topology.py) section
for more info.
4.1.2.2. no_net_num_nodes
¶
This determines the number of simulated nodes when you are using
topology=no_net_config
.
4.1.2.3. linklatency
¶
In a networked simulation, this allows you to specify the link latency of the simulated network in CYCLES. For example, 6405 cycles is roughly 2 microseconds at 3.2 GHz. A current limitation is that this value (in cycles) must be a multiple of 7. Furthermore, you must not exceed the buffer size specified in the NIC’s simulation widget.
4.1.2.4. switchinglatency
¶
In a networked simulation, this specifies the minimum port-to-port switching latency of the switch models, in CYCLES.
4.1.2.5. netbandwidth
¶
In a networked simulation, this specifies the maximum output bandwidth that a NIC is allowed to produce as an integer in Gbit/s. Currently, this must be a number between 1 and 200, allowing you to model NICs between 1 and 200 Gbit/s.
4.1.2.6. defaulthwconfig
¶
This sets the server configuration launched by default in the above topologies.
Heterogeneous configurations can be achieved by manually specifying different
names within the topology itself, but all the example_Nconfig
configurations
are homogeneous and use this value for all nodes.
You should set this to one of the hardware configurations you have defined already in
config_hwdb.ini
. You should set this to the NAME (section title) of the
hardware configuration from config_hwdb.ini
, NOT the actual agfi itself
(NOT something like agfi-XYZ...
).
4.1.3. [workload]
¶
This section defines the software that will run on the simulated system.
4.1.3.1. workloadname
¶
This selects a workload to run across the set of simulated nodes. A workload consists of a series of jobs that need to be run on simulated nodes (one job per node).
Workload definitions are located in firesim/deploy/workloads/*.json
.
Some sample workloads:
linux-uniform.json
: This runs the default FireSim Linux distro on as many nodes
as you specify when setting the [targetconfig]
parameters.
spec17-intrate.json
: This runs SPECint 2017’s rate benchmarks. In this type of
workload, you should launch EXACTLY the correct number of nodes necessary to run the
benchmark. If you specify fewer nodes, the manager will warn that not all jobs were
assigned to a simulation. If you specify too many simulations and not enough
jobs, the manager will not launch the jobs.
Others can be found in the aforementioned directory.
4.1.3.2. terminateoncompletion
¶
Set this to no
if you want your Run Farm to keep running once the workload
has completed. Set this to yes
if you want your Run Farm to be TERMINATED
after the workload has completed and results have been copied off.
4.2. config_build.ini
¶
Here is a sample of this configuration file:
# BUILDTIME/AGFI management configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.
[afibuild]
s3bucketname=firesim-yournamehere
buildinstancemarket=ondemand
spotinterruptionbehavior=terminate
spotmaxprice=ondemand
[builds]
# this section references builds defined in config_build_recipes.ini
# if you add a build here, it will be built when you run buildafi
firesim-singlecore-no-nic-lbp
firesim-singlecore-nic-lbp
firesim-quadcore-no-nic-lbp
firesim-quadcore-nic-lbp
firesim-quadcore-no-nic-ddr3-llc4mb
firesim-quadcore-nic-ddr3-llc4mb
fireboom-singlecore-no-nic-lbp
fireboom-singlecore-no-nic-ddr3-llc4mb
[agfistoshare]
firesim-singlecore-no-nic-lbp
firesim-singlecore-nic-lbp
firesim-quadcore-no-nic-lbp
firesim-quadcore-nic-lbp
firesim-quadcore-no-nic-ddr3-llc4mb
firesim-quadcore-nic-ddr3-llc4mb
fireboom-singlecore-no-nic-lbp
fireboom-singlecore-no-nic-ddr3-llc4mb
[sharewithaccounts]
somebodysname=123456789012
Below, we outline each section and parameter in detail.
4.2.1. [afibuild]
¶
This exposes options for AWS resources used in the process of building FireSim AGFIs (FPGA Images).
4.2.1.1. s3bucketname
¶
This is used behind the scenes in the AGFI creation process. You will only ever need to access this bucket manually if there is a failure in AGFI creation in Amazon’s backend.
Naming rules: this must be all lowercase and you should stick to letters and numbers.
The first time you try to run a build, the FireSim manager will try to create the bucket you name here. If the name is unavailable, it will complain and you will need to change this name. Once you choose a working name, you should never need to change it.
In general, firesim-yournamehere
is a good choice.
4.2.1.2. buildinstancemarket
¶
You can specify either spot
or ondemand
here, to use one of those
markets on AWS.
4.2.1.3. spotinterruptionbehavior
¶
When buildinstancemarket=spot
, this value determines what happens to an
instance if it receives the interruption signal from AWS. You can specify
either hibernate
, stop
, or terminate
.
4.2.1.4. spotmaxprice
¶
When buildinstancemarket=spot
, this value determines the max price you are
willing to pay per instance, in dollars. You can also set it to ondemand
to set your max to the on-demand price for the instance.
4.2.2. [builds]
¶
In this section, you can list as many build entries as you want to run
for a particular call to the buildafi
command (see
config_build_recipes.ini
below for how to define a build entry). For
example, if we want to run the builds named [awesome-firesim-config]
and [quad-core-awesome-firesim-config]
, we would
write:
[builds]
awesome-firesim-config
quad-core-awesome-firesim-config
4.3. config_build_recipes.ini
¶
Here is a sample of this configuration file:
# Build-time design configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.
# this file contains sections that describe hardware designs that /can/ be built.
# edit config_build.ini to actually "turn on" a config to be built when you run
# buildafi
[firesim-singlecore-nic-lbp]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipSingleCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-singlecore-no-nic-lbp]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipSingleCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-nic-lbp]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-no-nic-lbp]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-nic-ddr3-llc4mb]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-no-nic-ddr3-llc4mb]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
# BOOM-based targets
[fireboom-singlecore-no-nic-lbp]
DESIGN=FireBoomNoNIC
TARGET_CONFIG=FireSimBoomConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[fireboom-singlecore-no-nic-ddr3-llc4mb]
DESIGN=FireBoomNoNIC
TARGET_CONFIG=FireSimBoomConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
Below, we outline each section and parameter in detail.
4.3.1. Build definition sections, e.g. [awesome-firesim-config]
¶
In this file, you can specify as many build definition sections as you want,
each with a header like [awesome-firesim-config]
(i.e. a nice, short name
you made up). Such a section must contain the following fields:
4.3.1.1. DESIGN
¶
This specifies the basic target design that will be built. Unless you
are defining a custom system, this should either be FireSim
, for
systems with a NIC, or FireSimNoNIC
, for systems without a NIC. These
are defined in firesim/sim/src/main/scala/Targets.scala
.
4.3.1.2. TARGET_CONFIG
¶
This specifies the hardware configuration of the target being simulation. Some
examples include FireSimRocketChipConfig
and FireSimRocketChipQuadCoreConfig
.
These are defined in firesim/sim/src/main/scala/TargetConfigs.scala
.
4.3.1.3. PLATFORM_CONFIG
¶
This specifies hardware parameters of the simulation environment - for example,
selecting between a Latency-Bandwidth Pipe or DDR3 memory models.
These are defined in firesim/sim/src/main/scala/SimConfigs.scala
.
4.3.1.4. instancetype
¶
This defines the type of instance that the build will run on. Generally, running
on a c4.4xlarge
is sufficient. In our experience, using more powerful instances
than this provides little gain.
4.3.1.5. deploytriplet
¶
This allows you to override the deploytriplet
stored with the AGFI.
Otherwise, the DESIGN
/TARGET_CONFIG
/PLATFORM_CONFIG
you specify
above will be used. See the AGFI Tagging section for more details. Most likely,
you should leave this set to None
. This is usually only used if you have
proprietary RTL that you bake into an FPGA image, but don’t want to share with
users of the simulator.
4.4. config_hwdb.ini
¶
Here is a sample of this configuration file:
# Build-time design configuration for the FireSim Simulation Manager
# See docs/Advanced-Usage/Manager/Manager-Configuration-Files.rst for documentation of all of these params.
# this file contains sections that describe hardware designs that /can/ be built.
# edit config_build.ini to actually "turn on" a config to be built when you run
# buildafi
[firesim-singlecore-nic-lbp]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipSingleCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-singlecore-no-nic-lbp]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipSingleCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-nic-lbp]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-no-nic-lbp]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-nic-ddr3-llc4mb]
DESIGN=FireSim
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
[firesim-quadcore-no-nic-ddr3-llc4mb]
DESIGN=FireSimNoNIC
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
# BOOM-based targets
[fireboom-singlecore-no-nic-lbp]
DESIGN=FireBoomNoNIC
TARGET_CONFIG=FireSimBoomConfig
PLATFORM_CONFIG=FireSimConfig
instancetype=c4.4xlarge
deploytriplet=None
[fireboom-singlecore-no-nic-ddr3-llc4mb]
DESIGN=FireBoomNoNIC
TARGET_CONFIG=FireSimBoomConfig
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
instancetype=c4.4xlarge
deploytriplet=None
This file tracks hardware configurations that you can deploy as simulated nodes in FireSim. Each such configuration contains a name for easy reference in higher-level configurations, defined in the section header, an agfi, which represents the FPGA image, a custom runtime config, if one is needed, and a deploy triplet override if one is necessary.
When you build a new AGFI, you should put the default version of it in this file so that it can be referenced from your other configuration files.
The following is an example section from this file - you can add as many of these as necessary:
[firesim-quadcore-nic-ddr3-llc4mb]
# this is a comment that describes my favorite configuration!
agfi=agfi-0a6449b5894e96e53
deploytripletoverride=None
customruntimeconfig=None
4.4.1. [NAME_GOES_HERE]
¶
In this example, firesim-quadcore-nic-ddr3-llc4mb
is the name that will be
used to reference this hardware design in other configuration locations. The following
items describe this hardware configuration:
4.4.1.1. agfi
¶
This represents the AGFI (FPGA Image) used by this hardware configuration.
4.4.1.2. deploytripletoverride
¶
This is an advanced feature - under normal conditions, you should leave this set to None
, so that the
manager uses the configuration triplet that is automatically stored with the
AGFI at build time. Advanced users can set this to a different
value to build and use a different driver when deploying simulations. Since
the driver depends on logic now hardwired into the
FPGA bitstream, drivers cannot generally be changed without requiring FPGA
recompilation.
4.4.1.3. customruntimeconfig
¶
This is an advanced feature - under normal conditions, you can use the default
parameters generated automatically by the simulator by setting this field to
None
. If you want to customize runtime parameters for certain parts of
the simulation (e.g. the DRAM model’s runtime parameters), you can place
a custom config file in sim/custom-runtime-configs/
. Then, set this field
to the relative name of the config. For example,
sim/custom-runtime-configs/GREATCONFIG.conf
becomes
customruntimeconfig=GREATCONFIG.conf
.
4.4.2. Add more hardware config sections, like [NAME_GOES_HERE_2]
¶
You can add as many of these entries to config_hwdb.ini
as you want, following the format
discussed above (i.e. you provide agfi
, deploytripletoverride
, or customruntimeconfig
).