2.3. Setting up your Manager Instance¶
2.3.1. Launching a “Manager Instance”¶
Now, we need to launch a “Manager Instance” that acts as a
“head” node that we will ssh
or mosh
into to work from.
Since we will deploy the heavy lifting to separate c5.4xlarge
and
f1
instances later, the Manager Instance can be a relatively cheap instance.
In this guide, however, we will use a c5.4xlarge
,
running the AWS FPGA Developer AMI. (Be sure to subscribe to the AMI
if you have not done so. See Subscribe to the AWS FPGA Developer AMI. Note that it
might take a few minutes after subscribing to the AMI to be able to
launch instances using it.)
Head to the EC2 Management Console. In the top right corner, ensure that the correct region is selected.
To launch a manager instance, follow these steps:
From the main page of the EC2 Management Console, click
Launch Instance
. We use an on-demand instance here, so that your data is preserved when you stop/start the instance, and your data is not lost when pricing spikes on the spot market.When prompted to select an AMI, search in the
Community AMIs
tab forFPGA Developer AMI - 1.11.1-40257ab5-6688-4c95-97d1-e251a40fd1fc
and select the AMI that appears (there should be only one). DO NOT USE ANY OTHER VERSION.When prompted to choose an instance type, select the instance type of your choosing. A good choice is a
c5.4xlarge
.On the “Configure Instance Details” page:
First make sure that the
firesim
VPC is selected in the drop-down box next to “Network”. Any subnet within thefiresim
VPC is fine.Additionally, check the box for “Protect against accidental termination.” This adds a layer of protection to prevent your manager instance from being terminated by accident. You will need to disable this setting before being able to terminate the instance using usual methods.
Also on this page, expand “Advanced Details” and in the resulting text box, paste the following:
#!/bin/bash set -ex set -o pipefail echo "machine launch script started" > /home/centos/machine-launchstatus sudo chgrp centos /home/centos/machine-launchstatus sudo chown centos /home/centos/machine-launchstatus { sudo yum install -y ca-certificates sudo yum install -y mosh sudo yum groupinstall -y "Development tools" sudo yum install -y gmp-devel mpfr-devel libmpc-devel zlib-devel vim git java java-devel curl https://www.scala-sbt.org/sbt-rpm.repo | sudo tee /etc/yum.repos.d/scala-sbt-rpm.repo sudo yum install -y sbt texinfo gengetopt libffi-devel sudo yum install -y expat-devel libusb1-devel ncurses-devel cmake "perl(ExtUtils::MakeMaker)" # deps for poky sudo yum install -y python36 patch diffstat texi2html texinfo subversion chrpath git wget # deps for qemu sudo yum install -y gtk3-devel # deps for firesim-software (note that rsync is installed but too old) sudo yum install -y python36-pip python36-devel rsync # Install GNU make 4.x (needed to cross-compile glibc 2.28+) sudo yum install -y centos-release-scl sudo yum install -y devtoolset-8-make # install DTC sudo yum -y install dtc # get a proper version of git sudo yum -y remove git sudo yum -y install epel-release sudo yum -y install https://repo.ius.io/ius-release-el7.rpm sudo yum -y install git224 # install verilator git clone http://git.veripool.org/git/verilator cd verilator/ git checkout v4.034 autoconf && ./configure && make -j4 && sudo make install cd .. # bash completion for manager sudo yum -y install bash-completion # graphviz for manager sudo yum -y install graphviz python-devel # used for CI sudo yum -y install expect # upgrade pip sudo pip3 install --upgrade pip==21.3.1 # install requirements sudo python3 -m pip install fab-classic==1.19.1 sudo python3 -m pip install boto3==1.20.21 sudo python3 -m pip install colorama==0.4.3 sudo python3 -m pip install argcomplete==1.12.3 sudo python3 -m pip install graphviz==0.19 # for some of our workload plotting scripts sudo python3 -m pip install pyparsing==3.0.6 sudo python3 -m pip install numpy==1.19.5 sudo python3 -m pip install kiwisolver==1.3.1 sudo python3 -m pip install matplotlib==3.3.4 sudo python3 -m pip install pandas==1.1.5 sudo python3 -m pip install awscli==1.22.21 sudo python3 -m pip install pytest==6.2.5 sudo python3 -m pip install moto==2.2.17 sudo python3 -m pip install sure==2.0.0 # needed for the awstools cmdline parsing sudo python3 -m pip install pyyaml==5.4.1 # setup argcomplete activate-global-python-argcomplete } 2>&1 | tee /home/centos/machine-launchstatus.log # get a regular prompt echo "PS1='\u@\H:\w\\$ '" >> /home/centos/.bashrc echo "machine launch script completed" >> /home/centos/machine-launchstatus
This will pre-install all of the dependencies needed to run FireSim on your instance.
On the next page (“Add Storage”), increase the size of the root EBS volume to ~300GB. The default of 65GB can quickly become too small as you accumulate large Vivado reports/outputs, large waveforms, XSim outputs, and large root filesystems for simulations. You should get rid of the small (5GB) secondary volume that is added by default.
You can skip the “Add Tags” page, unless you want tags.
On the “Configure Security Group” page, select the
firesim
security group that was automatically created for you earlier.On the review page, click the button to launch your instance.
Make sure you select the firesim
key pair that we setup earlier.
2.3.1.1. Access your instance¶
We HIGHLY recommend using mosh instead
of ssh
or using ssh
with a screen/tmux session running on your
manager instance to ensure that long-running jobs are not killed by a
bad network connection to your manager instance. On this instance, the
mosh
server is installed as part of the setup script we pasted
before, so we need to first ssh into the instance and make sure the
setup is complete.
In either case, ssh
into your instance (e.g. ssh -i firesim.pem centos@YOUR_INSTANCE_IP
) and wait until the
~/machine-launchstatus
file contains all the following text:
centos@ip-172-30-2-140.us-west-2.compute.internal:~$ cat machine-launchstatus
machine launch script started
machine launch script completed!
Once this line appears, exit and re-ssh
into the system. If you want
to use mosh
, mosh
back into the system.
2.3.1.2. Key Setup, Part 2¶
Now that our manager instance is started, copy the private key that you
downloaded from AWS earlier (firesim.pem
) to ~/firesim.pem
on
your manager instance. This step is required to give the manager access
to the instances it launches for you.
2.3.2. Setting up the FireSim Repo¶
We’re finally ready to fetch FireSim’s sources. Run:
git clone https://github.com/firesim/firesim cd firesim # checkout latest official firesim release # note: this may not be the latest release if the documentation version != "stable" git checkout 1.13.4 ./build-setup.sh fast
The build-setup.sh
script will validate that you are on a tagged branch,
otherwise it will prompt for confirmation.
This will have initialized submodules and installed the RISC-V tools and
other dependencies.
Next, run:
source sourceme-f1-manager.sh
This will have initialized the AWS shell, added the RISC-V tools to your
path, and started an ssh-agent
that supplies ~/firesim.pem
automatically when you use ssh
to access other nodes. Sourcing this the
first time will take some time – however each time after that should be instantaneous.
Also, if your firesim.pem
key requires a passphrase, you will be asked for
it here and ssh-agent
should cache it.
Every time you login to your manager instance to use FireSim, you should ``cd`` into your firesim directory and source this file again.
2.3.3. Completing Setup Using the Manager¶
The FireSim manager contains a command that will interactively guide you through the rest of the FireSim setup process. To run it, do the following:
firesim managerinit
This will first prompt you to setup AWS credentials on the instance, which allows
the manager to automatically manage build/simulation nodes. See
https://docs.aws.amazon.com/cli/latest/userguide/tutorial-ec2-ubuntu.html#configure-cli-launch-ec2
for more about these credentials. When prompted, you should specify the same
region that you chose above and set the default output format to json
.
Next, it will create initial configuration files, which we will edit in later sections. Finally, it will prompt you for an email address, which is used to send email notifications upon FPGA build completion and optionally for workload completion. You can leave this blank if you do not wish to receive any notifications, but this is not recommended.
Now you’re ready to launch FireSim simulations! Hit Next to learn how to run single-node simulations.