2.3. Setting up your Manager Instance

2.3.1. Launching a “Manager Instance”

Now, we need to launch a “Manager Instance” that acts as a “head” node that we will ssh or mosh into to work from. Since we will deploy the heavy lifting to separate c5.4xlarge and f1 instances later, the Manager Instance can be a relatively cheap instance. In this guide, however, we will use a c5.4xlarge, running the AWS FPGA Developer AMI (be sure to subscribe if you have not done so. See Subscribe to the AWS FPGA Developer AMI).

Head to the EC2 Management Console. In the top right corner, ensure that the correct region is selected.

To launch a manager instance, follow these steps:

  1. From the main page of the EC2 Management Console, click Launch Instance. We use an on-demand instance here, so that your data is preserved when you stop/start the instance, and your data is not lost when pricing spikes on the spot market.

  2. When prompted to select an AMI, search in the Community AMIs tab for “FPGA” and select the option that starts with FPGA Developer AMI - 1.6.0. DO NOT USE ANY OTHER VERSION.

  3. When prompted to choose an instance type, select the instance type of your choosing. A good choice is a c5.4xlarge.

  4. On the “Configure Instance Details” page:

    1. First make sure that the firesim VPC is selected in the drop-down box next to “Network”. Any subnet within the firesim VPC is fine.

    2. Additionally, check the box for “Protect against accidental termination.” This adds a layer of protection to prevent your manager instance from being terminated by accident. You will need to disable this setting before being able to terminate the instance using usual methods.

    3. Also on this page, expand “Advanced Details” and in the resulting text box, paste the following:

      #!/bin/bash
      echo "machine launch script started" > /home/centos/machine-launchstatus
      sudo yum install -y mosh
      sudo yum groupinstall -y "Development tools"
      sudo yum install -y gmp-devel mpfr-devel libmpc-devel zlib-devel vim git java java-devel
      curl https://bintray.com/sbt/rpm/rpm | sudo tee /etc/yum.repos.d/bintray-sbt-rpm.repo
      sudo yum install -y sbt texinfo gengetopt
      sudo yum install -y expat-devel libusb1-devel ncurses-devel cmake "perl(ExtUtils::MakeMaker)"
      # deps for poky
      sudo yum install -y python36 patch diffstat texi2html texinfo subversion chrpath git wget
      # deps for qemu
      sudo yum install -y gtk3-devel
      # deps for firesim-software (note that rsync is installed but too old)
      sudo yum install -y python36-pip python36-devel rsync
      # Install GNU make 4.x (needed to cross-compile glibc 2.28+)
      sudo yum install -y centos-release-scl
      sudo yum install -y devtoolset-8-make
      
      # install DTC. it's not available in repos in FPGA AMI
      DTCversion=dtc-1.4.4
      wget https://git.kernel.org/pub/scm/utils/dtc/dtc.git/snapshot/$DTCversion.tar.gz
      tar -xvf $DTCversion.tar.gz
      cd $DTCversion
      make -j16
      make install
      cd ..
      rm -rf $DTCversion.tar.gz
      rm -rf $DTCversion
      
      # get a proper version of git
      sudo yum -y remove git
      sudo yum -y install epel-release
      sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm
      sudo yum -y install git2u
      
      # install verilator
      git clone http://git.veripool.org/git/verilator
      cd verilator/
      git checkout v4.002
      autoconf && ./configure && make -j16 && sudo make install
      cd ..
      
      # bash completion for manager
      sudo yum -y install bash-completion
      
      # graphviz for manager
      sudo yum -y install graphviz python-devel
      
      # these need to match what's in deploy/requirements.txt
      sudo pip2 install fabric==1.14.0
      sudo pip2 install boto3==1.6.2
      sudo pip2 install colorama==0.3.7
      sudo pip2 install argcomplete==1.9.3
      sudo pip2 install graphviz==0.8.3
      # for some of our workload plotting scripts
      sudo pip2 install --upgrade --ignore-installed pyparsing
      sudo pip2 install matplotlib==2.2.2
      sudo pip2 install pandas==0.22.0
      # new awscli on 1.6.0 AMI is broken with our versions of boto3
      sudo pip2 install awscli==1.15.76
      
      sudo activate-global-python-argcomplete
      
      # get a regular prompt
      echo "PS1='\u@\H:\w\\$ '" >> /home/centos/.bashrc
      echo "machine launch script completed" >> /home/centos/machine-launchstatus
      

      This will pre-install all of the dependencies needed to run FireSim on your instance.

  5. On the next page (“Add Storage”), increase the size of the root EBS volume to ~300GB. The default of 150GB can quickly become tight as you accumulate large Vivado reports/outputs, large waveforms, XSim outputs, and large root filesystems for simulations. You can get rid of the small (5GB) secondary volume that is added by default.

  6. You can skip the “Add Tags” page, unless you want tags.

  7. On the “Configure Security Group” page, select the firesim security group that was automatically created for you earlier.

  8. On the review page, click the button to launch your instance.

Make sure you select the firesim key pair that we setup earlier.

2.3.1.1. Access your instance

We HIGHLY recommend using mosh instead of ssh or using ssh with a screen/tmux session running on your manager instance to ensure that long-running jobs are not killed by a bad network connection to your manager instance. On this instance, the mosh server is installed as part of the setup script we pasted before, so we need to first ssh into the instance and make sure the setup is complete.

In either case, ssh into your instance (e.g. ssh -i firesim.pem centos@YOUR_INSTANCE_IP) and wait until the ~/machine-launchstatus file contains all the following text:

centos@ip-172-30-2-140.us-west-2.compute.internal:~$ cat machine-launchstatus
machine launch script started
machine launch script completed!

Once this line appears, exit and re-ssh into the system. If you want to use mosh, mosh back into the system.

2.3.1.2. Key Setup, Part 2

Now that our manager instance is started, copy the private key that you downloaded from AWS earlier (firesim.pem) to ~/firesim.pem on your manager instance. This step is required to give the manager access to the instances it launches for you.

2.3.2. Setting up the FireSim Repo

We’re finally ready to fetch FireSim’s sources. Run:

git clone https://github.com/firesim/firesim
cd firesim
./build-setup.sh fast

This will have initialized submodules and installed the RISC-V tools and other dependencies.

Next, run:

source sourceme-f1-manager.sh

This will have initialized the AWS shell, added the RISC-V tools to your path, and started an ssh-agent that supplies ~/firesim.pem automatically when you use ssh to access other nodes. Sourcing this the first time will take some time – however each time after that should be instantaneous. Also, if your firesim.pem key requires a passphrase, you will be asked for it here and ssh-agent should cache it.

Every time you login to your manager instance to use FireSim, you should ``cd`` into your firesim directory and source this file again.

2.3.3. Completing Setup Using the Manager

The FireSim manager contains a command that will interactively guide you through the rest of the FireSim setup process. To run it, do the following:

firesim managerinit

This will first prompt you to setup AWS credentials on the instance, which allows the manager to automatically manage build/simulation nodes. See https://docs.aws.amazon.com/cli/latest/userguide/tutorial-ec2-ubuntu.html#configure-cli-launch-ec2 for more about these credentials. When prompted, you should specify the same region that you chose above and set the default output format to json.

Next, it will create initial configuration files, which we will edit in later sections. Finally, it will prompt you for an email address, which is used to send email notifications upon FPGA build completion and optionally for workload completion. You can leave this blank if you do not wish to receive any notifications, but this is not recommended.

Now you’re ready to launch FireSim simulations! Hit Next to learn how to run single-node simulations.