Version 27 (modified by, 10 years ago) (diff)


When getting ready to run experiments in GENI, it is always a good idea to automate the setup of your experiment.

This page assumes you know

  • how to create a sliver with all the nodes that you need
  • how to login to your nodes
  • know what steps of your experiment you want to automate with the install scripts

In GENI RSpecs, you have the capability of instructing an Aggregate Manager to download software from the web, and execute install scripts at boot up time. This page provides step by step instructions about how to write and use install scripts.

1. Write your script

Things to remember

  • your script does not run in the context of your user, make sure that you change the permissions of anything installed in your home directory so that you have access to them when you login
  • your script does run as a user with sudo privileges. Make sure you use sudo when:
    • you execute privileged commands (e.g. yum, apt-get, service)
    • you place things in your home directory
    • you try to write files in protected directories (e.g. `/usr/locac/bin/, /local/)
  • the $PATH environment variable is not necessarily set, make sure you use the full path of commands in your scripts; for example don't use ifconfig, but use /sbin/ifconfig. This is also true for the interpreters used for your script.Make sure you start your script with a line like
    #!<path to your interpreter>
    If you don't know the path, while logged in run (this is an example to figure out where python is installed)
    $ which python
  • your script runs every time the host reboots, so if you reboot the host as part of your script you have to check if your script has already ran or you will end up in a cycle of reboots.
  • your script should be fully automated. You won't be able to interact with your script when it runs so make sure it doesn't need any input to run. For example if you use apt-get, use the -y flag so that apt-get assumes yes to any interactive question.

Choose a scripting language

You can use any scripting language you want, popular choices are:

  • shell scripts, there are many tutorials online that can guide you through it.
  • [ Perl]. Make sure that perl is installed by default in the Image you are using. Login to your nodes and type perl -h.
  • Python. Make sure that perl is installed by default in the Image you are using. Login to your nodes and type python -h.

If an interpreter for the scripting language of your choice is not installed, you can write a short shell script that will first install it

Parts of your script

Your script should have at least two parts:

  • the part that needs to be executed only once
  • the part that needs to be executed every time the node boots up

The easiest way to achieve this is by adding a file the first time your script runs and then check for the existence of that file before running one-time commands. Independent on which scripting language you decide to use, your script would look like:

go to execution directory
if <file> does not exist:
   create file
   run one-time commands
run boot-time commands

After you write your script make sure it is executable by doing:

chmod +x <script>

This is an example of a script that installs some packages, download and install a software, change some parameters and reboot the node in the end.

# Usual directory for downloading software in ProtoGENI hosts is `/local`
cd /local

##### Check if file is there #####
if [ ! -f "./installed.txt" ]
       #### Create the file ####
        sudo touch "./installed.txt"

       #### Run  one-time commands ####
       #Install necessary packages
        sudo apt-get update& EPID=$!
        wait $EPID
        sudo apt-get -y install git-core unzip cmake libpcap-dev libxerces-c2-dev libpcre3-dev flex bison g++ autoconf libtool pkg-config libboost-dev libboost1.40-all-dev gawk & EPID=$!
        wait $EPID

       # Install custom software
       sudo git clone git:// & EPID=$!
       wait $EPID
       cd myproject 
       sudo ./ & EPID=$!
       wait $EPID
       sudo ./configure & EPID=$!
       wait $EPID
       sudo make & EPID=$!
       wait $EPID
       sudo make install & EPID=$!
       # For my experiment I need to disable IPV6
       echo "#disable ipv6" | sudo tee -a /etc/sysctl.conf
       echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
       echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
       echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
       ## Reboot the OS to run a latest kernel without IPv6.
       sudo reboot
##### Run Boot-time commands
# Start my service -- assume it was installed at /usr/local/bin
sudo /usr/local/bin/myproject-server    

2. Upload your script

In order for the Aggregate Manager to find and run the install script, you will need to upload your script in a web server that is publicly available. Most institution provide this service to their users. However if you don't have access in a web server in order to upload your script, you can setup a web server in GENI. Please follow these instructions about setting up a web server in a PlanetLab node.

After you have ensured access to a web server, follow these simple steps:

  • For ProtoGENI you should place your script in a compressed file. Do :
    tar cvfz install-script.tar.gz <script>
  • Upload your file to your webserver. Make sure you have access to the file by pointing a browser at your file and ensuring that you can download it.

Tip: Keep your life simple! Keep the script file in its own tar file. f you have other software that you need installed on the nodes, package it up separately. You can instruct the AM to download and uncompress multiple pieces of software. This will keep the task of updating your script easy and independent from the rest of your experiment.

3. Test your script

Before you add the script to your RSpec and automate the setup of your experiment, make sure that your test script works. This will save you a lot of time with going back and forth, creating and delete slivers until you get it right. Reserve a node and after the node is ready login. In ProtoGENI the user-software is usually downloaded and installed in local. So login to your node and:

  • go to the /local/ directory
    cd /local
  • download and untar your script
    sudo wget <URL_FOR_YOUR_SCRIPT>
    sudo tar xvfz <your_script>.tar.gz

If you are planning to download additional software using install scripts, download this software as well.

  • run your script
    sudo ./<your_script>
  • fix any errors, upload your script and retest it.

4. Modify your RSpec

Now that you have a script that you believe works it is time to add the script so that it is executed at the creation time of your slice.

Useful tips

Where are the log files for my script?

Install a Web Server in PlanetLab

Reserver a PlanetLab node

In order to install a web server in a PlanetLab node, it is better if you create a separate slice just for the web server and use the attached rspec to reserve a PlanetLab node. You can use either Omni or Flack to do this. If you are using Flack, then you can see this video about how to create a slice with an existing rspec using Flack. While you follow the video make sure you also choose the plc manager so that you can reserver PlanetLab nodes.

Install and start a Web Server

Be patient, it might take up to 30-40 minutes for you sliver to be ready. Once your sliver is created you can login to your node. If you don't know how to login to your node you can follow these instructions.

After you login :

  • install the apache server by running :
    sudo yum install -y httpd
    • modify the default port by running the following commande where <NEW_PORT> is the port you want to use:
      sudo sed 's/^Listen 80$/Listen <NEW_PORT>/' /etc/httpd/conf/httpd.conf > httpd.conf ; sudo mv httpd.conf /etc/httpd/conf/httpd.conf
    • start the web server :
      sudo /sbin/service httpd start

If you get an error someone else might be using the port you used, change it again by using the following command where <OLD_PORT> is the port you used in the previous command:

sudo sed 's/^Listen <OLD_PORT>$/Listen <NEW_PORT>/' /etc/httpd/conf/httpd.conf > httpd.conf ; sudo mv httpd.conf /etc/httpd/conf/httpd.conf

In order to test that your webserver is up and running open a web brower and point it at<NEW_PORT>/ you should get the apache welcome page.

Upload files to your webserver

In order to make files publicly available in your server do these two simple steps :

  • transfer the file to the PlanetLab node. You can use scp to do this e.g.:
     scp -i /home/nriga/.ssh/geni_key install-mpich.tar.gz
  • login to your node and move the file to '/var/www/html', e.g.
     sudo mv install-mpich.tar.gz /var/www/html/