Version 26 (modified by 12 years ago) (diff) | ,
---|
When getting ready to run experiments in GENI, it is always a good idea to automate the setup of your experiment.
This page assumes you know
- how to create a sliver with all the nodes that you need
- how to login to your nodes
- know what steps of your experiment you want to automate with the install scripts
In GENI RSpecs, you have the capability of instructing an Aggregate Manager to download software from the web, and execute install scripts at boot up time. This page provides step by step instructions about how to write and use install scripts.
1. Write your script
Things to remember
- your script does not run in the context of your user, make sure that you change the permissions of anything installed in your home directory so that you have access to them when you login
- your script does run as a user with sudo privileges. Make sure you use
sudo
when:- you execute privileged commands (e.g.
yum
,apt-get
,service
) - you place things in your home directory
- you try to write files in protected directories (e.g. `/usr/locac/bin/, /local/)
- you execute privileged commands (e.g.
- the $PATH environment variable is not necessarily set, make sure you use the full path of commands in your scripts; for example don't use
ifconfig
, but use/sbin/ifconfig
. This is also true for the interpreters used for your script.Make sure you start your script with a line like#!<path to your interpreter>
If you don't know the path, while logged in run (this is an example to figure out where python is installed)$ which python
- your script runs every time the host reboots, so if you reboot the host as part of your script you have to check if your script has already ran or you will end up in a cycle of reboots.
- your script should be fully automated. You won't be able to interact with your script when it runs so make sure it doesn't need any input to run. For example if you use
apt-get
, use the-y
flag so thatapt-get
assumesyes
to any interactive question.
Choose a scripting language
You can use any scripting language you want, popular choices are:
- shell scripts, there are many tutorials online that can guide you through it.
- [ http://www.perl.org/ Perl]. Make sure that perl is installed by default in the Image you are using. Login to your nodes and type
perl -h
. - Python. Make sure that perl is installed by default in the Image you are using. Login to your nodes and type
python -h
.
If an interpreter for the scripting language of your choice is not installed, you can write a short shell script that will first install it
Parts of your script
Your script should have at least two parts:
- the part that needs to be executed only once
- the part that needs to be executed every time the node boots up
The easiest way to achieve this is by adding a file the first time your script runs and then check for the existence of that file before running one-time commands. Independent on which scripting language you decide to use, your script would look like:
go to execution directory if <file> does not exist: create file run one-time commands endif run boot-time commands
After you write your script make sure it is executable by doing:
chmod +x <script>
This is an example of a script that installs some packages, download and install a software, change some parameters and reboot the node in the end.
#!/bin/sh # Usual directory for downloading software in ProtoGENI hosts is `/local` cd /local ##### Check if file is there ##### if [ ! -f "./installed.txt" ] then #### Create the file #### sudo touch "./installed.txt" #### Run one-time commands #### #Install necessary packages sudo apt-get update& EPID=$! wait $EPID sudo apt-get -y install git-core unzip cmake libpcap-dev libxerces-c2-dev libpcre3-dev flex bison g++ autoconf libtool pkg-config libboost-dev libboost1.40-all-dev gawk & EPID=$! wait $EPID # Install custom software sudo git clone git://github.com/myproject.git & EPID=$! wait $EPID cd myproject sudo ./boot.sh & EPID=$! wait $EPID sudo ./configure & EPID=$! wait $EPID sudo make & EPID=$! wait $EPID sudo make install & EPID=$! # For my experiment I need to disable IPV6 echo "#disable ipv6" | sudo tee -a /etc/sysctl.conf echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf ## Reboot the OS to run a latest kernel without IPv6. sudo reboot fi ##### Run Boot-time commands # Start my service -- assume it was installed at /usr/local/bin sudo /usr/local/bin/myproject-server
2. Upload your script
In order for the Aggregate Manager to find and run the install script, you will need to upload your script in a web server that is publicly available. Most institution provide this service to their users. However if you don't have access in a web server in order to upload your script, you can setup a web server in GENI. Please follow these instructions about setting up a web server in a PlanetLab node.
After you have ensured access to a web server, follow these simple steps:
- For ProtoGENI you should place your script in a compressed file. Do :
tar cvfz install-script.tar.gz <script>
- Upload your file to your webserver. Make sure you have access to the file by pointing a browser at your file and ensuring that you can download it.
3. Test your script
Before you add the script to your RSpec and automate the setup of your experiment, make sure that your test script works. This will save you a lot of time with going back and forth, creating and delete slivers until you get it right. Reserve a node and after the node is ready login. In ProtoGENI the user-software is usually downloaded and installed in local. So login to your node and:
- go to the /local/ directory
cd /local
- download and untar your script
sudo wget <URL_FOR_YOUR_SCRIPT> sudo tar xvfz <your_script>.tar.gz
If you are planning to download additional software using install scripts, download this software as well.
- run your script
sudo ./<your_script>
- fix any errors, upload your script and retest it.
4. Modify your RSpec
Now that you have a script that you believe works it is time to add the script so that it is executed at the creation time of your slice.
Useful tips
Where are the log files for my script?
Install a Web Server in PlanetLab
Reserver a PlanetLab node
In order to install a web server in a PlanetLab node, it is better if you create a separate slice just for the web server and use the attached rspec to reserve a PlanetLab node. You can use either Omni or Flack to do this. If you are using Flack, then you can see this video about how to create a slice with an existing rspec using Flack. While you follow the video make sure you also choose the plc
manager so that you can reserver PlanetLab nodes.
Install and start a Web Server
Be patient, it might take up to 30-40 minutes for you sliver to be ready. Once your sliver is created you can login to your node. If you don't know how to login to your node you can follow these instructions.
After you login :
- install the apache server by running :
sudo yum install -y httpd
- modify the default port by running the following commande where <NEW_PORT> is the port you want to use:
sudo sed 's/^Listen 80$/Listen <NEW_PORT>/' /etc/httpd/conf/httpd.conf > httpd.conf ; sudo mv httpd.conf /etc/httpd/conf/httpd.conf
- start the web server :
sudo /sbin/service httpd start
- modify the default port by running the following commande where <NEW_PORT> is the port you want to use:
If you get an error someone else might be using the port you used, change it again by using the following command where <OLD_PORT> is the port you used in the previous command:
sudo sed 's/^Listen <OLD_PORT>$/Listen <NEW_PORT>/' /etc/httpd/conf/httpd.conf > httpd.conf ; sudo mv httpd.conf /etc/httpd/conf/httpd.conf
In order to test that your webserver is up and running open a web brower and point it at http://plnode-03.gpolab.bbn.com:<NEW_PORT>/ you should get the apache welcome page.
Upload files to your webserver
In order to make files publicly available in your server do these two simple steps :
- transfer the file to the PlanetLab node. You can use scp to do this e.g.:
scp -i /home/nriga/.ssh/geni_key install-mpich.tar.gz pgenigpolabbbncom_inkisrv@plnode-03.gpolab.bbn.com:
- login to your node and move the file to '/var/www/html', e.g.
sudo mv install-mpich.tar.gz /var/www/html/