Version 16 (modified by 12 years ago) (diff) | ,
---|
Example Experiment - Layer 3 Deflect
In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use:
- 1 ProtoGENI host to run our OpenFlow controller
- 3 myPlc hosts, 1 will be the client and 2 will be the servers
- OpenFlow resources that will connect the three hosts
Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.
1. Add another user to your experiment
Omni gives you the capability of giving access to other users on your compute resources. Depending on which AM you are using to get resources from, this is done in a different way. Ask the team next to you about their username and do the following:
- While in a terminal, download their public key under
~/.ssh/
:cd ~/.ssh wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub
- Follow the instructions these instructions? and add another user for ProtoGENI AMs
1. Create your experiment
In this step, we are going to setup the experiment. In this tutorial we assume that you are sufficiently comfortable with omni to verify that a listresources command works and to know when your slice is ready using sliverstatus.
- Create a slice, use the slicename given to you in the paper slip:
omni.py createslice <slicename>
- Create all the slivers using the rspecs from the URL given in your paper slip. At this point do not create the OpenFlow sliver. Make sure to use the
-V1
flag for all the myPlc AMs.omni.py createsliver -a <AM_nickname> <slicename> <rspec_url>
For convenience here is a list of all the URLs so that you can copy and paste. Make sure to only use your URLs.http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/l3deflect/rspecs/l3deflect-pg.rspec http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/l3deflect/rspecs/l3deflect-myplc-stanford.rspec http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/l3deflect/rspecs/l3deflect-myplc-clemson.rspec http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/l3deflect/rspecs/l3deflect-myplc-gpo.rspec
- Check the status of your sliver
omni.py sliverstatus -a <AM_nickname> <slicename>
2. Install scripts
While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts. In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.
- Download your rspec you used in the
pg-utah
AM.cd /tmp wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/l3deflect/rspecs/l3deflect-pg.rspec
- Open your rspec and look for the
install
tag and copy the value of the URL attribute. - Download and untar the software
cd /tmp wget <software_url> tar xvfz <software_name>
- Look in your rspec and locate the
execute
tag. Note what script is being executed at boot time. - Locate the script and open it. Can you identify the different parts?
3. Configure your hosts
Once our slivers iare ready we will go ahead and configure our myPlc hosts. For this tutorial we are both going to login to nodes and use remote execution to send commands to our nodes, so make sure that both work.
3a. Login and remote execution
- Lets reset our environment first:
cd rm .ssh/config touch .ssh/config
- Run the
readyToLogin.py
script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want. Make sure you use the-V1
flag of the myPlc AMs.readyToLogin.py -a <AM_nickname> <slicename> > login.out 2>&1
You'll get a big chunk of information, but you're interested in the ssh configuration info information near the end.... <lots of output> ... ================================================================================ SSH CONFIGURATION INFO for User inki ================================================================================ Host ganel.gpolab.bbn.com Port 22 HostName ganel.gpolab.bbn.com User pgenigpolabbbncom_testdeflect IdentityFile /home/nriga/.ssh/geni_key Host sardis.gpolab.bbn.com Port 22 HostName sardis.gpolab.bbn.com User pgenigpolabbbncom_testdeflect IdentityFile /home/nriga/.ssh/geni_key ...<more output>...
- Copy all the above information and paste it into your
.ssh/config
file, and do the same for all the AMs, then you can very easily login into your nodes, just by using the name that is after theHost
attribute. Your~/.ssh/config
file should look likeIdentityFile /home/geni/.ssh/geni_key Host ganel.gpolab.bbn.com Port 22 HostName ganel.gpolab.bbn.com User pgenigpolabbbncom_testdeflect IdentityFile /home/nriga/.ssh/geni_key Host sardis.gpolab.bbn.com Port 22 HostName sardis.gpolab.bbn.com User pgenigpolabbbncom_testdeflect IdentityFile /home/nriga/.ssh/geni_key Host planetlab5.clemson.edu Port 22 HostName planetlab5.clemson.edu User pgenigpolabbbncom_testdeflect IdentityFile /home/nriga/.ssh/geni_key Host ofctrl Port 22 HostName pc104.emulab.net User inki IdentityFile /home/nriga/.ssh/geni_key
- Note down the
HostName
of your host atpg-utah
. In the above example this would bepc104.emulab.net
.
Test login
For each one of the three myPlc hosts, open a new terminal and login to each one of them. Substitute <pl_hostname> with the hostnames of pl nodes you were given on the paper slip.
ssh <pl_hostname>
Test remote execution
You can execute commands in a remote host using ssh
. To do this just follow your ssh command with the command you want to execute in quotes. We will use one
of the myPlc nodes for this, just choose one.
- In your local terminal type :
ssh -A sardis.gpolab.bbn.com "ls -a"
This will list all the files under the home directory on hosttop
. The output should look like:geni@geni-VirtualBox:~$ ssh -A planetlab5.clemson.edu "ls -a" . .. .bash_history .bash_logout .bash_profile .bashrc
If you get something similar you are all set for controlling your nodes from your computer.
3b. Configure your hosts
- On each of the terminals that you have logged in to a myPlc node type:
sudo yum install -y nc
4. Create your OpenFlow sliver
Now that we have our myPlc hosts and our PG host is being configured, it is time to reserve our OpenFlow sliver.
- Add the OpenFlow AM nickname in your
omni_config
file- Open file
~/.gcf/omni_config
- find the
[aggregate_nicknames]
section in the file and move to thOpenFlow MAs
- Add this line :
of-tut=,https://aquarion.gpolab.bbn.com:3626/foam/gapi/1
- Open file
- Download your OpenFlow rspec, use the<ofrspec_url> on the paper slip
wget <ofrspec_url>
i Edit the OpenFlow rspec. There are only three things you will need to edit :- <USERNAME> : use the information on the paper slip
- <HOSTNAME> : use the hostname of you ProtoGENI host that you note down before (e.g.
pc104.emulab.net
) - <SUBNET> : use the information on the paper slip
- Create your sliver using the filename of your rspec:
omni.py createsliver -a of-tut
4b. Modify the configuration file of the OpenFlow controller
By now your ProtoGENI host should have been configured. Let's login to the node and modify the configuration file of our OpenFlow controller.
- Open a new terminal and login to the ProtoGENI host:
ssh ofctrl
- Run :
ps -ef | grep nox_core
The output should look like:root 6498 1 0 18:33 ? 00:00:01 /usr/local/bin/noxgpo/src/.libs/lt-nox_core -d -i ptcp:6633 switch inki 6562 6524 0 18:45 pts/0 00:00:00 grep nox_core
If the first line is missing then the setup has not finished yet. You can move to the next step, but make sure the setup has finished before moving to step 5. - Copy the configuration file to your home directory:
cp /local/l3deflect.conf ~/
- Modify the configuration file to match the information in your slip. The should look like:
# Configuration file for the l3deflect controller [general] orig_mac = 00:1B:21:4B:3E:E9 orig_ip = 10.42.123.52 deflect_mac = 00:1B:21:4B:3F:AD deflect_ip = 10.42.123.51
5. Run the experiment
Now we have reserved and configured all hosts. We are ready to run our experiment.
- In the terminal of the remote and of the local server run the following command. Substitute <NC_PORT> with the information in your slip:
nc <nc_port>
- In the terminal of the client run this command, using your paper slip to fill in the information :
nc <IP_REMOTE_SERVER> <NC_PORT>
- In the terminal of the client run this command, using your paper slip to fill in the information :
- You now have a chat application between the remote server and the client. Try typing something in the client and see it pop up at the server side, and vice versa. The forwarding of your packets work because there is a default learning switch controller running on your ProtoGENI host.
- Let's stop the default learning switch controller and start our deflection controller. In a local terminal type the following two commands:
ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh" ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect"
The output should look like that:nriga@pella:~$ ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh" Stopping all OpenFlow controllers nriga@pella:~$ ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect" Starting OpenFlow controler l3deflect
- If the
nc
is still running on the client, terminate it by pressingCtrl-C
. i Run thenc
at the client again:nc <IP_REMOTE_SERVER> <NC_PORT>
i Type something on the client and notice in which terminal it appears.
If you want to switch back to the switch controller you need to again kill all the controllers and start the switch controller :
ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh" ssh ofctrl "/usr/local/bin/start-ctrl.sh switch"
You can play switching between the two controllers and notice how the text appears on a different terminal.
6. Looking under the hood
If you still have a few minutes, then you can poke around the OpenFlow controller, if you don't have enough time then make sure you move to the next step and cleanup your resources. You can always come back and do the tutorial again and poke around.
Congratulations you have successfully diverted the traffic from your client to the local server using OpenFlow!. But how did this work? Basically our openflow controller instructed the switch to rewrite the IP information on the packet so that
- every packet that was destined to the original server, it would be changed to be sent to the local one
- also all packets from the local server are rewritten so that the client thinks that it talks to the remote server.
Let's take a quick look at the controller. On the terminal that is logged in to the ProtoGENI host open file /usr/local/src/noxgpo/src/nox/coreapps/example/l3deflect.py
. This is the deflection module. Lets try and follow the logic now:
- Find the
install
function. This is the function that registers your module to receive OpenFlow events. i Look at which function is called when a packet is received. - Ignore the first checks and find the function that is responsible for forwarding packets.
- Look at the logic that is overwriting the packet. Basically there is a section like this :
# Check if the eth_type must be deflected if ethtype == 0x800 : # and if the dst mac address is the one that should be deflected if mac_to_str(packet.dst).lower() == self.orig_mac.lower() : # replace the destination mac address actions.append([openflow.OFPAT_SET_DL_DST, octstr_to_array(self.deflect_mac)]) logger.debug("NEW DST MAC %s" % ( self.deflect_mac)) dstaddr = octstr_to_array(self.deflect_mac).tostring() actions.append([openflow.OFPAT_SET_NW_DST, ipstr_to_int(self.deflect_ip)]) logger.debug("NEW DST IP %s" % ( self.deflect_ip)) logger.info("Packet %s is deflected") if mac_to_str(packet.src).lower() == self.deflect_mac.lower() : # replace the destination mac address actions.append([openflow.OFPAT_SET_DL_SRC, octstr_to_array(self.orig_mac)]) logger.debug("NEW SRC MAC %s" % ( self.orig_mac)) actions.append([openflow.OFPAT_SET_NW_SRC, ipstr_to_int(self.orig_ip)]) logger.debug("NEW DST IP %s" % ( self.orig_ip))
7. Clean up
When you're done, please release your resources so they'll be available to others. Make sure you delete all the resources in all AMs. You have a slice at :
- pg-utah
- at two myplc sites
- at of-tut
omni.py deletesliver -a <AM_nickname> <slicename>
Now go back to the first page and wrap up:-)