wiki:GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment

Version 5 (modified by nriga@bbn.com, 7 years ago) (diff)

--

Example Experiment - Layer 3 Deflect

In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use:

  • 1 ProtoGENI host to run our OpenFlow controller
  • 3 myPlc hosts, 1 will be the client and 2 will be the servers
  • OpenFlow resources that will connect the three hosts

Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.

1. Add another user to your experiment

Omni gives you the capability of giving access to other users on your compute resources. Depending on which AM you are using to get resources from, this is done in a different way. Ask the team next to you about their username and do the following:

  1. While in a terminal, download their public key under ~/.ssh/ :
      cd ~/.ssh
      wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub
    
  1. Follow the instructions these instructions? and add another user for ProtoGENI AMs

1. Create your experiment

In this step, we are going to setup the experiment. In this tutorial we assume that you are sufficiently comfortable with omni to verify that a listresources command works and to know when your slice is ready using sliverstatus.

  1. Create a slice, use the slicename given to you in the paper slip:
    omni.py createslice <slicename>
    
  2. Create all the slivers using the rspecs from the URL given in your paper slip:
    omni.py createsliver -a <AM_nickname> <slicename> <rspec_url>
    
  3. Check the status of your sliver
    omni.py sliverstatus -a <AM_nickname> <slicename>
    

2. Install scripts

While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts. In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.

  1. Download your rspec you used in the pg-utah AM.
     cd /tmp
     wget <rspec_url>
    
  2. Open your rspec and look for the install tag and copy the value of the URL attribute.
  3. Download and untar the software
    cd /tmp
    wget <software_url>
    tar xvfz <software_name>
    
  4. Look in your rspec and locate the execute tag. Note what script is being executed at boot time.
  5. Locate the script and open it. Can you identify the different parts?

3. Configure your hosts

Once our slivers iare ready we will go ahead and configure our myPlc hosts. For this tutorial we are both going to login to nodes and use remote execution to send commands to our nodes, so make sure that both work.

3a. Login and remote execution

  1. Run the readyToLogin.py script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want.
    readyToLogin.py -a <AM_nickname> <slicename> > login.out 2>&1
    
    You'll get a big chunk of information, but you're interested in the ssh configuration info information near the end.
    ... <lots of output> ...
    ================================================================================
    SSH CONFIGURATION INFO for User inki
    ================================================================================
     
    Host ganel.gpolab.bbn.com
      Port 22
      HostName ganel.gpolab.bbn.com
      User pgenigpolabbbncom_testdeflect 
      IdentityFile /home/nriga/.ssh/geni_key 
    
    Host sardis.gpolab.bbn.com
      Port 22
      HostName sardis.gpolab.bbn.com
      User pgenigpolabbbncom_testdeflect 
      IdentityFile /home/nriga/.ssh/geni_key 
    
    
    ...<more output>...
    
  1. Copy all the above information and paste it into your .ssh/config file, and do the same for all the AMs, then you can very easily login into your nodes, just by using the name that is after the Host attribute. Your ~/.ssh/config file should look like
    IdentityFile /home/geni/.ssh/geni_key
    
    Host ganel.gpolab.bbn.com
      Port 22
      HostName ganel.gpolab.bbn.com
      User pgenigpolabbbncom_testdeflect 
      IdentityFile /home/nriga/.ssh/geni_key 
     
    Host sardis.gpolab.bbn.com
      Port 22
      HostName sardis.gpolab.bbn.com
      User pgenigpolabbbncom_testdeflect 
      IdentityFile /home/nriga/.ssh/geni_key 
    
    Host planetlab5.clemson.edu
      Port 22
      HostName planetlab5.clemson.edu
      User pgenigpolabbbncom_testdeflect 
      IdentityFile /home/nriga/.ssh/geni_key 
    
    Host ofctrl
      Port 22
      HostName pc104.emulab.net
      User inki 
      IdentityFile /home/nriga/.ssh/geni_key
    
    
  1. Note down the HostName of your host at pg-utah. In the above example this would be pc104.emulab.net.

Test login

For each one of the three myPlc hosts, open a new terminal and login to each one of them. Substitute <pl_hostname> with the hostnames of pl nodes you were given on the paper slip.

  ssh <pl_hostname>

Test remote execution

You can execute commands in a remote host using ssh. To do this just follow your ssh command with the command you want to execute in quotes. We will use one of the myPlc nodes for this, just choose one.

  1. In your local terminal type :
    ssh -A sardis.gpolab.bbn.com "ls -a"
    
    This will list all the files under the home directory on host top. The output should look like:
    geni@geni-VirtualBox:~$ ssh -A planetlab5.clemson.edu "ls -a"
    .
    ..
    .bash_history
    .bash_logout
    .bash_profile
    .bashrc
    

If you get something similar you are all set for controlling your nodes from your computer.

3b. Configure your hosts

  1. On each of the terminals that you have logged in to a myPlc node type:
    sudo yum install -y nc
    

4. Create your OpenFlow sliver

Now that we have our myPlc hosts and our PG host is being configured, it is time to reserve our OpenFlow sliver.

  1. Add the OpenFlow AM nickname in your omni_config file
    1. Open file ~/.gcf/omni_config
    2. find the [aggregate_nicknames] section in the file and move to th OpenFlow MAs
    3. Add this line :
      of-tut=,https://aquarion.gpolab.bbn.com:3626/foam/gapi/1
      
  2. Download your OpenFlow rspec, use the<ofrspec_url> on the paper slip
    wget <ofrspec_url>
    
    i Edit the OpenFlow rspec. There are only two things you will need to edit :
    1. <USERNAME> : use the information on the paper slip
    2. <HOSTNAME> : use the hostname of you ProtoGENI host that you note down before (e.g. pc104.emulab.net)
  3. Create your sliver using the filename of your rspec:
      omni.py createsliver -a of-tut
    

4b. Modify the configuration file of the OpenFlow controller

By now your ProtoGENI host should have been configured. Let's login to the node and modify the configuration file of our OpenFlow controller.

  1. Open a new terminal and login to the ProtoGENI host:
    ssh ofctrl
    
  2. Run :
    ps -ef | grep nox_core
    
    The output should look like:
    root      6498     1  0 18:33 ?        00:00:01 /usr/local/bin/noxgpo/src/.libs/lt-nox_core -d -i ptcp:6633 switch
    inki      6562  6524  0 18:45 pts/0    00:00:00 grep nox_core
    
    If the first line is missing then the setup has not finished yet. You can move to the next step, but make sure the setup has finished before moving to step 5.
  3. Copy the configuration file to your home directory:
    cp /local/l3deflect.conf ~/
    
  4. Modify the configuration file to match the information in your slip. The should look like:
    # Configuration file for the l3deflect controller
    
    [general]
    orig_mac = 00:1B:21:4B:3E:E9
    orig_ip = 10.42.123.52
    deflect_mac = 00:1B:21:4B:3F:AD
    deflect_ip = 10.42.123.51
    

5. Run the experiment

Now we have reserved and configured all hosts. We are ready to run our experiment.

  1. In the terminal of the remote and of the local server run the following command. Substitute <NC_PORT> with the information in your slip:
    nc <nc_port>
    
    1. In the terminal of the client run this command, using your paper slip to fill in the information :
      nc <IP_REMOTE_SERVER> <NC_PORT>
      
  2. You now have a chat application between the remote server and the client. Try typing something in the client and see it pop up at the server side, and vice versa. The forwarding of your packets work because there is a default learning switch controller running on your ProtoGENI host.
  3. Let's stop the default learning switch controller and start our deflection controller. In a local terminal type the following two commands:
     ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
     ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect"
    
    The output should look like that:
    nriga@pella:~$ ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
    Stopping all OpenFlow controllers
    nriga@pella:~$ ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect"
    Starting OpenFlow controler l3deflect
    
  4. If the nc is still running on the client, terminate it by pressing Ctrl-C. i Run the nc at the client again:
     nc <IP_REMOTE_SERVER> <NC_PORT>
    
    i Type something on the client and notice in which terminal it appears.

If you want to switch back to the switch controller you need to again kill all the controllers and start the switch controller :

 ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
 ssh ofctrl "/usr/local/bin/start-ctrl.sh switch"

You can play switching between the two controllers and notice how the text appears on a different terminal.

6. Looking under the hood

If you still have a few minutes, then you can poke around the OpenFlow controller, if you don't have enough time then make sure you move to the next step and cleanup your resources. You can always come back and do the tutorial again and poke around.

Congratulations you have successfully diverted the traffic from your client to the local server using OpenFlow!. But how did this work? Basically our openflow controller instructed the switch to rewrite the IP information on the packet so that

  • every packet that was destined to the original server, it would be changed to be sent to the local one
  • also all packets from the local server are rewritten so that the client thinks that it talks to the remote server.

Let's take a quick look at the controller. On the terminal that is logged in to the ProtoGENI host open file /usr/local/src/noxgpo/src/nox/coreapps/example/l3deflect.py. This is the deflection module. Lets try and follow the logic now:

  1. Find the install function. This is the function that registers your module to receive OpenFlow events. i Look at which function is called when a packet is received.
  2. Ignore the first checks and find the function that is responsible for forwarding packets.
  3. Look at the logic that is overwriting the packet. Basically there is a section like this :
          # Check if the eth_type must be deflected
          if ethtype == 0x800 :
            # and if the dst mac address is the one that should be deflected
            if mac_to_str(packet.dst).lower() == self.orig_mac.lower() :
              # replace the destination mac address 
              actions.append([openflow.OFPAT_SET_DL_DST,
                                  octstr_to_array(self.deflect_mac)])
              logger.debug("NEW DST MAC %s" % ( self.deflect_mac))
              dstaddr = octstr_to_array(self.deflect_mac).tostring()
              actions.append([openflow.OFPAT_SET_NW_DST,
                                  ipstr_to_int(self.deflect_ip)])
              logger.debug("NEW DST IP %s" % ( self.deflect_ip))
              logger.info("Packet %s is deflected")
    
            if mac_to_str(packet.src).lower() == self.deflect_mac.lower() :
              # replace the destination mac address 
              actions.append([openflow.OFPAT_SET_DL_SRC,
                                  octstr_to_array(self.orig_mac)])
              logger.debug("NEW SRC MAC %s" % ( self.orig_mac))
              actions.append([openflow.OFPAT_SET_NW_SRC,
                                  ipstr_to_int(self.orig_ip)])
              logger.debug("NEW DST IP %s" % ( self.orig_ip))
    

7. Clean up

When you're done, please release your resources so they'll be available to others. Make sure you delete all the resources in all AMs. You have a slice at :

  • pg-utah
  • at two myplc sites
  • at of-tut
omni.py deletesliver -a <AM_nickname> <slicename>