Changes between Version 1 and Version 2 of GEC15Agenda/AdvancedGENITopoOmni/Instructions/TopologyExperiment


Ignore:
Timestamp:
10/21/12 22:38:52 (7 years ago)
Author:
nriga@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC15Agenda/AdvancedGENITopoOmni/Instructions/TopologyExperiment

    v1 v2  
    33= Example Experiment - Mesoscale Topologies =
    44
    5 In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use:
    6   * 1 ProtoGENI host to run our OpenFlow controller
    7   * 3 myPlc hosts, 1 will be the client and 2 will be the servers
    8   * OpenFlow resources that will connect the three hosts
    9 
    10 Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.
    11 
    12 == 1. Add another user to your experiment ==
    13 Omni gives you the capability of giving access  to other users on your compute resources. Depending on which AM you are using to get resources from,  this is done in a different way.  Ask the team next to you about their username and do the following:
    14   i. While in a terminal, download their public key under `~/.ssh/` :
    15   {{{
    16     cd ~/.ssh
    17     wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub
    18   }}}
    19 
    20   i. Follow the instructions [wiki:HowTo/MulitpleUsersWithOmni these instructions] and add another user for ProtoGENI AMs
     5In this experiment we are going to examine how the underlying network
     6topology can affect the topology of your experiment. We are going to use the two topologies that are available in Mesoscale.
     7The new for [wiki:TangoGENI#VLAN3715 VLAN 3715] and the one for [wiki:TangoGENI#VLAN3716 VLAN 3716].
    218
    229== 1. Create your experiment ==
     
    2613omni.py createslice <slicename>
    2714}}}
    28   i. '''Create all the slivers''' using the rspecs from the URL given in your paper slip:
     15  i. '''Create all the slivers''' using the rspecs from the URL given in your paper slip. You should create a sliver in two myPlc AMs:
    2916  {{{
    3017omni.py createsliver -a <AM_nickname> <slicename> <rspec_url>
     
    3522}}}
    3623
    37 == 2. Install scripts ==
    38 While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts.
    39 In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.
    40   i. Download your rspec you used in the `pg-utah` AM.
     24== 2. Login to your nodes ==
     25
     26The login information to your hosts is reported back in sliverstatus. Omni comes with an example script that calls sliverstatus and figures out all the information you need for login in to your hosts.
     27  i. First of all let's clean our `.ssh/config` file in case it contains information from previous experiments. Unless if you have added some information you care about in  your ssh configuration file, it is safe to remove it and recreate it.
    4128  {{{
    42    cd /tmp
    43    wget <rspec_url>
    44 }}}
    45   i. Open your rspec and look for the `install` tag and copy the value of the URL attribute.
    46   i. Download and untar the software
    47   {{{
    48   cd /tmp
    49   wget <software_url>
    50   tar xvfz <software_name>
    51 }}}
    52   i. Look in your rspec and locate the `execute` tag. Note what script is being executed at boot time.
    53   i. Locate the script and open it. Can you identify the different parts?
    54 
    55 == 3. Configure your hosts ==
    56 Once our slivers iare ready we will go ahead and configure our myPlc hosts. For this tutorial we are both going to login to nodes and use remote execution to send commands to our nodes, so make sure that both work.
    57 
    58 === 3a. Login and remote execution ===
    59   i. Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want.
     29  cd
     30  rm .ssh/config
     31  touch .ssh/config
     32  }}}
     33  i. Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want. Use the same AMs as you used in Step 1.
    6034  {{{
    6135readyToLogin.py -a <AM_nickname> <slicename> > login.out 2>&1
    6236}}}
    63   You'll get a big chunk of information, but you're interested in the '''ssh configuration info''' information near the end.
     37  i. Open the login.out file. You'll get a big chunk of information, but you're interested in the '''ssh configuration info''' information near the end.
    6438  {{{
    6539... <lots of output> ...
     
    10175  IdentityFile /home/nriga/.ssh/geni_key
    10276
    103 Host planetlab5.clemson.edu
    104   Port 22
    105   HostName planetlab5.clemson.edu
    106   User pgenigpolabbbncom_testdeflect
    107   IdentityFile /home/nriga/.ssh/geni_key
    108 
    109 Host ofctrl
    110   Port 22
    111   HostName pc104.emulab.net
    112   User inki
    113   IdentityFile /home/nriga/.ssh/geni_key
    114 
    115 }}}
    116 
    117   i. Note down the `HostName` of your host at `pg-utah`. In the above example this would be `pc104.emulab.net`.
    118 
    11977==== Test login ====
    120 For each one of the three myPlc hosts, open a new terminal and login to each one of them. Substitute <pl_hostname> with the hostnames of pl nodes you were given on the paper slip.
     78For each one of the myPlc hosts, open a new terminal and login to each one of them. Substitute <pl_hostname> with the hostnames of pl nodes you were given on the paper slip.
    12179{{{
    12280  ssh <pl_hostname>
    12381}}}
    12482
    125 ==== Test remote execution ====
    126 You can execute commands in a remote host using `ssh`. To do this just follow your ssh command with the command you want to execute in quotes. We will use one
    127 of the myPlc nodes for this, just choose one.
    128   i. In your local terminal type :
    129   {{{
    130   ssh -A sardis.gpolab.bbn.com "ls -a"
    131   }}}
    132   This will list all the files under the home directory on host `top`. The output should look like:
    133   {{{
    134 geni@geni-VirtualBox:~$ ssh -A planetlab5.clemson.edu "ls -a"
    135 .
    136 ..
    137 .bash_history
    138 .bash_logout
    139 .bash_profile
    140 .bashrc
    141   }}}
     83== 3. Run your Experiment  ==
     84The Mesoscale deployment can offer different topologies for communicating between hosts. We have provisioned two different IP subnets each one using a different topology.
    14285
    143 If you get something similar you are all set for controlling your nodes from your computer.
     86The two subnets that have been provisioned are '''10.42.112.0/24''' on 3715 and '''10.42.113.0/24''' on 3716.
    14487
    145 === 3b. Configure your hosts ===
    146   i. On each of the terminals that you have logged in to a myPlc node type:
    147   {{{
    148   sudo yum install -y nc
    149   }}}
     88First of all let's see how we can figure out the IP of the hosts we reserved :
    15089
    151 == 4. Create your OpenFlow sliver ==
    152 Now that we have our myPlc hosts and our PG host is being configured, it is time to reserve our OpenFlow sliver.
    153   i. Add the OpenFlow AM nickname in your `omni_config` file
    154     a. Open file `~/.gcf/omni_config`
    155     b. find the `[aggregate_nicknames]` section in the file and move to th `OpenFlow MAs`
    156     c. Add this line :
    157     {{{
    158 of-tut=,https://aquarion.gpolab.bbn.com:3626/foam/gapi/1
    159 }}}
    160   i. Download your OpenFlow rspec, use the<ofrspec_url> on the paper slip
    161   {{{
    162   wget <ofrspec_url>
    163   }}}
    164   i Edit the OpenFlow rspec. There are only two things you will need to edit :
    165     a. <USERNAME> : use the information on the paper slip
    166     b. <HOSTNAME> : use the hostname of you ProtoGENI host that you note down before (e.g. `pc104.emulab.net`)
    167   i. Create your sliver using the filename of your rspec:
    168   {{{
    169     omni.py createsliver -a of-tut
    170   }}}
     90   * List all the interfaces on your host. You will see that there are many interfaces of the form eth1.XXXX
     91   {{{
     92   /sbin/ifconfig
     93   }}}
     94   Part of the output would look like :
     95   {{{
     96   eth1.1750:42147 Link encap:Ethernet  HWaddr 00:B0:D0:E1:6F:78 
     97          inet addr:10.42.147.90  Bcast:10.42.147.255  Mask:255.255.255.0
     98          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
    17199
    172 === 4b. Modify the configuration file of the OpenFlow controller ===
    173 By now your ProtoGENI host should have been configured. Let's login to the node and modify the configuration file of our OpenFlow controller.
    174   i. Open a new terminal and login to the ProtoGENI host:
    175   {{{
    176 ssh ofctrl
    177 }}}
    178   i. Run :
    179   {{{
    180   ps -ef | grep nox_core
    181 }}}
    182   The output should look like:
    183   {{{
    184 root      6498     1  0 18:33 ?        00:00:01 /usr/local/bin/noxgpo/src/.libs/lt-nox_core -d -i ptcp:6633 switch
    185 inki      6562  6524  0 18:45 pts/0    00:00:00 grep nox_core
    186 }}}
    187   If the first line is missing then the setup has not finished yet. You can move to the next step, but make sure the setup has finished before moving to step 5.
    188   i. Copy the configuration file to your home directory:
    189   {{{
    190   cp /local/l3deflect.conf ~/
    191   }}}
    192   i. Modify the configuration file to match the information in your slip. The should look like:
    193   {{{
    194 # Configuration file for the l3deflect controller
     100   eth1.1750:42148 Link encap:Ethernet  HWaddr 00:B0:D0:E1:6F:78 
     101          inet addr:10.42.148.90  Bcast:10.42.148.255  Mask:255.255.255.0
     102          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
     103   }}}
    195104
    196 [general]
    197 orig_mac = 00:1B:21:4B:3E:E9
    198 orig_ip = 10.42.123.52
    199 deflect_mac = 00:1B:21:4B:3F:AD
    200 deflect_ip = 10.42.123.51
    201 }}}
    202 == 5. Run the experiment ==
    203 Now we have reserved and configured all hosts. We are ready to run our experiment.
    204   i. In the terminal of the remote and of the local server run the following command. Substitute <NC_PORT> with the information in your slip:
    205   {{{
    206   nc <nc_port>
    207 }}}
    208    i. In the terminal of the client run this command, using your paper slip to fill in the information :
    209   {{{
    210   nc <IP_REMOTE_SERVER> <NC_PORT>
    211 }}}
    212   i. You now have a chat application between the remote server and the client. Try typing something in the client and see it pop up at the server side, and vice versa. The forwarding of your packets work because there is a default learning switch controller running on your ProtoGENI host.
    213   i. Let's stop the default learning switch controller and start our deflection controller. In a local terminal type the following two commands:
    214   {{{
    215    ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
    216    ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect"
    217   }}}
    218   The output should look like that:
    219   {{{
    220 nriga@pella:~$ ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
    221 Stopping all OpenFlow controllers
    222 nriga@pella:~$ ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect"
    223 Starting OpenFlow controler l3deflect
    224   }}}
    225   i. If the `nc` is still running on the client, terminate it by pressing `Ctrl-C`.
    226   i Run the `nc` at the client again:
    227   {{{
    228  nc <IP_REMOTE_SERVER> <NC_PORT>
    229 }}}
    230   i Type something on the client and notice in which terminal it appears.
     105   * Figure out the last octet of the IP address of your hosts. The way these MyPLC hosts are setup, is that they have multiple subinterfaces, each configured to be part of a different IP subnet. Almost all subnets in the hosts are in the form of 10.42.Y.0/24. You will notice that for all these subnets, your host has the same last octet.  In the example above all the subinterfaces of eth1, will have an IP address that will end in '''90''' (10.42.147.'''90''', 10.42.148.'''90''')
    231106
    232 If you want to switch back to the switch controller you need to again kill all the controllers and start the switch controller :
    233 {{{
    234  ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh"
    235  ssh ofctrl "/usr/local/bin/start-ctrl.sh switch"
    236 }}}
    237 You can play switching between the two controllers and notice how the text appears on a different terminal.
     107   * Ping over 3715. After logging in to your hosts, ping from host1 to host2. Assuming that host2 has a last octet of YYY you should:
     108   {{{
     109ping 10.42.112.YYY
     110   }}}
     111   Notice the RTT on the packets.
    238112
    239 == 6. Looking under the hood ==
    240 If you still have a few minutes, then you can poke around the OpenFlow controller, if you don't have enough time then make sure you move to the next step
    241 and cleanup your resources. You can always come back and do the tutorial again and poke around.
     113   * Ping over 3716. After logging in to your hosts, ping from one host2 to host1. Assuming that host1 has XXX as it's last octet you should ping
     114   {{{
     115ping 10.42.113.XXX
     116   }}}
     117   * Notice the RTT on the packets and compare it with the above ping.
    242118
    243 Congratulations you have successfully diverted the traffic from your client to the local server using OpenFlow!. But how did this work? Basically our openflow controller
    244 instructed the switch to rewrite the IP information on the packet so that
    245   * every packet that was destined to the original server, it would be changed to be sent to the local one
    246   * also all packets from the local  server are rewritten so that the client thinks that it talks to the remote server.
    247 
    248 Let's take a quick look at the controller. On the terminal that is logged in to the ProtoGENI host open file `/usr/local/src/noxgpo/src/nox/coreapps/example/l3deflect.py`. This is the deflection module. Lets try and follow the logic now:
    249   i. Find the `install` function. This is the function that registers your module to receive OpenFlow events.
    250   i Look at which function is called when a packet is received.
    251   i. Ignore the first checks and find the function that is responsible for forwarding packets.
    252   i. Look at the logic that is overwriting the packet. Basically there is a section like this :
    253   {{{
    254         # Check if the eth_type must be deflected
    255         if ethtype == 0x800 :
    256           # and if the dst mac address is the one that should be deflected
    257           if mac_to_str(packet.dst).lower() == self.orig_mac.lower() :
    258             # replace the destination mac address
    259             actions.append([openflow.OFPAT_SET_DL_DST,
    260                                 octstr_to_array(self.deflect_mac)])
    261             logger.debug("NEW DST MAC %s" % ( self.deflect_mac))
    262             dstaddr = octstr_to_array(self.deflect_mac).tostring()
    263             actions.append([openflow.OFPAT_SET_NW_DST,
    264                                 ipstr_to_int(self.deflect_ip)])
    265             logger.debug("NEW DST IP %s" % ( self.deflect_ip))
    266             logger.info("Packet %s is deflected")
    267 
    268           if mac_to_str(packet.src).lower() == self.deflect_mac.lower() :
    269             # replace the destination mac address
    270             actions.append([openflow.OFPAT_SET_DL_SRC,
    271                                 octstr_to_array(self.orig_mac)])
    272             logger.debug("NEW SRC MAC %s" % ( self.orig_mac))
    273             actions.append([openflow.OFPAT_SET_NW_SRC,
    274                                 ipstr_to_int(self.orig_ip)])
    275             logger.debug("NEW DST IP %s" % ( self.orig_ip))
    276   }}}
    277119== 7. Clean up ==
    278 
    279 When you're done, please release your resources so they'll be available to others. Make sure you delete all the resources in all AMs. You have a slice at :
    280   * pg-utah
    281   * at two myplc sites
    282   * at of-tut
     120Congratulations! You're done with this exercise, please release your resources before moving on so they'll be available to others. Make sure you delete all the resources in all AMs you used when you reserved resources in Step 1.
    283121
    284122{{{