Changes between Initial Version and Version 1 of GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment


Ignore:
Timestamp:
10/21/12 13:55:22 (10 years ago)
Author:
nriga@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment

    v1 v1  
     1[[PageOutline]]
     2
     3= Example Experiment -Layer 3 Deflect =
     4
     5In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use:
     6  * 1 ProtoGENI host to run our OpenFlow controller
     7  * 3 myPlc hosts, 1 will be the client and 2 will be the servers
     8  * OpenFlow resources that will connect the three hosts
     9
     10Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.
     11
     12== 1. Add another user to your experiment ==
     13Omni gives you the capability of giving access  to other users on your compute resources. Depending on which AM you are using to get resources from,  this is done in a different way.  Ask the team next to you about their username and do the following:
     14  i. While in a terminal, download their public key under `~/.ssh/` :
     15  {{{
     16    cd ~/.ssh
     17    wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub
     18  }}}
     19
     20  i. Follow the instructions [wiki:HowTo/MulitpleUsersWithOmni these instructions] and add another user for ProtoGENI AMs
     21
     22== 1. Create your experiment ==
     23In this step, we are going to setup the experiment. In this tutorial we assume that you are sufficiently comfortable with omni to verify that a '''listresources''' command works and to know when your slice is ready using '''sliverstatus'''.
     24  i. '''Create a slice''', use the slicename given to you in the paper slip:
     25  {{{
     26omni.py createslice <slicename>
     27}}}
     28  i. '''Create a sliver''' using the rspec from the URL given in your paper slip:
     29  {{{
     30omni.py createsliver -a pg-utah <slicename> <rspec_url>
     31}}}
     32  i. '''Check the status of your sliver'''
     33  {{{
     34omni.py sliverstatus -a pg-utah <slicename>
     35}}}
     36
     37== 2. Install scripts ==
     38While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts.
     39In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.
     40  i. Download your rspec
     41  {{{
     42   cd /tmp
     43   wget <rspec_url>
     44}}}
     45  i. Open your rspec and look for the `install` tag and copy the value of the URL attribute.
     46  i. Download and untar the software
     47  {{{
     48  cd /tmp
     49  wget <software_url>
     50  tar xvfz <software_name>
     51}}}
     52  i. Look in your rspec and locate the `execute` tag. Note what script is being executed at boot time.
     53  i. Locate the script and open it. Can you identify the different parts?
     54
     55== 3. Configure your routers ==
     56Once our sliver is ready we will go ahead and configure our click routers. In this example we have 4 routers, so instead of logging into each one of them and configuring it, we are going to use remote execution and configure them from our VM.
     57
     58=== 3a. Login and remote execution ===
     59Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want.
     60
     61{{{
     62readyToLogin.py -a pg-utah <slicename> > login.out 2>&1
     63}}}
     64
     65You'll get a big chunk of information, but you're interested in the '''ssh configuration info''' information near the end.
     66
     67{{{
     68... <lots of output> ...
     69================================================================================
     70SSH CONFIGURATION INFO for User inki
     71================================================================================
     72 
     73Host left
     74  Port 30778
     75  HostName pc403.emulab.net
     76  User inki
     77  IdentityFile /home/geni/.ssh/geni_key
     78
     79Host hostB
     80  Port 30779
     81  HostName pc490.emulab.net
     82  User inki
     83  IdentityFile /home/geni/.ssh/geni_key
     84
     85Host hostA
     86  Port 30778
     87  HostName pc545.emulab.net
     88  User inki
     89  IdentityFile /home/geni/.ssh/geni_key
     90
     91Host bottom
     92  Port 30778
     93  HostName pc490.emulab.net
     94  User inki
     95  IdentityFile /home/geni/.ssh/geni_key
     96 
     97Host right
     98  Port 30778
     99  HostName pc411.emulab.net
     100  User inki
     101  IdentityFile /home/geni/.ssh/geni_key
     102 
     103Host top
     104  Port 30779
     105  HostName pc545.emulab.net
     106  User inki
     107  IdentityFile /home/geni/.ssh/geni_key
     108
     109...<more output>...
     110}}}
     111
     112Copy all the above information and paste it into your `.ssh/config` file, then you can very easily login into your nodes, just by using the nickname (client_id) of the nodes.
     113
     114Your `~/.ssh/config` file should look like
     115{{{
     116IdentityFile /home/geni/.ssh/geni_key
     117Host left
     118  Port 30778
     119  HostName pc403.emulab.net
     120  User inki
     121  IdentityFile /home/geni/.ssh/geni_key
     122 
     123Host hostB
     124  Port 30779
     125  HostName pc490.emulab.net
     126  User inki
     127  IdentityFile /home/geni/.ssh/geni_key
     128
     129Host hostA
     130  Port 30778
     131  HostName pc545.emulab.net
     132  User inki
     133  IdentityFile /home/geni/.ssh/geni_key
     134
     135Host bottom
     136  Port 30778
     137  HostName pc490.emulab.net
     138  User inki
     139  IdentityFile /home/geni/.ssh/geni_key
     140
     141Host right
     142  Port 30778
     143  HostName pc411.emulab.net
     144  User inki
     145  IdentityFile /home/geni/.ssh/geni_key
     146
     147Host top
     148  Port 30779
     149  HostName pc545.emulab.net
     150  User inki
     151  IdentityFile /home/geni/.ssh/geni_key
     152
     153}}}
     154
     155Let's login to our two hosts, the nicknames are `hostA` and `hostB`
     156  i. Open two new terminals
     157  i. In one terminal type
     158  {{{
     159  ssh -A hostA
     160  }}}
     161  and in the other
     162  {{{
     163  ssh -A hostB
     164  }}}
     165==== Test remote execution ====
     166You can execute commands in a remote host using `ssh`. To do this just follow your ssh command with the command you want to execute in quotes.
     167  i. In your local terminal type :
     168  {{{
     169  ssh -A top "ls -a"
     170  }}}
     171  This will list all the files under the home directory on host `top`. The output should look like:
     172  {{{
     173geni@geni-VirtualBox:~$ ssh -A top "ls -a"
     174.
     175..
     176.bash_logout
     177.bash_profile
     178.bashrc
     179.forward
     180.kshrc
     181.ssh
     182.zshrc
     183  }}}
     184
     185If you get something similar you are all set for controlling your nodes from your computer.
     186
     187=== 3b. Configure your routers ===
     188We are going to use remote execution to configure our routers.
     189  i. On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
     190  {{{
     191  geni@geni-VirtualBox:~$ ssh -A <router_nickname> "/local/click-example/extractClickConfig.py "
     192  }}}
     193  You'll get output something like this:
     194  {{{
     195Your host information:
     196        hostA: hostA.StupidSliceName.emulab-net.emulab.net pc347.emulab.net
     197        top: top.StupidSliceName.emulab-net.emulab.net pc336.emulab.net
     198        left: left.StupidSliceName.emulab-net.emulab.net pc358.emulab.net
     199        right: right.StupidSliceName.emulab-net.emulab.net pc278.emulab.net
     200        bottom: bottom.StupidSliceName.emulab-net.emulab.net pc348.emulab.net
     201        hostB: hostB.StupidSliceName.emulab-net.emulab.net pc353.emulab.net
     202Done.
     203}}}
     204  (If you are prompted for a password, check to make sure that you provided the -A switch in your ssh command above.)
     205
     206  i. The extractClickConfig script produces router configurations for your experiment. It also creates a diagram of your experiment. Get a copy locally from one of the routers, by typing in a local terminal:
     207  {{{
     208 scp top:myslice.png
     209}}}
     210  i. View the diagram by typing :
     211  {{{
     212  eog myslice.png &
     213}}}
     214  Your slice will look something like the one below (see [attachment:myslice.png]). The overall configuration should be the same, with two end hosts, named hostA and hostB, and four routers (top, left, right, bottom) in a diamond configuration. The host names, interface names, and MAC addresses will be different, depending on the actual resources assigned to your slice.
     215
     216[[Image(myslice.png, 25%)]]
     217
     218The four routers interconnected by solid lines are your "core network," which will run a non-standard, non-IP protocol. The dashed lines out to the end hosts carry standard IP traffic.
     219
     220== 4. Turn off internet protocol ==
     221
     222At this point, your network is still running IP. You can check by running a ping. In your '''hosta''' terminal window, run this command.
     223
     224{{{
     225ping -c 3 hostb
     226}}}
     227
     228The command should succeed, with output like this:
     229
     230{{{
     231PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
     23264 bytes from hostB-link-B (10.10.6.2): icmp_seq=1 ttl=61 time=1.38 ms
     23364 bytes from hostB-link-B (10.10.6.2): icmp_seq=2 ttl=61 time=1.19 ms
     23464 bytes from hostB-link-B (10.10.6.2): icmp_seq=3 ttl=61 time=1.53 ms
     235
     236--- hostB-link-B ping statistics ---
     2373 packets transmitted, 3 received, 0% packet loss, time 2004ms
     238rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms
     239}}}
     240
     241Since our experiment doesn't want IP, let's turn it off :
     242  i. On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
     243  {{{
     244  ssh -A <router_nickname> "sh ./stopIP.sh"
     245}}}
     246
     247  You'll get output like this (the interface names may be different):
     248  {{{
     249Disabling IP on interface mv10.9
     250Disabling IP on interface mv10.10
     251}}}
     252
     253  i. Verify that IP is really off, try another ping. On '''hosta''':
     254  {{{
     255  ping -c 3 hostb
     256}}}
     257  The command should take twelve seconds to time out, then fail with output like this:
     258  {{{
     259PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
     260
     261--- hostB-link-B ping statistics ---
     2623 packets transmitted, 0 received, 100% packet loss, time 11999ms
     263}}}
     264
     265== 5. Start your routers ==
     266
     267The extractor script produces a click configuration file for each of your routers. 
     268  i. On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
     269  {{{
     270  ssh -A <router_nickname> "sh ./startClick.sh"
     271}}}
     272  You'll get output like this. (Don't worry about the warning messages, Click is just reminding you that you have no IP addresses in your core network.) The output of the click router is redirected to `/tmp/click.out` on each host.
     273
     274  {{{
     275Stopping any running Click routers
     276Starting Click router
     277top.click:34: While initializing ‘FromDevice@18 :: FromDevice’:
     278  warning: eth2: no IPv4 address assigned
     279top.click:35: While initializing ‘FromDevice@21 :: FromDevice’:
     280  warning: eth4: no IPv4 address assigned
     281}}}
     282
     283Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.
     284
     285== 6. Send some traffic ==
     286
     287Now you'll use your two edge hosts, '''hostA''' and '''hostB''' to send traffic along your network.  Since these end hosts are not running your modified protocol, they'll rely on the '''top''' and '''bottom''' routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.
     288  i. In your terminal window on '''hostB''', instruct '''nc''' to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
     289  {{{
     290[mberman@hostb ~]$ nc -ul 24565
     291}}}
     292  i. Connect to it from your terminal window on '''hostA''':
     293  {{{
     294[mberman@hosta ~]$ nc -u hostb 24565
     295}}}
     296
     297You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.
     298
     299That's it!  Now, let's look inside to see what's going on.
     300
     301== 7. Looking under the hood ==
     302
     303Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.
     304
     305Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)
     306
     307=== 7b. Packet transformation ===
     308
     309  i. The more interesting configuration appears here, in the '''top.click''' configuration file. In a local terminal type:
     310  {{{
     311 ssh -A top "cat top.click"
     312}}}
     313  The output will look like :
     314  {{{
     315// This portion accepts IP packets,
     316// reformats them, and routes them
     317// to an internal router.
     318route :: Classifier(27/01%01,-);
     319
     320modify :: Unstrip(2) ->
     321    StoreData(0, "AliceWasHere3546") ->
     322    route;
     323
     324FromDevice(eth3, PROMISC true) ->
     325    Classifier(12/0800) ->
     326    modify;
     327
     328route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
     329    SimpleQueue ->
     330    Print(outL) ->
     331    ToDevice(eth2);
     332
     333route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
     334    SimpleQueue ->
     335    Print(outR) ->
     336    ToDevice(eth4);
     337
     338// This portion accepts non-IP packets
     339// with an ether type of 0x7744
     340// from an internal router, restores
     341// them to IP format, and forwards.
     342restore :: SimpleQueue ->
     343    Strip(30) ->
     344    EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->
     345    ToDevice(eth3);
     346
     347FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
     348FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;
     349}}}
     350
     351  As indicated in the comments, the top portion of the configuration listens ('''!FromDevice''') for IP packets arriving on the interface connected to '''hostA''' (that's '''eth3''' in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the '''Unstrip''' operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti ('''!StoreData'''). The '''route''' operation then routes the packet via either the '''left''' or '''right''' router toward '''hostB'''. In either case, it wraps the packet in a fresh Ethernet header ('''!EtherEncap''') with a distinctive ether type code (0x7744), logs the new packet on its way out ('''Print''') and sends it out on the correct interface ('''!ToDevice''').
     352
     353  The bottom portion of the configuration is intended for packets coming out of the core network to '''hostA'''.  It accepts packets from either the '''left''' or '''right''' router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to '''hostA'''.
     354
     355  The configuration for the '''bottom''' router is exactly symmetric, routing packets between '''hostB''' and the core network, but using different graffiti.
     356
     357=== 7b. Simple Forwarding ===
     358
     359The '''left''' router configuration is much simpler.  In a local terminal type:
     360{{{
     361 ssh -A left "cat left.click"
     362}}}
     363  The output will look like :
     364
     365{{{
     366// Copy packets from top to bottom.
     367FromDevice(eth2) ->
     368    StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
     369    StoreEtherAddress(00:04:23:b7:18:fb, src) ->
     370    SimpleQueue ->
     371    Print(top) ->
     372    ToDevice(eth3);
     373// Copy packets from bottom to top.
     374FromDevice(eth3) ->
     375    StoreEtherAddress(00:04:23:b7:14:76, dst) ->
     376    StoreEtherAddress(00:04:23:b7:18:fa, src) ->
     377    SimpleQueue ->
     378    Print(bottom) ->
     379    ToDevice(eth2);
     380}}}
     381
     382This configuration just blindly forwards packets. It picks up any packet from the '''top''' router, updates the Ethernet header, and passes it along to the '''bottom''' router. The same applies in the reverse direction.  Again, the configuration for the '''right''' router is exactly analogous.
     383
     384=== 7c. Monitoring your core network ===
     385
     386Let's watch how the packets travel through the network.
     387  i. In a local terminal type:
     388  {{{
     389  ssh -A top "tail -f /tmp/click.log"
     390}}}
     391  i. Go to your window for hostA, where your '''nc''' command is still running. Type a message into this window. You should see a log message in three of your four router windows. In this example, you might see:
     392  i. In the local terminal you will see:
     393  {{{
     394outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
     395}}}
     396  This log entry says that the '''top''' router received a packet from '''hostA''', modified it, and sent it out to the '''right''' router. If the entry started with '''outL''', that would indicate that it sent the packet out to the '''left''' router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of '''eth4''' on '''right'''. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or '''eth4''' on '''top'''. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "!AliceWasHe" in ASCII.
     397  i. Try typing a few different lines to hostA. You should see some packets routed to the left and some to the right. The routing decision is based on the '''route :: Classifier(27/01%01,-);''' entry in the '''top''' router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.
     398
     399== 9. Clean up ==
     400
     401When you're done, please release your resources so they'll be available to others.
     402
     403{{{
     404omni.py deletesliver -a pg-utah <slicename>
     405}}}