Changes between Version 2 and Version 3 of GENIExperimenter/Tutorials/ClickExample


Ignore:
Timestamp:
03/17/13 02:42:12 (11 years ago)
Author:
nriga@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIExperimenter/Tutorials/ClickExample

    v2 v3  
    6060}}}
    6161
    62 = Example Experiment - Click Routers =
    63 
    64 
    65 
    66 == 1. Add another user to your experiment ==
    67 Omni gives you the capability of giving access  to other users on your compute resources. Depending on which AM you are using to get resources from,  this is done in a different way.  Ask the team next to you about their username and do the following:
    68   i. While in a terminal, download their public key under `~/.ssh/` :
    69   {{{
    70     cd ~/.ssh
    71     wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub
    72   }}}
    73 
    74   i. Follow the instructions [http://trac.gpolab.bbn.com/gcf/wiki/OmniConfigure/Manual#B1.MultipleusersinProtoGENIAMs these instructions] and add another user for ProtoGENI AMs
    75 
    76 == 1. Create your experiment ==
    77 In this step, we are going to setup the experiment. In this tutorial we assume that you are sufficiently comfortable with omni to verify that a '''listresources''' command works and to know when your slice is ready using '''sliverstatus'''.
    78   i. '''Create a slice''', use the slicename given to you in the paper slip:
    79   {{{
    80 omni.py createslice <slicename>
    81 }}}
    82   i. '''Create a sliver''' using the rspec from the URL given in your paper slip:
    83   {{{
    84 omni.py createsliver -a pg-utah <slicename> <rspec_url>
    85 }}}
    86   i. '''Check the status of your sliver'''
    87   {{{
    88 omni.py sliverstatus -a pg-utah <slicename>
    89 }}}
    90 
    91 == 2. Install scripts ==
    92 While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts.
    93 In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.
    94   i. Download your rspec
    95   {{{
    96    cd /tmp
    97    wget <rspec_url>
    98 }}}
    99   i. Open your rspec and look for the `install` tag and copy the value of the URL attribute.
    100   i. Download and untar the software
    101   {{{
    102   cd /tmp
    103   wget <software_url>
    104   tar xvfz <software_name>
    105 }}}
    106   i. Look in your rspec and locate the `execute` tag. Note what script is being executed at boot time.
    107   i. Locate the script and open it. Can you identify the different parts?
    108 
    109 == 3. Configure your routers ==
    110 Once our sliver is ready we will go ahead and configure our click routers. In this example we have 4 routers, so instead of logging into each one of them and configuring it, we are going to use remote execution and configure them from our VM.
    111 
    112 First lets reset our environment:
    113 {{{
    114 cd
    115 rm .ssh/config
    116 touch .ssh/config
    117 }}}
    118 
    119 === 3a. Login and remote execution ===
    120 Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want.
    121 
    122 {{{
    123 readyToLogin.py -a pg-utah <slicename> > login.out 2>&1
    124 }}}
    125 
    126 You'll get a big chunk of information, but you're interested in the '''ssh configuration info''' information near the end.
    127 
    128 {{{
    129 ... <lots of output> ...
    130 ================================================================================
    131 SSH CONFIGURATION INFO for User inki
    132 ================================================================================
    133  
    134 Host left
    135   Port 30778
    136   HostName pc403.emulab.net
    137   User inki
    138   IdentityFile /home/geni/.ssh/geni_key
    139 
    140 Host hostB
    141   Port 30779
    142   HostName pc490.emulab.net
    143   User inki
    144   IdentityFile /home/geni/.ssh/geni_key
    145 
    146 Host hostA
    147   Port 30778
    148   HostName pc545.emulab.net
    149   User inki
    150   IdentityFile /home/geni/.ssh/geni_key
    151 
    152 Host bottom
    153   Port 30778
    154   HostName pc490.emulab.net
    155   User inki
    156   IdentityFile /home/geni/.ssh/geni_key
    157  
    158 Host right
    159   Port 30778
    160   HostName pc411.emulab.net
    161   User inki
    162   IdentityFile /home/geni/.ssh/geni_key
    163  
    164 Host top
    165   Port 30779
    166   HostName pc545.emulab.net
    167   User inki
    168   IdentityFile /home/geni/.ssh/geni_key
    169 
    170 ...<more output>...
    171 }}}
    172 
    173 Copy all the above information and paste it into your `.ssh/config` file, then you can very easily login into your nodes, just by using the nickname (client_id) of the nodes.
    174 
    175 Your `~/.ssh/config` file should look like
    176 {{{
    177 IdentityFile /home/geni/.ssh/geni_key
    178 Host left
    179   Port 30778
    180   HostName pc403.emulab.net
    181   User inki
    182   IdentityFile /home/geni/.ssh/geni_key
    183  
    184 Host hostB
    185   Port 30779
    186   HostName pc490.emulab.net
    187   User inki
    188   IdentityFile /home/geni/.ssh/geni_key
    189 
    190 Host hostA
    191   Port 30778
    192   HostName pc545.emulab.net
    193   User inki
    194   IdentityFile /home/geni/.ssh/geni_key
    195 
    196 Host bottom
    197   Port 30778
    198   HostName pc490.emulab.net
    199   User inki
    200   IdentityFile /home/geni/.ssh/geni_key
    201 
    202 Host right
    203   Port 30778
    204   HostName pc411.emulab.net
    205   User inki
    206   IdentityFile /home/geni/.ssh/geni_key
    207 
    208 Host top
    209   Port 30779
    210   HostName pc545.emulab.net
    211   User inki
    212   IdentityFile /home/geni/.ssh/geni_key
    213 
    214 }}}
    215 
    216 Let's login to our two hosts, the nicknames are `hostA` and `hostB`
    217   i. Open two new terminals
    218   i. In one terminal type
    219   {{{
    220   ssh -A hostA
    221   }}}
    222   and in the other
    223   {{{
    224   ssh -A hostB
    225   }}}
    226 ==== Test remote execution ====
    227 You can execute commands in a remote host using `ssh`. To do this just follow your ssh command with the command you want to execute in quotes.
    228   i. In your local terminal type :
    229   {{{
    230   ssh -A top "ls -a"
    231   }}}
    232   This will list all the files under the home directory on host `top`. The output should look like:
    233   {{{
    234 geni@geni-VirtualBox:~$ ssh -A top "ls -a"
    235 .
    236 ..
    237 .bash_logout
    238 .bash_profile
    239 .bashrc
    240 .forward
    241 .kshrc
    242 .ssh
    243 .zshrc
    244   }}}
    245 
    246 If you get something similar you are all set for controlling your nodes from your computer.
    247 
    248 === 3b. Configure your routers ===
    249 We are going to use remote execution to configure our routers.
    250   i. On a local terminal run the following command four times, each time substituting the <router_nickname> with one of the top, bottom, left, right:
    251   {{{
    252   geni@geni-VirtualBox:~$ ssh -A <router_nickname> "/local/click-example/extractClickConfig.py "
    253   }}}
    254   You'll get output something like this:
    255   {{{
    256 Your host information:
    257         hostA: hostA.StupidSliceName.emulab-net.emulab.net pc347.emulab.net
    258         top: top.StupidSliceName.emulab-net.emulab.net pc336.emulab.net
    259         left: left.StupidSliceName.emulab-net.emulab.net pc358.emulab.net
    260         right: right.StupidSliceName.emulab-net.emulab.net pc278.emulab.net
    261         bottom: bottom.StupidSliceName.emulab-net.emulab.net pc348.emulab.net
    262         hostB: hostB.StupidSliceName.emulab-net.emulab.net pc353.emulab.net
    263 Done.
    264 }}}
    265   (If you are prompted for a password, check to make sure that you provided the -A switch in your ssh command above.)
    266 
    267   i. The extractClickConfig script produces router configurations for your experiment. It also creates a diagram of your experiment. Get a copy locally from one of the routers, by typing in a local terminal:
    268   {{{
    269  scp top:myslice.png ./
    270 }}}
    271   i. View the diagram by typing :
    272   {{{
    273   eog myslice.png &
    274 }}}
    275   Your slice will look something like the one below (see [attachment:myslice.png]). The overall configuration should be the same, with two end hosts, named hostA and hostB, and four routers (top, left, right, bottom) in a diamond configuration. The host names, interface names, and MAC addresses will be different, depending on the actual resources assigned to your slice.
    276 
    277 [[Image(myslice.png, 25%)]]
    278 
    279 The four routers interconnected by solid lines are your "core network," which will run a non-standard, non-IP protocol. The dashed lines out to the end hosts carry standard IP traffic.
    280 
    281 == 4. Turn off internet protocol ==
    282 
    283 At this point, your network is still running IP. You can check by running a ping. In your '''hosta''' terminal window, run this command.
    284 
    285 {{{
    286 ping -c 3 hostb
    287 }}}
    288 
    289 The command should succeed, with output like this:
    290 
    291 {{{
    292 PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
    293 64 bytes from hostB-link-B (10.10.6.2): icmp_seq=1 ttl=61 time=1.38 ms
    294 64 bytes from hostB-link-B (10.10.6.2): icmp_seq=2 ttl=61 time=1.19 ms
    295 64 bytes from hostB-link-B (10.10.6.2): icmp_seq=3 ttl=61 time=1.53 ms
    296 
    297 --- hostB-link-B ping statistics ---
    298 3 packets transmitted, 3 received, 0% packet loss, time 2004ms
    299 rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms
    300 }}}
    301 
    302 Since our experiment doesn't want IP, let's turn it off :
    303   i. On a local terminal run the following command four times, each time substituting the <router_nickname> with one of the top, bottom, left, right:
    304   {{{
    305   ssh -A <router_nickname> "sh ./stopIP.sh"
    306 }}}
    307 
    308   You'll get output like this (the interface names may be different):
    309   {{{
    310 Disabling IP on interface mv10.9
    311 Disabling IP on interface mv10.10
    312 }}}
    313 
    314   i. Verify that IP is really off, try another ping. On '''hosta''':
    315   {{{
    316   ping -c 3 hostb
    317 }}}
    318   The command should take twelve seconds to time out, then fail with output like this:
    319   {{{
    320 PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
    321 
    322 --- hostB-link-B ping statistics ---
    323 3 packets transmitted, 0 received, 100% packet loss, time 11999ms
    324 }}}
    325 
    326 == 5. Start your routers ==
    327 
    328 The extractor script produces a click configuration file for each of your routers. 
    329   i. On a local terminal run the following command four times, each time substituting the <router_nickname> with one of the top, bottom, left, right:
    330   {{{
    331   ssh -A <router_nickname> "sh ./startClick.sh"
    332 }}}
    333   You'll get output like this. (Don't worry about the warning messages, Click is just reminding you that you have no IP addresses in your core network.) The output of the click router is redirected to `/tmp/click.out` on each host.
    334 
    335   {{{
    336 Stopping any running Click routers
    337 Starting Click router
    338 top.click:34: While initializing ‘FromDevice@18 :: FromDevice’:
    339   warning: eth2: no IPv4 address assigned
    340 top.click:35: While initializing ‘FromDevice@21 :: FromDevice’:
    341   warning: eth4: no IPv4 address assigned
    342 }}}
    343 
    344 Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.
    345 
    346 == 6. Send some traffic ==
    347 
    348 Now you'll use your two edge hosts, '''hostA''' and '''hostB''' to send traffic along your network.  Since these end hosts are not running your modified protocol, they'll rely on the '''top''' and '''bottom''' routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.
    349   i. In your terminal window on '''hostB''', instruct '''nc''' to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
    350   {{{
    351 [mberman@hostb ~]$ nc -ul 24565
    352 }}}
    353   i. Connect to it from your terminal window on '''hostA''':
    354   {{{
    355 [mberman@hosta ~]$ nc -u hostb 24565
    356 }}}
    357 
    358 You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.
    359 
    360 That's it!  Now, let's look inside to see what's going on.
    361 
    362 == 7. Looking under the hood ==
    363 
    364 Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.
    365 
    366 Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)
    367 
    368 === 7b. Packet transformation ===
    369 
    370   i. The more interesting configuration appears here, in the '''top.click''' configuration file. In a local terminal type:
    371   {{{
    372  ssh -A top "cat top.click"
    373 }}}
    374   The output will look like :
    375   {{{
    376 // This portion accepts IP packets,
    377 // reformats them, and routes them
    378 // to an internal router.
    379 route :: Classifier(27/01%01,-);
    380 
    381 modify :: Unstrip(2) ->
    382     StoreData(0, "AliceWasHere3546") ->
    383     route;
    384 
    385 FromDevice(eth3, PROMISC true) ->
    386     Classifier(12/0800) ->
    387     modify;
    388 
    389 route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
    390     SimpleQueue ->
    391     Print(outL) ->
    392     ToDevice(eth2);
    393 
    394 route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
    395     SimpleQueue ->
    396     Print(outR) ->
    397     ToDevice(eth4);
    398 
    399 // This portion accepts non-IP packets
    400 // with an ether type of 0x7744
    401 // from an internal router, restores
    402 // them to IP format, and forwards.
    403 restore :: SimpleQueue ->
    404     Strip(30) ->
    405     EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->
    406     ToDevice(eth3);
    407 
    408 FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
    409 FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;
    410 }}}
    411 
    412   As indicated in the comments, the top portion of the configuration listens ('''!FromDevice''') for IP packets arriving on the interface connected to '''hostA''' (that's '''eth3''' in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the '''Unstrip''' operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti ('''!StoreData'''). The '''route''' operation then routes the packet via either the '''left''' or '''right''' router toward '''hostB'''. In either case, it wraps the packet in a fresh Ethernet header ('''!EtherEncap''') with a distinctive ether type code (0x7744), logs the new packet on its way out ('''Print''') and sends it out on the correct interface ('''!ToDevice''').
    413 
    414   The bottom portion of the configuration is intended for packets coming out of the core network to '''hostA'''.  It accepts packets from either the '''left''' or '''right''' router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to '''hostA'''.
    415 
    416   The configuration for the '''bottom''' router is exactly symmetric, routing packets between '''hostB''' and the core network, but using different graffiti.
    417 
    418 === 7b. Simple Forwarding ===
    419 
    420 The '''left''' router configuration is much simpler.  In a local terminal type:
    421 {{{
    422  ssh -A left "cat left.click"
    423 }}}
    424   The output will look like :
    425 
    426 {{{
    427 // Copy packets from top to bottom.
    428 FromDevice(eth2) ->
    429     StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
    430     StoreEtherAddress(00:04:23:b7:18:fb, src) ->
    431     SimpleQueue ->
    432     Print(top) ->
    433     ToDevice(eth3);
    434 // Copy packets from bottom to top.
    435 FromDevice(eth3) ->
    436     StoreEtherAddress(00:04:23:b7:14:76, dst) ->
    437     StoreEtherAddress(00:04:23:b7:18:fa, src) ->
    438     SimpleQueue ->
    439     Print(bottom) ->
    440     ToDevice(eth2);
    441 }}}
    442 
    443 This configuration just blindly forwards packets. It picks up any packet from the '''top''' router, updates the Ethernet header, and passes it along to the '''bottom''' router. The same applies in the reverse direction.  Again, the configuration for the '''right''' router is exactly analogous.
    444 
    445 === 7c. Monitoring your core network ===
    446 
    447 Let's watch how the packets travel through the network.
    448   i. In a local terminal type:
    449   {{{
    450   ssh -A top "tail -f /tmp/click.log"
    451 }}}
    452   i. Go to your window for hostA, where your '''nc''' command is still running. Type a message into this window. You should see a log message in three of your four router windows. In this example, you might see:
    453   i. In the local terminal you will see:
    454   {{{
    455 outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
    456 }}}
    457   This log entry says that the '''top''' router received a packet from '''hostA''', modified it, and sent it out to the '''right''' router. If the entry started with '''outL''', that would indicate that it sent the packet out to the '''left''' router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of '''eth4''' on '''right'''. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or '''eth4''' on '''top'''. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "!AliceWasHe" in ASCII.
    458   i. Try typing a few different lines to hostA. You should see some packets routed to the left and some to the right. The routing decision is based on the '''route :: Classifier(27/01%01,-);''' entry in the '''top''' router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.
    459 
    460 == 9. Clean up ==
    461 
    462 When you're done, please release your resources so they'll be available to others.
    463 
    464 {{{
    465 omni.py deletesliver -a pg-utah <slicename>
    466 }}}
    467 
    468 Now go [wiki:GEC15Agenda/AdvancedGENITopoOmni/Instructions#Wrapup back to the first page] and wrap up:-)