Changes between Initial Version and Version 1 of GEC15Agenda/AdvancedGENITopoOmni/Instructions/ClickExampleExperiment


Ignore:
Timestamp:
10/18/12 22:14:26 (7 years ago)
Author:
nriga@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC15Agenda/AdvancedGENITopoOmni/Instructions/ClickExampleExperiment

    v1 v1  
     1[[PageOutline]]
     2
     3= Example Experiment - Click Routers =
     4
     5In this example experiment, you will configure and run a non-IP software routing configuration, using the [http://read.cs.ucla.edu/click/click Click] modular router and [http://protogeni.net ProtoGENI] hosts. In this example, we'll be running click in user mode. Once you have the prerequisites in place, you should be able to complete this example experiment in under an hour.
     6
     7Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.
     8
     9== Prerequisites ==
     10
     11Before beginning this experiment, you should:
     12
     13 * Have a GENI credential. If you don't, check out SignMeUp.
     14 * Configure omni (version 1.5.2 or later) on your machine. Be sufficiently comfortable with omni to verify that a '''listresources''' command works and to know when your slice is ready using '''sliverstatus'''.
     15 
     16Information on obtaining GENI credentials and omni is available at SignMeUp and [wiki:HowToUseOmni] or by contacting [mailto:help@geni.net].
     17
     18== Setup ==
     19
     20Create a new directory, '''click-example''', on your machine. You will do the rest of your work from this directory.
     21
     22{{{
     23mkdir click-example
     24cd click-example
     25}}}
     26
     27Download [attachment:click-example.rspec?format=raw the attached rspec file] and save it in your '''click-example''' directory, using the name '''click-example.rspec'''. Be sure to use raw mode.
     28
     29If you haven't already, add your omni source and examples directories to your PATH:
     30
     31{{{
     32export PATH=$PATH:/path/to/gcf-1.x.x/src:/path/to/gcf-1.x.x/examples
     33}}}
     34
     35If you haven't already, add your omni source directory to your PYTHONPATH:
     36
     37{{{
     38export PYTHONPATH=$PYTHONPATH:/path/to/gcf-1.x.x/src
     39}}}
     40
     41(Optional) You'll be making heavy use of your private key to log into your ProtoGENI hosts in the steps below. If your key is encrypted, you might want to take a look at ways for [wiki:HowTo/LoginToNodes#ManagingSSHKeys managing your ssh keys] to make this easier.
     42
     43== Obtain your resources ==
     44
     45Create your slice. Please don't use my stupid slice name.  This example will use the Utah ProtoGENI site. You can choose a different site by selecting a different aggregate manager with the -a switch.
     46
     47{{{
     48omni.py createslice -a http://www.emulab.net/protogeni/xmlrpc/am StupidSliceName
     49}}}
     50
     51Create a sliver and add resources. (You changed the slice name, right?)
     52
     53{{{
     54omni.py createsliver -a http://www.emulab.net/protogeni/xmlrpc/am StupidSliceName click-example.rspec
     55}}}
     56
     57Wait until your sliver is ready, typically a few minutes. (You can monitor your sliver with [http://protogeni.net Flack] or use omni's '''sliverstatus''' command as shown below.)
     58
     59{{{
     60omni.py sliverstatus -a http://www.emulab.net/protogeni/xmlrpc/am StupidSliceName
     61}}}
     62
     63When your sliver is ready, run the '''readyToLogin''' script to get login information for your nodes. Use the same aggregate manager and slice name that you used for your '''sliverstatus''' command above.
     64{{{
     65readyToLogin.py -a http://www.emulab.net/protogeni/xmlrpc/am StupidSliceName
     66}}}
     67
     68You'll get a big chunk of information, but you're interested in the '''ssh''' command information near the end.
     69
     70{{{
     71... <lots of output> ...
     72================================================================================
     73Aggregate [http://www.emulab.net/protogeni/xmlrpc/am] has a ProtoGENI sliver.
     74
     75
     76pc278.emulab.net's geni_status is: ready
     77Login using:
     78        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc278.emulab.net &
     79
     80pc336.emulab.net's geni_status is: ready
     81Login using:
     82        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc336.emulab.net &
     83
     84pc347.emulab.net's geni_status is: ready
     85Login using:
     86        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc347.emulab.net &
     87
     88pc358.emulab.net's geni_status is: ready
     89Login using:
     90        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc358.emulab.net &
     91
     92pc348.emulab.net's geni_status is: ready
     93Login using:
     94        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc348.emulab.net &
     95
     96pc353.emulab.net's geni_status is: ready
     97Login using:
     98        xterm -e ssh -i /Users/mberman/.ssh/id_rsa mberman@pc353.emulab.net &
     99
     100================================================================================
     101}}}
     102
     103The six lines that start with '''xterm -e ssh ...''' contain the information you need to connect to your nodes. There are six nodes in this experiment, and you will want to open a window for each. Copy and paste the commands, '''adding the -A switch to each ssh command''' as shown below. You will get six new terminal windows, one on each node of your experiment. (You should say yes if you are prompted to accept the RSA key of the remote host.)
     104
     105{{{
     106xterm -e ssh -A -i /Users/mberman/.ssh/id_rsa mberman@pc336.emulab.net &
     107}}}
     108
     109The shell prompt in each of your new terminal windows will tell you the host name. The four routers in your experiment are named '''top''', '''left''', '''right''', and '''bottom'''. The two end hosts are named '''hosta''' and '''hostb'''. You may want to rearrange the windows on your screen, putting the router hosts in the locations indicated, with the end hosts off to the side.
     110
     111'''Tip''': If you want to login to your hosts from a different host than the one you are running Omni on, take a look [wiki:HowTo/LoginToNodes here] for some tips about how to do this.
     112
     113== Configure your routers ==
     114
     115''In each of your four router hosts,'' run the extractClickConfig script as shown below.
     116
     117{{{
     118[mberman@top ~]$ /local/click-example/extractClickConfig.py
     119}}}
     120
     121You'll get output something like this:
     122{{{
     123Your host information:
     124        hostA: hostA.StupidSliceName.emulab-net.emulab.net pc347.emulab.net
     125        top: top.StupidSliceName.emulab-net.emulab.net pc336.emulab.net
     126        left: left.StupidSliceName.emulab-net.emulab.net pc358.emulab.net
     127        right: right.StupidSliceName.emulab-net.emulab.net pc278.emulab.net
     128        bottom: bottom.StupidSliceName.emulab-net.emulab.net pc348.emulab.net
     129        hostB: hostB.StupidSliceName.emulab-net.emulab.net pc353.emulab.net
     130Done.
     131}}}
     132
     133(If you are prompted for a password, check to make sure that you provided the -A switch in your xterm command above.)
     134
     135The extractClickConfig script produces router configurations for your experiment. It also creates a diagram of your experiment. Copy it back to the '''click-example''' directory on your local host so you can view it. There's a copy on each of your router hosts. I've chosen pc336.emulab.net, which is the top router, but it doesn't matter. You'll need to find the host name in the script output above, and then run this command ''on your local machine''.
     136
     137{{{
     138scp pc336.emulab.net:myslice.png .
     139}}}
     140
     141Once you have '''myslice.png''' on your local machine, open it in a browser or other viewer program. Your slice will look something like the one below (see [attachment:myslice.png]). The overall configuration should be the same, with two end hosts, named hostA and hostB, and four routers (top, left, right, bottom) in a diamond configuration. The host names, interface names, and MAC addresses will be different, depending on the actual resources assigned to your slice.
     142
     143[[Image(myslice.png, 25%)]]
     144
     145The four routers interconnected by solid lines are your "core network," which will run a non-standard, non-IP protocol. The dashed lines out to the end hosts carry standard IP traffic.
     146
     147== Turn off internet protocol ==
     148
     149At this point, your network is still running IP. You can check by running a ping. In your '''hosta''' terminal window, run this command.
     150
     151{{{
     152ping -c 3 hostb
     153}}}
     154
     155The command should succeed, with output like this:
     156
     157{{{
     158PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
     15964 bytes from hostB-link-B (10.10.6.2): icmp_seq=1 ttl=61 time=1.38 ms
     16064 bytes from hostB-link-B (10.10.6.2): icmp_seq=2 ttl=61 time=1.19 ms
     16164 bytes from hostB-link-B (10.10.6.2): icmp_seq=3 ttl=61 time=1.53 ms
     162
     163--- hostB-link-B ping statistics ---
     1643 packets transmitted, 3 received, 0% packet loss, time 2004ms
     165rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms
     166}}}
     167
     168Since our experiment doesn't want IP, let's turn it off. ''On each of your four router hosts,'' run this command:
     169
     170{{{
     171sh ./stopIP.sh
     172}}}
     173
     174You'll get output like this (the interface names may be different):
     175
     176{{{
     177Disabling IP on interface eth2
     178Disabling IP on interface eth4
     179}}}
     180
     181To verify that IP is really off, try another ping. On '''hosta''':
     182{{{
     183ping -c 3 hostb
     184}}}
     185
     186The command should take twelve seconds to time out, then fail with output like this:
     187
     188{{{
     189PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
     190
     191--- hostB-link-B ping statistics ---
     1923 packets transmitted, 0 received, 100% packet loss, time 11999ms
     193}}}
     194
     195== Start your routers ==
     196
     197The extractor script produces a click configuration file for each of your routers.  ''On each of your four router hosts,'' run this command:
     198
     199{{{
     200sh ./startClick.sh
     201}}}
     202
     203You'll get output like this. (Don't worry about the warning messages, Click is just reminding you that you have no IP addresses in your core network.) The shell prompt won't come back, but you'll see the debugging output of the Click router in the terminal window.
     204
     205{{{
     206Stopping any running Click routers
     207Starting Click router
     208top.click:34: While initializing ‘FromDevice@18 :: FromDevice’:
     209  warning: eth2: no IPv4 address assigned
     210top.click:35: While initializing ‘FromDevice@21 :: FromDevice’:
     211  warning: eth4: no IPv4 address assigned
     212}}}
     213
     214Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.
     215
     216== Send some traffic ==
     217
     218Now you'll use your two edge hosts, '''hostA''' and '''hostB''' to send traffic along your network.  Since these end hosts are not running your modified protocol, they'll rely on the '''top''' and '''bottom''' routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.
     219
     220In your terminal window on '''hostB''', instruct '''nc''' to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
     221{{{
     222[mberman@hostb ~]$ nc -ul 24565
     223}}}
     224
     225Then connect to it from your terminal window on '''hostA''':
     226{{{
     227[mberman@hosta ~]$ nc -u hostb 24565
     228}}}
     229
     230You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.
     231
     232That's it!  Now, let's look inside to see what's going on.
     233
     234== Looking under the hood ==
     235
     236Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.
     237
     238Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)
     239
     240=== Packet transformation ===
     241
     242The more interesting configuration appears here, in the '''top.click''' configuration file.
     243
     244{{{
     245// This portion accepts IP packets,
     246// reformats them, and routes them
     247// to an internal router.
     248route :: Classifier(27/01%01,-);
     249
     250modify :: Unstrip(2) ->
     251    StoreData(0, "AliceWasHere3546") ->
     252    route;
     253
     254FromDevice(eth3, PROMISC true) ->
     255    Classifier(12/0800) ->
     256    modify;
     257
     258route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
     259    SimpleQueue ->
     260    Print(outL) ->
     261    ToDevice(eth2);
     262
     263route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
     264    SimpleQueue ->
     265    Print(outR) ->
     266    ToDevice(eth4);
     267
     268// This portion accepts non-IP packets
     269// with an ether type of 0x7744
     270// from an internal router, restores
     271// them to IP format, and forwards.
     272restore :: SimpleQueue ->
     273    Strip(30) ->
     274    EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->
     275    ToDevice(eth3);
     276
     277FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
     278FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;
     279}}}
     280
     281As indicated in the comments, the top portion of the configuration listens ('''!FromDevice''') for IP packets arriving on the interface connected to '''hostA''' (that's '''eth3''' in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the '''Unstrip''' operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti ('''!StoreData'''). The '''route''' operation then routes the packet via either the '''left''' or '''right''' router toward '''hostB'''. In either case, it wraps the packet in a fresh Ethernet header ('''!EtherEncap''') with a distinctive ether type code (0x7744), logs the new packet on its way out ('''Print''') and sends it out on the correct interface ('''!ToDevice''').
     282
     283The bottom portion of the configuration is intended for packets coming out of the core network to '''hostA'''.  It accepts packets from either the '''left''' or '''right''' router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to '''hostA'''.
     284
     285The configuration for the '''bottom''' router is exactly symmetric, routing packets between '''hostB''' and the core network, but using different graffiti.
     286
     287=== Simple Forwarding ===
     288
     289The '''left''' router configuration is much simpler.  Here's the '''left.click''' file:
     290
     291{{{
     292// Copy packets from top to bottom.
     293FromDevice(eth2) ->
     294    StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
     295    StoreEtherAddress(00:04:23:b7:18:fb, src) ->
     296    SimpleQueue ->
     297    Print(top) ->
     298    ToDevice(eth3);
     299// Copy packets from bottom to top.
     300FromDevice(eth3) ->
     301    StoreEtherAddress(00:04:23:b7:14:76, dst) ->
     302    StoreEtherAddress(00:04:23:b7:18:fa, src) ->
     303    SimpleQueue ->
     304    Print(bottom) ->
     305    ToDevice(eth2);
     306}}}
     307
     308This configuration just blindly forwards packets. It picks up any packet from the '''top''' router, updates the Ethernet header, and passes it along to the '''bottom''' router. The same applies in the reverse direction.  Again, the configuration for the '''right''' router is exactly analogous.
     309
     310=== Monitoring your core network ===
     311
     312Let's watch how the packets travel through the network. The terminal windows on your four router hosts show the log output from the '''Print''' statements shown above, one line per packet.
     313
     314Type <enter> a few times in each router window to create a space below the existing log entries, so you can identify new log messages as they appear.
     315
     316Now go to your window for hostA, where your '''nc''' command is still running. Type a message into this window. You should see a log message in three of your four router windows. In this example, you might see:
     317
     318In the '''top''' router log:
     319{{{
     320outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
     321}}}
     322
     323This log entry says that the '''top''' router received a packet from '''hostA''', modified it, and sent it out to the '''right''' router. If the entry started with '''outL''', that would indicate that it sent the packet out to the '''left''' router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of '''eth4''' on '''right'''. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or '''eth4''' on '''top'''. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "!AliceWasHe" in ASCII.
     324
     325Since this packet was routed to the right, there's an entry in the '''right''' router log. This entry indicates that a packet was received from '''top'''. The logged contents show the packet sent to '''bottom''', with rewritten MAC addresses, corresponding to '''eth5''' on '''bottom''' and '''eth5''' on '''right'''.
     326
     327{{{
     328top:   76 | 000423b7 42590004 23b7192f 7744416c 69636557 61734865
     329}}}
     330
     331Finally, here's the entry on '''bottom''':
     332{{{
     333inR:   76 | 000423b7 42590004 23b7192f 7744416c 69636557 61734865
     334}}}
     335
     336It shows the same packet received from '''right'''. After logging, the packet is rewritten into IP and sent to '''hostB'''.
     337
     338Try typing a few different lines to hostA. You should see some packets routed to the left and some to the right. The routing decision is based on the '''route :: Classifier(27/01%01,-);''' entry in the '''top''' router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.
     339
     340Try typing a few lines to hostB. You should see similar behavior, but starting from '''bottom''' and working up. You'll also see the different value inserted in the new header field.
     341
     342== Clean up ==
     343
     344When you're done, please release your resources so they'll be available to others.
     345
     346{{{
     347omni.py deletesliver -a http://www.emulab.net/protogeni/xmlrpc/am StupidSliceName
     348}}}
     349
     350== Moving forward with your experiment ==
     351
     352This sample experiment is just a very simple demonstration of how to run a Click-based routing configuration using ProtoGENI. For a more meaningful experiment, you may want to try some of the variations described below. We'd love to hear what you're doing with Click and GENI, and we're here to help. Please let us know at [mailto:help@geni.net].
     353
     354=== Improved routing ===
     355
     356Instead of writing "!AliceWasHere" in your packets, perhaps include some real routing instructions. Modify the Click configurations to route packets accordingly.
     357
     358=== Richer topology ===
     359
     360Incorporate additional hosts into a core network topology that's more interesting than a simple diamond.
     361
     362=== Improved performance with kernel-level Click ===
     363
     364The Click router supports kernel-level operation. The principles are the same, but the setup is a bit more involved. To use kernel-level Click, you will probably want first to develop and debug your setup at the user level. Drop us a line at [mailto:help@geni.net] for help setting up Click in kernel mode.
     365
     366=== Wide-area routing ===
     367
     368Modify your rspec to include some ProtoGENI hosts on the Internet2 backbone or at multiple ProtoGENI sites. Additional information available at http://protogeni.net or [mailto:help@geni.net].