Changes between Version 3 and Version 4 of GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment

10/21/12 21:46:54 (9 years ago)



  • GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment

    v3 v4  
    170170  }}}
    172 == 4. Turn off internet protocol ==
    174 At this point, your network is still running IP. You can check by running a ping. In your '''hosta''' terminal window, run this command.
     172=== 4b. Modify the configuration file of the OpenFlow controller ===
     173By now your ProtoGENI host should have been configured. Let's login to the node and modify the configuration file of our OpenFlow controller.
     174  i. Open a new terminal and login to the ProtoGENI host:
     175  {{{
     176ssh ofctrl
     178  i. Run :
     179  {{{
     180  ps -ef | grep nox_core
     182  The output should look like:
     183  {{{
     184root      6498     1  0 18:33 ?        00:00:01 /usr/local/bin/noxgpo/src/.libs/lt-nox_core -d -i ptcp:6633 switch
     185inki      6562  6524  0 18:45 pts/0    00:00:00 grep nox_core
     187  If the first line is missing then the setup has not finished yet. You can move to the next step, but make sure the setup has finished before moving to step 5.
     188  i. Copy the configuration file to your home directory:
     189  {{{
     190  cp /local/l3deflect.conf ~/
     191  }}}
     192  i. Modify the configuration file to match the information in your slip. The should look like:
     193  {{{
     194# Configuration file for the l3deflect controller
     197orig_mac = 00:1B:21:4B:3E:E9
     198orig_ip =
     199deflect_mac = 00:1B:21:4B:3F:AD
     200deflect_ip =
     202== 5. Run the experiment ==
     203Now we have reserved and configured all hosts. We are ready to run our experiment.
     204  i. In the terminal of the remote and of the local server run the following command. Substitute <NC_PORT> with the information in your slip:
     205  {{{
     206  nc <nc_port>
     208   i. In the terminal of the client run this command, using your paper slip to fill in the information :
     209  {{{
     210  nc <IP_REMOTE_SERVER> <NC_PORT>
     212  i. You now have a chat application between the remote server and the client. Try typing something in the client and see it pop up at the server side, and vice versa. The forwarding of your packets work because there is a default learning switch controller running on your ProtoGENI host.
     213  i. Let's stop the default learning switch controller and start our deflection controller. In a local terminal type the following two commands:
     214  {{{
     215   ssh ofctrl "/usr/local/bin/"
     216   ssh ofctrl "/usr/local/bin/ l3deflect"
     217  }}}
     218  The output should look like that:
     219  {{{
     220nriga@pella:~$ ssh ofctrl "/usr/local/bin/"
     221Stopping all OpenFlow controllers
     222nriga@pella:~$ ssh ofctrl "/usr/local/bin/ l3deflect"
     223Starting OpenFlow controler l3deflect
     224  }}}
     225  i. If the `nc` is still running on the client, terminate it by pressing `Ctrl-C`.
     226  i Run the `nc` at the client again:
     227  {{{
     228 nc <IP_REMOTE_SERVER> <NC_PORT>
     230  i Type something on the client and notice in which terminal it appears.
     232If you want to switch back to the switch controller you need to again kill all the controllers and start the switch controller :
    177 ping -c 3 hostb
    178 }}}
    180 The command should succeed, with output like this:
    182 {{{
    183 PING hostB-link-B ( 56(84) bytes of data.
    184 64 bytes from hostB-link-B ( icmp_seq=1 ttl=61 time=1.38 ms
    185 64 bytes from hostB-link-B ( icmp_seq=2 ttl=61 time=1.19 ms
    186 64 bytes from hostB-link-B ( icmp_seq=3 ttl=61 time=1.53 ms
    188 --- hostB-link-B ping statistics ---
    189 3 packets transmitted, 3 received, 0% packet loss, time 2004ms
    190 rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms
    191 }}}
    193 Since our experiment doesn't want IP, let's turn it off :
    194   i. On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
    195   {{{
    196   ssh -A <router_nickname> "sh ./"
    197 }}}
    199   You'll get output like this (the interface names may be different):
    200   {{{
    201 Disabling IP on interface mv10.9
    202 Disabling IP on interface mv10.10
    203 }}}
    205   i. Verify that IP is really off, try another ping. On '''hosta''':
    206   {{{
    207   ping -c 3 hostb
    208 }}}
    209   The command should take twelve seconds to time out, then fail with output like this:
    210   {{{
    211 PING hostB-link-B ( 56(84) bytes of data.
    213 --- hostB-link-B ping statistics ---
    214 3 packets transmitted, 0 received, 100% packet loss, time 11999ms
    215 }}}
    217 == 5. Start your routers ==
    219 The extractor script produces a click configuration file for each of your routers. 
    220   i. On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
    221   {{{
    222   ssh -A <router_nickname> "sh ./"
    223 }}}
    224   You'll get output like this. (Don't worry about the warning messages, Click is just reminding you that you have no IP addresses in your core network.) The output of the click router is redirected to `/tmp/click.out` on each host.
    226   {{{
    227 Stopping any running Click routers
    228 Starting Click router
    229 While initializing ‘FromDevice@18 :: FromDevice’:
    230   warning: eth2: no IPv4 address assigned
    231 While initializing ‘FromDevice@21 :: FromDevice’:
    232   warning: eth4: no IPv4 address assigned
    233 }}}
    235 Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.
    237 == 6. Send some traffic ==
    239 Now you'll use your two edge hosts, '''hostA''' and '''hostB''' to send traffic along your network.  Since these end hosts are not running your modified protocol, they'll rely on the '''top''' and '''bottom''' routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.
    240   i. In your terminal window on '''hostB''', instruct '''nc''' to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
    241   {{{
    242 [mberman@hostb ~]$ nc -ul 24565
    243 }}}
    244   i. Connect to it from your terminal window on '''hostA''':
    245   {{{
    246 [mberman@hosta ~]$ nc -u hostb 24565
    247 }}}
    249 You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.
    251 That's it!  Now, let's look inside to see what's going on.
    253 == 7. Looking under the hood ==
    255 Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.
    257 Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)
    259 === 7b. Packet transformation ===
    261   i. The more interesting configuration appears here, in the '''''' configuration file. In a local terminal type:
    262   {{{
    263  ssh -A top "cat"
    264 }}}
    265   The output will look like :
    266   {{{
    267 // This portion accepts IP packets,
    268 // reformats them, and routes them
    269 // to an internal router.
    270 route :: Classifier(27/01%01,-);
    272 modify :: Unstrip(2) ->
    273     StoreData(0, "AliceWasHere3546") ->
    274     route;
    276 FromDevice(eth3, PROMISC true) ->
    277     Classifier(12/0800) ->
    278     modify;
    280 route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
    281     SimpleQueue ->
    282     Print(outL) ->
    283     ToDevice(eth2);
    285 route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
    286     SimpleQueue ->
    287     Print(outR) ->
    288     ToDevice(eth4);
    290 // This portion accepts non-IP packets
    291 // with an ether type of 0x7744
    292 // from an internal router, restores
    293 // them to IP format, and forwards.
    294 restore :: SimpleQueue ->
    295     Strip(30) ->
    296     EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->
    297     ToDevice(eth3);
    299 FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
    300 FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;
    301 }}}
    303   As indicated in the comments, the top portion of the configuration listens ('''!FromDevice''') for IP packets arriving on the interface connected to '''hostA''' (that's '''eth3''' in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the '''Unstrip''' operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti ('''!StoreData'''). The '''route''' operation then routes the packet via either the '''left''' or '''right''' router toward '''hostB'''. In either case, it wraps the packet in a fresh Ethernet header ('''!EtherEncap''') with a distinctive ether type code (0x7744), logs the new packet on its way out ('''Print''') and sends it out on the correct interface ('''!ToDevice''').
    305   The bottom portion of the configuration is intended for packets coming out of the core network to '''hostA'''.  It accepts packets from either the '''left''' or '''right''' router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to '''hostA'''.
    307   The configuration for the '''bottom''' router is exactly symmetric, routing packets between '''hostB''' and the core network, but using different graffiti.
    309 === 7b. Simple Forwarding ===
    311 The '''left''' router configuration is much simpler.  In a local terminal type:
    312 {{{
    313  ssh -A left "cat"
    314 }}}
    315   The output will look like :
    317 {{{
    318 // Copy packets from top to bottom.
    319 FromDevice(eth2) ->
    320     StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
    321     StoreEtherAddress(00:04:23:b7:18:fb, src) ->
    322     SimpleQueue ->
    323     Print(top) ->
    324     ToDevice(eth3);
    325 // Copy packets from bottom to top.
    326 FromDevice(eth3) ->
    327     StoreEtherAddress(00:04:23:b7:14:76, dst) ->
    328     StoreEtherAddress(00:04:23:b7:18:fa, src) ->
    329     SimpleQueue ->
    330     Print(bottom) ->
    331     ToDevice(eth2);
    332 }}}
    334 This configuration just blindly forwards packets. It picks up any packet from the '''top''' router, updates the Ethernet header, and passes it along to the '''bottom''' router. The same applies in the reverse direction.  Again, the configuration for the '''right''' router is exactly analogous.
    336 === 7c. Monitoring your core network ===
    338 Let's watch how the packets travel through the network.
    339   i. In a local terminal type:
    340   {{{
    341   ssh -A top "tail -f /tmp/click.log"
    342 }}}
    343   i. Go to your window for hostA, where your '''nc''' command is still running. Type a message into this window. You should see a log message in three of your four router windows. In this example, you might see:
    344   i. In the local terminal you will see:
    345   {{{
    346 outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
    347 }}}
    348   This log entry says that the '''top''' router received a packet from '''hostA''', modified it, and sent it out to the '''right''' router. If the entry started with '''outL''', that would indicate that it sent the packet out to the '''left''' router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of '''eth4''' on '''right'''. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or '''eth4''' on '''top'''. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "!AliceWasHe" in ASCII.
    349   i. Try typing a few different lines to hostA. You should see some packets routed to the left and some to the right. The routing decision is based on the '''route :: Classifier(27/01%01,-);''' entry in the '''top''' router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.
    351 == 9. Clean up ==
     234 ssh ofctrl "/usr/local/bin/"
     235 ssh ofctrl "/usr/local/bin/ switch"
     237You can play switching between the two controllers and notice how the text appears on a different terminal.
     239== 6. Looking under the hood ==
     240If you still have a few minutes, then you can poke around the OpenFlow controller, if you don't have enough time then make sure you move to the next step
     241and cleanup your resources. You can always come back and do the tutorial again and poke around.
     243Congratulations you have successfully diverted the traffic from your client to the local server using OpenFlow!. But how did this work? Basically our openflow controller
     244instructed the switch to rewrite the IP information on the packet so that
     245  * every packet that was destined to the original server, it would be changed to be sent to the local one
     246  * also all packets from the local  server are rewritten so that the client thinks that it talks to the remote server.
     248Let's take a quick look at the controller. On the terminal that is logged in to the ProtoGENI host open file `/usr/local/src/noxgpo/src/nox/coreapps/example/`. This is the deflection module. Lets try and follow the logic now:
     249  i. Find the `install` function. This is the function that registers your module to receive OpenFlow events.
     250  i Look at which function is called when a packet is received.
     251  i. Ignore the first checks and find the function that is responsible for forwarding packets.
     252  i. Look at the logic that is overwriting the packet. Basically there is a section like this :
     253  {{{
     254        # Check if the eth_type must be deflected
     255        if ethtype == 0x800 :
     256          # and if the dst mac address is the one that should be deflected
     257          if mac_to_str(packet.dst).lower() == self.orig_mac.lower() :
     258            # replace the destination mac address
     259            actions.append([openflow.OFPAT_SET_DL_DST,
     260                                octstr_to_array(self.deflect_mac)])
     261            logger.debug("NEW DST MAC %s" % ( self.deflect_mac))
     262            dstaddr = octstr_to_array(self.deflect_mac).tostring()
     263            actions.append([openflow.OFPAT_SET_NW_DST,
     264                                ipstr_to_int(self.deflect_ip)])
     265            logger.debug("NEW DST IP %s" % ( self.deflect_ip))
     266  "Packet %s is deflected")
     268          if mac_to_str(packet.src).lower() == self.deflect_mac.lower() :
     269            # replace the destination mac address
     270            actions.append([openflow.OFPAT_SET_DL_SRC,
     271                                octstr_to_array(self.orig_mac)])
     272            logger.debug("NEW SRC MAC %s" % ( self.orig_mac))
     273            actions.append([openflow.OFPAT_SET_NW_SRC,
     274                                ipstr_to_int(self.orig_ip)])
     275            logger.debug("NEW DST IP %s" % ( self.orig_ip))
     276  }}}
     277== 7. Clean up ==
    353279When you're done, please release your resources so they'll be available to others.