Click Example

Hello GENI index Hello GENI index Hello GENI index

1. Configure and Initialize Services: Configure the Click Routers

Once our sliver is ready we will go ahead and configure our click routers.

1a. Login

  1. First if you have an existing ssh config, save it now:
    mv ~/.ssh/config ~/.ssh/config.keep
  2. Run the readyToLogin script to get information about logging in to nodes. The script has a lot of output so lets put that in a file (using -o) so that we can easily search for the information we want.
    readyToLogin <slicename> --useSliceAggregates -o 

    This will save all the information to different files.

  3. We want to use the ssh configuration file that the script produced:
    mv ./sshconfig.txt ~/.ssh/config
  4. Make sure the ~/.ssh/config file only includes the section for your user. Edit it if necessary.
    Note This custom .ssh/config will allow you to login to each host in your topology using the client_id specified in the RSpec (e.g. server) instead of the fully qualified domain name (e.g It will also allow you to login without specifying your ssh key via the command line or via ssh-add.
  5. Let's login to our two hosts with client_ids of hostA and hostB
  6. On your local machine, open two new terminals
  7. In one terminal type
    ssh -A hostA
    and in the other
    ssh -A hostB
    Tip The -A option allows you to forward the authentication agent connection so you can log into other hosts in your topology from hostA and hostB.

1b. Configure your routers

We are going to configure our routers using ssh.

  1. On a local terminal run the following command four times, each time substituting the <router_name> with one of top, bottom, left, right:
    ssh -A <router_name> "/local/click-example/"

    You'll get output something like this:

    Your host information:
            hostA: hostA.<slicename>
            top: top.<slicename>
            left: left.<slicename>
            right: right.<slicename>
            bottom: bottom.<slicename>
            hostB: hostB.<slicename>

    (If you are prompted for a password, check to make sure that you provided the -A switch in your ssh command above.)

  1. The extractClickConfig script produces router configurations for your experiment. It also creates a diagram of your experiment. Get a copy locally from one of the routers, by typing in a local terminal:
    scp top:myslice.png ./
  2. Open the diagram in your browser or your favorite image viewing program.

Your slice will look something like the one below. The overall configuration should be the same, with two end hosts, named hostA and hostB, and four routers (top, left, right, bottom) in a diamond configuration. The host names, interface names, and MAC addresses will be different, depending on the actual resources assigned to your slice.

The four routers interconnected by solid lines are your "core network," which will run a non-standard, non-IP protocol. The dashed lines out to the end hosts carry standard IP traffic.

1c. Turn off internet protocol

Note In the code snippets below, the color of the background corresponds to the terminal you should enter the text into. The yellow background is your local terminal and purple and red indicate `hostA` and `hostB` respectively.
  1. At this point, your network is still running IP. You can check by running a ping. In your hostA terminal window, run this command:
    [mberman@hostA ~]$ ping -c 3 hostB

    The command should succeed, with output like this:

    PING hostB-link-B ( 56(84) bytes of data.
    64 bytes from hostB-link-B ( icmp_seq=1 ttl=61 time=1.38 ms
    64 bytes from hostB-link-B ( icmp_seq=2 ttl=61 time=1.19 ms
    64 bytes from hostB-link-B ( icmp_seq=3 ttl=61 time=1.53 ms
    --- hostB-link-B ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2004ms
    rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms
  2. Since our experiment doesn't use IP, let's turn it off.
  3. On a local terminal run the following command four times, each time substituting the <router_name> with one of top, bottom, left, right:
    ssh -A <router_name> "sh ./"

    You'll get output like this (the interface names may be different):

    Disabling IP on interface mv10.9
    Disabling IP on interface mv10.10
  4. Verify that IP is really off, try another ping. On hostA:
    [mberman@hostA ~]$ ping -c 3 hostb

    The command should take twelve seconds to time out, then fail with output like this:

    PING hostB-link-B ( 56(84) bytes of data.
    --- hostB-link-B ping statistics ---
    3 packets transmitted, 0 received, 100% packet loss, time 11999ms

2. Execute Experiment: Use custom routing to forward traffic over multi-path topology

2a. Start Click Routers produces a click configuration file for each of your routers.

  1. On a local terminal run the following command four times, each time substituting the <router_name> with one of top, bottom, left, right:
    ssh -A <router_name> "sh ./"

    You'll get output like this.

    Stopping any running Click routers
    Starting Click router While initializing ‘FromDevice@18 :: FromDevice’:
      warning: eth2: no IPv4 address assigned While initializing ‘FromDevice@21 :: FromDevice’:
      warning: eth4: no IPv4 address assigned

    Don't worry if you get warning messages, Click is just reminding you that you have no IP addresses in your core network.

    The output of the click router is redirected to /tmp/click.out on each host.

Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.

2b. Send some traffic

Now you'll use your two edge hosts, hostA and hostB to send traffic along your network. Since these end hosts are not running your modified protocol, they'll rely on the top and bottom routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.

  1. In your terminal window on hostB, instruct nc to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
    [mberman@hostB ~]$ nc -ul 24565
  2. Connect to it from your terminal window on hostA:
    [mberman@hostA ~]$ nc -u hostB 24565

You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.

That's it! Now, let's look inside to see what's going on.

2c. Looking under the hood

Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.

Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)

Packet transformation

  1. The more interesting configuration appears in the configuration file.

In a local terminal type:

ssh top "cat"

The output will look like :

// This portion accepts IP packets,
// reformats them, and routes them
// to an internal router.
route :: Classifier(27/01%01,-);

modify :: Unstrip(2) ->
    StoreData(0, "AliceWasHere3546") ->

FromDevice(eth3, PROMISC true) -> 
    Classifier(12/0800) ->

route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
    SimpleQueue ->
    Print(outL) ->

route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
    SimpleQueue ->
    Print(outR) ->

// This portion accepts non-IP packets
// with an ether type of 0x7744
// from an internal router, restores
// them to IP format, and forwards.
restore :: SimpleQueue ->
    Strip(30) ->
    EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->

FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;

Look in the Click documentation for more information about the elements in the above script.

As indicated in the comments, the first portion of the configuration listens (FromDevice) for IP packets arriving on the interface connected to hostA (that's eth3 in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the Unstrip operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti (StoreData). The route operation then routes the packet via either the left or right router toward hostB. In either case, it wraps the packet in a fresh Ethernet header (EtherEncap) with a distinctive ether type code (0x7744), logs the new packet on its way out (Print) and sends it out on the correct interface (ToDevice).

The last portion of the configuration is intended for packets coming out of the core network to hostA. It accepts packets from either the left or right router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to hostA. The configuration for the bottom router is exactly symmetric, routing packets between hostB and the core network, but using different graffiti.

Simple Forwarding

The left router configuration is much simpler.

In a local terminal type:

ssh left "cat"

The output will look like :

// Copy packets from top to bottom.
FromDevice(eth2) ->
    StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
    StoreEtherAddress(00:04:23:b7:18:fb, src) ->
    SimpleQueue ->
    Print(top) ->
// Copy packets from bottom to top.
FromDevice(eth3) ->
    StoreEtherAddress(00:04:23:b7:14:76, dst) ->
    StoreEtherAddress(00:04:23:b7:18:fa, src) ->
    SimpleQueue ->
    Print(bottom) ->

This configuration just blindly forwards packets. It picks up any packet from the top router, updates the Ethernet header, and passes it along to the bottom router. The same applies in the reverse direction. Again, the configuration for the right router is exactly analogous.

Monitoring your core network

Let's watch how the packets travel through the network.

  1. In a local terminal type:
    ssh top "tail -f /tmp/click.log"
  1. Go to your window for hostA, where your nc command is still running. Type a message into this window. Log messages should be recorded on three of your four routers.
    [mberman@hosta ~]$ nc -u hostb 24565
    your message here
  2. In the local terminal you will see:
    outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
    This log entry says that the top router received a packet from hostA, modified it, and sent it out to the right router. If the entry started with outL, that would indicate that it sent the packet out to the left router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of eth4 on right. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or eth4 on top. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "AliceWasHe" in ASCII.
  1. Try typing a few different lines to hostA.
    [mberman@hostb ~]$ nc -ul 24565
    your message here
    four score
    seven years
  2. In the local terminal you should see some packets routed to the left and some to the right.

The routing decision is based on the route :: Classifier(27/01%01,-); entry in the top router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.


  • Run tcpdump on an interface in the core network.
    • Which packets say AliceWasHere? Which say BobWasHere? Why?

Next: Teardown

Last modified 10 years ago Last modified on 07/21/14 15:09:59