wiki:GEC15Agenda/AdvancedGENITopoOmni/Instructions/L3DeflectExperiment

Context Navigation

Version 3 (modified by nriga@bbn.com, 12 years ago) (diff)
--

Example Experiment - Layer 3 Deflect

Example Experiment - Layer 3 Deflect

In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use:

1 ProtoGENI host to run our OpenFlow controller
3 myPlc hosts, 1 will be the client and 2 will be the servers
OpenFlow resources that will connect the three hosts

Please note that you can't just cut and paste all of the commands. There are additional instructions in the text.

1. Add another user to your experiment

Omni gives you the capability of giving access to other users on your compute resources. Depending on which AM you are using to get resources from, this is done in a different way. Ask the team next to you about their username and do the following:

While in a terminal, download their public key under ~/.ssh/ :

  cd ~/.ssh
  wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub

Follow the instructions these instructions? and add another user for ProtoGENI AMs

1. Create your experiment

In this step, we are going to setup the experiment. In this tutorial we assume that you are sufficiently comfortable with omni to verify that a listresources command works and to know when your slice is ready using sliverstatus.

Create a slice, use the slicename given to you in the paper slip:
```
omni.py createslice <slicename>
```
Create all the slivers using the rspecs from the URL given in your paper slip:
```
omni.py createsliver -a <AM_nickname> <slicename> <rspec_url>
```

Check the status of your sliver

omni.py sliverstatus -a <AM_nickname> <slicename>

2. Install scripts

While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts. In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts.

Download your rspec you used in the pg-utah AM.
```
 cd /tmp
 wget <rspec_url>
```
Open your rspec and look for the install tag and copy the value of the URL attribute.

Download and untar the software

cd /tmp
wget <software_url>
tar xvfz <software_name>

Look in your rspec and locate the execute tag. Note what script is being executed at boot time.
Locate the script and open it. Can you identify the different parts?

3. Configure your hosts

Once our slivers iare ready we will go ahead and configure our myPlc hosts. For this tutorial we are both going to login to nodes and use remote execution to send commands to our nodes, so make sure that both work.

3a. Login and remote execution

Run the readyToLogin.py script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want.

readyToLogin.py -a <AM_nickname> <slicename> > login.out 2>&1

You'll get a big chunk of information, but you're interested in the ssh configuration info information near the end.

... <lots of output> ...
================================================================================
SSH CONFIGURATION INFO for User inki
================================================================================
 
Host ganel.gpolab.bbn.com
  Port 22
  HostName ganel.gpolab.bbn.com
  User pgenigpolabbbncom_testdeflect 
  IdentityFile /home/nriga/.ssh/geni_key 

Host sardis.gpolab.bbn.com
  Port 22
  HostName sardis.gpolab.bbn.com
  User pgenigpolabbbncom_testdeflect 
  IdentityFile /home/nriga/.ssh/geni_key 


...<more output>...

Copy all the above information and paste it into your .ssh/config file, and do the same for all the AMs, then you can very easily login into your nodes, just by using the name that is after the Host attribute. Your ~/.ssh/config file should look like

IdentityFile /home/geni/.ssh/geni_key

Host ganel.gpolab.bbn.com
  Port 22
  HostName ganel.gpolab.bbn.com
  User pgenigpolabbbncom_testdeflect 
  IdentityFile /home/nriga/.ssh/geni_key 
 
Host sardis.gpolab.bbn.com
  Port 22
  HostName sardis.gpolab.bbn.com
  User pgenigpolabbbncom_testdeflect 
  IdentityFile /home/nriga/.ssh/geni_key 

Host planetlab5.clemson.edu
  Port 22
  HostName planetlab5.clemson.edu
  User pgenigpolabbbncom_testdeflect 
  IdentityFile /home/nriga/.ssh/geni_key 

Host ofctrl
  Port 22
  HostName pc104.emulab.net
  User inki 
  IdentityFile /home/nriga/.ssh/geni_key

Note down the HostName of your host at pg-utah. In the above example this would be pc104.emulab.net.

Test login

For each one of the three myPlc hosts, open a new terminal and login to each one of them. Substitute <pl_hostname> with the hostnames of pl nodes you were given on the paper slip.

  ssh <pl_hostname>

Test remote execution

You can execute commands in a remote host using ssh. To do this just follow your ssh command with the command you want to execute in quotes. We will use one of the myPlc nodes for this, just choose one.

In your local terminal type :

ssh -A sardis.gpolab.bbn.com "ls -a"

This will list all the files under the home directory on host top. The output should look like:

geni@geni-VirtualBox:~$ ssh -A planetlab5.clemson.edu "ls -a"
.
..
.bash_history
.bash_logout
.bash_profile
.bashrc

If you get something similar you are all set for controlling your nodes from your computer.

3b. Configure your hosts

On each of the terminals that you have logged in to a myPlc node type:
```
sudo yum install -y nc
```

4. Create your OpenFlow sliver

Now that we have our myPlc hosts and our PG host is being configured, it is time to reserve our OpenFlow sliver.

Add the OpenFlow AM nickname in your omni_config file
1. Open file ~/.gcf/omni_config
2. find the [aggregate_nicknames] section in the file and move to th OpenFlow MAs
3. Add this line :
```
of-tut=,https://aquarion.gpolab.bbn.com:3626/foam/gapi/1
```
Download your OpenFlow rspec, use the<ofrspec_url> on the paper slip
```
wget <ofrspec_url>
```
i Edit the OpenFlow rspec. There are only two things you will need to edit :
1. <USERNAME> : use the information on the paper slip
2. <HOSTNAME> : use the hostname of you ProtoGENI host that you note down before (e.g. pc104.emulab.net)
Create your sliver using the filename of your rspec:
```
  omni.py createsliver -a of-tut
```

4. Turn off internet protocol

At this point, your network is still running IP. You can check by running a ping. In your hosta terminal window, run this command.

ping -c 3 hostb

The command should succeed, with output like this:

PING hostB-link-B (10.10.6.2) 56(84) bytes of data.
64 bytes from hostB-link-B (10.10.6.2): icmp_seq=1 ttl=61 time=1.38 ms
64 bytes from hostB-link-B (10.10.6.2): icmp_seq=2 ttl=61 time=1.19 ms
64 bytes from hostB-link-B (10.10.6.2): icmp_seq=3 ttl=61 time=1.53 ms

--- hostB-link-B ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.193/1.370/1.531/0.138 ms

Since our experiment doesn't want IP, let's turn it off :

On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
```
ssh -A <router_nickname> "sh ./stopIP.sh"
```

You'll get output like this (the interface names may be different):
Disabling IP on interface mv10.9
Disabling IP on interface mv10.10

Verify that IP is really off, try another ping. On hosta:

ping -c 3 hostb

The command should take twelve seconds to time out, then fail with output like this:

PING hostB-link-B (10.10.6.2) 56(84) bytes of data.

--- hostB-link-B ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 11999ms

5. Start your routers

The extractor script produces a click configuration file for each of your routers.

On a local terminal run the following command four time, each time substituting the <router_nickname> with one of the top, bottom, left, right:
```
ssh -A <router_nickname> "sh ./startClick.sh"
```
You'll get output like this. (Don't worry about the warning messages, Click is just reminding you that you have no IP addresses in your core network.) The output of the click router is redirected to /tmp/click.out on each host.

Stopping any running Click routers
Starting Click router
top.click:34: While initializing ‘FromDevice@18 :: FromDevice’:
  warning: eth2: no IPv4 address assigned
top.click:35: While initializing ‘FromDevice@21 :: FromDevice’:
  warning: eth4: no IPv4 address assigned

Congratulations! You are now running a non-IP core network on your four routers, along with a (primitive) non-IP multipath routing algorithm. You're ready to experiment with this configuration.

6. Send some traffic

Now you'll use your two edge hosts, hostA and hostB to send traffic along your network. Since these end hosts are not running your modified protocol, they'll rely on the top and bottom routers to transform their IP packets into your modified protocol on entry to the core network and back into IP packets on exit.

In your terminal window on hostB, instruct nc to listen for a UDP connection on port 24565 (or some other port that catches your fancy).
```
[mberman@hostb ~]$ nc -ul 24565
```
Connect to it from your terminal window on hostA:
```
[mberman@hosta ~]$ nc -u hostb 24565
```

You've established a simple text chat connection. Enter a line of text in either window, and it should appear in the other. Of course to do this, the text is travelling through your core network, using your non-standard protocol and routing. So type a message into each window, and make sure it appears in the other.

That's it! Now, let's look inside to see what's going on.

7. Looking under the hood

Please note: the interface names and MAC addresses below are for the sample configuration shown in the figure above. You will want to refer to your network diagram to get the correct interfaces and addresses for your configuration.

Let's take a look at what's happening in the four routers in your configuration. There are two basic router configurations. (You can find all of these files on any of your router hosts.)

7b. Packet transformation

The more interesting configuration appears here, in the top.click configuration file. In a local terminal type:

 ssh -A top "cat top.click"

The output will look like :

// This portion accepts IP packets,
// reformats them, and routes them
// to an internal router.
route :: Classifier(27/01%01,-);

modify :: Unstrip(2) ->
    StoreData(0, "AliceWasHere3546") ->
    route;

FromDevice(eth3, PROMISC true) -> 
    Classifier(12/0800) ->
    modify;

route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) ->
    SimpleQueue ->
    Print(outL) ->
    ToDevice(eth2);

route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) ->
    SimpleQueue ->
    Print(outR) ->
    ToDevice(eth4);

// This portion accepts non-IP packets
// with an ether type of 0x7744
// from an internal router, restores
// them to IP format, and forwards.
restore :: SimpleQueue ->
    Strip(30) ->
    EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) ->
    ToDevice(eth3);

FromDevice(eth2) -> Classifier(12/7744) -> Print(inL) -> restore;
FromDevice(eth4) ->  Classifier(12/7744) -> Print(inR) -> restore;

As indicated in the comments, the top portion of the configuration listens (FromDevice) for IP packets arriving on the interface connected to hostA (that's eth3 in this example). It then creates a new 16-byte field at the head of the packet (two bytes added by the Unstrip operation, plus the existing 14-byte Ethernet header. It fills that field with what could be important routing instructions, but in this case is just graffiti (StoreData). The route operation then routes the packet via either the left or right router toward hostB. In either case, it wraps the packet in a fresh Ethernet header (EtherEncap) with a distinctive ether type code (0x7744), logs the new packet on its way out (Print) and sends it out on the correct interface (ToDevice).

The bottom portion of the configuration is intended for packets coming out of the core network to hostA. It accepts packets from either the left or right router, logs them, strips off thirty bytes (Ethernet header plus your 16-byte new header field), puts on a fresh Ethernet header, and sends them along to hostA.

The configuration for the bottom router is exactly symmetric, routing packets between hostB and the core network, but using different graffiti.

7b. Simple Forwarding

The left router configuration is much simpler. In a local terminal type:

 ssh -A left "cat left.click"

The output will look like :

// Copy packets from top to bottom.
FromDevice(eth2) ->
    StoreEtherAddress(00:04:23:b7:42:b6, dst) ->
    StoreEtherAddress(00:04:23:b7:18:fb, src) ->
    SimpleQueue ->
    Print(top) ->
    ToDevice(eth3);
// Copy packets from bottom to top.
FromDevice(eth3) ->
    StoreEtherAddress(00:04:23:b7:14:76, dst) ->
    StoreEtherAddress(00:04:23:b7:18:fa, src) ->
    SimpleQueue ->
    Print(bottom) ->
    ToDevice(eth2);

This configuration just blindly forwards packets. It picks up any packet from the top router, updates the Ethernet header, and passes it along to the bottom router. The same applies in the reverse direction. Again, the configuration for the right router is exactly analogous.

7c. Monitoring your core network

Let's watch how the packets travel through the network.

In a local terminal type:
```
ssh -A top "tail -f /tmp/click.log"
```
Go to your window for hostA, where your nc command is still running. Type a message into this window. You should see a log message in three of your four router windows. In this example, you might see:
In the local terminal you will see:
```
outR:   76 | 000423b7 192e0004 23b71ce0 7744416c 69636557 61734865
```
This log entry says that the top router received a packet from hostA, modified it, and sent it out to the right router. If the entry started with outL, that would indicate that it sent the packet out to the left router. Let's look a bit at the start of the packet (the first 24 bytes are logged). It starts with an Ethernet header. The first six bytes are the MAC address of the destination interface, that's 00:04:23:B7:19:2E, the MAC address of eth4 on right. The next six bytes are the MAC address of the source interface, 00:04:23:B7:1C:E0, or eth4 on top. Next comes your ether type, 0x7744. The remaining bytes, "416c 69636557 61734865" are the start of the first field in your new protocol, "AliceWasHe" in ASCII.
Try typing a few different lines to hostA. You should see some packets routed to the left and some to the right. The routing decision is based on the route :: Classifier(27/01%01,-); entry in the top router configuration. Here, the router is looking at the low-order bit of the checksum on the initial IP packet (now at byte position 27 with the addition of the new sixteen byte field at the start of the header). Packets with odd checksums go to the left; those with even checksums go right.

9. Clean up

When you're done, please release your resources so they'll be available to others.

omni.py deletesliver -a pg-utah <slicename>

Download in other formats:

Plain Text