| 5 | | In this example experiment, you will configure and run an OpenFlow experiment. In this experiment we are going to deflect service requests from one server to another. We are going to use: |
| 6 | | * 1 ProtoGENI host to run our OpenFlow controller |
| 7 | | * 3 myPlc hosts, 1 will be the client and 2 will be the servers |
| 8 | | * OpenFlow resources that will connect the three hosts |
| 9 | | |
| 10 | | Please note that you can't just cut and paste all of the commands. There are additional instructions in the text. |
| 11 | | |
| 12 | | == 1. Add another user to your experiment == |
| 13 | | Omni gives you the capability of giving access to other users on your compute resources. Depending on which AM you are using to get resources from, this is done in a different way. Ask the team next to you about their username and do the following: |
| 14 | | i. While in a terminal, download their public key under `~/.ssh/` : |
| 15 | | {{{ |
| 16 | | cd ~/.ssh |
| 17 | | wget http://www.gpolab.bbn.com/experiment-support/gec15/adv-omni/pub-keys/<username>_key.pub |
| 18 | | }}} |
| 19 | | |
| 20 | | i. Follow the instructions [wiki:HowTo/MulitpleUsersWithOmni these instructions] and add another user for ProtoGENI AMs |
| | 5 | In this experiment we are going to examine how the underlying network |
| | 6 | topology can affect the topology of your experiment. We are going to use the two topologies that are available in Mesoscale. |
| | 7 | The new for [wiki:TangoGENI#VLAN3715 VLAN 3715] and the one for [wiki:TangoGENI#VLAN3716 VLAN 3716]. |
| 37 | | == 2. Install scripts == |
| 38 | | While you wait for your sliver to become ready, we will see how we can automate the installation of our experiment with install scripts. |
| 39 | | In this experiment we are going to use software routers in order to write our own forwarding scheme. This means that in any experiment we are going to run we want the basic installation of the software router to always be present. The configuration might change from run to run, but the software should always be installed. The software to be installed, and the scripts to be executed at boot time, are defined in the rspecs. Follow these steps to locate your install script and identify the different parts. |
| 40 | | i. Download your rspec you used in the `pg-utah` AM. |
| | 24 | == 2. Login to your nodes == |
| | 25 | |
| | 26 | The login information to your hosts is reported back in sliverstatus. Omni comes with an example script that calls sliverstatus and figures out all the information you need for login in to your hosts. |
| | 27 | i. First of all let's clean our `.ssh/config` file in case it contains information from previous experiments. Unless if you have added some information you care about in your ssh configuration file, it is safe to remove it and recreate it. |
| 42 | | cd /tmp |
| 43 | | wget <rspec_url> |
| 44 | | }}} |
| 45 | | i. Open your rspec and look for the `install` tag and copy the value of the URL attribute. |
| 46 | | i. Download and untar the software |
| 47 | | {{{ |
| 48 | | cd /tmp |
| 49 | | wget <software_url> |
| 50 | | tar xvfz <software_name> |
| 51 | | }}} |
| 52 | | i. Look in your rspec and locate the `execute` tag. Note what script is being executed at boot time. |
| 53 | | i. Locate the script and open it. Can you identify the different parts? |
| 54 | | |
| 55 | | == 3. Configure your hosts == |
| 56 | | Once our slivers iare ready we will go ahead and configure our myPlc hosts. For this tutorial we are both going to login to nodes and use remote execution to send commands to our nodes, so make sure that both work. |
| 57 | | |
| 58 | | === 3a. Login and remote execution === |
| 59 | | i. Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want. |
| | 29 | cd |
| | 30 | rm .ssh/config |
| | 31 | touch .ssh/config |
| | 32 | }}} |
| | 33 | i. Run the `readyToLogin.py` script to get information about logging in to nodes. The script has a lot of output so lets put that in a file so that we can easily search for the information we want. Use the same AMs as you used in Step 1. |
| 125 | | ==== Test remote execution ==== |
| 126 | | You can execute commands in a remote host using `ssh`. To do this just follow your ssh command with the command you want to execute in quotes. We will use one |
| 127 | | of the myPlc nodes for this, just choose one. |
| 128 | | i. In your local terminal type : |
| 129 | | {{{ |
| 130 | | ssh -A sardis.gpolab.bbn.com "ls -a" |
| 131 | | }}} |
| 132 | | This will list all the files under the home directory on host `top`. The output should look like: |
| 133 | | {{{ |
| 134 | | geni@geni-VirtualBox:~$ ssh -A planetlab5.clemson.edu "ls -a" |
| 135 | | . |
| 136 | | .. |
| 137 | | .bash_history |
| 138 | | .bash_logout |
| 139 | | .bash_profile |
| 140 | | .bashrc |
| 141 | | }}} |
| | 83 | == 3. Run your Experiment == |
| | 84 | The Mesoscale deployment can offer different topologies for communicating between hosts. We have provisioned two different IP subnets each one using a different topology. |
| 151 | | == 4. Create your OpenFlow sliver == |
| 152 | | Now that we have our myPlc hosts and our PG host is being configured, it is time to reserve our OpenFlow sliver. |
| 153 | | i. Add the OpenFlow AM nickname in your `omni_config` file |
| 154 | | a. Open file `~/.gcf/omni_config` |
| 155 | | b. find the `[aggregate_nicknames]` section in the file and move to th `OpenFlow MAs` |
| 156 | | c. Add this line : |
| 157 | | {{{ |
| 158 | | of-tut=,https://aquarion.gpolab.bbn.com:3626/foam/gapi/1 |
| 159 | | }}} |
| 160 | | i. Download your OpenFlow rspec, use the<ofrspec_url> on the paper slip |
| 161 | | {{{ |
| 162 | | wget <ofrspec_url> |
| 163 | | }}} |
| 164 | | i Edit the OpenFlow rspec. There are only two things you will need to edit : |
| 165 | | a. <USERNAME> : use the information on the paper slip |
| 166 | | b. <HOSTNAME> : use the hostname of you ProtoGENI host that you note down before (e.g. `pc104.emulab.net`) |
| 167 | | i. Create your sliver using the filename of your rspec: |
| 168 | | {{{ |
| 169 | | omni.py createsliver -a of-tut |
| 170 | | }}} |
| | 90 | * List all the interfaces on your host. You will see that there are many interfaces of the form eth1.XXXX |
| | 91 | {{{ |
| | 92 | /sbin/ifconfig |
| | 93 | }}} |
| | 94 | Part of the output would look like : |
| | 95 | {{{ |
| | 96 | eth1.1750:42147 Link encap:Ethernet HWaddr 00:B0:D0:E1:6F:78 |
| | 97 | inet addr:10.42.147.90 Bcast:10.42.147.255 Mask:255.255.255.0 |
| | 98 | UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 |
| 172 | | === 4b. Modify the configuration file of the OpenFlow controller === |
| 173 | | By now your ProtoGENI host should have been configured. Let's login to the node and modify the configuration file of our OpenFlow controller. |
| 174 | | i. Open a new terminal and login to the ProtoGENI host: |
| 175 | | {{{ |
| 176 | | ssh ofctrl |
| 177 | | }}} |
| 178 | | i. Run : |
| 179 | | {{{ |
| 180 | | ps -ef | grep nox_core |
| 181 | | }}} |
| 182 | | The output should look like: |
| 183 | | {{{ |
| 184 | | root 6498 1 0 18:33 ? 00:00:01 /usr/local/bin/noxgpo/src/.libs/lt-nox_core -d -i ptcp:6633 switch |
| 185 | | inki 6562 6524 0 18:45 pts/0 00:00:00 grep nox_core |
| 186 | | }}} |
| 187 | | If the first line is missing then the setup has not finished yet. You can move to the next step, but make sure the setup has finished before moving to step 5. |
| 188 | | i. Copy the configuration file to your home directory: |
| 189 | | {{{ |
| 190 | | cp /local/l3deflect.conf ~/ |
| 191 | | }}} |
| 192 | | i. Modify the configuration file to match the information in your slip. The should look like: |
| 193 | | {{{ |
| 194 | | # Configuration file for the l3deflect controller |
| | 100 | eth1.1750:42148 Link encap:Ethernet HWaddr 00:B0:D0:E1:6F:78 |
| | 101 | inet addr:10.42.148.90 Bcast:10.42.148.255 Mask:255.255.255.0 |
| | 102 | UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 |
| | 103 | }}} |
| 196 | | [general] |
| 197 | | orig_mac = 00:1B:21:4B:3E:E9 |
| 198 | | orig_ip = 10.42.123.52 |
| 199 | | deflect_mac = 00:1B:21:4B:3F:AD |
| 200 | | deflect_ip = 10.42.123.51 |
| 201 | | }}} |
| 202 | | == 5. Run the experiment == |
| 203 | | Now we have reserved and configured all hosts. We are ready to run our experiment. |
| 204 | | i. In the terminal of the remote and of the local server run the following command. Substitute <NC_PORT> with the information in your slip: |
| 205 | | {{{ |
| 206 | | nc <nc_port> |
| 207 | | }}} |
| 208 | | i. In the terminal of the client run this command, using your paper slip to fill in the information : |
| 209 | | {{{ |
| 210 | | nc <IP_REMOTE_SERVER> <NC_PORT> |
| 211 | | }}} |
| 212 | | i. You now have a chat application between the remote server and the client. Try typing something in the client and see it pop up at the server side, and vice versa. The forwarding of your packets work because there is a default learning switch controller running on your ProtoGENI host. |
| 213 | | i. Let's stop the default learning switch controller and start our deflection controller. In a local terminal type the following two commands: |
| 214 | | {{{ |
| 215 | | ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh" |
| 216 | | ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect" |
| 217 | | }}} |
| 218 | | The output should look like that: |
| 219 | | {{{ |
| 220 | | nriga@pella:~$ ssh ofctrl "/usr/local/bin/stop-all-ctrls.sh" |
| 221 | | Stopping all OpenFlow controllers |
| 222 | | nriga@pella:~$ ssh ofctrl "/usr/local/bin/start-ctrl.sh l3deflect" |
| 223 | | Starting OpenFlow controler l3deflect |
| 224 | | }}} |
| 225 | | i. If the `nc` is still running on the client, terminate it by pressing `Ctrl-C`. |
| 226 | | i Run the `nc` at the client again: |
| 227 | | {{{ |
| 228 | | nc <IP_REMOTE_SERVER> <NC_PORT> |
| 229 | | }}} |
| 230 | | i Type something on the client and notice in which terminal it appears. |
| | 105 | * Figure out the last octet of the IP address of your hosts. The way these MyPLC hosts are setup, is that they have multiple subinterfaces, each configured to be part of a different IP subnet. Almost all subnets in the hosts are in the form of 10.42.Y.0/24. You will notice that for all these subnets, your host has the same last octet. In the example above all the subinterfaces of eth1, will have an IP address that will end in '''90''' (10.42.147.'''90''', 10.42.148.'''90''') |
| 243 | | Congratulations you have successfully diverted the traffic from your client to the local server using OpenFlow!. But how did this work? Basically our openflow controller |
| 244 | | instructed the switch to rewrite the IP information on the packet so that |
| 245 | | * every packet that was destined to the original server, it would be changed to be sent to the local one |
| 246 | | * also all packets from the local server are rewritten so that the client thinks that it talks to the remote server. |
| 247 | | |
| 248 | | Let's take a quick look at the controller. On the terminal that is logged in to the ProtoGENI host open file `/usr/local/src/noxgpo/src/nox/coreapps/example/l3deflect.py`. This is the deflection module. Lets try and follow the logic now: |
| 249 | | i. Find the `install` function. This is the function that registers your module to receive OpenFlow events. |
| 250 | | i Look at which function is called when a packet is received. |
| 251 | | i. Ignore the first checks and find the function that is responsible for forwarding packets. |
| 252 | | i. Look at the logic that is overwriting the packet. Basically there is a section like this : |
| 253 | | {{{ |
| 254 | | # Check if the eth_type must be deflected |
| 255 | | if ethtype == 0x800 : |
| 256 | | # and if the dst mac address is the one that should be deflected |
| 257 | | if mac_to_str(packet.dst).lower() == self.orig_mac.lower() : |
| 258 | | # replace the destination mac address |
| 259 | | actions.append([openflow.OFPAT_SET_DL_DST, |
| 260 | | octstr_to_array(self.deflect_mac)]) |
| 261 | | logger.debug("NEW DST MAC %s" % ( self.deflect_mac)) |
| 262 | | dstaddr = octstr_to_array(self.deflect_mac).tostring() |
| 263 | | actions.append([openflow.OFPAT_SET_NW_DST, |
| 264 | | ipstr_to_int(self.deflect_ip)]) |
| 265 | | logger.debug("NEW DST IP %s" % ( self.deflect_ip)) |
| 266 | | logger.info("Packet %s is deflected") |
| 267 | | |
| 268 | | if mac_to_str(packet.src).lower() == self.deflect_mac.lower() : |
| 269 | | # replace the destination mac address |
| 270 | | actions.append([openflow.OFPAT_SET_DL_SRC, |
| 271 | | octstr_to_array(self.orig_mac)]) |
| 272 | | logger.debug("NEW SRC MAC %s" % ( self.orig_mac)) |
| 273 | | actions.append([openflow.OFPAT_SET_NW_SRC, |
| 274 | | ipstr_to_int(self.orig_ip)]) |
| 275 | | logger.debug("NEW DST IP %s" % ( self.orig_ip)) |
| 276 | | }}} |
