wiki:GENIExperimenter/Tutorials/WiMAXOpenFlow/Design-Setup

Version 18 (modified by Ryan Izard, 10 years ago) (diff)

--

OpenFlow-Based Vertical Handoff over WiFi and WiMAX in the Orbit Testbed

Image Map

1. Design the Experiment

The goal of this tutorial is to perform a vertical handoff over WiFi and WiMAX in the Orbit testbed. Let's talk specifics with repect to each Orbit node. As briefly mentioned in the Introduction, the experiment will be conducted with three Orbit nodes -- the client, the server, and the AP. Up first, we have the client node. The client is equipped with an Intel 6250 WiFi/WiMAX network adapter and an Atheros 5000 series WiFi network adapter. The Intel 6250 has a limitation – it can support the operation of either WiFi or WiMAX but not both simultaneously. As such, the Intel 6250 will serve as the WiMAX adapter and the Atheros 5000 series will serve as the WiFi adapter on the client. The OpenFlow-based handoff solution will choose which of these network interfaces it would like to use in order to relay messages to the server. Speaking of which, the server is equipped with the same wireless cards as the client. In order to receive packets from the client over both WiFi and WiMAX, the server will listen for the client's traffic over both its WiFi and WiMAX interfaces. To do this seamlessly, the server will also implement an OpenFlow-based solution. More details about how OpenFlow is used in both the server and the client are provided as the tutorial progresses. The third and final Orbit node necessary for this tutorial is the AP. Like the client and the server, the AP also contains both an Intel 6250 and an Atheros 5000 series wireless card; however, it will only use the Atheros card. The Atheros card on the AP is used by the software package hostapd to provide a WiFi network for both the client and the server machines. If you are interested in expanding upon this experiment, it is worth noting that Atheros cards play very well with hostapd-based networks; however, some Intel cards (such as the Intel 6250) are not directly supported. This is the reason we choose to use the Atheros over the Intel 6250 to provide the WiFi network.

Now, in order to provide a seamless handoff, the application should not be aware that the handoff takes place. The "seamlessness" of the handoff is still in development by the SDN team at Clemson. But for starters, from a networking point of view, if a network interface is brought up or is taken down, the IP address will be added to or removed from that network interface. This means the application's network socket will be broken if a break-before-make handoff occurs (which is oftentimes the case). When an application socket breaks, that means the application can no longer communicate over then network with it, and if this happens, the application must have some scheme to recreate the socket, otherwise the connection will be lost. In order to put as little reliance on the application layer as possible, a virtual tap interface is used. All applications will bind to the IP on this tap interface, and since tap interfaces are not physical, the only time it can “go down” is if we as programmers/network administrators take it down ourselves. Thus, it is a reliable way to make sure the application's socket stays “up”. In this tutorial, all application traffic will originate from and terminate at the tap interface.

The end-user will use the virtual tap interface (created by OpenVPN) for network connections, while the handoff execution will handle which physical interface to use. Now, the metric that decides which interface to use and when to perform the handoff is beyond the scope of this tutorial. Instead, this tutorial is designed to provide a basic framework for a handoff. To conduct the handoff, the Static Flow Pusher API in Floodlight, the OpenFlow controller, is used to insert OpenFlow flows manually, as determined by the handoff decision (e.g. a novel and super-cool metric a GENI experimenter has implemented). A Python script leverages the Static Flow Pusher API to add and remove flows.

The next logical question is where should this Static Flow Pusher insert the flows? In order to leverage the capabilities of OpenFlow, we need to have hardware and/or software that supports its use. Hence, Open vSwitch (OVS) is used in this tutorial. OVS is a software package that implements a virtual OpenFlow switch. This tutorial uses a pre-installed and pre-configured OVS network that incorporates the WiFi, WiMAX, and tap interfaces of the client as ports of virtual switches. The following is a picture depicting the general OVS topology used in this tutorial.

Each physical and tap interface has a corresponding OVS bridge. (In OVS terminology, an "OVS bridge" is essentially an "OpenFlow switch." It is called a bridge rather than a switch, since from the Linux kernel's perspective, the OVS bridge "bridges" the physical interface it is associated with.) These OVS bridges are linked together as if you actually plugged a physical switch into another with an Ethernet cable. The interfaces of the machine appear as ports on the OVS bridges, as do the internal links between the OVS bridges. With OVS, one can build a vitual network contained entirely within a single entity -- namely the client and server nodes of this tutorial. Each will employ internal OVS networks to alter the default behavior of the kernel with regard to networking. In essence, OVS allows the experimenter to circumvent the networking kernel and allow OpenFlow flows to route and modify packets instead.

2. Establish the Environment

  1. Using the Orbit images for the client, server, and AP, load each onto a separate node in an Orbit testbed. Make sure the node supports the hardware requirements for the experiment (Intel 6250 WiMAX and Atheros 5000 or 9000 series for the client and server, and Atheros 5000 or 9000 series for the AP). Boot these images after they are loaded and SSH into each machine.
  1. In the client, remove the Forwarding module from the Floodlight OpenFlow controller. Floodlight uses what it calls a module loading system, where the user can write modules to perform a certain task or set of tasks. Each module can register for certain events. For example, the Forwarding module registers for PACKET_IN events where the controller is sent a packet from a connected switch. Upon such an event, the Forwarding module will send the packet out the correct port(s) depending on the destination. This module essentially implements a standard learning switch function where the OpenFlow-enabled switch behaves as if it were a standard network switch. We do not want this functionality, since we would like to have exclusive control over which port(s) our packets get forwarded.
    1. Remove the Forwarding module.
    2. The module loading system maintains a list of the modules to be loaded at runtime. To remove the Forwarding module from this list (and thus disable it), open the /root/floodlight-0.90/src/main/resources/floodlightdefault.properties file and remove the line net.floodlightcontroller.forwarding.Forwarding,\.
    3. Browse to the root directory of the Floodlight project -- /root/floodlight-0.90. Execute ant in the terminal. ant is a Java-based build tool to build and compile Java projects. Upon success, ant will produce an updated executable jar file in the /root/floodlight-0.90/target directory named floodlight.jar.
  1. In the client, determine the MAC addresses of the WiMAX and WiFi network interface cards (NICs). These will be used in OpenFlow flows and will be discussed shortly.
    1. Load the kernel modules for each of the NICs by executing modprobe i2400m_usb and modprobe ath5k, respectively.
    2. After the kernel modules are loaded, the devices should be visible in the system. Execute ifconfig wmx0 and ifconfig wlan1, respectively and note the MAC addresses of each NIC. These will be used in later steps.
      $ ifconfig wmx0
         wmx0      Link encap:Ethernet  HWaddr '''00:0c:29:04:5c:41'''  
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:0 errors:0 dropped:0 overruns:0 frame:0
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:1000 
             RX bytes:0 (0 KiB)  TX bytes:0 (0 KiB)
             Interrupt:18 Base address:0x1424 
      $ ifconfig wlan1
         wlan1      Link encap:Ethernet  HWaddr '''00:0c:29:04:5c:4b'''  
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:0 errors:0 dropped:0 overruns:0 frame:0
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:1000 
             RX bytes:0 (0 KiB)  TX bytes:0 (0 KiB)
             Interrupt:19 Base address:0x14a4 
      
  1. Also in the client, customize the setup script gec18_setup.sh in /root/StartupScripts. This script is designed to (1) define user variables, (2) configure the tap interface with OpenVPN, (3) start Floodlight, (4) initialize and start Open vSwitch, and (5) configure Linux networking.
    1. Open the script with the text editor of your choice (vim, gedit, pico, nano, etc):
      $ vim gec18_setup.sh
      
    2. There are numerous user defined variables at the top of the script. These are placeholders for commonly used system and configuration specific information throughout the script. We need to change a few of them to suit our needs for this tutorial. Modify the MAC address for IFACE_D_MAC to match that of the WiMAX interface noted from ifconfig wmx0. Also, modify the IFACE_bridge_E_IP variable to be an IP in the same subnet as the server node (yet to be configured). For the tutorial at GEC18, the server node will be configured for you and you will be assigned an IP to give your client node. Assign the IP you have been provided with to the IFACE_bridge_E_IP variable. Another variable that needs to be configured in the client is the MAC address, SSID, and channel of the AP node. The client needs these pieces of information in order to successfully associate with the AP. For this tutorial, the AP will also be pre-configured. Use the information about the AP that is provided.
      ###################
      #USR DEF VARIABLES#
      ###################
      
      SERVER_IP=10.41.14.3
      SERVER_MAC=00:1d:e1:3b:62:a6
      
      #IFACE_bridge_A=br_eth
      IFACE_bridge_B=br_wifi1
      #IFACE_bridge_C=br_wifi1
      IFACE_bridge_D=br_wimax
      IFACE_bridge_E=br_tap
      IFACE_bridge_E_IP=10.41.14.1/16
      
      #IFACE_A=eth0
      IFACE_B=wlan1
      IFACE_B_AP_ESSID=GENI_WiFi_AP
      IFACE_B_AP_MAC=00:60:B3:B0:C6:6C
      IFACE_B_AP_channel=11
      IFACE_B_AP_mode=Managed
      IFACE_B_kernel_module=ath5k
      #IFACE_C=wlan1
      IFACE_D=wmx0
      IFACE_D_network_ID=51
      IFACE_D_DEV_NAME=/dev/sg1
      IFACE_D_kernel_module=i2400m_usb
      IFACE_D_MAC=00:1d:e1:3b:48:1d
      IFACE_E=tap0
      IFACE_E_MAC=12:51:16:90:8f:ee
      
      #IFACE_patch_bridge_A_to_bridge_E=eth-tap
      #IFACE_patch_bridge_E_to_bridge_A=tap-eth
      IFACE_patch_bridge_B_to_bridge_E=wlan1-tap
      IFACE_patch_bridge_E_to_bridge_B=tap-wlan1
      #IFACE_patch_bridge_C_to_bridge_E=wlan0-tap
      #IFACE_patch_bridge_E_to_bridge_C=tap-wlan0
      IFACE_patch_bridge_D_to_bridge_E=wimax-tap
      
      OVS_kernel_module_path=/root/openvswitch-1.9.0/datapath/linux/openvswitch.ko
      #OVS_switchDPID_A=0000000000000001
      OVS_switchDPID_B=0000000000000002
      #OVS_switchDPID_C=0000000000000003
      OVS_switchDPID_D=0000000000000004
      OVS_switchDPID_E=0000000000000005
      OVS_controllerIP=127.0.0.1:6633
      
      FL_contRESTIP=127.0.0.1
      FL_path=/root/floodlight-0.90/target/floodlight.jar
      FL_log=/root/floodlight-output
      FL_clear_flows_script=/root/SwitchingScripts/clear_flows.sh
      FL_initial_flows_script=/root/SwitchingScripts/gec18_switch_to_wifi.sh
      ...
      
    3. Next, we need to remove any running instances of Floodlight and OVS. We do not want more than one instance of these processes running, otherwise chaos might ensue. So, a simple loop and kill works to terminate all running processes.
      ...
      ##########
      #CLEAN UP#
      ##########
      
      while [ -n "$(ps aux | grep floodlight | grep -v grep | awk '{print $2}')" ];
      do
              echo "PRE: Cleaning up previous Floodlight..."
              kill -9 $(ps aux | grep floodlight | grep -v grep | awk '{print $2}')
      done
      while [ -n "$(ps aux | grep ovs[^_] | grep -v grep | awk '{print $2}')" ];
      do
              echo "PRE: Cleaning up previous OVS..."
              kill -9 $(ps aux | grep ovs[^_] | grep -v grep | awk '{print $2}')
      done
      ...
      
    4. The next thing to do is load our WiFi and WiMAX kernel modules. We have already done this to get the MAC addresses of the interfaces, but we should also do this here so that this script can be run without any prerequisites aside from this initial configuration process.
      ...
      #################
      #ADD WIMAX IFACE#
      #################
      
      echo "WM: Installing WiMAX kernel module..."
      modprobe $IFACE_D_kernel_module
      
      sleep 3
      
      echo "WM: Testing WiMAX connection to basestation..."
      wimaxcu roff
      sleep 2
      wimaxcu ron
      sleep 2
      wimaxcu connect network $IFACE_D_network_ID
      
      echo "WiFi: Installing WiFi kernel module..."
      modprobe $IFACE_B_kernel_module
      ...
      
    5. Next, we need to create our tap interface. This is the network interface that will funnel all outbound packets from the applications on our client and send them into our OpenVswitch network.
      ...
      ###############
      #ADD TAP IFAC1#
      ###############
      
      echo "OVPN: Installing tap interface, $IFACE_E"
      openvpn --mktun --dev $IFACE_E --lladdr $IFACE_E_MAC
      ...
      
    6. After the tap interface, we need to start the Floodlight controller. The path to the controller executable generated by ant previously is defined as the variable FL_path. Once Floodlight starts, it will not be connected to anything -- we haven't configured our OVS network yet. Once we do this, Floodlight will automatically detect and begin talking to the OVS bridges.
      ...
      ##################
      #START FLOODLIGHT#
      ##################
      
      echo "FL: Starting Floodlight..."
      (java -jar $FL_path) > $FL_log 2>&1 &
      echo "FL: Finished!"
      ...
      
    7. Now that Floodlight is running, we need to give the OpenFlow controller something to connect to, which is out OVS network. The first thing to to wirth regard to OVS is to load its kernel module and initialize it's database.
      ...
      ###################
      #START OPENVSWITCH#
      ###################
      
      echo "OVS: Configuring OVS..."
      echo "OVS: Checking for kernel module..."
      if [ -e $(lsmod | grep openvswitch) ]
      then
              echo "OVS: ...inserting kernel module"
              insmod $OVS_kernel_module_path
      else
              echo "OVS: ...kernel module already present"
      fi
      ##################################################################
      echo "OVS: Creating database"
      ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
      --remote=db:Open_vSwitch,manager_options \
      --private-key=db:SSL,private_key \
      --certificate=db:SSL,certificate \
      --bootstrap-ca-cert=db:SSL,ca_cert \
      --pidfile --detach
      echo "OVS: Initializing OVS..."
      ovs-vsctl --no-wait init
      echo "OVS: Starting OVS..."
      ovs-vswitchd --pidfile --detach
      ...
      
  1. Now, we need to take down any pre-existing OVS bridges and create new ones. If we later try to create bridges with the same names, OVS will produce an error. There are options called --if-exists and --may-exists that are supposed to allow the creation of an OVS bridge/port under the condition it does not exist already. I have not been able to get these options to work, so we will use Bash scripting instead:
    echo "OVS: Removing any existing bridge, $IFACE_bridge_A $IFACE_bridge_B"
    echo "OVS: $IFACE_bridge_C $IFACE_bridge_D ..."
    
    ## Ethernet
    #if [ -n "$(ovs-vsctl show | grep $IFACE_bridge_A)" ]
    #then
    #       echo "OVS: ...removing $IFACE_bridge_A"
    #       ovs-vsctl del-br $IFACE_bridge_A
    #fi
    
    ## Wlan0
    echo "OVS: Removing any existing bridge, $IFACE_bridge_B..."
    if [ -n "$(ovs-vsctl show | grep $IFACE_bridge_B)" ]
    then
            echo "OVS: ...removing $IFACE_bridge_B"
            ovs-vsctl del-br $IFACE_bridge_B
    fi
    
    ## Wlan1
    #if [ -n "$(ovs-vsctl show | grep $IFACE_bridge_C)" ]
    #then
    #       echo "OVS: ...removing $IFACE_bridge_C"
    #       ovs-vsctl del-br $IFACE_bridge_C
    #fi
    
    ## WiMAX
    if [ -n "$(ovs-vsctl show | grep $IFACE_bridge_D)" ]
    then
            echo "OVS: ...removing $IFACE_bridge_D"
            ovs-vsctl del-br $IFACE_bridge_D
    fi
    
    ## Tap
    if [ -n "$(ovs-vsctl show | grep $IFACE_bridge_E)" ]
    then
            echo "OVS: ...removing $IFACE_bridge_E"
            ovs-vsctl del-br $IFACE_bridge_E
    fi
    
    ## Wlan0
    echo "OVS: Adding interface bridge, $IFACE_bridge_B..."
    ovs-vsctl add-br $IFACE_bridge_B
    echo "OVS: ...with port $IFACE_B"
    ovs-vsctl add-port $IFACE_bridge_B $IFACE_B
    echo "OVS: ...with port $IFACE_patch_bridge_B_to_bridge_E"
    ovs-vsctl add-port $IFACE_bridge_B $IFACE_patch_bridge_B_to_bridge_E
    
    ## Wlan1
    #echo "OVS: Adding interface bridge, $IFACE_bridge_C..."
    #ovs-vsctl add-br $IFACE_bridge_C
    #echo "OVS: ...with port $IFACE_C"
    #ovs-vsctl add-port $IFACE_bridge_C $IFACE_C
    #echo "OVS: ...with port $IFACE_patch_bridge_C_to_bridge_E"
    #ovs-vsctl add-port $IFACE_bridge_C $IFACE_patch_bridge_C_to_bridge_E
    
    ## WiMAX
    echo "OVS: Adding interface bridge, $IFACE_bridge_D..."
    ovs-vsctl add-br $IFACE_bridge_D
    echo "OVS: ...with port $IFACE_D"
    ovs-vsctl add-port $IFACE_bridge_D $IFACE_D
    echo "OVS: ...with port $IFACE_patch_bridge_D_to_bridge_E"
    ovs-vsctl add-port $IFACE_bridge_D $IFACE_patch_bridge_D_to_bridge_E
    
    ## Tap
    echo "OVS: Adding interface bridge, $IFACE_bridge_E..."
    ovs-vsctl add-br $IFACE_bridge_E
    echo "OVS: ...with port $IFACE_E"
    ovs-vsctl add-port $IFACE_bridge_E $IFACE_E
    #echo "OVS: ...with port $IFACE_patch_bridge_E_to_bridge_A"
    #ovs-vsctl add-port $IFACE_bridge_E $IFACE_patch_bridge_E_to_bridge_A
    echo "OVS: ...with port $IFACE_patch_bridge_E_to_bridge_B"
    ovs-vsctl add-port $IFACE_bridge_E $IFACE_patch_bridge_E_to_bridge_B
    #echo "OVS: ...with port $IFACE_patch_bridge_E_to_bridge_C"
    #ovs-vsctl add-port $IFACE_bridge_E $IFACE_patch_bridge_E_to_bridge_C
    echo "OVS: ...with port $IFACE_patch_bridge_E_to_bridge_D"
    ovs-vsctl add-port $IFACE_bridge_E $IFACE_patch_bridge_E_to_bridge_D
    ...
    
  2. At this point, we're ready to set the patch ports between the OVS bridges. These create links between the OVS tap bridge and the OVS WiFi and WiMAX bridges in order to facilitate the flow of packets from the tap bridge to the physical interface of choice -- WiFi or WiMAX.
    ...
    #################
    #SET PATCH PORTS#
    #################
    
    ## Set patch ports
    #echo "OVS: Patching ports $IFACE_patch_bridge_E_to_bridge_A, $IFACE_patch_bridge_A_to_bridge_E"
    #ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_A type=patch
    #ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_A options:peer=$IFACE_patch_bridge_A_to_bridge_E
    #ovs-vsctl set interface $IFACE_patch_bridge_A_to_bridge_E type=patch
    #ovs-vsctl set interface $IFACE_patch_bridge_A_to_bridge_E options:peer=$IFACE_patch_bridge_E_to_bridge_A
    
    echo "OVS: Patching ports $IFACE_patch_bridge_E_to_bridge_B, $IFACE_patch_bridge_B_to_bridge_E"
    ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_B type=patch
    ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_B options:peer=$IFACE_patch_bridge_B_to_bridge_E
    ovs-vsctl set interface $IFACE_patch_bridge_B_to_bridge_E type=patch
    ovs-vsctl set interface $IFACE_patch_bridge_B_to_bridge_E options:peer=$IFACE_patch_bridge_E_to_bridge_B
    
    #echo "OVS: Patching ports $IFACE_patch_bridge_E_to_bridge_C, $IFACE_patch_bridge_C_to_bridge_E"
    #ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_C type=patch
    #ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_C options:peer=$IFACE_patch_bridge_C_to_bridge_E
    #ovs-vsctl set interface $IFACE_patch_bridge_C_to_bridge_E type=patch
    #ovs-vsctl set interface $IFACE_patch_bridge_C_to_bridge_E options:peer=$IFACE_patch_bridge_E_to_bridge_C
    
    echo "OVS: Patching ports $IFACE_patch_bridge_E_to_bridge_D, $IFACE_patch_bridge_D_to_bridge_E"
    ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_D type=patch
    ovs-vsctl set interface $IFACE_patch_bridge_E_to_bridge_D options:peer=$IFACE_patch_bridge_D_to_bridge_E
    ovs-vsctl set interface $IFACE_patch_bridge_D_to_bridge_E type=patch
    ovs-vsctl set interface $IFACE_patch_bridge_D_to_bridge_E options:peer=$IFACE_patch_bridge_E_to_bridge_D
    
    
  3. Now, we need to assign each OVS bridge a unique ID (DPID) and point them to the address of the Floodlight controller. Floodlight will be run on the localhost, so the loopback address is defined within a the variable OVS_controllerIP and is set as the loopback address 127.0.0.1:6633. Port 6633 is the default OpenFlow port. As long as Floodlight and OVS are configured to use the same port number, any available port will suffice. We also need to tell the OVS bridges what to do if and when they lose connection to the Floodlight controller. The standalone mode allows them to default to a learning switch state if the link between the controller and the bridge is broken.
    ## Set Eth DPID
    #echo "OVS: Setting $IFACE_bridge_A DPID to $OVS_switchDPID_A..."
    #ovs-vsctl set bridge $IFACE_bridge_A other-config:datapath-id=$OVS_switchDPID_A
    
    ## Set Wlan0 DPID
    echo "OVS: Setting $IFACE_bridge_B DPID to $OVS_switchDPID_B..."
    ovs-vsctl set bridge $IFACE_bridge_B other-config:datapath-id=$OVS_switchDPID_B
    
    ## Set Wlan1 DPID
    #echo "OVS: Setting $IFACE_bridge_C DPID to $OVS_switchDPID_C..."
    #ovs-vsctl set bridge $IFACE_bridge_C other-config:datapath-id=$OVS_switchDPID_C
    
    ## Set WiMAX DPID
    echo "OVS: Setting $IFACE_bridge_D DPID to $OVS_switchDPID_D..."
    ovs-vsctl set bridge $IFACE_bridge_D other-config:datapath-id=$OVS_switchDPID_D
    
    ## Set Tap DPID
    echo "OVS: Setting $IFACE_bridge_E DPID to $OVS_switchDPID_E..."
    ovs-vsctl set bridge $IFACE_bridge_E other-config:datapath-id=$OVS_switchDPID_E
    
    ## Ethernet
    #echo "OVS: Connecting $IFACE_bridge_A to controller at $OVS_controllerIP"
    #ovs-vsctl set-controller $IFACE_bridge_A tcp:$OVS_controllerIP
    #ovs-vsctl set-fail-mode $IFACE_bridge_A standalone
    
    ## Wlan0
    echo "OVS: Connecting $IFACE_bridge_B to controller at $OVS_controllerIP"
    ovs-vsctl set-controller $IFACE_bridge_B tcp:$OVS_controllerIP
    ovs-vsctl set-fail-mode $IFACE_bridge_B standalone
    echo "OVS: Setting MAC of $IFACE_bridge_B to $IFACE_D"
    ovs-vsctl set bridge $IFACE_bridge_B other-config:hwaddr=$IFACE_D_MAC
    
    ## Wlan1
    #echo "OVS: Connecting $IFACE_bridge_C to controller at $OVS_controllerIP"
    #ovs-vsctl set-controller $IFACE_bridge_C tcp:$OVS_controllerIP
    #ovs-vsctl set-fail-mode $IFACE_bridge_C standalone
    
    ## WiMAX
    echo "OVS: Connecting $IFACE_bridge_D to controller at $OVS_controllerIP"
    ovs-vsctl set-controller $IFACE_bridge_D tcp:$OVS_controllerIP
    ovs-vsctl set-fail-mode $IFACE_bridge_D standalone
    echo "OVS: Setting MAC of $IFACE_bridge_D to $IFACE_D"
    ovs-vsctl set bridge $IFACE_bridge_D other-config:hwaddr=$IFACE_D_MAC
    
    ## Tap
    echo "OVS: Connecting $IFACE_bridge_E to controller at $OVS_controllerIP"
    ovs-vsctl set-controller $IFACE_bridge_E tcp:$OVS_controllerIP
    ovs-vsctl set-fail-mode $IFACE_bridge_E standalone
    echo "OVS: Setting MAC of $IFACE_bridge_E to $IFACE_E"
    ovs-vsctl set bridge $IFACE_bridge_E other-config:hwaddr=$IFACE_E_MAC
    
    echo "OVS: Finished!"
    ...
    
  4. Now that we have OVS and Floodlight running, the next thing to do in the setup script is to configure our network connections. We first need to disable IP forwarding, so that the kernel does not try to route packets around our OVS network. We then need to establish a link on each of the interface we would like to participate in the handoff -- WiFi and WiMAX. Note that WiFi is assigned to IFACE_B and WiMAX is assigned to IFACE_D. Also note the use of the WiFi AP variables we set in the first step of configuring this script. After WiFi and WiMAX each connect to their respective networks, we need to remove any IP addresses assigned by DHCP when the interfaces were brought up. In the case of this tutorial, we should not get an IP via DHCP for WiFi, since we're managing our own AP. It is possible that we received an IP over WiMAX though, so we need to remove it, just in case. This involves killing the running dhclient process for WiMAX. At this point, we're ready to assign our single IP to the tap OVS bridge over which applications can send and receive data. This will allow us to inject data/packets into our OVS network as well as pull packets destined for an application out of our OVS network.
    ##########################
    #CONFIGURE NETWORK ACCESS#
    ##########################
    
    echo "NTWK: Making sure loopback is up (it has gone down for some reason before)..."
    ifconfig lo up
    
    ## Turn off IP Forwarding
    echo "NTWK: Disabling IP Forwarding..."
    echo "0" > /proc/sys/net/ipv4/ip_forward
    echo "0" > /proc/sys/net/ipv4/conf/all/forwarding
    
    ## Disable IPv6
    echo "NTWK: Disabling IPv6..."
    echo "0" > /proc/sys/net/ipv6/conf/all/disable_ipv6
    
    ## Disable IP on interfaces
    #echo "NTWK: Taking down $IFACE_A..."
    #ifconfig $IFACE_A down
    echo "NTWK: Taking down $IFACE_D..."
    ifconfig $IFACE_D down
    echo "NTWK: Taking down $IFACE_B..."
    ifconfig $IFACE_B down
    #echo "NTWK: Taking down $IFACE_C..."
    #ifconfig $IFACE_C down
    #echo "NTWK: Taking down $IFACE_bridge_A..."
    #ifconfig $IFACE_bridge_A down
    echo "NTWK: Taking down $IFACE_bridge_B..."
    ifconfig $IFACE_bridge_B down
    #echo "NTWK: Taking down $IFACE_bridge_C..."
    #ifconfig $IFACE_bridge_C down
    echo "NTWK: Taking down $IFACE_bridge_D..."
    ifconfig $IFACE_bridge_D down
    sleep 2
    
    # The use of ip link over ifconfig is necessary is some cases, especially for WiFi (haven't fully debugged yet)
    ip link set dev $IFACE_B down
    
    echo "NTWK: Bringing all non-bridge interfaces back up..."
    ip link set dev $IFACE_B promisc on up
    ip link set dev $IFACE_D promisc on up
    ip link set dev $IFACE_E promisc on up
    ifconfig $IFACE_B 0.0.0.0
    ifconfig $IFACE_D 0.0.0.0
    ifconfig $IFACE_E 0.0.0.0
    
    echo "NTWK: Setting MAC of $IFACE_B to $IFACE_D..."
    ifconfig $IFACE_B down
    ifconfig $IFACE_B hw ether $IFACE_D_MAC
    ifconfig $IFACE_B up promisc multicast
    echo "NTWK: Connecting $IFACE_B to AP via iwconfig..."
    iwconfig $IFACE_B essid $IFACE_B_AP_ESSID ap $IFACE_B_AP_MAC channel $IFACE_B_AP_channel mode $IFACE_B_AP_mode
    
    echo "NTWK: Connecting $IFACE_D to BS via wimaxcu..."
    wimaxcu roff
    sleep 2
    wimaxcu ron
    sleep 2
    wimaxcu connect network $IFACE_D_network_ID
    
    echo "NTWK: Bringing all non-IP OVS bridges up..."
    ifconfig $IFACE_bridge_B 0.0.0.0 up
    ifconfig $IFACE_bridge_D 0.0.0.0 up
    
    echo "NTWK: Setting IP of $IFACE_bridge_E_IP on $IFACE_bridge_E"
    ifconfig $IFACE_bridge_E $IFACE_bridge_E_IP up
    
    # We can't kill every dhclient process here, since on Orbit, we have a control interface (eth1?) over DHCP it seems
    echo "NTWK: Sleeping for a bit while dhclient has a chance to auto-start..."
    sleep 10
    
    echo "NTWK: Killing any running dhclient processes on $IFACE_D"
    pgrep -f dhclient.$IFACE_D
    pkill -9 -f dhclient.$IFACE_D
    ifconfig $IFACE_D 0.0.0.0
    
    echo "NTWK: Inserting static ARP entry to server as $SERVER_IP $SERVER_MAC"
    arp -s $SERVER_IP $SERVER_MAC
    
    echo "NTWK: Pushing initial flows..."
    $FL_clear_flows_script
    $FL_initial_flows_script
    
    exit 0
    
  5. At this point, all components of the gec18_setup.sh script have been configured for the client. The last thing to do is allow the flow of packets from the tap interface to the interface of our choice using Floodlight's Static Flow Pusher API. There is the variable FL_initial_flows_script that is defined as the path to the Bash script that will insert the initial flows. Now, at this point, we will insert these flows, but we need to need check to ensure they are configured correctly. For this, we will need to take note of the WiMAX MAC address determined in an earlier step.
  6. Save the script using your text editor and close it.
  1. The last thing to configure in the Client are the flows themselves. OpenFlow flows are designed to match certain types of packets, and then based on a successful match, do something to those packets -- e.g. output on port 1, rewrite to a new destination IP, etc. For this tutorial, we need to do two things -- (1) we need to output the packet to the correct port, and (2) we need to rewrite the source MAC for any outbound packets and the destination MAC for any inbound packets. Why? Well, we configured our OVS switches and linked them together as depicted in the figure at the top of this page. Each link of each switch has a port number associated with it. Any physical or tap interface has the port number 1, and any OVS patch port (a port linking two OVS bridges) has an arbitrarily assigned port number. In order to switch packets in our OVS network, we need to know the correct patch ports over which to send packets. For example, if the client wants to send a packet from the tap OVS bridge to the WiFi interface, when the packet is in the tap OVS bridge, the switch needs to know which port to send the packet out on. Well, since we are manually telling the client which interface to use, our static flows must specify the port number that leads to the physical interface of choice. In the case of our example, we would want to insert a flow to direct all packets out the patch port number leading to the WiFi OVS bridge. To determine these port numbers, we need to query Floodlight for information about connected switches.
    1. There is a script in the directory /root/SwitchingScripts called getJSONPorts.sh. This script will send an HTTP request to Floodlight's REST API, requesting information about the connected switches. It pipes this information (in JSON format) to a python JSON printer (i.e. it "pretty-prints" the JSON string for us). Based on this output, we can determine and confirm the port numbers assigned to each OVS bridge, and thus, we can correctly compose our flows such that they match and forward packets out the correct port numbers. Since we are using the same Orbit image for each client machine in this tutorial, all the port numbers should be configured correctly as-is, but it's still worth checking just to be certain.
    2. The following is an example output from the client's /root/SwitchingScripts/getJSONPorts.sh script. Based on this information, note the port numbers for each port of each OVS bridge. The WiFi and WiMAX OVS bridges should have three ports -- the physical interface port, the OVS patch port to the tap OVS bridge, and the port of the OVS bridge itself. And, the tap OVS bridge should have four ports -- the "physical" tap interface port, the OVS patch ports to the WiFi and WiMAX OVS bridges, and the OVS bridge itself.
      $ ./getJSONports.sh 
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      100  3388    0  3388    0     0  74701      0 --:--:-- --:--:-- --:--:-- 77000
      [
          {
              "actions": 4095, 
              "attributes": {
                  "DescriptionData": {
                      "datapathDescription": "None", 
                      "hardwareDescription": "Open vSwitch", 
                      "length": 1056, 
                      "manufacturerDescription": "Nicira, Inc.", 
                      "serialNumber": "None", 
                      "softwareDescription": "1.9.0"
                  }, 
                  "FastWildcards": 4194303, 
                  "supportsOfppFlood": true, 
                  "supportsOfppTable": true
              }, 
              "buffers": 256, 
              "capabilities": 199, 
              "connectedSince": 1382901469877, 
              "dpid": "00:00:00:00:00:00:00:05", 
              "featuresReplyFromSwitch": {
                  "cancelled": false, 
                  "done": true, 
                  "transactionId": 2
              }, 
              "inetAddress": "/127.0.0.1:60016", 
              "ports": [
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:00:00:00:00:05", 
                      "name": "tap-wlan1", 
                      "peerFeatures": 0, 
                      "portNumber": 6, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "12:51:16:90:8f:ee", 
                      "name": "br_tap", 
                      "peerFeatures": 0, 
                      "portNumber": 65534, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:00:00:00:00:05", 
                      "name": "tap-wimax", 
                      "peerFeatures": 0, 
                      "portNumber": 10, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 130, 
                      "hardwareAddress": "12:51:16:90:8f:ee", 
                      "name": "tap0", 
                      "peerFeatures": 0, 
                      "portNumber": 1, 
                      "state": 1, 
                      "supportedFeatures": 0
                  }
              ], 
              "role": null, 
              "tables": -1
          }, 
          {
              "actions": 4095, 
              "attributes": {
                  "DescriptionData": {
                      "datapathDescription": "None", 
                      "hardwareDescription": "Open vSwitch", 
                      "length": 1056, 
                      "manufacturerDescription": "Nicira, Inc.", 
                      "serialNumber": "None", 
                      "softwareDescription": "1.9.0"
                  }, 
                  "FastWildcards": 4194303, 
                  "supportsOfppFlood": true, 
                  "supportsOfppTable": true
              }, 
              "buffers": 256, 
              "capabilities": 199, 
              "connectedSince": 1382901469877, 
              "dpid": "00:00:00:00:00:00:00:02", 
              "featuresReplyFromSwitch": {
                  "cancelled": false, 
                  "done": false, 
                  "transactionId": 2
              }, 
              "inetAddress": "/127.0.0.1:60015", 
              "ports": [
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:00:00:00:00:02", 
                      "name": "wlan1-tap", 
                      "peerFeatures": 0, 
                      "portNumber": 12, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:1d:e1:3b:48:1d", 
                      "name": "br_wifi1", 
                      "peerFeatures": 0, 
                      "portNumber": 65534, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:1d:e1:3b:48:1d", 
                      "name": "wlan1", 
                      "peerFeatures": 0, 
                      "portNumber": 1, 
                      "state": 1, 
                      "supportedFeatures": 0
                  }
              ], 
              "role": null, 
              "tables": -1
          }, 
          {
              "actions": 4095, 
              "attributes": {
                  "DescriptionData": {
                      "datapathDescription": "None", 
                      "hardwareDescription": "Open vSwitch", 
                      "length": 1056, 
                      "manufacturerDescription": "Nicira, Inc.", 
                      "serialNumber": "None", 
                      "softwareDescription": "1.9.0"
                  }, 
                  "FastWildcards": 4194303, 
                  "supportsOfppFlood": true, 
                  "supportsOfppTable": true
              }, 
              "buffers": 256, 
              "capabilities": 199, 
              "connectedSince": 1382901469877, 
              "dpid": "00:00:00:00:00:00:00:04", 
              "featuresReplyFromSwitch": {
                  "cancelled": false, 
                  "done": false, 
                  "transactionId": 2
              }, 
              "inetAddress": "/127.0.0.1:60014", 
              "ports": [
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:00:00:00:00:04", 
                      "name": "wimax-tap", 
                      "peerFeatures": 0, 
                      "portNumber": 13, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                      "currentFeatures": 0, 
                      "hardwareAddress": "00:1d:e1:3b:48:1d", 
                      "name": "br_wimax", 
                      "peerFeatures": 0, 
                      "portNumber": 65534, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }, 
                  {
                      "advertisedFeatures": 0, 
                      "config": 0, 
                     "currentFeatures": 0, 
                      "hardwareAddress": "00:00:00:00:00:04", 
                      "name": "wmx0", 
                      "peerFeatures": 0, 
                      "portNumber": 1, 
                      "state": 0, 
                      "supportedFeatures": 0
                  }
              ], 
              "role": null, 
              "tables": -1
          }
      ]
      
  1. Using your favorite text editor, open the /root/SwitchingScripts/gec18_switch_to_wifi.py script. (Do not open the /root/SwitchingScripts/gec18_switch_to_wifi.sh script. This is simply a wrapper for the python script with the flows themselves.) In the python script, browse down to were you see the definition of flow1. Right before this definition are a few variables we need to check/set. They define the MAC addresses of the physical and tap interfaces. Recall that we need to rewrite these in our flows. Why is this so? Well, WiFi and WiMAX have an association process. This process provides the AP/BS with the MAC address of the associated client. The AP/BS will filter incoming packets by associate MACs. If an ingress packet contains a source MAC that is not known to the AP/BS, it will be dropped. Thus, any packets we send from our tap interface need to have their source MAC addresses rewritten so that the AP or BS will recognize the source and thus accept the packets. Using the MAC of the WiMAX interface noted in prior steps, set the MAC address of the wifi_mac and wimax_mac to the MAC address of the WiMAX MAC. This might seem odd, but in order to simplify this tuturial and experiment, we have "spoofed" the actual MAC of the WiFi interface to that of the WiMAX interface, thus making both interfaces appear to have the same MAC. You can see this for yourselves in the gec18_setup.sh script of the client. This allows the server to address the client with a single MAC.
  2. Next, save the WiFi switching script and open /root/SwithingScripts/gec_18_switch_to_wimax.py. Repeat step 5b.
  1. The configuration of the client is now complete. The next step is to configure the server node. For the tutorial at GEC18, the server has been configured for you. Not only does this save time, but it also allows multiple clients to share the same server node. The process is nearly identical to that of the client node. The difference is that you need to assign a different IP to the tap interface and provide the IP of the client instead of the the IP of the server. Another key difference is how the flows work. On the client, the flows inserted will switch packets to the interface the client wishes to use. As a simplification to this experiment and tutorial, the server's flows are configured to always listen and send packets out its WiFi and WiMAX interfaces. In other words, no switching takes place on the server node. The flows that are inserted when the server is configured and run are the flows that will remain for the lifetime of the experiment.
  1. Last but not least, after the server is configured, the AP needs to be configured and brought up. This has also been done for you at GEC18, since all client nodes will share a single AP. The steps to configuring the AP are mostly related to setting up hostapd. On the AP, there is a script /root/gec18_setup.sh. This script simply inserts the kernel module for the Atheros 5000 series card (modprobe ath5k) and starts hostapd. All configuration parameters related to the WiFi network (e.g. SSID, encryption, channel, etc) are located in the /root/hostapd.conf configuration file. hostapd will parse this file and read these parameters when it is executed. The AP is set by default to broadcast an open (unsecured) network on channel 11 and with SSID GENI_WiFi_AP. It is also configured by default in AP mode, not ad-hoc mode. No changes should be made to this configuration for this experiment.

Warnings

Warning Be on the lookout for typos in your scripts!

Notes

Note Write down your interface names, IP addresses, and subnet masks. All subnets must be the same for a Layer-2 handoff.

Tips

Tip If you need assistance, please ask for help!

Introduction

Next: Execute

Attachments (2)

Download all attachments as: .zip