Changes between Version 43 and Version 44 of GEC17Agenda/AdvancedOpenFlow/Procedure/Execute


Ignore:
Timestamp:
03/09/14 19:40:11 (10 years ago)
Author:
nriga@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC17Agenda/AdvancedOpenFlow/Procedure/Execute

    v43 v44  
    133133 - Did you see any difference from the graphs plotted on !LabWiki, compared with the graphs plotted in the first experiment? why?
    134134 - Check out the output of the Load Balancer on node "Switch" and tell how many flows are directed to the left path and how many are on the right path, why?
    135  - To answer the above question, you need to understand the Load Balancing controller. Check out the "load-balancer.rb" file in your home directory on node "Switch". Check [#A.AbouttheOpenFlowcontrollerload-balancer.rb Appendix B] for hints/explanations about this OpenFlow Controller.
     135 - To answer the above question, you need to understand the Load Balancing controller. Check out the "load-balancer.rb" file in your home directory on node "Switch". Check [wiki:GEC17Agenda/AdvancedOpenFlow/Procedure/Appendices#A.AbouttheOpenFlowcontrollerload-balancer.rb Appendix B] for hints/explanations about this OpenFlow Controller.
    136136
    137137=== 3.3 Modify the OpenFlow Controller to balance throughput among all the TCP flows ===
     
    176176  - Load balancers that begin with "load-balancer-throughput" picks path based on the total throughput sent out to each path: the one with '''more throughput''' is picked
    177177
    178 = Appendix: Hints and Explanations =
    179 == A. About the OpenFlow controller [http://www.gpolab.bbn.com/experiment-support/OpenFlowExampleExperiment/ExoGENI/load-balancer.rb load-balancer.rb] ==
    180   - Trema web site: http://trema.github.io/trema/
    181   - Treme ruby API document: http://rubydoc.info/github/trema/trema/master/frames
    182   - '''Functions used in our tutorial:'''
    183     - '''start()''': is the function that will be called when the OpenFlow Controller is started. Here in our case, we read the file /tmp/portmap and figures out which OpenFlow port points to which path
    184     - '''switch_ready()''': is the function that will be called each time a switch connects to the OpenFlow Controller. Here in our case, we allow all non-TCP flows to pass (including ARP and ICMP packets) and ask new inbound TCP flow to go to the controller. We also starts a "timer" function that calls "query_stats()" once every 2 seconds.
    185     - '''query_stats()''': is the function that sends out a flow_stats_request to get the current statistics about each flow.
    186     - '''packet_in()''': is the function that will be called each time a packet arrives at the controller. Here in our case, we call "decide_path()" to get path decisions, then send flow entry back to the OpenFlow Switch to instruct the switch which path to take for this new TCP flow.
    187     - '''stats_reply()''': is the function that will be called when the OpenFlow Controller receives a flow_stats_reply message from the OpenFlow Switch. Here in our case, we update the flow statistics so that "decide_path()" can make the right decision.
    188     - '''send_flow_mod_add()''': is the function that you should use to add a flow entry into an OpenFlow Switch.
    189     - '''decide_path()''': is the function that makes path decisions. It returns the path choices based on flow statistics.
    190   - '''The Whole Process: '''
    191     - When the OpenFlow switch is ready, our controller starts a function that asks for flow stats once every 2 seconds.
    192     - The OpenFlow switch will reply with statistics information about all flows in its flow table.
    193     - This flow statistics message will be fetched by the "stats_reply" function in the OpenFlow controller implemented by the user on node "Switch".
    194     - As a result, our controller updates its knowledge about both left and right path once every 2 seconds.
    195     - Upon the arrival of a new TCP flow, the OpenFlow controller decides which path to send the new flow to, based on the updated flow statistics.
    196178
    197   The !FlowStatsReply message is in the following format:
    198 {{{
    199 FlowStatsReply.new(
    200   :length => 96,
    201   :table_id => 0,
    202   :match => Match.new
    203   :duration_sec => 10,
    204   :duration_nsec => 106000000,
    205   :priority => 0,
    206   :idle_timeout => 0,
    207   :hard_timeout => 0,
    208   :cookie => 0xabcd,
    209   :packet_count => 1,
    210   :byte_count => 1,
    211   :actions => [ ActionOutput.new ]
    212 )
    213 }}}
    214 
    215 == B. About The Rspec file [http://www.gpolab.bbn.com/experiment-support/OpenFlowExampleExperiment/openflow-loadbalancer-kvm.rspec OpenFlowLBExo.rspec] ==
    216   - The Rspec file describes a topology we showed earlier--each node is assigned with certain number of interfaces with pre-defined IP addresses
    217   - Some of the nodes are loaded with softwares and post-scripts. We will take node "Switch" as an example since it is the most complicated one.
    218    - The following section in the Rspec file for node "Switch":
    219    {{{
    220      <install url="http://www.gpolab.bbn.com/experiment-support/OpenFlowExampleExperiment/software/of-switch-exo.tar.gz"
    221                          install_path="/"/>
    222    }}}
    223    means it is going to download that tar ball from the specified URL and extract to directory "/"
    224    - The following section in the Rspec file for node "Switch":
    225    {{{
    226      <execute shell="bash" command="/tmp/postboot_script_exo.sh $sliceName $self.Name() ;
    227                            /tmp/of-topo-setup/lb-setup"/>
    228    }}}
    229    names the post-boot script that ExoGENI is going to run for you after the nodes are booted. 
    230   - More information about "/tmp/postboot_script_exo.sh":
    231    It is a "hook" to the !LabWiki interface. Experimenter run this so that !LabWiki knows the name of the slice and the hostname of the particular node that OML/OMF toolkits are running on.
    232   - More information about "/tmp/of-topo-setup/lb-setup":
    233    "lb-setup" is to setup the load balancing switch. The source code as well as explanation is as follows:
    234    {{{
    235    #!/bin/sh
    236 
    237    /tmp/of-topo-setup/prep-trema       # install all libraries for trema
    238    /tmp/of-topo-setup/ovs-start           # create ovs bridge
    239 
    240    cp /usr/bin/trace-oml2 /usr/bin/trace        # a hack to the current LabWiki --> needs to be fixed
    241    cp /usr/bin/nmetrics-oml2 /usr/bin/nmetrics       # a hack to the current LabWiki --> needs to be fixed
    242    # download the load balancing openflow controller source code to user directory
    243    wget http://www.gpolab.bbn.com/experiment-support/OpenFlowExampleExperiment/ExoGENI/load-balancer.rb -O /root/load-balancer.rb
    244 
    245    INTERFACES="192.168.1.1 192.168.2.1 192.168.3.1"
    246 
    247    # wait until all interfaces are up, then fetch the mapping from interface name to its ip/MAC address and save this info in a file /tmp/ifmap
    248    /tmp/of-topo-setup/writeifmap3
    249 
    250    # add port to the ovs bridge
    251    /tmp/of-topo-setup/find-interfaces $INTERFACES | while read iface; do
    252        ovs-vsctl add-port br0 $iface < /dev/null
    253    done
    254 
    255    # create port map save it to /tmp/portmap
    256    ovs-ofctl show tcp:127.0.0.1:6634 \
    257        | /tmp/of-topo-setup/ovs-id-ports 192.168.1.1=outside 192.168.2.1=left 192.168.3.1=right \
    258        > /tmp/portmap
    259    }}}
    260 
    261 == C. About the GIMI script you run on !LabWiki ==
    262  - Line 1 to Line 128: the definition of oml trace and oml nmetrics library. It basically defines the command line options for oml2-trace and oml2-nmetrics, as well as the output (the monitoring data that is going to be stored into the oml server)
    263   - users are not supposed to modify them
    264   - the definition here we used is not the same as what is provided by the latest OML2 2.10.0 library because there is some version mis-match between the OMF that !LabWiki is using and the OML2 toolkit that we are using. It is a temporary hack for now --> to be fixed
    265   - we added the definition of option "--oml-config" for trace app (Line 27-28) so that oml2-trace accepts configuration files:
    266   {{{
    267   app.defProperty('config', 'config file to follow', '--oml-config',
    268                   :type => :string, :default => '"/tmp/monitor/conf.xml"')
    269   }}}
    270  - Line 134 to Line 137: user defines the monitoring interfaces here. In our case, we want to monitor the interface on node "Switch" that connects to the left path (with IP 192.168.2.2) and to the right path (with IP 192.168.3.1)
    271  - Line 139 to Line 169: defines on which node the user wants to run which monitoring app; and the "display graph" option.
    272   - group "Monitor" monitors the left path statistics using nmetrics and trace.
    273   - group "Monitor1" monitors the right path statistics using nmetrics and trace.
    274   - To monitor the throughput information, we used oml2-trace with the option of "--oml-config" which uses the configuration file we created at /tmp/monitor/conf.xml, which simply sums up the number of tcp_packet_size (in Bytes) for each second and save the info into the OML Server (in a Postgre database):
    275   {{{
    276 <omlc id="switch" encoding="binary">
    277   <collect url="tcp:emmy9.casa.umass.edu:3004" name="traffic">
    278     <stream mp="tcp" interval="1">
    279       <filter field="tcp_packet_size" operation="sum" rename="tcp_throughput" />
    280     </stream>
    281   </collect>
    282 </omlc>
    283   }}}
    284   - More information about nmetrics and trace can be found here: http://oml.mytestbed.net/projects/omlapp/wiki/OML-instrumented_Applications#Packet-tracer-trace-oml2
    285  - Line 173 to Line 218: defines the experiment:
    286   - Line 175-177: starts the monitoring app
    287   - Line 179-181: starts the TCP receiver (using iperf)
    288   - Line 183-189: starts the load balancer and connects ovs switch to the load balancer (controller)
    289   - Line 191-200: starts 20 TCP flows, with 5 seconds interval between the initial of each Flow
    290   - Line 205-209: stop the load balancer controller, disconnect the ovs switch from the controller and finish the experiment
    291  - Line 217 to Line 234: defines the two graphs we want to plot:
    292   - The first uses the monitoring data from oml2-nmetrics to display the cumulated number of bytes observed from each of the interfaces;
    293   - The second graph uses the monitoring results from oml2-trace to display the throughput observed from each of the interfaces.
    294 
    295 = D. Tips: Debugging an OpenFlow Controller =
    296 You will find it helpful to know what is going on inside your OpenFlow controller and its associated switch when implementing these exercises. [[BR]]
    297 This section contains a few tips that may help you out if you are using the Open vSwitch implementation provided with this tutorial.
    298 If you are using a hardware OpenFlow switch, your instructor can help you find equivalent commands. [[BR]]
    299 The Open vSwitch installation provided by the RSpec included in this tutorial is located in ''/opt/openvswitch-1.6.1-F15''. You will find Open vSwitch commands in ''/opt/openvswitch-1.6.1-F15/bin'' and ''/opt/openvswitch-1.6.1-F15/sbin''. Some of these commands may be helpful to you. If you add these paths to your shell’s ''$PATH'', you will be able to access their manual pages with man. Note that ''$PATH'' will not affect sudo, so you will still have to provide the absolute path to sudo; the absolute path is omitted from the following examples for clarity and formatting.
    300 
    301  - '''ovs-vsctl'''[[BR]]
    302  Open vSwitch switches are primarily configured using the ''ovs-vsctl'' command. For exploring, you may find the ''ovs-vsctl show'' command useful, as it dumps the status of all virtual switches on the local Open vSwitch instance. Once you have some information on the local switch configurations, ''ovs-vsctl'' provides a broad range of capabilities that you will likely find useful for expanding your network setup to more complex configurations for testing and verification. In particular, the subcommands ''add-br'', ''add-port'', and ''set-controller'' may be of interest.
    303  - '''ovs-ofctl''' [[BR]]
    304  The switch host configured by the given rspec listens for incoming OpenFlow connections on localhost port 6634.
    305  You can use this to query the switch state using the ''ovs-ofctl'' command. In particular, you may find the ''dump-tables'' and ''dump-flows'' subcommands useful. For example, ''sudo ovs-ofctl dump-flows tcp:127.0.0.1:6634'' will output lines that look like this:
    306  {{{
    307 cookie=0x4, duration=6112.717s, table=0, n packets=1, n bytes=74, idle age=78,priority=5,tcp,
    308 nw src=10.10.10.0/24 actions=CONTROLLER:65535
    309  }}}
    310  This indicates that any TCP segment with source IP in the 10.10.10.0/24 subnet should be sent to the OpenFlow controller for processing, that it has been 78 seconds since such a segment was last seen, that one such segment has been seen so far, and the total number of bytes in packets matching this rule is 74. The other fields are perhaps interesting, but you will probably not need them for debugging. (Unless, of course, you choose to use multiple tables — an exercise in OpenFlow 1.1 functionality left to the reader.)
    311  - '''Unix utilities'''[[BR]]
    312  You will want to use a variety of Unix utilities, in addition to the tools listed in [http://groups.geni.net/geni/wiki/GENIEducation/SampleAssignments/OpenFlowAssignment/ExerciseLayout ExerciseLayout], to test your controllers. The standard ping and ''/usr/sbin/arping'' tools are useful for debugging connectivity (but make sure your controller passes ''ICMP ECHO REQUEST'' and ''REPLY'' packets and ''ARP'' traffic, respectively!), and the command ''netstat -an'' will show all active network connections on a Unix host; the TCP connections of interest in this exercise will be at the top of the listing. The format of netstat output is out of the scope of this tutorial, but information is available online and in the manual pages.
    313  - '''Linux netem''' [[BR]]
    314  Use the ''tc'' command to enable and configure delay and lossrate constraints on the outgoing interfaces for traffic traveling from the OpenFlow switch to the Aggregator node. To configure a path with a 20 ms delay and 10% lossrate on eth2, you would issue the command:
    315 {{{
    316 sudo tc qdisc add dev eth2 root handle 1:0 netem delay 20ms loss 2%
    317 }}}
    318  Use the "tc qdisc change" command to reconfigure existing links,instead of "tc qdisc add". [[BR]]
    319 
    320 = D. !LabWiki Script for the LoadBalancer =
    321 
    322 {{{
    323 
    324 }}}
    325179
    326180