wiki:OMF5.3Experiment

Version 2 (modified by hmussman@bbn.com, 13 years ago) (diff)

--

7. Writing experiment script, and throughput experiment example

Throughput experiment design, and flow

Here we briefly describe an OMF experiment that:

  • runs on a mobile node with stand-alone OMF installation.
  • connects to the GENI WiMAX base-station using the OMF wimax driver.
  • measures RSSI (Received Signal Strength Indicator), CINR (Carrier to Interference-plus-Noise Ratio), and Avg. Transmit Power using the OMF wimax driver.
  • Asks the experimenter for a manual prompt to start the iperf server/client.
  • The experimenter uses the iperf dual mode experiments which do not require coordination between the client and server.
  • Log GPS Coordinates of the MS while it performs simultaneous or non-simultaneous uplink and downlink UDP throughput/jitter/loss measurements.
  • At the end of the experiment,Copy measurements to a given folder.

Throughput experiment script

Experiments in OMF are written in a domain-specific language called OEDL which is based on Ruby. More information about OEDL can be found at: http://mytestbed.net/projects/omf/wiki/OEDL-5-3

Here we briefly describe an OMF experiment that:

  • runs on a mobile node with stand-alone OMF installation.
  • connects to the GENI WiMAX base-station using the OMF wimax driver.
  • measures RSSI, CINR, and Avg. Transmit Power using the OMF wimax driver.
  • Asks the experimenter for a manual prompt to start the iperf server/client.
  • The experimenter uses the iperf dual mode experiments which do not require coordination between the client and server.
  • Log GPS Coordinates of the MS while it performs simultaneous or non-simultaneous uplink and downlink UDP throughput/jitter/loss measurements.
  • At the end of the experiment,Copy measurements to a given folder.

The following is the OEDL corresponding to this experiment. Next, we will go over different parts of it in more detail. Other similar experiments performing downlink/uplink TCP tests are available at http://software.geni.net/local-sw/

#----------------------------------------------------------------------
# Copyright (c) 2011 Raytheon BBN Technologies
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and/or hardware specification (the "Work") to
# deal in the Work without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Work, and to permit persons to whom the Work
# is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Work.
#
# THE WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE WORK OR THE USE OR OTHER DEALINGS
# IN THE WORK.
#----------------------------------------------------------------------


defProperty('res1', "omf.bbn.node1", "ID of a node")

# Added support for GPS
defApplication('gpslogger','gps') do |app|
  app.path="gpslogger"
  app.appPackage = "gpslogger.tar"
  app.version(1,0,0)
  app.shortDescription = "GPS Logger"
  app.description = "Retrieves GPS fix data from gpsd and feeds it into OML."
  app.defMeasurement('gps_data') do |mp|
    mp.defMetric('lat', :float)
    mp.defMetric('lon', :float)
    mp.defMetric('ele', :float)
    mp.defMetric('fix', :string)
    mp.defMetric('time', :string)
  end
end

# Added support for link status
defApplication('linkstatus_app','linkstatus') do |app|
  app.path="wiwrapper.rb"
  app.appPackage = "wiwrapper.tar"
  app.version(1,0,0)
  app.shortDescription = "Wrapper around wimaxcu link status"
  app.description = <<TEXT
This is a wrapper around wimaxcu link status command
TEXT

  app.defProperty('sampling','Sampling interval in sec', 's', {:type => :integer, :dynamic => false})
  app.defMeasurement('wimaxstat') do |m|
    m.defMetric('CenterFrequency', :long)
    m.defMetric('RSSI', :long)
    m.defMetric('CINR', :long)
    m.defMetric('TXPWR', :long)  
  end
end


defGroup('Actor', property.res1) {|node|
  node.prototype("test:proto:new_iperfudpdual", {
  'client' => '192.1.240.126',
  'interval' => '1',
  'time' => 30,
  'len' => '1440',
  'bandwidth' => '20M'
  })

  node.addApplication("gpslogger") {|app|
    app.measure('gps_data', :interval => 3)
  }
  
  node.addApplication("linkstatus_app") { |app|
    app.setProperty('sampling', 5)
    app.measure('wimaxstat')
  }

  node.net.x0.profile = '51'

}

onEvent(:ALL_UP_AND_INSTALLED) do |event|
  wait 5
  info "Press enter to continue"
  STDIN.gets
  group('Actor').startApplications
  wait 40
  group('Actor').stopApplications
  info "Experiment is done. Copying measurements to /home/geni/experiment_logs"
  info "Enter prefix: "
  prefix=STDIN.gets
  prefix.strip!
  FileUtils.cp "/tmp/#{Experiment.ID}.sq3" , "/home/geni/experiment_logs/#{prefix}_#{Experiment.ID}.sq3"
  info "files copied."
  Experiment.done
end


# support for visualization of data
addTab(:defaults)
addTab(:graph2) do |tab|
  opts = { :postfix => %{This graph shows the throughput.}, :updateEvery => 1 }
  tab.addGraph("Throughput", opts) do |g|
    data = []
    mp = ms('iperf_transfer')
    mp.project(:oml_ts_server, :size).each do |sample|
      time, tr = sample.tuple
      data << [time, tr]
    end
    
    g.addLine(data, :label => "Bandwidth")
    
  end
end

Support for GPS logging

This section adds support for tracking measurement locations using a GPS device attached to the mobile node.

# Added support for GPS
defApplication('gpslogger','gps') do |app|
  app.path="gpslogger"
  app.appPackage = "gpslogger.tar"
  app.version(1,0,0)
  app.shortDescription = "GPS Logger"
  app.description = "Retrieves GPS fix data from gpsd and feeds it into OML."
  app.defMeasurement('gps_data') do |mp|
    mp.defMetric('lat', :float)
    mp.defMetric('lon', :float)
    mp.defMetric('ele', :float)
    mp.defMetric('fix', :string)
    mp.defMetric('time', :string)
  end
end

A tar ball containing the gpslogger application has to be present in the same folder where you will run your experiment from.

Support for Link Status Measurements

This section of the experiment OEDL adds support for logging link status parameters such as RSSI, CINR, and Transmit Power.

# Added support for link status
defApplication('linkstatus_app','linkstatus') do |app|
  app.path="wiwrapper.rb"
  app.appPackage = "wiwrapper.tar"
  app.version(1,0,0)
  app.shortDescription = "Wrapper around wimaxcu link status"
  app.description = <<TEXT
This is a wrapper around wimaxcu link status command
TEXT

  app.defProperty('sampling','Sampling interval in sec', 's', {:type => :integer, :dynamic => false})
  app.defMeasurement('wimaxstat') do |m|
    m.defMetric('CenterFrequency', :long)
    m.defMetric('RSSI', :long)
    m.defMetric('CINR', :long)
    m.defMetric('TXPWR', :long)  
  end
end

The link status information is typically provided by tools associated with the specific WiMAX driver you're using. Our mobile node is equipped with an Intel WiMAX card for which a tool named 'wimaxcu' is available that provides link info.

In order to be able to properly use wimaxcu link status reports in the experiment, we will have to write an OMF wrapper around it as 'wimaxcu' does not come with OML support. To create the wrapper:

  • Create a folder named wiwrapper
  • Put the following code segment in a separate file named wiwrapper.rb, and place it in the wiwrapper folder
  • Get the file 'oml4r.rb' from http://mytestbed.net/repositories/changes/oml/lib/client/ruby/oml4r.rb
  • Put the file 'oml4r.rb' in wiwrapper folder
  • Make a tar ball of wiwrapper folder(wiwrapper.tar).
  • Put the tar ball, wiwrapper.rb, and oml4r.rb in the same directory as that of your experiment (we assume this is the directory you run your experiments from).

This is the code for wiwrapper.rb:

#----------------------------------------------------------------------
# Copyright (c) 2011 Raytheon BBN Technologies
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and/or hardware specification (the "Work") to
# deal in the Work without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Work, and to permit persons to whom the Work
# is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Work.
#
# THE WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE WORK OR THE USE OR OTHER DEALINGS
# IN THE WORK.
#----------------------------------------------------------------------

#!/usr/bin/env ruby
$LOAD_PATH<<'/usr/bin'
require 'oml4r'



APPNAME = "wimaxcu"
APPPATH = "/usr/bin/wimaxcu"
APPVERSION = "1.0"



class MPStat < OML4R::MPBase
    name :wimaxstat
    param :CenterFrequency, :type => :long
    param :RSSI, :type => :long
    param :CINR, :type => :long
	param :TXPWR,:type => :long

    # wimaxcu returns other metrics which we ignore here
end

class Wrapper

  def process_output(output)
    # wimaxcu returns a sequence of lines
    # The 1st line is a list of labels for the fields of the remaining lines
    # Each remaining line is for a detected stations, and follows the format:
    # ADDR AID CHAN RATE RSSI DBM IDLE TXSEQ RXSEQ CAPS ACAPS ERP STATE MODE
    lines = output.split("\n")
    lines.delete_at(0)
    lines.delete_at(1)
    lines.delete_at(4)
    column = Array.new
    lines.each { |row|
      nums = row.scan(/\d+|-\d+/)
      column << nums
    }

    # Inject the measurements into OML 
    MPStat.inject(column[0], column[1], column[2], column[3])
  end


  def initialize(args)
    
    # Initialise some variable specific to this wrapper
    @interval = 5

    # Now call the Init of OML4R with the command line arguments (args)
    # and a block defining the arguments specific to this wrapper
    OML4R::init(args, :appID => APPNAME) { |argParser|
      argParser.banner = "\nExecute a wrapper around #{APPNAME}\n" +
                         "Use -h or --help for a list of options\n\n" 
      argParser.on("-s", "--sampling DURATION", 
                   "Interval in second between collected samples") { |time| 
                      @interval = time 
                   }
      argParser.on_tail("-v", "--version", 
                        "Show the version\n") { |v| 
                           puts "Version: #{APPVERSION}"; exit 
                        }
    }

    # Finally do some checking specific to this wrapper
    # e.g. here we do not proceed if the user did not give us a 
    # valid interface to monitor
    #unless @interface != nil
    #  raise "You did not specify an interface to monitor! (-i option)"
    #end
  end

  def start()
    while true
      # Run the wlanconfig command with the following syntax
      #   "wlanconfig <interface> list"
      cmd = "#{APPPATH} status link"
      output = `#{cmd}`
      # Process its output
      process_output(output)
      # Wait for a given duration and loop again
      sleep @interval.to_i
    end
  end
end

begin
  app = Wrapper.new(ARGV)
  app.start()
rescue Exception => ex
  puts "Received an Exception when executing the wrapper!"
  puts "The Exception is: #{ex}\n"
end

For more information on writing wrappers in OMF/OML see: http://mytestbed.net/wiki/omf/OMLWrapperHowTo

Support for Bandwidth Measurements

We use an OMLized version of iperf to perform bi-directional simultaneous uplink/downlink tests.

Installation of the new iperf Application

The following installation works with OML-2.5.0 and OMF-5.3.

  • Remove all existing versions of iperf from the node, if they exist
  • Make sure that automake and autoconf are installed, if not,
     sudo apt-get install automake autoconf
    

Configure the new code

 cd iperf
 ./autogen.sh
 ./configure

  • check the config.log file to make sure ldd finds the required oml2 libraries.
  • Make and install the application
 make
 sudo make install
  • Upon successful compilation
     cd oml
     cat iperf.app > new_iperf.rb
    
  • Open new_iperf.rb and replace all occurrences of ':uint32' with ':long'
  • In the same file, replace all occurences of ':double' with ':float'
  • Copy the file to the application directory in OML
 cp new_iperf.rb /usr/share/omf-expctl-5.3/repository/test/app/
Creating OMF prototypes for iperf udp dualtest

cd /usr/share/omf-expctl-5.3/repository/test/proto/
  • create a file named new_iperfudpdual.rb and copy the following code in it:
    # Define a prototype
    #
    defPrototype("test:proto:new_iperfudpdual") { |p|
      p.name = "Iperf UDP Dual Send/Receive Test"
      p.description = "Nodes which send a stream of packets"
    
      # Define the properties (and their default values) that can be configured 
      # for this prototype
      #
      p.defProperty('client', 'Host to send packets to','localhost')
      p.defProperty('len', 'Size of packets','1440')
      p.defProperty('time', 'Experiment duration (sec)', 10)
      p.defProperty('dualtest','Do simultaneous Bidirectional Test', true)
      p.defProperty('interval', 'Interval between reports',1)
      p.defProperty('udp','Use UDP',true)
      p.defProperty('reportstyle','report using OML','O')
      p.defProperty('bandwidth','target udp bandwidth','20M')
      #p.defProperty('stdin','input data to be transmitted from stdin',true)  
    
      # Define the application to be installed on this type of node,
      # bind the application properties to the prototype properties,
      # and finally, define what measurements should be collected
      # for each application.
      #
      p.addApplication("test:app:new_iperf"){|new_iperf|
        new_iperf.bindProperty('client', 'client')
        new_iperf.bindProperty('time')
        new_iperf.bindProperty('dualtest')
        new_iperf.bindProperty('len')
        new_iperf.bindProperty('udp')
        new_iperf.bindProperty('interval')
        new_iperf.bindProperty('reportstyle')
        new_iperf.bindProperty('bandwidth')
     
        new_iperf.measure('transfer', :samples => 1)
        new_iperf.measure('packets', :samples =>1)
        new_iperf.measure('application', :samples =>1)
        new_iperf.measure('settings', :samples=>1)
        new_iperf.measure('connection', :samples=>1)
        new_iperf.measure('losses', :samples=>1)
        new_iperf.measure('jitter', :samples=>1) 
      }
    }
    

To gain a better understanding of prototypes and wrappers in OMF, please refer to http://mytestbed.net/wiki/1/How_to_run_an_application_using_Prototypes

Putting Everything Together

The following segment of the OEDL, puts all the pieces we described so far together. It basically defines a one-node group that is prepared to run the gps-logging, link-status-monitoring, and bandwidth tests together.

defGroup('Actor', property.res1) {|node|
  node.prototype("test:proto:new_iperfudpdual", {
  'client' => '192.1.240.126',
  'interval' => '1',
  'time' => 30,
  'len' => '1440',
  'bandwidth' => '20M'
  })

  node.addApplication("gpslogger") {|app|
    app.measure('gps_data', :interval => 3)
  }
  
  node.addApplication("linkstatus_app") { |app|
    app.setProperty('sampling', 5)
    app.measure('wimaxstat')
  }

  node.net.x0.profile = '51'

}

onEvent(:ALL_UP_AND_INSTALLED) do |event|
  wait 5
  info "Press enter to continue"
  STDIN.gets
  group('Actor').startApplications
  wait 40
  group('Actor').stopApplications
  info "Experiment is done. Copying measurements to /home/geni/experiment_logs"
  info "Enter prefix: "
  prefix=STDIN.gets
  prefix.strip!
  FileUtils.cp "/tmp/#{Experiment.ID}.sq3" , "/home/geni/experiment_logs/#{prefix}_#{Experiment.ID}.sq3"
  info "files copied."
  Experiment.done
end

Note how we have used the prototype and specified bandwidth measurement parameters. The last line, "node.net.x0.profile='51'" is a call to the OMF WiMAX driver to asking to establish a connection with the GENI base-station.

Note that at the end of the experiment, we give the chance to the user to prefix the sqlite3 file created by OML. This provides a simple but useful means to annotate the collected data. For example, we have used a naming convention to ease the post-processing of collected data. As we will discuss later, our post-analysis script assumes that the name of the collected sq3 files have the following pattern: point#{i}_*.sq3 . This prefixing chance at the end of the experiment, allows the user to follow this naming convention.

Using the Visualization Service

OMF provides means to visualize the data being collected in real-time. This web based service is available by connecting to port 4000 on the MS (http://localhost:4000) to view the live experiment data being plotted. The next segment of OEDL shows an example of using the visualization service:

# support for visualization of data
addTab(:defaults)
addTab(:graph2) do |tab|
  opts = { :postfix => %{This graph shows the amount of data being transferred.}, :updateEvery => 1 }
  tab.addGraph("Transfers", opts) do |g|
    data = []
    mp = ms('iperf_transfer')
    mp.project(:oml_ts_server, :size).each do |sample|
      time, tr = sample.tuple
      data << [time, tr]
    end
    
    g.addLine(data, :label => "Transfer (bytes)")
    
  end
end

Saving the experiment image

The experiment described above is a sample. The motivation is for experimenters to develop new experiments using the framework described above. Once the experimenters have modified the scripts and created new experiments on the Wimax MS, they will need to save their work for repeating experiments.

  • On the OMF aggregate manager machine
omf save -n <node_name>
  • Restart the Wimax MS and the MS will boot from PXE
  • The image will be saved on the OMF aggregate manager machine under /var/lib/omf-images-5.3
  • Rename the image and re-use if needed.