Changes between Initial Version and Version 1 of OTM-CustomMetric


Ignore:
Timestamp:
11/12/11 10:34:24 (12 years ago)
Author:
Prasad Calyam
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OTM-CustomMetric

    v1 v1  
     1= !OnTimeMeasure - Custom Metric Integration Documentation =
     2'''Prasad Calyam, Yingxiao Xu'''[[BR]]
     3'''October  24, 2011'''
     4
     5''Software License:'' [http://www.geni.net/wp-content/uploads/2009/02/geniprojlic.pdf ''GENI Project License (GPL)'']
     6
     7== 1. Background ==
     8This document describes the concept, steps and case studies on how experimenters can add custom measurements (at host or network levels) in their application into !OnTimeMeasure.
     9
     10!OnTimeMeasure is an extensible measurement framework for both active and passive measurements within GENI experiment slices. It provides measurement data aggregation, visualization, analysis and archive for GENI experimenters and operators. For more information, please see - [http://groups.geni.net/geni/wiki/OnTimeMeasure Project Wiki]
     11
     12To integrate your custom measurement metrics in your application into !OnTimeMeasure, you will need to provide information on: '''''Control Actors''''', '''''Control Actions''''', '''''Data Sources''''' and '''''Data Structures'''''.
     13
     14 [[Image(arch1.png, 50%)]]
     15
     16 * '''''Control Actors''''': Project members or your research experiment who can provide the “slice password” to the !OnTimeMeasure instance in an experiment slice, and would access the I&M data or share the data with other actors
     17 * '''''Data Sources''''': Data generation tools deployed in slice nodes; the tool would communicate with other nodes to perform active measurements or inherently collects passive measurements in on-going and on-demand manners
     18 * '''''Data Structures/Data Types''''': Measurement data that needs to be stored in a database with appropriate data structure and data types
     19 * '''''Control Actions''''': Control both start and stop of the data generation tool, control how to retrieve and utilize the data e.g., dashboard, plots, time series files with anomaly annotation
     20
     21Above information would be specified in a configuration file, and integrated into !OnTimeMeasure through web services.
     22
     23== 2. Steps for Custom Metric Integration ==
     24
     25=== 2.1 Specify custom metric information ===
     26
     27Let us consider an example of "!FrameLength" metric for illustration purposes below.
     28{{{
     29Metric
     30    FrameLength corresponding to size of HTTP data packets
     31Control Actor
     32    Experimenters at Purdue University who have the slice password of their !OnTimeMeasure nodes
     33Data Source
     34    Wireshark packet capture tool; command line option to collect measurements is:
     35           tshark -i eth0 -V -R "http" -T fields -e ip.src -e ip.dst -e frame.len -a duration:
     36Data Structures/Data Types
     37    The measurement results are stored in a database table with the following fields:
     38           tsharkDt (timestamp), SrcAddress (varchar(25)), DstAddress (varchar(25)), Framelen (double)
     39    The Framelen field of measurement data can also be stored in the Graphite dashboard software by specifying -
     40           !SrcAddress.Framelen
     41Control Actions
     42    Tool execution time is 100 seconds     
     43    Tool must be invoked periodically once every 10 minutes
     44    Tool execution does not conflict with other any tools
     45}}}
     46
     47=== 2.2 Compose a configuration file and add custom metric specifications ===
     48
     49In a new configuration file !FrameLengthSpecs.yaml, add the following:
     50
     51{{{
     52Metrics:
     53  FrameLength:
     54     description: Frame Length
     55     tool: Tshark
     56     metrictype: host
     57Tools:
     58  Tshark:
     59    command: sudo tshark -i eth0 -V -R "http" -T fields -e ip.src -e ip.dst -e frame.len -a duration:100
     60    runat: <SRC>
     61    dbtable:
     62          - field: [tsharkDt, SrcAddress, DstAddress, Framelen]
     63            type:  [timestamp, varchar(25), varchar(25), double]
     64    graphite:
     65          - send: [Avgframelen,SrcAddress_DstAddress.Framelen]
     66    time: 10
     67    duration: 100
     68    conflict: []
     69}}}
     70
     71=== 2.3 Compose a parser file to extract the custom metric "timeseries record" from output of measurement tool  ===
     72Please see !FrameLengthParser.pm example written in Perl.
     73
     74=== 2.4: Add custom metric configuration file and parser file into !OnTimeMeasure framework ===
     75
     76From Researcher Web-portal (!OnTimeBeacon):
     77  Login to http://ontime.oar.net. In "Request Specification" page, select the configuration file/parser file and click the "Add Metric" button.
     78 
     79  [[Image(metric.png, 50%)]]
     80
     81Or, From Command-line (!OnTimeControl):
     82  Use add_metric.py script that comes with the [http://ontime.oar.net/download/OnTimeMeasure_latest.php OnTimeControl software package].
     83
     84Options for add_metric.py script are:
     85{{{
     86-c <configuration file name>
     87-p <parser file name>
     88}}}
     89Example:
     90{{{
     91python add_metric.py –c FrameLengthSpecs.yaml -p FrameLengthParser.pm
     92}}}
     93   
     94The custom metric !FrameLength is now integrated into !OnTimeMeasure!
     95
     96=== 2.5 Add measurement tasks with the custom metric ===
     97From Researcher Web-portal (!OnTimeBeacon):
     98The custom metric will appear in the "Measurement Request Specification" page to add related measurement tasks.
     99
     100 [[Image(specification.png, 50%)]]
     101 
     102Or, From Command-line (!OnTimeControl):
     103"task_manage.py" in the [http://ontime.oar.net/download/OnTimeMeasure_latest.php OnTimeControl software package] manages the measurement tasks (i.e., requests), which are configured in a .yaml file.
     104
     105An example "measurement.yaml" is included in the !OnTimeControl package with contents:
     106{{{
     107# Measurement task settings
     108# Hint: A space is required after ':'
     109
     110##################################################
     111# Supported sampling patterns:
     112# - Periodic
     113# - RandomPoisson
     114# - RandomExponential
     115# - RandomGaussian
     116# - StratifiedRandom
     117# - Adaptive
     118##################################################
     119pattern: Periodic
     120
     121##################################################
     122# Measurement links
     123# Please use node names as source and destination
     124# Supported metrics:
     125# - RoundtripDelay
     126# - Throughput
     127# - Loss
     128# - Jitter
     129# - RouteChanges
     130##################################################
     131links:
     132- source: WASH
     133  destination: SALT
     134  metric: [Throughput, RoundtripDelay, Jitter, RouteChanges]
     135
     136- source: SALT
     137  destination: WASH
     138  metric: [Jitter, RoundtripDelay, Throughput, RouteChanges]
     139
     140- source: SALT
     141  metric: [FrameLength]
     142
     143}}}
     144
     145Create your own measurement configuration file by following the above example and comments, and then run the "task_manage.py" with parameter "-c <measurement_config_file.yaml>".
     146
     147An example:
     148{{{
     149python task_manage.py -c measurement.yaml
     150}}}
     151
     152=== 2.6: Query measurement results of the integrated custom metric ===
     153
     154From Researcher Web-portal (!OnTimeBeacon):
     155
     156 [[Image(Query.png, 50%)]]
     157
     158Or, From Command-line (!OnTimeControl):
     159Use the "query.py" in the [http://ontime.oar.net/download/OnTimeMeasure_latest.php OnTimeControl software package] that has following query options:
     160{{{
     161-f from time "YYYY-MM-DD HH:MM:SS" in UTC
     162-t to time "YYYY-MM-DD HH:MM:SS" in UTC
     163-s source node name
     164-d destination node name
     165-m metric {RountTripDelay|Throughput|Loss|Jitter}
     166-r result type {RawFiles|TimeSeries|TimeSeriesPlusAnomalies|TimeSeriesPlusForecasts}
     167}}}
     168
     169Measurement results queried are automatically downloaded into your local folder.
     170
     171An example query:
     172{{{
     173python query.py -f "2011-10-07 18:00:00" -t "2011-11-01 20:00:00" -s SALT -m FrameLength -r TimeSeries
     174}}}
     175
     176== 3. Underlying Process for Custom Metric Integration within !OnTimeMeasure ==
     177
     178[[Image(arch2.png,60%)]]
     179
     180With the slice password, the Control Actor (person or program) has access to Root Beacon and Node Beacons through the Researcher web-portal (http://ontime.oar.net) or command-line tool (!OnTimeControl).
     181
     182The information of custom metric in the configuration file will be used to appropriately create database table schema, tool wrapper and tool data parser through a "Configure Service". More specifically, the data source information is used along with the corresponding tool wrapper to invoke your measurement tool with proper arguments. The parser you uploaded will be used to interpret the raw measurement data and convert it to a time series format that corresponds to the data structure/data types you specified. The converted time series data is stored into the database. The sampling rate specified as part of control action information is used by the measurement scheduler when generating measurement schedules to provision active measurements or inherently collect passive measurements in on-going and on-demand manners.
     183
     184== 4. Case Study  ==
     185
     186== 4.1 Case Study 1: Emulating cloud dynamics for performance sensitive applications ==
     187
     188A GENI experiment being led by Purdue University aims to enable performance aware enterprise applications on the cloud and is developing adaptive systems tuned to cloud dynamics. The experiment application needs measurements of link delay and frame length across all links within a data center and across data centers.
     189
     190The experiment is conducted on machines from two different data centers in Utah and Kentucky. As illustrated in figure below, node fe1, bl1 and be1 are located in Utah; Node fe2 and bl2 are located in Kentucky. A delay node (bridge-0) is deployed on the path between fe1 and bl1 nodes to change the link delays. Instrumented web applications are running on all nodes. MySQL server is installed on be1 to host the application database.
     191
     192[[Image(Purdue.png)]]
     193
     194To see this experiment case study of !OnTimeMeasure in action, please see [http://ontime.oar.net/demo/framelength.htm demo video]. The experiment uses !OnTimeMeasure to measure all links delay across inter and intra federated sites periodically using Ping tool, and measures "!FrameLength" of http requests across all links using Tshark periodically. The measurement results are compared and studied with various other aspects of data from the experiment application. To accomplish these measurements with !OnTimeMeasure, the !RootBeacon is installed on an extra node in the experiment slice, and !NodeBeacons are installed on nodes f1, bl1, be1, fe2 and bl2 within the experiment slice. 
     195
     196=== 4.2 Case Study 2: Resource allocation in virtual desktop clouds ===
     197
     198A GENI experiment being led by The Ohio State University aims to develop utility-directed resource allocation schemes within virtual desktop clouds (VDCs). The schemes allocate resources (i.e., CPU, memory, network bandwidth) such that the global utility (i.e., combined utility of all virtual desktops at a data center) is maximized under the constraint that each virtual desktop at least meets it minimum quality requirement specified in service level agreements. The allocation adaptation is based on feedback of network health from the client, and CPU/memory load on the server.
     199
     200To see this experiment case study of !OnTimeMeasure in action, please see [http://ontime.oar.net/demo/new_metric_integration.htm demo video]. The detailed steps of !OnTimeMeasure configuration for the custom metrics in the VDC experiment are as follows.
     201
     202=== Step 1: Specify custom metric information ===
     203{{{
     204Control Actors
     205A VDC application script that requires end-to-end active measurements, passive measurements (i.e., packet capture) and VMware tools measurements to run the resource allocation schemes
     206
     207Data Source
     208        VMware provides tools to access memory, CPU, network, etc. of each host.
     209        OSU developed a wrapper script to use the VMware tools to obtain experiment measurements.
     210        The command line arguments for this script are:
     211            VDMeasurement.exe –m cpu –m mem
     212Data Structure/Data type
     213        For CPU
     214            Field: cpuDt,HostAddress, CPU
     215            Datatype: timestamp, varchar(25), varchar(25), double
     216        For Memory
     217            Field: memDt,HostAddress,MEM
     218            Datatype: timestamp, varchar(25), varchar(25), double
     219Control Action
     220        Measurements from VMware tools are queried for 10 seconds once every 30 minutes
     221}}}
     222
     223=== Step 2: Compose a configuration file and add custom metric specifications ===
     224
     225{{{
     226Metrics:
     227  CPU:
     228     description: CPU
     229     tool: PowershellVMWareScript
     230     metrictype: host
     231  Memory:
     232     description: Memeory
     233     tool: PowershellVMWareScript
     234     metrictype: host
     235
     236Tools:
     237  PowershellVMWareScript
     238    command: VDMeasurement.exe –m cpu –m mem
     239    runat: <SRC>
     240    dbtable:
     241          - field: [cpuDt,HostAddress, CPU]
     242            type:  [timestamp, varchar(25), varchar(25), double]
     243          - field: [memDt,HostAddress, MEM]
     244            type:  [timestamp, varchar(25), varchar(25), double]
     245    graphite:
     246         - send: [CPU,HostAddress.CPU]
     247         - send: [MEM,HostAddress.MEM]
     248    time: 30
     249    duration: 10
     250    conflict: []
     251
     252}}}
     253
     254=== Step 3: Add custom metric configuration file and parser file into !OnTimeMeasure framework ===
     255From Researcher Web-portal (!OnTimeBeacon): 
     256      Login to http://ontime.oar.net.  In "Request Specification" page, select the configuration file/parser file and click the "Add Metric" button.
     257
     258Or, From Command-line (!OnTimeControl):
     259      Invoke python add_metric.py –c CPUMEM.yaml -p CPUMEM.pm
     260
     261The custom metrics for the VDC GENI experiment are now integrated into !OnTimeMeasure!