Changes between Version 4 and Version 5 of GEC21Agenda/OEDL/Experiment1


Ignore:
Timestamp:
10/20/14 14:41:47 (9 years ago)
Author:
thierry.rakotoarivelo@nicta.com.au
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC21Agenda/OEDL/Experiment1

    v4 v5  
    66{{{
    77#!html
    8 
    98  <div id="wrapper">
    10       <h1 id="gec21-oedltutorial2">GEC21 - OEDL Tutorial 2</h1>
     9      <h1 id="gec21-oedltutorial1">GEC21 - OEDL Tutorial 1</h1>
    1110
    1211<h1 id="overview">Overview</h1>
    1312
    14 <p>This second part shows how to design, execute, and view the results of an experiment, which dynamically reacts to measurements that are collected during its execution.</p>
     13<p>This first part shows how to design, execute, and view the results of an experiment, which dynamically reacts to a user-defined event.</p>
    1514
    16 <p>This experiment demonstrate OEDL&#8217;s capability to allow user to define events which will trigger based on measurements produced by resources invovled in the experiment. In other words, if one or more collected measurements reach a specific condition, then the event will trigger and some user-defined tasks will be executed.</p>
     15<p>This experiment demonstrate OEDL&#8217;s capability to allow user to define events which will trigger based on the state of the used resources. In other words, if one or more resources reaches a specific state, the event will trigger and some user-defined tasks will be executed.</p>
    1716
    1817<p>In this experiment:</p>
    1918
    2019<ul>
    21 <li>we use only 1 resources, which will run a sine signal generator</li>
    22 <li>this signal generators is configured to collect and report the sine values</li>
    23 <li>we define a custom event which will trigger when the absolute value of the last measurement is above an arbitrary threshold of 0.99. The Experiment Controller will periodically monitor the collected measurements to check for this condition</li>
    24 <li>we also define some tasks to execute when this event is triggered. In this case, the task is to send one ICMP ping packet to a target host (using the same instrumented ping application as in the first part of this tutorial).</li>
    25 <li>we finally display both measurements from the sine generator and the ICMP ping packets, as a graph showing the sine values and a table showing ping&#8217;s timestamp and RTT, respectively. Thus the table&#8217;s number of rows should be equal to the graph&#8217;s number of positive and negative peaks.</li>
     20<li>we start with 4 resources, i.e. 2 initial &#8216;workers&#8217; and 2 backup ones</li>
     21<li>all workers have a ping application associated to them</li>
     22<li>the 2 initial workers starts their ping applications</li>
     23<li>we define a custom event which will trigger when a running ping application is stopped. The Experiment Controller will periodically monitor the state of all resources to check for this condition</li>
     24<li>furthermore, we define a set of tasks to execute if the event is triggered. In this case, the task is to start a new ping application on one of the backup resources.</li>
     25<li>every 20 seconds, we purposely stop the ping application running on one of the initial workers</li>
     26<li>we display a graph of the ping&#8217;s RTT for each of the 4 resources and observe that a new ping instance starts when a previously running one is stopped</li>
    2627</ul>
    2728
    2829<figure>
    29 <img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_measure.fig1.png" alt="Experiment 2 Overview" />
    30 <figcaption>Experiment 2 Overview</figcaption></figure>
     30<img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_state.fig1.png" alt="Experiment 1 Overview" />
     31<figcaption>Experiment 1 Overview</figcaption></figure>
    3132
    3233
     
    4041<ul>
    4142<li>First, if you have not done it yet, login into LabWiki</li>
    42 <li>Load the &#8216;tut_event_measure.oedl&#8217; experiment file in the &#8216;Prepare&#8217; Panel of LabWiki. This file contains the OEDL script for this 1st experiment</li>
    43 <li>If you are not reading this using LabWiki, you can view this OEDL file online at: <a href="http://git.io/31AM-w">http://git.io/31AM-w</a></li>
     43<li>Load the &#8216;tut_event_state.oedl&#8217; experiment file in the &#8216;Prepare&#8217; Panel of LabWiki. This file contains the OEDL script for this 1st experiment</li>
     44<li>If you are not reading this using LabWiki, you can view this OEDL file online at: <a href="http://git.io/xr4pag">http://git.io/xr4pag</a></li>
    4445</ul>
    4546
    4647<figure>
    47 <img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_measure.fig2.png" alt="Experiment 2 OEDL Extract" />
    48 <figcaption>Experiment 2 OEDL Extract</figcaption></figure>
     48<img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_state.fig2.png" alt="Experiment 1 OEDL Extract" />
     49<figcaption>Experiment 1 OEDL Extract</figcaption></figure>
    4950
    5051
     
    5455<ol>
    5556<li><p>First, a reminder that all details on OEDL are available in the <a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6">OEDL reference page</a></p></li>
    56 <li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#loadOEDL"><strong>loadOEDL</strong></a> (line 16&#8211;17). This command includes in your OEDL experiment other external OEDL scripts. In this example, we are loading the definition of a signal generator and ping application (both instrumented with <a href="http://oml.mytestbed.net">OML</a>)</p></li>
    57 <li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defProperty-38-property-38-ensureProperty"><strong>defProperty</strong></a> (line 19&#8211;20). This command defines experiment properties (aka variables), you can set the values of these properties as parameters for each experiment trials, and access them throughout the entire experiment run. In this example, we are defining 2 properties, to hold the name of the resource to use and the target for the ping application.</p></li>
    58 <li><p><strong>Some internal variables</strong> (line 21). This is a simple Ruby command that defines an array to hold the list of signal peaks throughout the experiment&#8217;s execution. As opposed to the above defProperty variables, internal variables cannot be set at the start of each experiment trial (without having to change the content of the OEDL script itself)</p></li>
    59 <li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGroup"><strong>defGroup</strong></a> (line 23&#8211;36). This command is used to define a group of resources which we will use in this experiment. A group may contain many resources or any other group, and a resource may be included in many groups. This commands may also be used to associate a set of configurations and applications to all resources in a group. In this example, we define a first group &#8216;Generator&#8217; with only one resource, then we associate an instrumented signal generator to it. This association is done using the <a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGroup"><strong>addApplication</strong></a>. Furthermore, we also define a second group &#8216;Pinger&#8217;, which contains the same resource as the first group, but which has an instrumented ping application associated to it.</p></li>
    60 <li><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defEvent"><strong>defEvent</strong></a> (line 40&#8211;62). This command defines the name of a user&#8217;s custom event and the block of conditions which will be used to check if this event should be triggered.
     57<li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#loadOEDL"><strong>loadOEDL</strong></a> (line 12). This command is used to include in your OEDL experiment other external OEDL scripts. In this example, we are loading the definition of a ping application, which has been instrumented with <a href="http://oml.mytestbed.net">OML</a></p></li>
     58<li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defProperty-38-property-38-ensureProperty"><strong>defProperty</strong></a> (line 14&#8211;18). This command is used to define experiment properties (aka variables), you can set the values of these properties as parameters for each experiment trials, and access them throughout the entire experiment run. In this example, we are defining 5 properties, to hold the names of each of the resources that we will use and the target for the ping application.</p></li>
     59<li><p><strong>Some internal variables</strong> (line 21&#8211;29). These are classic simple Ruby commands that allow us put all our resource in a single list, then split that list into one holding the &#8216;initial&#8217; resources, and one holding the &#8216;backup&#8217; resources. As opposed to the above defProperty variables, these internal variables cannot be set at the start of each experiment trial (without having to change the content of the OEDL script itself)</p></li>
     60<li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGroup"><strong>defGroup</strong></a> (line 32&#8211;41). This command is used to define a group of resources which we will use in this experiment. A group may contain many resources or any other group, and a resource may be included in many groups. This commands may also be used to associate a set of configurations and applications to all resources in a group. In this example, we first define 4 groups (e.g. &#8216;Worker_X&#8217;), each with only one resource, then we are associating an instrumented ping to the unique resource in each group. This association is done using the <a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGroup"><strong>addApplication</strong></a>. Furthermore, we also define a final group (&#8216;Initial_Worker&#8217;), which will contain the &#8216;Worker&#8217; groups with the initial resources.</p></li>
     61<li><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defEvent"><strong>defEvent</strong></a> (line 23&#8211;53). This command defines the name of a user&#8217;s custom event and the block of conditions which will be used to check if this event should be triggered.
    6162
    6263<ul>
    63 <li>First since the measurements are being collected as the experiment is running, we need to specify how frequently the Experiment Controller should query the collected measurements to check for the conditions. This is set using the &#8216;every:&#8217; parameter of <strong>defEvent</strong> (in second).</li>
    64 <li>Within the condition block we have two different syntax to access some of the collected measurements:
    65 
    66 <ul>
    67 <li>one option (line 51) is to use the SQL syntax to directly write a query for the measurement database using the command <strong>defQuery</strong>. As many experimenter may be familiar with SQL, this option gives them a very flexible mechanism to access any collected data. The caveat is that they need to know the names of the database tables corresponding to their experiment&#8217;s measurement points. This is usually of the form &#8220;ApplicationName_MeasurementPoint&#8221; (e.g. signalgen_sin in line 51.</li>
    68 <li>the other option (line 44&#8211;45) is to use a <a href="http://sequel.jeremyevans.net/rdoc/files/doc/querying_rdoc.html">Ruby Sequel syntax</a>. The method <strong>ms(arg)</strong> returns a Measurement Point model, on which any Sequel querying methods can be applied. The resulting query is then passed to the <strong>defQuery</strong> command.</li>
     64<li>Within the condition block we have access to the &#8216;state&#8217; variable, which holds a array. Each element of that array represents a resource and is a hash of key/value pairs corresponding to each properties of that resource.</li>
     65<li>In this example, in our condition block we check for each resource if it failed before. If not we check if is an application and if it is currently stopped. If so then we add it to the list of failed resource, and we trigger the event.</li>
    6966</ul></li>
    70 <li>The returned data is an array where each element represents a measurement sample. Each sample is in turn a hash where each key represents a metric of that sample. The remaining of the condition block in this example (line 53&#8211;59) checks if the absolute value of the last sample is greather than 0.99, and triggers the event if so.</li>
    71 </ul></li>
    72 <li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#onEvent"><strong>onEvent</strong></a> (line 64&#8211;66). This command declares the set of actions to perform when a specific event is triggered. In this example, the event is our previously defined &#8220;SINE_ABOVE_THRESHOLD&#8221;. The actions to perform in this case is to select start a ping application. There is another <strong>onEvent</strong> declaration further (line 68&#8211;75), for the event &#8220;APP_UP_AND_INSTALLED&#8221;, i.e. when all resources are ready to receive commands and all applications associated to them are installed. When this event triggers, we start the signal generator application on the resource within the &#8216;Generator&#8217; group, then after 60 seconds we stop all applications and terminate the experiment trial.</p></li>
    73 <li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGraph"><strong>defGraph</strong></a> (line 77&#8211;91). This commands defines the graphs that will be displayed while the experiment trial is running. In this example, we define one graph showing the generated sine values against time, and one table showing the timestamp and RTT of from each sent ICMP ping packet. These graph and table will be updated using measurements enabled in the previous defGroup blocks.</p></li>
     67<li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#onEvent"><strong>onEvent</strong></a> (line 55&#8211;61). This command declares the set of actions to perform when a specific event is triggered. In this example, the event is our previously defined &#8220;APP_EXITED&#8221;. The actions to perform in this case is to select a backup resource and start its ping application. There is another <strong>onEvent</strong> declaration further (line 68&#8211;74), for the event &#8220;APP_UP_AND_INSTALLED&#8221;, i.e. when all resources are ready to receive commands and all applications associated to them are installed. When this event triggers, we start the ping application on the resources within the &#8216;Initial_Worker&#8217; group, then after 60 seconds we stop all applications and terminate the experiment trial.</p></li>
     68<li><p><a href="http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#defGraph"><strong>defGraph</strong></a> (line 76&#8211;83). This commands defines the graphs that will be displayed while the experiment trial is running. In this example, we define 1 graph showing the RTT values from the ping applications against time for each resources in our experiment. This graph will be drawn using measurements enabled in the previous defGroup blocks.</p></li>
    7469</ol>
    7570
     
    7873<ul>
    7974<li>After reviewing this OEDL experiment description, drag-and-drop it from the &#8220;Prepare&#8221; panel to the &#8220;Execute&#8221; panel, as described on the <a href="[http://groups.geni.net/geni/wiki/GEC21Agenda/OEDL/Introduction#Execute">LabWiki introduction page</a></li>
    80 <li>Set the values of the properties &#8216;res1&#8217; to the name of your allocated resource. Similarly set the &#8216;Slice&#8217; property to your own slice.
     75<li>Set the values of the properties &#8216;res1&#8217; to &#8216;res4&#8217; to the names of your allocated resources. Similarly set the &#8216;Slice&#8217; property to your own slice.
    8176(You can optionally decide to give a name to your experiment, if not LabWiki will assign a default unique name to it.)</li>
    8277<li>Click on the &#8220;Start Experiment&#8221; button. You will soon see output messages under the &#8220;Logging&#8221; section. Some of these messages are from the OMF Experiment Controller, which interprets your OEDL experiment description and sends corresponding commands to the resources. Other messages are from the resources themselves (either the VM nodes or the applications), reporting on configuration and command results.</li>
     
    8479
    8580<figure>
    86 <img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_measure.fig3.png" alt="Experiment 2 Execute Screenshot" />
    87 <figcaption>Experiment 2 Execute Screenshot</figcaption></figure>
     81<img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_state.fig3.png" alt="Experiment 1 Execute Screenshot" />
     82<figcaption>Experiment 1 Execute Screenshot</figcaption></figure>
    8883
    8984
     
    9489
    9590<figure>
    96 <img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_measure.fig4.png" alt="Experiment 2 Running Screenshot" />
    97 <figcaption>Experiment 2 Running Screenshot</figcaption></figure>
     91<img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_state.fig4.png" alt="Experiment 1 Running Screenshot" />
     92<figcaption>Experiment 1 Running Screenshot</figcaption></figure>
    9893
    9994
     
    10297
    10398<ul>
    104 <li>A message in the &#8220;Execute&#8221; panel will appear to inform you that the experiment execution has finished. At this stage, you should have the complete graph and table for this experiment in that panel, which should look as follows.</li>
     99<li>A message in the &#8220;Execute&#8221; panel will appear to inform you that the experiment execution has finished. At this stage, you should have the complete graphs for this experiment in that panel, which should look as follows.</li>
    105100</ul>
    106101
    107102<figure>
    108 <img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_measure.fig5.png" alt="Experiment 2 Result Screenshot" />
    109 <figcaption>Experiment 2 Result Screenshot</figcaption></figure>
     103<img src="https://raw.githubusercontent.com/mytestbed/gec_demos_tutorial/master/gec21_oedl_tutorial/wiki/tut_event_state.fig5.png" alt="Experiment 1 Result Screenshot" />
     104<figcaption>Experiment 1 Result Screenshot</figcaption></figure>
    110105
    111106
    112107
    113108<ul>
    114 <li><p>You may interact to with the graph, e.g. hover the pointer above a graph point to display the underlying data point, drag-and-drop the graph via its icon to the &#8220;Plan&#8221; panel as described in the <a href="http://groups.geni.net/geni/wiki/GEC21Agenda/OEDL/Introduction#Execute">LabWiki introduction page</a></p></li>
     109<li><p>You may interact to with these graphs, e.g. tick or un-tick the legend&#8217;s keys to display only results from the first or/and second resource, hover the pointer above a graph point to display the underlying data point, drag-and-drop the graph via its icon to the &#8220;Plan&#8221; panel as described in the <a href="http://groups.geni.net/geni/wiki/GEC21Agenda/OEDL/Introduction#Execute">LabWiki introduction page</a></p></li>
    115110<li><p>The complete data set holding the measurements collected from this experiment is stored in an SQL database. You can retrieve a copy of that database by clicking on the &#8216;Database Dump&#8217; buttom in the &#8216;Execute&#8217; panel. The format of that copy is depends on your LabWiki&#8217;s deployment configuration. It could be an iRODS dump, a Zipped archive of CSV files, a SQLite3 dump or a PostgreSQL dump. By default, it is a PostgreSQL dump.</p></li>
    116111</ul>