Changes between Version 1 and Version 2 of GEMINI/Tutorial/GEC18/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment


Ignore:
Timestamp:
10/18/13 18:33:03 (6 years ago)
Author:
Jeanne Ohren
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC18/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment

    v1 v2  
    1 tbd
     1Navigation: [wiki:GEMINI/Tutorial/GEC18/GENI_Desktop_and_GEMINI_blipp Up]
     2
     3[[PageOutline]]
     4{{{
     5#!html
     6<h1> <font color="Blue">BLiPP and NetLogger:</font> Access active measurement probe data </h1>
     7}}}
     8
     9
     10== Overview ==
     11
     12The goal of this exercise is to configure BLiPP (Basic Lightweight Periscope Probes) and enable active measurements within your slice.  An example !NetLogger script will be used to generate data that will be collected with an active BLiPP probe.
     13
     14
     15=== The role of BLiPP ===
     16
     17BLiPP is a measurement agent that runs on every GEMINI experiment node within your slice.  Through a web interface and/or command line, you can configure BLiPP to collect both passive and active measurements from the given node.  BLiPP retrieves its configuration from a central information store known as UNIS, the contents of which we update through the configuration interfaces.  BLiPP will then store measurement data to a measurement store (MS) running on the GEMINI global node within your slice.  Each measurement is associated with one or more "event types" that identify what metric(s) are collected for a given measurement.  These event types are displayed in a drop down box next to the configured measurements with an option for graphing the collected data.
     18
     19
     20=== Configure BLiPP measurements ===
     21
     22From the GENI Desktop page, click on the [[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:launcher.png)]] and then select 'GEMINI GN' from [[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:launchermenu.png)]].  Select 'Open PS Config' from the new GN window: [[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:psconfig.png)]]
     23
     24You should then see a page similar to the below image.  Select 'Schedule BLiPP Test' from the menu.
     25
     26----
     27[[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:psconfig_main.png, 70%)]]
     28----
     29
     30Select a node to configure from the dropdown menu [[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:select_node.png, 30%)]], then select an active test to configure.
     31
     32For example, below we configure a ping test. At the moment, only a single target may be configured per test.  You can also add external hosts (i.e., outside of the slice) to ping by entering an IP or hostname in the text box.
     33
     34[[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:ping_test.png, 50%)]]
     35----
     36
     37Once configured, BLiPP will poll UNIS, update its configuration, and begin collecting the desired measurements.  The event types for each measurement will be displayed alongside each scheduled test row.  Once data is available (give it a few minutes to begin collecting!), you can select an event type and display the associated graph.
     38
     39NOTE: For throughput testing, no assumption is made about listening servers.  For example, an iperf test will attempt to connect to the specified host at each interval, but if no iperf server is listening, no result will be collected.  Future work will allow BLiPP to start a throughput listener on another node within the slice.
     40
     41
     42== Using !NetLogger ==
     43
     44!NetLogger is a set of software tools that can be used to log application events and perform analysis and debugging of the collected log data.  The python netlogger tools and libraries are installed on your slice nodes by default with GEMINI.  More details and software can be obtained from the !NetLogger website [[http://netlogger.lbl.gov/]]
     45
     46An application instrumented with !NetLogger can output log messages to a file in a standardized format.  BLiPP has a netlogger probe that can parse this log file and report the measurements to the measurement store on your global node.  In this way, measurement data collected from experiments in your slice will be in the same format as the other GEMINI measurements.  This also means they are accessible in a common location and will be archived to iRODS along with any other measurements collected through the GEMINI system.
     47
     48To configure a !NetLogger probe, you simply add another BLiPP test as shown below.
     49
     50[[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:nl_probe.png, 50%)]]
     51----
     52
     53Specify the file on the given node where you are collecting !NetLogger log messages.  BLiPP will then check the file for new messages on the specified interval and report the collected values to the measurement store.
     54
     55The event type(s) for !NetLogger probes depend on what is found in the log messages.  Since they are defined by the programmer, BLiPP does not know about them until it begins parsing the log file.  Once data is available, BLiPP will update the measurement information and display the event types in the drop down next to the scheduled test.  Give it a few minutes and then refresh the page.  Once the event types appear, you can select 'Graph' and display a graph of the collected measurements from the log file.  The following exercise will ask you to use !NetLogger on your experiment node and collect the measurement data with BLiPP.
     56
     57[[Image(wiki:GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment:chart.png, 40%)]]
     58
     59
     60== __Exercise Tasks__ ==
     61
     62For the following exercise, if you do not have your own source code that you want to try and instrument with !NetLogger, you can simply use the
     63following [http://groups.geni.net/geni/attachment/wiki/GEMINI/Tutorial/GEC17/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment/netlogger_test.py netlogger_test.py] program.
     64
     65{{{
     66#!/usr/bin/python       
     67import sys
     68import logging
     69import time
     70import random
     71from netlogger.nllog import get_logger
     72from netlogger import nlapi, nllog
     73
     74log = nllog.get_logger(".mynamespace")
     75handler = logging.StreamHandler(sys.stdout)
     76log.addHandler(handler)
     77log.set_meta(host=nlapi.get_host())
     78log.setLevel(logging.INFO)
     79
     80while True:
     81    log.info("random.number", VAL=random.random())
     82    time.sleep(1)
     83}}}
     84
     85
     86 a. '''Task 1:''' Upload the !NetLogger application/script to one of your experiment nodes.
     87 a. '''Task 2:''' Execute the application on the node.  For example:
     88{{{
     89./netlogger_test.py > /tmp/my_random
     90}}}
     91 a. '''Task 3:''' Configure a !NetLogger probe to read the log file and graph the measurements.