Changes between Version 8 and Version 9 of GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData


Ignore:
Timestamp:
06/20/14 14:15:05 (10 years ago)
Author:
mkeele@indiana.edu
Comment:

move some words around

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData

    v8 v9  
    44{{{
    55#!html
    6 <h1> Retrieving data from periscope supporting backend </h1>
     6<h1> Retrieving Data from the Periscope Supported Backend </h1>
    77}}}
    88
    99== Overview ==
    1010
    11 In the previous exercise, you may already have configured measurement probes via the GENI Desktop GUI. Here we are going to show you what happens in UNIS, the topology services that actually support and direct the measurements.
     11In the previous exercise, you may have already configured measurement probes via the GENI Desktop GUI. Here we are going to show you what happens in UNIS, which is the topology service that actually supports and directs the measurements.
    1212
    1313[[BR]]
    1414
    15 == How Components Are Connected ==
     15== How the Components are Connected ==
    1616
    1717[[Image(wiki:GEMINI/Tutorial/Images:IandM_diagram.png)]]
    1818[[BR]]
    19 The above diagram shows the components of GEMINI framework. When a user reserve a slice and the slivers on the slice, hosts are created and connections between the hosts are built as well. Next step is to initialize and instrumentize these raw resources, and during this procedure, several tools are installed to your machines. BLiPP, the probe agent is installed at each measurement points. One task that BLiPP carries out is to run various kinds of probes including ping, iperf, traceroute etc. An users can either let these agents run some default probe or manually start probes at selected times.
    20 here's the link [https://github.com/periscope-ps/blipp] to the BLiPP repo and wiki page where you can investigate more about using blipp for your special probes.
     19The above diagram shows the components of the GEMINI framework. When a user reserves a slice, along with the slivers on that slice, hosts are created and connections between these hosts are built as well. Next step is to Initialize and Instrumentize these raw resources, which will install the tools that the GEMINI framework uses. BLiPP is the probe agent that is installed at each measurement point. The major task that BLiPP carries out is running various types of probes including ping, iperf, traceroute, etc. A user can either let these agents run a default probe or manually start the probes at selected times.
     20Here's the link [https://github.com/periscope-ps/blipp] to the BLiPP code repository and here's the wiki page [https://github.com/GENI-GEMINI/GEMINI/wiki/BLiPP-Documentation] where you can investigate more about using BLiPP for your custom probes.
    2121
    2222[[BR]]
    2323
    24 == BLiPP generates Metadata and Data ==
     24== BLiPP Generates Metadata and Data from Measurements ==
    2525
    26 BLiPP runs user probes, parses the command line output and eventually uploads the result data to the measurement storage (MS). The MS is running at the Global Node and collecting data for the slice. Because there could be many kinds of measurement data collected at different experiment period, and they all can be large. We separate the data into Metadata and Data. The metadata should describe which particular measurement event a real data set belongs to and the metadata is stored at UNIS as bookkeeping information. The real data, on the other hands will be stored at the Global Node on the slices, so it should not overwhelm UNIS service. Realizing that there are three UNIS objects involved in the business: the measurement, the metadata of the measurement and the data of the metadata.
     26BLiPP runs user measurements via probes, parses the command line output, and once completed BLiPP uploads the result data to the Measurement Store (MS). The MS is running on the Global Node which will act as a collection point for the data generated on the user slice. Due to the fact that there could be many different kinds of measurement data collected at different periods in the experiment, we separate the data into Metadata and Data. The Metadata will describe which particular Measurement Event Type a real data set belongs to and this is stored in UNIS for bookkeeping and for easy retrieval of the desired information. The actual data, will be stored on the Global Node via the Measurement Store on a per user slice basis, this allows us to tie data to a slice instance. The three UNIS objects you need to be familiar with for getting a full picture of your experiment: the measurement, the metadata of the measurement and the data of the metadata.
    2727
    2828[[BR]]
    29 * The measurement object: a measurement object depict what probe the user want to use, and how to run this probe. It configures the probe command line arguments, user preference (e.g. how many times to repeat and what the interval is).[[BR]]
    30 * The metadata object: it is a handle stored at UNIS to retrieve the real data on the Global Node on each slice. It is also the key information users should specify if they need to access their raw data.[[BR]]
    31 * The data is the real output stored in the format of JSON
     29* The Measurement object: A measurement object depicts what probe the user want to use and how that probe will run. For example, probe command line arguments for user preference (e.g. how many times to repeat? and at what interval?).[[BR]]
     30* The Metadata object: Is a handle stored at UNIS to retrieve the actual data on a users Global Node, per slice. Also, this is the key information a user should specify should they need access to the raw data.[[BR]]
     31* The Data object: Is a series of key value pairs in JSON format.
    3232
    3333[[BR]]
    3434
    35 == How to Access the Data through the RESTful API (step through) ==
     35== How to Access the Data through the RESTful API ==
    3636
    3737step-by-step process of scanning the topology, looking for metadata, finding the eventTypes you are interested in (using unis_client.py or a browser), and then ms_plot.py as a simple graphing tool tht can be customized for whatever graphing the user needs