Changes between Version 22 and Version 23 of GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData


Ignore:
Timestamp:
06/20/14 23:54:13 (5 years ago)
Author:
mkeele@indiana.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData

    v22 v23  
    4848Above is a sample server topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply.
    4949
    50 1. Access GN0:   {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]]
    51 2. Access VM:     {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]]
    52 3. Access VM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}}
     50=== Login to the Global Node ===
     51
     52{{{
     53$ ssh username@pcvm2-4.instageni.illinois.edu
     54 > Global Node
     55$ ssh username@pc2.instageni.illinois.edu -p 33339
     56 > Measurement Point
     57$ ssh username@pc2.instageni.illinois.edu -p 33340
     58 > Measurement Point
     59}}}
    5360
    5461  * If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys.
    5562 
    56 Now lets login to the Global Node. Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine. [[BR]]
     63Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine.
     64
    5765 
    58661. The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]]
     
    6876If you would like to try a periscope tool that handles your key, cert and slice uuid for you, then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js].
    6977
    70 Install Dependencies [[BR]]
     78=== Install Dependencies ===
    7179{{{
    7280$ cd /usr/src
     
    8088}}}
    8189
    82 Test Dependencies [[BR]]
     90=== Test Dependencies ===
    8391{{{
    8492$ node -v
     
    9098}}}
    9199
    92 Clone Periscope Tool [[BR]]
     100=== Clone Periscope Tool ===
    93101{{{
    94102$ cd ~
     
    97105}}}
    98106
    99 Install Peri-JS Dependencies [[BR]]
     107=== Install Peri-JS Dependencies ===
    100108{{{
    101109$ sudo npm install
     
    104112}}}
    105113
    106 Start Peri-JS [[BR]]
     114=== Start Peri-JS ===
    107115{{{
    108116$ sudo npm start
     
    111119Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool provides a RESTful API that represents your subsection of UNIS. You can query this tool using the browser or command line.
    112120
    113 1. http://pcvm2-4.instageni.illinois.edu:42424/api
     1211. http://pcvm2-4.instageni.illinois.edu:42424/api [[BR]]
    1141222. $ curl pcvm2-4.instageni.illinois.edu:42424/api
    115123
    116124Either way, when you request this URL you will see a list of routes you can query. These routes accept the same HTTP verbs as the UNIS and MS API's but Peri-JS only displays the data for your slice.
    117125
    118 Routes [[BR]]
     126=== Routes ===
    119127{{{
    120128GET /api/slice
     
    137145}}}
    138146
    139 Once some measurements are posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net. Now, lets step through the process of gathering memory usage data.
     1471. http://pcvm2-4.instageni.illinois.edu:42424/api/measurements
    140148
    141 Navigate to {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/metadata }}} to see a list of all metadata.
     149[[Image(measurement.png)]]
     150
     151Once some measurements have been posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net.
     152
     153Now, lets step through the process of gathering memory usage data.
     154
     1552. http://pcvm2-4.instageni.illinois.edu:42424/api/metadata
     156  * list of all metadata
    142157
    143158Search for "memory" and you will see every event types such as: [[BR]]
    144 "ps:tools:blipp:linux:memory:utilization:free" [[BR]]
    145 "ps:tools:blipp:linux:memory:utilization:kernel" [[BR]]
     159{{{
     160"ps:tools:blipp:linux:memory:utilization:free"
     161"ps:tools:blipp:linux:memory:utilization:kernel"
    146162"ps:tools:blipp:linux:memory:utilization:used"
     163}}}
    147164
    148 For the memory free event type copy the value for the id key and add that value to your route, {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/metadata/53a486ad377f972d2e1b8e7b }}}. This will display the single metadata object for memory free. To see the actual memory free data alter the URL to, {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/data/53a486ad377f972d2e1b8e7b }}}.
     165For the memory used event type copy the value for the id key and add that value to your route
    149166
    150 Now you should see all your data in JSON format. To reiterate, each metadata object represents an event type for a particular measurement. Navigating to a particular event type/metadata object by the metadata ID will get the single metadata object. Using that metadata ID, you can query the data route to get the data associated with that event type.
     1673. http://pcvm2-4.instageni.illinois.edu:42424/api/metadata/53a486ad377f972d2e1b8e7b
     168  * single metadata object for memory used
     169
     170[[Image(metadata.png)]]
     171
     172Replace metadata with data in the URL to get the data associated with the metadata event type
     173
     1744. http://pcvm2-4.instageni.illinois.edu:42424/api/data/53a486ad377f972d2e1b8e7b
     175  * actual memory used data
     176
     177[[Image(data.png)]]
     178
     179Now you will see all your data in JSON format. To reiterate, each metadata object represents an event type for a particular measurement. Navigating to a particular event type/metadata object by the metadata ID will get the single metadata object. Using that metadata ID, you can query the data route to get the data associated with that event type.
    151180
    152181Once you have your data you might like to visualize it to help make discoveries from the data. At this repository, [https://github.com/GENI-GEMINI/UW/blob/master/msplot_python/ms_plot.py] we provide a script to help you with data visualization.