Changes between Version 21 and Version 22 of GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData


Ignore:
Timestamp:
06/20/14 23:12:03 (5 years ago)
Author:
mkeele@indiana.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData

    v21 v22  
    3737Before we provide a step-by-step guide to retrieving data, lets go more in depth with these components. UNIS is a topology service that is used to describe a network but underlying this concept is a RESTful API. As you might expect UNIS supports the standard HTTP Verbs: GET, POST, PUT, and DELETE.
    3838
    39 * For example, if we wanted to query all the nodes on our slice we could: GET unis_host:port/nodes [[BR]]
    40 * For example, if we wanted to query one particular node on our slice we could: GET unis_host:port/nodes/:id
     391. if we wanted to query all the nodes on our slice we could: GET unis_host:port/nodes [[BR]]
     402. if we wanted to query one particular node on our slice we could: GET unis_host:port/nodes/:id
    4141
    4242These HTTP requests to UNIS can be executed through the browser, a script, or via the command line. For a more detailed explaination of the UNIS RESTful API see [https://github.com/GENI-GEMINI/GEMINI/wiki/UNIS-REST-API]. A user can supply the same HTTP verbs to interact with the MS but remember that the MS is located on the Global Node for a given slice. For a more detailed explaination of the MS RESTful API see [https://github.com/GENI-GEMINI/GEMINI/wiki/MS-REST-API]. If you would like more information about how UNIS represents the individual components of a network see this detailed resource [http://monitor.incntre.iu.edu/docs/].
     
    4646[[Image(slice.png)]]
    4747
    48 Above is a sample web server topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply.
     48Above is a sample server topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply.
    4949
    50 * To access the GN0:   {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]]
    51 * To access the VM:     {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]]
    52 * To access the VM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}}
     501. Access GN0:   {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]]
     512. Access VM:     {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]]
     523. Access VM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}}
    5353
    54 If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys.
     54  * If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys.
    5555 
    5656Now lets login to the Global Node. Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine. [[BR]]
    5757 
    58 * The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]]
     581. The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]]
    5959
    60 * The file {{{ /usr/local/etc/certs/unis-proxy.pem }}} is the key, cert used to query the UNIS API, which is also created during the Instrumentation process.
     602. The file {{{ /usr/local/etc/certs/unis-proxy.pem }}} is the key, cert used to query the UNIS API, which is also created during the Instrumentation process.
    6161
    6262When making a request to the UNIS API you will need to supply your key, cert and also the UUID of your slice so that you only see your subsection of UNIS. The slice UUID can be found in the "node.info" file with the following format, "auth_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b10".
    6363
    64 * For example, to query all the nodes on your slice: GET /nodes?properties.geni.slice_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b10
     641. query all the nodes on your slice: GET /nodes?properties.geni.slice_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b10
    6565
    66 If you're going to be using a command line tool, such as curl, you will need to follow this structure to retrieve information from UNIS. Fortunately, we provide a python script [https://github.com/periscope-ps/unis/blob/master/example/unis_client.py] to help facilitate this process.
     66If you're going to be using a command line tool, such as curl, you will need to follow this structure to retrieve information from UNIS. Here, we provide a python script [https://github.com/periscope-ps/unis/blob/master/example/unis_client.py] to help facilitate this process.
    6767
    68 If you would like to try a periscope tool that handles your key, cert and slice uuid for you then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js].
     68If you would like to try a periscope tool that handles your key, cert and slice uuid for you, then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js].
    6969
    7070Install Dependencies [[BR]]
    71 {{{ cd /usr/src }}} [[BR]]
    72 {{{ sudo wget http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz }}} [[BR]]
    73 {{{ sudo tar -xvzf node-v0.10.28.tar.gz }}} [[BR]]
    74 {{{ cd node-v0.10.28 }}} [[BR]]
    75 {{{ sudo ./configure }}} [[BR]]
    76 {{{ sudo make }}} [[BR]]
    77 {{{ sudo make install }}} [[BR]]
    78 
    79 {{{ sudo yum install git }}}
     71{{{
     72$ cd /usr/src
     73$ sudo wget http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz
     74$ sudo tar -xvzf node-v0.10.28.tar.gz
     75$ cd node-v0.10.28
     76$ sudo ./configure
     77$ sudo make
     78$ sudo make install
     79$ sudo yum install git
     80}}}
    8081
    8182Test Dependencies [[BR]]
    82 {{{ node -v }}} [[BR]]
    83 v0.10.28 [[BR]]
    84 {{{ npm -v }}} [[BR]]
    85 1.4.9 [[BR]]
    86 {{{ git --version }}} [[BR]]
    87 git version 1.7.6.5
     83{{{
     84$ node -v
     85 > v0.10.28
     86$ npm -v
     87 > 1.4.9
     88$ git --version
     89 > git version 1.7.6.5
     90}}}
    8891
    8992Clone Periscope Tool [[BR]]
    90 {{{ cd ~ }}} [[BR]]
    91 {{{ git clone https://github.com/periscope-ps/periscope.git }}} [[BR]]
    92 {{{ cd periscope/peri-js/ }}}
     93{{{
     94$ cd ~
     95$ git clone https://github.com/periscope-ps/periscope.git
     96$ cd periscope/peri-js/
     97}}}
    9398
    9499Install Peri-JS Dependencies [[BR]]
    95 {{{ sudo npm install }}} [[BR]]
    96 {{{ sudo npm install -g bower }}} [[BR]]
    97 {{{ bower install }}}
     100{{{
     101$ sudo npm install
     102$ sudo npm install -g bower
     103$ bower install
     104}}}
    98105
    99106Start Peri-JS [[BR]]
    100 {{{ sudo npm start }}}
     107{{{
     108$ sudo npm start
     109}}}
    101110
    102 Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool is a RESTful API itself that represents your subsection of UNIS. You can query this tool using the browser or command line.
     111Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool provides a RESTful API that represents your subsection of UNIS. You can query this tool using the browser or command line.
    103112
    104 Browser: http://pcvm2-4.instageni.illinois.edu:42424/api [[BR]]
    105 CMD: curl pcvm2-4.instageni.illinois.edu:42424/api
     1131. http://pcvm2-4.instageni.illinois.edu:42424/api
     1142. $ curl pcvm2-4.instageni.illinois.edu:42424/api
    106115
    107116Either way, when you request this URL you will see a list of routes you can query. These routes accept the same HTTP verbs as the UNIS and MS API's but Peri-JS only displays the data for your slice.
    108117
    109118Routes [[BR]]
    110 {{{ GET /api/slice }}} [[BR]]
    111 {{{ GET /api/nodes }}} [[BR]]
    112 {{{ GET /api/nodes/:id }}} [[BR]]
    113 {{{ GET /api/services }}} [[BR]]
    114 {{{ GET /api/services/:id }}} [[BR]]
    115 {{{ GET /api/measurements }}} [[BR]]
    116 {{{ GET /api/measurements/:id }}} [[BR]]
    117 {{{ POST /api/measurements/:id }}} [[BR]]
    118 {{{ GET /api/data/:id }}} [[BR]]
    119 {{{ GET /api/links }}} [[BR]]
    120 {{{ GET /api/links/:id }}} [[BR]]
    121 {{{ GET /api/ports }}} [[BR]]
    122 {{{ GET /api/ports/:id }}} [[BR]]
    123 {{{ GET /api/domains }}} [[BR]]
    124 {{{ GET /api/domains/:id }}}
     119{{{
     120GET /api/slice
     121GET /api/nodes
     122GET /api/nodes/:id
     123GET /api/services
     124GET /api/services/:id
     125GET /api/measurements
     126GET /api/measurements/:id
     127POST /api/measurements/:id
     128PUT /api/measurements/:id
     129DELETE /api/measurements/:id
     130GET /api/data/:id
     131GET /api/links
     132GET /api/links/:id
     133GET /api/ports
     134GET /api/ports/:id
     135GET /api/domains
     136GET /api/domains/:id
     137}}}
    125138
    126139Once some measurements are posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net. Now, lets step through the process of gathering memory usage data.