Changes between Version 21 and Version 22 of GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData
- Timestamp:
- 06/20/14 23:12:03 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData
v21 v22 37 37 Before we provide a step-by-step guide to retrieving data, lets go more in depth with these components. UNIS is a topology service that is used to describe a network but underlying this concept is a RESTful API. As you might expect UNIS supports the standard HTTP Verbs: GET, POST, PUT, and DELETE. 38 38 39 * For example,if we wanted to query all the nodes on our slice we could: GET unis_host:port/nodes [[BR]]40 * For example,if we wanted to query one particular node on our slice we could: GET unis_host:port/nodes/:id39 1. if we wanted to query all the nodes on our slice we could: GET unis_host:port/nodes [[BR]] 40 2. if we wanted to query one particular node on our slice we could: GET unis_host:port/nodes/:id 41 41 42 42 These HTTP requests to UNIS can be executed through the browser, a script, or via the command line. For a more detailed explaination of the UNIS RESTful API see [https://github.com/GENI-GEMINI/GEMINI/wiki/UNIS-REST-API]. A user can supply the same HTTP verbs to interact with the MS but remember that the MS is located on the Global Node for a given slice. For a more detailed explaination of the MS RESTful API see [https://github.com/GENI-GEMINI/GEMINI/wiki/MS-REST-API]. If you would like more information about how UNIS represents the individual components of a network see this detailed resource [http://monitor.incntre.iu.edu/docs/]. … … 46 46 [[Image(slice.png)]] 47 47 48 Above is a sample webserver topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply.48 Above is a sample server topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply. 49 49 50 * To access theGN0: {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]]51 * To access theVM: {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]]52 * To access theVM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}}50 1. Access GN0: {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]] 51 2. Access VM: {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]] 52 3. Access VM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}} 53 53 54 If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys.54 * If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys. 55 55 56 56 Now lets login to the Global Node. Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine. [[BR]] 57 57 58 *The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]]58 1. The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]] 59 59 60 *The file {{{ /usr/local/etc/certs/unis-proxy.pem }}} is the key, cert used to query the UNIS API, which is also created during the Instrumentation process.60 2. The file {{{ /usr/local/etc/certs/unis-proxy.pem }}} is the key, cert used to query the UNIS API, which is also created during the Instrumentation process. 61 61 62 62 When making a request to the UNIS API you will need to supply your key, cert and also the UUID of your slice so that you only see your subsection of UNIS. The slice UUID can be found in the "node.info" file with the following format, "auth_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b10". 63 63 64 * For example, toquery all the nodes on your slice: GET /nodes?properties.geni.slice_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b1064 1. query all the nodes on your slice: GET /nodes?properties.geni.slice_uuid=fc028e53-cfbf-463d-9bf3-d26c35333b10 65 65 66 If you're going to be using a command line tool, such as curl, you will need to follow this structure to retrieve information from UNIS. Fortunately, we provide a python script [https://github.com/periscope-ps/unis/blob/master/example/unis_client.py] to help facilitate this process.66 If you're going to be using a command line tool, such as curl, you will need to follow this structure to retrieve information from UNIS. Here, we provide a python script [https://github.com/periscope-ps/unis/blob/master/example/unis_client.py] to help facilitate this process. 67 67 68 If you would like to try a periscope tool that handles your key, cert and slice uuid for you then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js].68 If you would like to try a periscope tool that handles your key, cert and slice uuid for you, then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js]. 69 69 70 70 Install Dependencies [[BR]] 71 {{{ cd /usr/src }}} [[BR]] 72 {{{ sudo wget http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz }}} [[BR]] 73 {{{ sudo tar -xvzf node-v0.10.28.tar.gz }}} [[BR]] 74 {{{ cd node-v0.10.28 }}} [[BR]] 75 {{{ sudo ./configure }}} [[BR]] 76 {{{ sudo make }}} [[BR]] 77 {{{ sudo make install }}} [[BR]] 78 79 {{{ sudo yum install git }}} 71 {{{ 72 $ cd /usr/src 73 $ sudo wget http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz 74 $ sudo tar -xvzf node-v0.10.28.tar.gz 75 $ cd node-v0.10.28 76 $ sudo ./configure 77 $ sudo make 78 $ sudo make install 79 $ sudo yum install git 80 }}} 80 81 81 82 Test Dependencies [[BR]] 82 {{{ node -v }}} [[BR]] 83 v0.10.28 [[BR]] 84 {{{ npm -v }}} [[BR]] 85 1.4.9 [[BR]] 86 {{{ git --version }}} [[BR]] 87 git version 1.7.6.5 83 {{{ 84 $ node -v 85 > v0.10.28 86 $ npm -v 87 > 1.4.9 88 $ git --version 89 > git version 1.7.6.5 90 }}} 88 91 89 92 Clone Periscope Tool [[BR]] 90 {{{ cd ~ }}} [[BR]] 91 {{{ git clone https://github.com/periscope-ps/periscope.git }}} [[BR]] 92 {{{ cd periscope/peri-js/ }}} 93 {{{ 94 $ cd ~ 95 $ git clone https://github.com/periscope-ps/periscope.git 96 $ cd periscope/peri-js/ 97 }}} 93 98 94 99 Install Peri-JS Dependencies [[BR]] 95 {{{ sudo npm install }}} [[BR]] 96 {{{ sudo npm install -g bower }}} [[BR]] 97 {{{ bower install }}} 100 {{{ 101 $ sudo npm install 102 $ sudo npm install -g bower 103 $ bower install 104 }}} 98 105 99 106 Start Peri-JS [[BR]] 100 {{{ sudo npm start }}} 107 {{{ 108 $ sudo npm start 109 }}} 101 110 102 Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool is a RESTful API itselfthat represents your subsection of UNIS. You can query this tool using the browser or command line.111 Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool provides a RESTful API that represents your subsection of UNIS. You can query this tool using the browser or command line. 103 112 104 Browser: http://pcvm2-4.instageni.illinois.edu:42424/api [[BR]] 105 CMD:curl pcvm2-4.instageni.illinois.edu:42424/api113 1. http://pcvm2-4.instageni.illinois.edu:42424/api 114 2. $ curl pcvm2-4.instageni.illinois.edu:42424/api 106 115 107 116 Either way, when you request this URL you will see a list of routes you can query. These routes accept the same HTTP verbs as the UNIS and MS API's but Peri-JS only displays the data for your slice. 108 117 109 118 Routes [[BR]] 110 {{{ GET /api/slice }}} [[BR]] 111 {{{ GET /api/nodes }}} [[BR]] 112 {{{ GET /api/nodes/:id }}} [[BR]] 113 {{{ GET /api/services }}} [[BR]] 114 {{{ GET /api/services/:id }}} [[BR]] 115 {{{ GET /api/measurements }}} [[BR]] 116 {{{ GET /api/measurements/:id }}} [[BR]] 117 {{{ POST /api/measurements/:id }}} [[BR]] 118 {{{ GET /api/data/:id }}} [[BR]] 119 {{{ GET /api/links }}} [[BR]] 120 {{{ GET /api/links/:id }}} [[BR]] 121 {{{ GET /api/ports }}} [[BR]] 122 {{{ GET /api/ports/:id }}} [[BR]] 123 {{{ GET /api/domains }}} [[BR]] 124 {{{ GET /api/domains/:id }}} 119 {{{ 120 GET /api/slice 121 GET /api/nodes 122 GET /api/nodes/:id 123 GET /api/services 124 GET /api/services/:id 125 GET /api/measurements 126 GET /api/measurements/:id 127 POST /api/measurements/:id 128 PUT /api/measurements/:id 129 DELETE /api/measurements/:id 130 GET /api/data/:id 131 GET /api/links 132 GET /api/links/:id 133 GET /api/ports 134 GET /api/ports/:id 135 GET /api/domains 136 GET /api/domains/:id 137 }}} 125 138 126 139 Once some measurements are posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net. Now, lets step through the process of gathering memory usage data.