Changes between Version 22 and Version 23 of GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData
- Timestamp:
- 06/20/14 23:54:13 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GEMINI/Tutorial/GEC20/GENI_Desktop_and_GEMINI_data/RetrieveData
v22 v23 48 48 Above is a sample server topology that will be used in this example. The GN0 node represents the Global Node and is where the MS runs and where data is collectioned. VM and VM-0 are the Measurement Points where BLiPP executes the probes we supply. 49 49 50 1. Access GN0: {{{ ssh username@pcvm2-4.instageni.illinois.edu }}} [[BR]] 51 2. Access VM: {{{ ssh username@pc2.instageni.illinois.edu -p 33339 }}} [[BR]] 52 3. Access VM-0: {{{ ssh username@pc2.instageni.illinois.edu -p 33340 }}} 50 === Login to the Global Node === 51 52 {{{ 53 $ ssh username@pcvm2-4.instageni.illinois.edu 54 > Global Node 55 $ ssh username@pc2.instageni.illinois.edu -p 33339 56 > Measurement Point 57 $ ssh username@pc2.instageni.illinois.edu -p 33340 58 > Measurement Point 59 }}} 53 60 54 61 * If one of the machines asks for a password your ssh keys may not be in place. Visit [https://portal.geni.net/secure/profile.php] and click SSH Keys to download your keys. 55 62 56 Now lets login to the Global Node. Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine. [[BR]] 63 Your command prompt should be similar to "[username@GN0]" for confirmation. There are a few important files to note on this machine. 64 57 65 58 66 1. The file {{{ /usr/local/etc/node.info }}} contains information about your slice and what services it's using. This file can be found on any Global Node as it is created during the Instrumentation process. [[BR]] … … 68 76 If you would like to try a periscope tool that handles your key, cert and slice uuid for you, then you will need to install a dependency on the Global Node. For this example my Global Node is Fedora 15 but if you're using another image you can see more installation instructions here [https://github.com/periscope-ps/periscope/tree/master/peri-js]. 69 77 70 Install Dependencies [[BR]] 78 === Install Dependencies === 71 79 {{{ 72 80 $ cd /usr/src … … 80 88 }}} 81 89 82 Test Dependencies [[BR]] 90 === Test Dependencies === 83 91 {{{ 84 92 $ node -v … … 90 98 }}} 91 99 92 Clone Periscope Tool [[BR]] 100 === Clone Periscope Tool === 93 101 {{{ 94 102 $ cd ~ … … 97 105 }}} 98 106 99 Install Peri-JS Dependencies [[BR]] 107 === Install Peri-JS Dependencies === 100 108 {{{ 101 109 $ sudo npm install … … 104 112 }}} 105 113 106 Start Peri-JS [[BR]] 114 === Start Peri-JS === 107 115 {{{ 108 116 $ sudo npm start … … 111 119 Upon starting this tool you will see the UNIS instance you are connecting to, the MS store instance, and various information about the Global Node. The Peri-JS tool provides a RESTful API that represents your subsection of UNIS. You can query this tool using the browser or command line. 112 120 113 1. http://pcvm2-4.instageni.illinois.edu:42424/api 121 1. http://pcvm2-4.instageni.illinois.edu:42424/api [[BR]] 114 122 2. $ curl pcvm2-4.instageni.illinois.edu:42424/api 115 123 116 124 Either way, when you request this URL you will see a list of routes you can query. These routes accept the same HTTP verbs as the UNIS and MS API's but Peri-JS only displays the data for your slice. 117 125 118 Routes [[BR]] 126 === Routes === 119 127 {{{ 120 128 GET /api/slice … … 137 145 }}} 138 146 139 Once some measurements are posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net. Now, lets step through the process of gathering memory usage data. 147 1. http://pcvm2-4.instageni.illinois.edu:42424/api/measurements 140 148 141 Navigate to {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/metadata }}} to see a list of all metadata. 149 [[Image(measurement.png)]] 150 151 Once some measurements have been posted and executed by BLiPP, metadata will be generated. Every measurement object has a list of event types associated with it and you will use this event type of find the data you desire. For example, if you were to schedule a throughput test you could expect an event type for bandwidth. BLiPP executes a few measurement probes directly after Instrumentation, such as cpu, mem, and net. 152 153 Now, lets step through the process of gathering memory usage data. 154 155 2. http://pcvm2-4.instageni.illinois.edu:42424/api/metadata 156 * list of all metadata 142 157 143 158 Search for "memory" and you will see every event types such as: [[BR]] 144 "ps:tools:blipp:linux:memory:utilization:free" [[BR]] 145 "ps:tools:blipp:linux:memory:utilization:kernel" [[BR]] 159 {{{ 160 "ps:tools:blipp:linux:memory:utilization:free" 161 "ps:tools:blipp:linux:memory:utilization:kernel" 146 162 "ps:tools:blipp:linux:memory:utilization:used" 163 }}} 147 164 148 For the memory free event type copy the value for the id key and add that value to your route, {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/metadata/53a486ad377f972d2e1b8e7b }}}. This will display the single metadata object for memory free. To see the actual memory free data alter the URL to, {{{ http://pcvm2-4.instageni.illinois.edu:42424/api/data/53a486ad377f972d2e1b8e7b }}}.165 For the memory used event type copy the value for the id key and add that value to your route 149 166 150 Now you should see all your data in JSON format. To reiterate, each metadata object represents an event type for a particular measurement. Navigating to a particular event type/metadata object by the metadata ID will get the single metadata object. Using that metadata ID, you can query the data route to get the data associated with that event type. 167 3. http://pcvm2-4.instageni.illinois.edu:42424/api/metadata/53a486ad377f972d2e1b8e7b 168 * single metadata object for memory used 169 170 [[Image(metadata.png)]] 171 172 Replace metadata with data in the URL to get the data associated with the metadata event type 173 174 4. http://pcvm2-4.instageni.illinois.edu:42424/api/data/53a486ad377f972d2e1b8e7b 175 * actual memory used data 176 177 [[Image(data.png)]] 178 179 Now you will see all your data in JSON format. To reiterate, each metadata object represents an event type for a particular measurement. Navigating to a particular event type/metadata object by the metadata ID will get the single metadata object. Using that metadata ID, you can query the data route to get the data associated with that event type. 151 180 152 181 Once you have your data you might like to visualize it to help make discoveries from the data. At this repository, [https://github.com/GENI-GEMINI/UW/blob/master/msplot_python/ms_plot.py] we provide a script to help you with data visualization.