| 1 | [[PageOutline]] |
| 2 | |
| 3 | = INSTOOLS Project Status Report = |
| 4 | |
| 5 | Period: Post-GEC11 Report |
| 6 | |
| 7 | == I. Major accomplishments == |
| 8 | |
| 9 | The following highlights our accomplishments during the last reporting period. |
| 10 | |
| 11 | === A. Milestones achieved === |
| 12 | |
| 13 | During the last reporting period we achieved the following milestones: |
| 14 | |
| 15 | * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the |
| 16 | use of: |
| 17 | * * a portal that provides single point of entry/control across various |
| 18 | aggregates, and |
| 19 | * * an archival system with features to archive data for future use and |
| 20 | analysis. |
| 21 | |
| 22 | * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the |
| 23 | * (milestone s3.d) We identified multiple GENI experiments |
| 24 | that will use our Instrumentation Tools. We made contact |
| 25 | with these experimenters and have been providing support as they |
| 26 | begin to use the tools. We established a web site |
| 27 | with documentation/tutorials/examples to help experimenters get started. |
| 28 | |
| 29 | * (milestone s3.e) We gave a tutorial of the instrumentation tools |
| 30 | (together with the Utah ProtoGENI tutorial) at GEC 11 to experimenters and |
| 31 | educators. |
| 32 | |
| 33 | * (milestone s3.f) We made the latest stable version of our software |
| 34 | available through the ProtoGENI GIT repository. |
| 35 | |
| 36 | * Other achievements include: |
| 37 | * * Modifying our design to separate out the user interface from the backend |
| 38 | services that deploy the instrumentation infrastructure, thereby enabling |
| 39 | a wider range of user interfaces. |
| 40 | * * Integrating the INSTOOLS software into the FLACK client/GUI. |
| 41 | * * Adding several enhancements to our GENI Monitoring Portal (GMP). |
| 42 | * * Simplifying the authentication process for users. |
| 43 | * * Adding support for creating archives on the UK Archive Service (UKAS) |
| 44 | and the CNRI archive service. |
| 45 | * * Adding support for both version 1 and version 2 RSPECs. |
| 46 | * * Adding support for Ubuntu Node OS images. |
| 47 | * * Improving the robustness/resilience of the tools to errors. |
| 48 | |
| 49 | === B. Deliverables made === |
| 50 | |
| 51 | * We made the latest stable version of our software available via the |
| 52 | ProtoGENI GIT repository. |
| 53 | * We posted new documentation about INSTOOLS at |
| 54 | [http://groups.geni.net/geni/wiki/INSTOOLSSummary http://groups.geni.net/geni/wiki/INSTOOLSSummary] |
| 55 | * We posted a tutorial providing step-by-step instructions |
| 56 | on how to use INSTOOLS at |
| 57 | [http://www.netlab.uky.edu/p/instools/ http://www.netlab.uky.edu/p/instools/] |
| 58 | |
| 59 | == II. Description of work performed during last quarter == |
| 60 | |
| 61 | The following provides a description of our activities and results |
| 62 | during the last report period. |
| 63 | |
| 64 | === A. Activities and findings === |
| 65 | |
| 66 | During the last reporting period we made a major effort to integrate |
| 67 | the instrumentation tools with the ProtoGENI FLACK client. |
| 68 | |
| 69 | Figure 1: The FLACK Client. |
| 70 | [[Image(flack.png)]] |
| 71 | |
| 72 | Instead of requiring users to run a set of scripts to instrumentize their |
| 73 | experiment, we modified the FLACK GUI so that users can instrumentize their |
| 74 | slice simply by clicking on a button in the GUI. |
| 75 | In particular, we added two new buttons to the FLACK GUI |
| 76 | (Figure 1), one to add |
| 77 | instrumentation to a slice (Figure 2) and the other to |
| 78 | access the portal site (Figure 3). |
| 79 | To instrumentize an existing slice, a user simply needs to click on the |
| 80 | instrumentize button in the FLACK GUI. The GUI will then talk with the |
| 81 | backend instrumentation servers to instrumentize the slice. While this is |
| 82 | occuring, information about the progress/status of the instrumentize process |
| 83 | is sent back to the GUI and can be viewed by the user. |
| 84 | After the slice has been instrumentized, the ``Go to portal" button |
| 85 | (Figure 3) is enabled, allowing the user to view the topology |
| 86 | and the traffic moving across various nodes and links. |
| 87 | The user can pick any node or link in the experiment to observe the |
| 88 | measurement data he/she is interested in, as shown in Figure 4. |
| 89 | |
| 90 | Figure 2: The Button for Instrumentation. |
| 91 | [[Image(goto_instrument.png)]] |
| 92 | |
| 93 | Figure 3: The Button for the Portal. |
| 94 | [[Image(goto_portal.png)]] |
| 95 | |
| 96 | Figure 4: The Portal Site. |
| 97 | [[Image(portal.png)]] |
| 98 | |
| 99 | The integration of the INTOOLS with the FLACK interface greatly |
| 100 | simplifies the instrumentation process for users, making it much easier |
| 101 | to use the instrumentation tools. |
| 102 | From a user's perspective, the integration also enables a sort of single |
| 103 | sign-on service in which the user authenticates to the FLACK client, but |
| 104 | then the FLACK client and the backend instrumentation manager handles |
| 105 | all the authentication on behalf of the user to the long lists of |
| 106 | services that comprise the instrumentation system. |
| 107 | The portal button on the FLACK client takes the user |
| 108 | directly to our GENI Monitoring Portal (GMP) so that the user |
| 109 | does not need to remember URLs or have to login to the GMP |
| 110 | on their own. |
| 111 | |
| 112 | To integrate the instrumentation tools with the FLACK client, |
| 113 | we redesigned our software so that the installation and deployment |
| 114 | of the instrumentation software is done by a backend instrumentation |
| 115 | manager that, like a component manager, is responsible for allocating |
| 116 | and setting up instrumentation infrastructure. As a result, the |
| 117 | user interface components of our system could be separated out |
| 118 | and integrated into the FLACK client. This involved rewriting |
| 119 | our Python scripts as flash code making ProtoGENI API calls as |
| 120 | well as XML-RPC calls to our instrumentation manager (IM). |
| 121 | We currently run an instrumentation manager (IM) at each aggregate |
| 122 | in concert with the component manager at each aggregate. |
| 123 | The IM is responsible for sshing to experimental nodes, running |
| 124 | initialization scripts, setting up the MC nodes, etc and, like a component |
| 125 | manager, has the authority needed to carry out these tasks. |
| 126 | |
| 127 | The integration of INSTOOLS with the FLACK client is |
| 128 | one of our efforts to make the INSTOOLS interface more user friendly |
| 129 | and attract more experimenters to use our tools. During this period |
| 130 | of time, with the help of GPO, we identified and provided support for |
| 131 | experiments (e.g, RIT, Buffalo) to use our instrumentation tools. |
| 132 | We updated our web site to reflect the current status of the project |
| 133 | and provided documentation/tutorials/examples to help experimenters |
| 134 | use our tools better. |
| 135 | |
| 136 | On another front, we also connected our INSTOOLS system with two |
| 137 | archival systems: (1) the University of Kentucky Archive Service (UKAS) |
| 138 | and (2) the CNRI Archive Service. (We have begun discussions with the |
| 139 | GMOC to connect to their archive service, but do not yet have support |
| 140 | for that in our implementation). |
| 141 | |
| 142 | Our UKAS is the most powerful of the services because it not only |
| 143 | provides a repository for data, but it also provides a computational |
| 144 | environment where we can recreate the same look-and-feel the user |
| 145 | had when viewing the live data. In particular, we |
| 146 | use OpenVZ containers to provide a virtualized environment in which |
| 147 | to run the drupal content management systems that was running on the MC |
| 148 | at the the time the archive was taken. As a result, the user can visit |
| 149 | the same web pages that were offered by the MC at the time the archive |
| 150 | was taken. |
| 151 | |
| 152 | We have also incorporated support for the CNRI archive service and its |
| 153 | concept of workspaces. In particular, our system is able to store |
| 154 | the data files containing measurements information into a CNRI workspace |
| 155 | associated with a slice. After the data has been stored in the workspace, |
| 156 | the system adds the necessary metadata needed by the CNRI archive to move |
| 157 | the measurement files from the workspace to the CNRI archive for permanent |
| 158 | storage. The files can then be access via the CNRI web interface. |
| 159 | |
| 160 | Together with our Utah colleagues, we gave a tutorial on the instrumentation |
| 161 | tools at GEC 11 to experimenters and potential users of our tools. |
| 162 | The response to the integrated tool was extremely positive, and we have |
| 163 | already begun to see several user using the tool since the GEC. |
| 164 | |
| 165 | |
| 166 | Finally, as in past reporting periods, we upgraded our ProtGENI software to |
| 167 | incorporate the latest changes made by the Utah group, making the necessary |
| 168 | changes to our INSTOOLs software in order to run on the upgraded aggregate. |
| 169 | We also added support for both version 1 and version 2 RSPECs, |
| 170 | we added support for Ubuntu Node OS images (something requested by many |
| 171 | users), and we performed quite a bit of testing in preparation for the |
| 172 | tutorial that greatly improved the robustness/resilience of our tools to |
| 173 | errors. The latest stable version of our software can be obtained via |
| 174 | the ProtoGENI GIT repository. |
| 175 | |
| 176 | |
| 177 | === B. Project participants === |
| 178 | |
| 179 | The following individuals have helped with the project in one way or another |
| 180 | during the last quarter: |
| 181 | * Jim Griffioen - Project PI |
| 182 | * Zongming Fei - Project Co-PI |
| 183 | * Hussamuddin Nasir - Technician/programmer |
| 184 | * Xiongqi Wu - Research Assistant |
| 185 | * Jeremy Reed - Research Assistant |
| 186 | * Lowell Pike - Network administrator |
| 187 | * Woody Marvel - Assists in Emulab administration |
| 188 | |
| 189 | |
| 190 | === C. Publications (individual and organizational) === |
| 191 | |
| 192 | None this quarter. |
| 193 | |
| 194 | === D. Outreach activities === |
| 195 | |
| 196 | * We gave a tutorial about INSTOOLS at GEC 11. |
| 197 | * We have been providing support for the early adopters of our tools. |
| 198 | |
| 199 | === E. Collaborations === |
| 200 | |
| 201 | Most of our collaborations continue to be with the Utah ProtoGENI team, and |
| 202 | we continue to be actively involved in the bi-weekly meetings of the ProtoGENI |
| 203 | cluster. |
| 204 | |
| 205 | === F. Other Contributions === |
| 206 | |