Changes between Initial Version and Version 1 of InstrumentationTools-August2011-status


Ignore:
Timestamp:
08/22/11 17:25:10 (13 years ago)
Author:
griff@netlab.uky.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • InstrumentationTools-August2011-status

    v1 v1  
     1[[PageOutline]]
     2
     3= INSTOOLS Project Status Report =
     4
     5Period: Post-GEC11 Report
     6
     7== I. Major accomplishments ==
     8
     9The following highlights our accomplishments during the last reporting period.
     10
     11=== A. Milestones achieved ===
     12
     13During the last reporting period we achieved the following milestones:
     14
     15 * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the
     16  use of:
     17 * *  a portal that provides single point of entry/control across various
     18      aggregates, and
     19 * * an archival system with features to archive data for future use and
     20     analysis.
     21
     22 * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the
     23 * (milestone s3.d) We identified multiple GENI experiments
     24   that will use our Instrumentation Tools.  We made contact
     25   with these experimenters and have been providing support as they
     26   begin to use the tools.  We established a web site
     27   with documentation/tutorials/examples to help experimenters get started.
     28 
     29 * (milestone s3.e) We gave a tutorial of the instrumentation tools
     30   (together with the Utah ProtoGENI tutorial) at GEC 11 to experimenters and
     31   educators.
     32     
     33 * (milestone s3.f) We made the latest stable version of our software
     34   available through the ProtoGENI GIT repository.   
     35
     36 * Other achievements include:
     37 * * Modifying our design to separate out the user interface from the backend
     38     services that deploy the instrumentation infrastructure, thereby enabling
     39     a wider range of user interfaces.
     40 * * Integrating the INSTOOLS software into the FLACK client/GUI.
     41 * * Adding several enhancements to our GENI Monitoring Portal (GMP).
     42 * * Simplifying the authentication process for users.
     43 * * Adding support for creating archives on the UK Archive Service (UKAS)
     44      and the CNRI archive service.
     45 * * Adding support for both version 1 and version 2 RSPECs.
     46 * * Adding support for Ubuntu Node OS images.
     47 * * Improving the robustness/resilience of the tools to errors.
     48     
     49=== B. Deliverables made ===
     50
     51 * We made the latest stable version of our software available via the
     52   ProtoGENI GIT repository.
     53 * We posted new documentation about INSTOOLS at
     54   [http://groups.geni.net/geni/wiki/INSTOOLSSummary http://groups.geni.net/geni/wiki/INSTOOLSSummary]
     55 * We posted a tutorial providing step-by-step instructions
     56   on how to use INSTOOLS at
     57   [http://www.netlab.uky.edu/p/instools/ http://www.netlab.uky.edu/p/instools/]
     58
     59== II. Description of work performed during last quarter ==
     60
     61The following provides a description of our activities and results
     62during the last report period.
     63
     64=== A. Activities and findings ===
     65
     66During the last reporting period we made a major effort to integrate
     67the instrumentation tools with the ProtoGENI FLACK client.
     68
     69Figure 1: The FLACK Client.
     70[[Image(flack.png)]]
     71
     72Instead of requiring users to run a set of scripts to instrumentize their
     73experiment, we modified the FLACK GUI so that users can instrumentize their
     74slice simply by clicking on a button in the GUI.
     75In particular, we added two new buttons to the FLACK GUI
     76(Figure 1), one to add
     77instrumentation to a slice (Figure 2) and the other to
     78access the portal site (Figure 3).
     79To instrumentize an existing slice, a user simply needs to click on the
     80instrumentize button in the FLACK GUI.  The GUI will then talk with the
     81backend instrumentation servers to instrumentize the slice.  While this is
     82occuring, information about the progress/status of the instrumentize process
     83is sent back to the GUI and can be viewed by the user.
     84After the slice has been instrumentized, the ``Go to portal" button
     85(Figure 3) is enabled, allowing the user to view the topology
     86and the traffic moving across various nodes and links.
     87The user can pick any node or link in the experiment to observe the
     88measurement data he/she is interested in, as shown in Figure 4.
     89
     90Figure 2: The Button for Instrumentation.
     91[[Image(goto_instrument.png)]]
     92
     93Figure 3: The Button for the Portal.
     94[[Image(goto_portal.png)]]
     95
     96Figure 4: The Portal Site.
     97[[Image(portal.png)]]
     98
     99The integration of the INTOOLS with the FLACK interface greatly
     100simplifies the instrumentation process for users, making it much easier
     101to use the instrumentation tools.
     102From a user's perspective, the integration also enables a sort of single
     103sign-on service in which the user authenticates to the FLACK client, but
     104then the FLACK client and the backend instrumentation manager handles
     105all the authentication on behalf of the user to the long lists of
     106services that comprise the instrumentation system.
     107The portal button on the FLACK client takes the user
     108directly to our GENI Monitoring Portal (GMP) so that the user
     109does not need to remember URLs or have to login to the GMP
     110on their own.
     111
     112To integrate the instrumentation tools with the FLACK client,
     113we redesigned our software so that the installation and deployment
     114of the instrumentation software is done by a backend instrumentation
     115manager that, like a component manager, is responsible for allocating
     116and setting up instrumentation infrastructure.  As a result, the
     117user interface components of our system could be separated out
     118and integrated into the FLACK client.  This involved rewriting
     119our Python scripts as flash code making ProtoGENI API calls as
     120well as XML-RPC calls to our instrumentation manager (IM). 
     121We currently run an instrumentation manager (IM) at each aggregate
     122in concert with the component manager at each aggregate.
     123The IM is responsible for sshing to experimental nodes, running
     124initialization scripts, setting up the MC nodes, etc and, like a component
     125manager, has the authority needed to carry out these tasks.
     126
     127The integration of INSTOOLS with the FLACK client is
     128one of our efforts to make the INSTOOLS interface more user friendly
     129and attract more experimenters to use our tools. During this period
     130of time, with the help of GPO, we identified and provided support for
     131experiments (e.g, RIT, Buffalo) to use our instrumentation tools.
     132We updated our web site to reflect the current status of the project
     133and provided documentation/tutorials/examples to help experimenters
     134use our tools better.
     135
     136On another front, we also connected our INSTOOLS system with two
     137archival systems: (1) the University of Kentucky Archive Service (UKAS)
     138and (2) the CNRI Archive Service.  (We have begun discussions with the
     139GMOC to connect to their archive service, but do not yet have support
     140for that in our implementation).
     141
     142Our UKAS is the most powerful of the services because it not only
     143provides a repository for data, but it also provides a computational
     144environment where we can recreate the same look-and-feel the user
     145had when viewing the live data.  In particular, we
     146use OpenVZ containers to provide a virtualized environment in which
     147to run the drupal content management systems that was running on the MC
     148at the the time the archive was taken.  As a result, the user can visit
     149the same web pages that were offered by the MC at the time the archive
     150was taken.
     151
     152We have also incorporated support for the CNRI archive service and its
     153concept of workspaces.  In particular, our system is able to store
     154the data files containing measurements information into a CNRI workspace
     155associated with a slice.  After the data has been stored in the workspace,
     156the system adds the necessary metadata needed by the CNRI archive to move
     157the measurement files from the workspace to the CNRI archive for permanent
     158storage.  The files can then be access via the CNRI web interface.
     159
     160Together with our Utah colleagues, we gave a tutorial on the instrumentation
     161tools at GEC 11 to experimenters and potential users of our tools.
     162The response to the integrated tool was extremely positive, and we have
     163already begun to see several user using the tool since the GEC.
     164
     165
     166Finally, as in past reporting periods, we upgraded our ProtGENI software to
     167incorporate the latest changes made by the Utah group, making the necessary
     168changes to our INSTOOLs software in order to run on the upgraded aggregate.
     169We also added support for both version 1 and version 2 RSPECs,
     170we added support for Ubuntu Node OS images (something requested by many
     171users), and we performed quite a bit of testing in preparation for the
     172tutorial that greatly improved the robustness/resilience of our tools to
     173errors.  The latest stable version of our software can be obtained via
     174the ProtoGENI GIT repository.
     175
     176
     177=== B. Project participants ===
     178
     179The following individuals have helped with the project in one way or another
     180during the last quarter:
     181 * Jim Griffioen - Project PI
     182 * Zongming Fei - Project Co-PI
     183 * Hussamuddin Nasir - Technician/programmer
     184 * Xiongqi Wu - Research Assistant
     185 * Jeremy Reed - Research Assistant
     186 * Lowell Pike - Network administrator
     187 * Woody Marvel - Assists in Emulab administration
     188
     189
     190=== C. Publications (individual and organizational) ===
     191
     192None this quarter.
     193
     194=== D. Outreach activities ===
     195
     196 * We gave a tutorial about INSTOOLS at GEC 11.
     197 * We have been providing support for the early adopters of our tools.
     198
     199=== E. Collaborations ===
     200
     201Most of our collaborations continue to be with the Utah ProtoGENI team, and
     202we continue to be actively involved in the bi-weekly meetings of the ProtoGENI
     203cluster.
     204
     205=== F. Other Contributions ===
     206