Changes between Initial Version and Version 1 of InstrumentationTools-April2011-status

04/05/11 17:13:07 (11 years ago)
Vic Thomas



  • InstrumentationTools-April2011-status

    v1 v1  
     4= INSTOOLS Project Status Report =
     6Period: Post-GEC10 Report
     8== I. Major accomplishments ==
     10The following highlights our accomplishments during the last reporting period.
     12=== A. Milestones achieved ===
     14During the last reporting period we achieved the following milestones:
     16 * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the
     17  use of:
     18    *  a portal that provides single point of entry/control across various
     19      aggregates, and
     20    * an archival system with features to archive data for future use and
     21     analysis. 
     23=== B. Deliverables made ===
     27== II. Description of work performed during last quarter ==
     29The following provides a description of our activities and results
     30during the last report period.
     32=== A. Activities and findings ===
     34During this period of time we made significant progress on several fronts. We
     35developed a portal system to give users the look-and-feel of a single
     36interface to their data. We also implemented an archival system that can
     37store the measurement data for later retrieval and viewing.
     39In order to attract more experimenters to use the GENI facilities, improving
     40the usability of the system was one of the priorities.  In particular, the
     41instrumentation and measurement tool needed to collect various information from
     42different nodes of an experiment, possibly scattered across multiple
     43aggregates. The interface for presenting the information to the users is as
     44important as collecting and organizing the data.  Earlier versions of the
     45INSTOOLs provided a web interface to view the measurement data collected at a
     46measurement controller (MC), which presented the data using the Drupal
     47content management system through an Apache web server. To minimize the load
     48imposed by measurement traffic, INSTOOLs allocates one MC per aggregate,
     49localizing the measurement traffic within an aggregate.  This implies that
     50users must interact with multiple aggregates' MCs in order to "see" all their
     51measurement data (i.e., data being collected from all aggregages spanned by
     52the slice/experiment).
     54To simplify access to a user's measurement data we developed a "portal"
     55system.  The portal is a one-stop shop that allows a users to access all
     56their measurement data via a single interface.  Instead of requiring the user
     57to visit multiple URLs, where each URL refers to one MC, the user visits a
     58single URL representing the portal.  The portal presents a visualization of
     59the topology and allows users to select the measurement data they would like
     60to see. Given the longitude and latitude information for each resource in the
     61slice, the portal can show each resource's actual location on a map with
     62links connecting the resources.  If nodes within an aggregate are too close
     63to be distinguished, the portal provides an expanded view to show the logical
     64links among them. By clicking on a node or a link, the users can get a list
     65of measurement information available. They can then choose what data to
     66view. The portal will then present the corresponding tables and/or graphs on
     67the screen. The links can also be color-coded to show the current levels of
     68traffic over the links.
     71Another major development we performed is to implement an archival system for
     72INSTOOLs.  The measurement controller records the most recent measurement
     73data collected from an experiment.  However, as the measurement data streams
     74in, we need a longer-term storage solution to archive the data for future
     75reference and analysis.  Users may want to take several "snapshots" of
     76their running experiment over the lifetime of the experiment.  As more
     77experimental data is collected, it is in the public interest to share these
     78data to avoid repeating experiments. The archival service provides an easy way
     79for users to store the measurement data for future use and sharing.  The
     80INSTOOLs portal site provides a link to allow users to make an archive at any
     81time. It can include measurement data from a single MC or from all MCs. The
     82archival data is uniquely identified by user name, slice name, component
     83manager and timestamp.  Currently our implementation uses a local server at
     84Kentucky as the archival location.  However, we have been in conversation
     85with the GMOC group and the CNRI group to explore the potential of using
     86their archival services for long term measurement data storage.
     88Rather than store processed measurement data (e.g., graphs and charts), our
     89system simply records the raw list of measurement numbers and then generates
     90the graphs on demand.  Of course, redrawing (rendering) the graphs requires
     91reestablishing the environment in which they were originally generated. Our
     92current implementation stores the raw RRD data for the archival purpose
     93because of their smaller sizes, but then when the users retrieve these data
     94from the archival server, our archival system provides a tool to load the RRD
     95files into a (virtual) environment where the graphs and charts can be
     96recreated (i.e., by creating the context of the data that existed at the time
     97the data was archived).
     99During this last reporting period we also upgraded our ProtGENI software to
     100incorporate the latest changes made by the Utah group, making the necessary
     101changes to our INSTOOLs software in order to run on the upgraded
     102aggregate. We also worked with the Utah group to support their stitching
     103effort of setting up a level 2 connection (ION) between the Utah aggregate
     104and the Kentucky aggregate.
     106Finally, we gave a demo of the newly implemented portal and archival service
     107at GEC-10.
     110=== B. Project participants ===
     112The following individuals have helped with the project in one way or another
     113during the last quarter:
     114 * Jim Griffioen - Project PI
     115 * Zongming Fei - Project Co-PI
     116 * Hussamuddin Nasir - Technician/programmer
     117 * Xiongqi Wu - Research Assistant
     118 * Jeremy Reed - Research Assistant (half time)
     119 * Lowell Pike - Network administrator
     120 * Woody Marvel - Assists in Emulab administration
     123=== C. Publications (individual and organizational) ===
     125None this quarter.
     127=== D. Outreach activities ===
     129 * Zongming Fei gave a talk on ``Monitoring GENI Networks" at the Internet2 Joint Techs conference held at Clemson University.
     131=== E. Collaborations ===
     133Most of our collaborations continue to be with the Utah ProtoGENI team, and
     134we continue to be actively involved in the bi-weekly meetings of the ProtoGENI
     137=== F. Other Contributions ===