Changes between Initial Version and Version 1 of InstrumentationTools-April2011-status

Timestamp:

04/05/11 17:13:07 (13 years ago)

Author:

Vic Thomas

Comment:

--

InstrumentationTools-April2011-status

@@ +1 @@
+
+[[PageOutline]]
+
+= INSTOOLS Project Status Report = 
+
+Period: Post-GEC10 Report
+
+== I. Major accomplishments ==
+
+The following highlights our accomplishments during the last reporting period. 
+
+=== A. Milestones achieved ===
+
+During the last reporting period we achieved the following milestones:
+
+ * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the
+  use of:
+    *  a portal that provides single point of entry/control across various
+      aggregates, and
+    * an archival system with features to archive data for future use and
+     analysis.  
+      
+=== B. Deliverables made ===
+
+None.
+
+== II. Description of work performed during last quarter ==
+
+The following provides a description of our activities and results
+during the last report period.
+
+=== A. Activities and findings ===
+
+During this period of time we made significant progress on several fronts. We
+developed a portal system to give users the look-and-feel of a single
+interface to their data. We also implemented an archival system that can
+store the measurement data for later retrieval and viewing.
+
+In order to attract more experimenters to use the GENI facilities, improving
+the usability of the system was one of the priorities.  In particular, the
+instrumentation and measurement tool needed to collect various information from
+different nodes of an experiment, possibly scattered across multiple
+aggregates. The interface for presenting the information to the users is as
+important as collecting and organizing the data.  Earlier versions of the
+INSTOOLs provided a web interface to view the measurement data collected at a 
+measurement controller (MC), which presented the data using the Drupal
+content management system through an Apache web server. To minimize the load
+imposed by measurement traffic, INSTOOLs allocates one MC per aggregate,
+localizing the measurement traffic within an aggregate.  This implies that
+users must interact with multiple aggregates' MCs in order to "see" all their
+measurement data (i.e., data being collected from all aggregages spanned by
+the slice/experiment).
+
+To simplify access to a user's measurement data we developed a "portal"
+system.  The portal is a one-stop shop that allows a users to access all
+their measurement data via a single interface.  Instead of requiring the user
+to visit multiple URLs, where each URL refers to one MC, the user visits a
+single URL representing the portal.  The portal presents a visualization of
+the topology and allows users to select the measurement data they would like
+to see. Given the longitude and latitude information for each resource in the
+slice, the portal can show each resource's actual location on a map with
+links connecting the resources.  If nodes within an aggregate are too close
+to be distinguished, the portal provides an expanded view to show the logical
+links among them. By clicking on a node or a link, the users can get a list
+of measurement information available. They can then choose what data to
+view. The portal will then present the corresponding tables and/or graphs on
+the screen. The links can also be color-coded to show the current levels of
+traffic over the links.
+
+
+Another major development we performed is to implement an archival system for
+INSTOOLs.  The measurement controller records the most recent measurement
+data collected from an experiment.  However, as the measurement data streams
+in, we need a longer-term storage solution to archive the data for future
+reference and analysis.  Users may want to take several "snapshots" of
+their running experiment over the lifetime of the experiment.  As more
+experimental data is collected, it is in the public interest to share these
+data to avoid repeating experiments. The archival service provides an easy way
+for users to store the measurement data for future use and sharing.  The
+INSTOOLs portal site provides a link to allow users to make an archive at any
+time. It can include measurement data from a single MC or from all MCs. The
+archival data is uniquely identified by user name, slice name, component
+manager and timestamp.  Currently our implementation uses a local server at
+Kentucky as the archival location.  However, we have been in conversation
+with the GMOC group and the CNRI group to explore the potential of using
+their archival services for long term measurement data storage.
+
+Rather than store processed measurement data (e.g., graphs and charts), our
+system simply records the raw list of measurement numbers and then generates
+the graphs on demand.  Of course, redrawing (rendering) the graphs requires
+reestablishing the environment in which they were originally generated. Our
+current implementation stores the raw RRD data for the archival purpose
+because of their smaller sizes, but then when the users retrieve these data
+from the archival server, our archival system provides a tool to load the RRD
+files into a (virtual) environment where the graphs and charts can be
+recreated (i.e., by creating the context of the data that existed at the time
+the data was archived).
+
+During this last reporting period we also upgraded our ProtGENI software to
+incorporate the latest changes made by the Utah group, making the necessary
+changes to our INSTOOLs software in order to run on the upgraded
+aggregate. We also worked with the Utah group to support their stitching
+effort of setting up a level 2 connection (ION) between the Utah aggregate
+and the Kentucky aggregate. 
+
+Finally, we gave a demo of the newly implemented portal and archival service 
+at GEC-10.
+
+
+=== B. Project participants ===
+
+The following individuals have helped with the project in one way or another
+during the last quarter:
+ * Jim Griffioen - Project PI
+ * Zongming Fei - Project Co-PI
+ * Hussamuddin Nasir - Technician/programmer
+ * Xiongqi Wu - Research Assistant
+ * Jeremy Reed - Research Assistant (half time)
+ * Lowell Pike - Network administrator
+ * Woody Marvel - Assists in Emulab administration
+
+
+=== C. Publications (individual and organizational) ===
+
+None this quarter.
+
+=== D. Outreach activities ===
+
+ * Zongming Fei gave a talk on ``Monitoring GENI Networks" at the Internet2 Joint Techs conference held at Clemson University.
+
+=== E. Collaborations ===
+
+Most of our collaborations continue to be with the Utah ProtoGENI team, and
+we continue to be actively involved in the bi-weekly meetings of the ProtoGENI
+cluster.
+
+=== F. Other Contributions ===