wiki:InstrumentationTools-April2011-status

Version 1 (modified by Vic Thomas, 13 years ago) (diff)

--

INSTOOLS Project Status Report

Period: Post-GEC10 Report

I. Major accomplishments

The following highlights our accomplishments during the last reporting period.

A. Milestones achieved

During the last reporting period we achieved the following milestones:

  • (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the use of:
    • a portal that provides single point of entry/control across various aggregates, and
    • an archival system with features to archive data for future use and analysis.

B. Deliverables made

None.

II. Description of work performed during last quarter

The following provides a description of our activities and results during the last report period.

A. Activities and findings

During this period of time we made significant progress on several fronts. We developed a portal system to give users the look-and-feel of a single interface to their data. We also implemented an archival system that can store the measurement data for later retrieval and viewing.

In order to attract more experimenters to use the GENI facilities, improving the usability of the system was one of the priorities. In particular, the instrumentation and measurement tool needed to collect various information from different nodes of an experiment, possibly scattered across multiple aggregates. The interface for presenting the information to the users is as important as collecting and organizing the data. Earlier versions of the INSTOOLs provided a web interface to view the measurement data collected at a measurement controller (MC), which presented the data using the Drupal content management system through an Apache web server. To minimize the load imposed by measurement traffic, INSTOOLs allocates one MC per aggregate, localizing the measurement traffic within an aggregate. This implies that users must interact with multiple aggregates' MCs in order to "see" all their measurement data (i.e., data being collected from all aggregages spanned by the slice/experiment).

To simplify access to a user's measurement data we developed a "portal" system. The portal is a one-stop shop that allows a users to access all their measurement data via a single interface. Instead of requiring the user to visit multiple URLs, where each URL refers to one MC, the user visits a single URL representing the portal. The portal presents a visualization of the topology and allows users to select the measurement data they would like to see. Given the longitude and latitude information for each resource in the slice, the portal can show each resource's actual location on a map with links connecting the resources. If nodes within an aggregate are too close to be distinguished, the portal provides an expanded view to show the logical links among them. By clicking on a node or a link, the users can get a list of measurement information available. They can then choose what data to view. The portal will then present the corresponding tables and/or graphs on the screen. The links can also be color-coded to show the current levels of traffic over the links.

Another major development we performed is to implement an archival system for INSTOOLs. The measurement controller records the most recent measurement data collected from an experiment. However, as the measurement data streams in, we need a longer-term storage solution to archive the data for future reference and analysis. Users may want to take several "snapshots" of their running experiment over the lifetime of the experiment. As more experimental data is collected, it is in the public interest to share these data to avoid repeating experiments. The archival service provides an easy way for users to store the measurement data for future use and sharing. The INSTOOLs portal site provides a link to allow users to make an archive at any time. It can include measurement data from a single MC or from all MCs. The archival data is uniquely identified by user name, slice name, component manager and timestamp. Currently our implementation uses a local server at Kentucky as the archival location. However, we have been in conversation with the GMOC group and the CNRI group to explore the potential of using their archival services for long term measurement data storage.

Rather than store processed measurement data (e.g., graphs and charts), our system simply records the raw list of measurement numbers and then generates the graphs on demand. Of course, redrawing (rendering) the graphs requires reestablishing the environment in which they were originally generated. Our current implementation stores the raw RRD data for the archival purpose because of their smaller sizes, but then when the users retrieve these data from the archival server, our archival system provides a tool to load the RRD files into a (virtual) environment where the graphs and charts can be recreated (i.e., by creating the context of the data that existed at the time the data was archived).

During this last reporting period we also upgraded our ProtGENI software to incorporate the latest changes made by the Utah group, making the necessary changes to our INSTOOLs software in order to run on the upgraded aggregate. We also worked with the Utah group to support their stitching effort of setting up a level 2 connection (ION) between the Utah aggregate and the Kentucky aggregate.

Finally, we gave a demo of the newly implemented portal and archival service at GEC-10.

B. Project participants

The following individuals have helped with the project in one way or another during the last quarter:

  • Jim Griffioen - Project PI
  • Zongming Fei - Project Co-PI
  • Hussamuddin Nasir - Technician/programmer
  • Xiongqi Wu - Research Assistant
  • Jeremy Reed - Research Assistant (half time)
  • Lowell Pike - Network administrator
  • Woody Marvel - Assists in Emulab administration

C. Publications (individual and organizational)

None this quarter.

D. Outreach activities

  • Zongming Fei gave a talk on Monitoring GENI Networks" at the Internet2 Joint Techs conference held at Clemson University.

E. Collaborations

Most of our collaborations continue to be with the Utah ProtoGENI team, and we continue to be actively involved in the bi-weekly meetings of the ProtoGENI cluster.

F. Other Contributions