wiki:InstrumentationTools-2Q10-status

Version 1 (modified by Vic Thomas, 14 years ago) (diff)

--

INSTOOLS Project Status Report

Period: Q2 2010 - (4/1/10-6/30/10)

I. Major accomplishments

The following highlights our accomplishments during Q2 2010.

A. Milestones achieved

In the past quarter we continued to make progress toward our milestones. Some of our accomplishments include:

  • continuing to support operation of the Kentucky ProtoGENI prototype, enabling basic experimentation through the ProtoGENI clearinghouse.
  • released an initial version of our code and helped BBN install/test the code release.
  • continued to give input to the security group and discussed ways to integrate the emerging ABAC system.
  • attempted using ProtoGENI in our classes.

B. Deliverables made

  • We released an initial version of our INSTOOLS software.

II. Description of work performed during last quarter

The following provides a description of our activities and results from the last quarter.

A. Activities and findings

During this quarter we made significant progress on several fronts, including support for NetFlow data, web interface improvements, the ability to instrument an existing slice, the ability to span aggregates, an initial code release, and additional testing/evaluation.

Building on the groundwork established last quarter, we completed our implementation of support for NetFlow data collected within a user's slice. In addition to setting up the instrumentation and measurement infrastructure to collect packet data rates, we also set up the NetFlow services needed to capture (and categorize) data on a per-flow basis. Similar to our SNMP services, the NetFlow capture services send their data to the measurement controller (MC) for processing and viewing. Currently, the flows to be monitored are preconfigured for the user so that they can simply go to the web interface to see most common types of flows (e.g., between well known ports). Changing what is monitored is a manual task, but we plan to modify that in a future release.

The experience of adding a new type of information source (NetFlow) to our measurement system caused us to think about the problem of adding new information sources in the future. In particular, we wanted it to be easy for users to modify the web interface to display the data they collect on each node or link. To make this possible, we created a web page on the MC where a user can enter a parameterized command that is then used to (automatically) generate all the web pages needed to view that type of data. As a result, it is now relatively easy to incorporate new types of (collected) information into the web interface.

To provide authorized access to the data, we have temporarily added user accounts and passwords to the web interface. While we don't expect this to be a long-term solution, we needed some way to protect the data being collected now that we are beginning to have users of the system. We have been in discussions with the ABAC group and hope to make use of their authentication system to regulate access to the data. We have also looked into supporting certificates as a way to regulate access to the data and may offer that as well.

Another enhancement we made this past quarter was to make the instrumentation and measurement deployment and teardown independent of the slice setup and teardown. Previously, we had modified the Utah slice creation scripts to also startup the instrumentation system. In our most recent version of the code, slice creation and instrumentation are not tied together. Users can create a slice using any of the ProtoGENI approved ways (e.g., via scripts or via the Utah flash GUI). After the slice has been established, our new script will discover the topology of the slice and instrument it appropriately. We also created scripts to remove instrumentation from a slice when it is no longer needed.

Perhaps the most important advance we made this past quarter is the ability to instrument slices that span aggregates. Our new code identifies all the aggregates that comprise a slice, locates the component managers for those aggregates, discovers the resources used in each aggregate, and then proceeds to set up an MC in each of the aggregates. Finally, our code installs and initiates the measurement software on the resources in each of the aggregates, with measurement data from each resource being directed to the appropriate MC. Our scripts also allow a user to instrument portions of a slice by selecting which aggregates should be instrumented and which ones should not be instrumented. We are currently in the process of developing a portal that will give users the look-and-feel of a single interface to their data. In the meantime, the list of URLs representing the various aggregate web servers is returned to the user so they can go to individual MCs directly. Note that getting our code to run across aggregates also involved modifying our code to handle differences among the aggregates. At this point our code runs across the Utah and Kentucky aggregates and can be made to run across any aggregate that support OS loading of Fedora 8.

We also gave an assignment using the ProtoGENI system in our undergraduate networking class. The assignment required students to set up an experiment and collect and view measurement data using the INSTOOLS system. It took much longer than we expected to develop the assignment, and we ended up giving out the assignment very late in the semester. As a result, no students were able to finish the assignment. Although we did not learn as much as we would have liked, we did learn that it took us much longer than we expected to develop the step-by-step instructions need by the students. In short, the interface to ProtoGENI is still quite challenging to use. Presumably the new flash GUI will make this much better.

As in previous quarters, we upgraded our ProtGENI software to incorporate the latest changes made by the Utah group, making the necessary changes to our INSTOOLs software in order to run on the upgraded aggregate. This is an ongoing task.

We also created an initial release of our code which we gave to our colleagues at BBN and also released via the Utah GIT repository. BBN has installed and tested our software and was able to get it to work on our Kentucky aggregate. They were also able to get it to work on the Utah aggregate. The next step is to try it on their own internal aggregate.

B. Project participants

The following individuals have helped with the project in one way or another during the last quarter:

  • Jim Griffioen - Project PI
  • Zongming Fei - Project Co-PI
  • Hussamuddin Nasir - Technician/programmer
  • Xiongqi Wu - Research Assistant
  • Jeremy Reed - Research Assistant (half time)
  • Lowell Pike - Network administrator
  • Woody Marvel - Assists in Emulab administration

C. Publications (individual and organizational)

None this quarter.

D. Outreach activities

We released a version of our code to BBN and help them install, configure, and test the software. They were able to successfully use the system and provided feedback to improve our documentation.

Jim Griffioen attended the 2nd GENI Instrumentation and Measurement Workshop June 8-9, 2010 and participated in the discussions regarding the design of the instrumentation and measurement plane.

E. Collaborations

Most of our collaborations continue to be with the Utah ProtoGENI team, and we continue to be actively involved in the bi-weekly meetings of the ProtoGENI cluster.

We have also continued our discussions with other measurement groups including the OnTimeMeasure group at Ohio State and the S3 Monitor group at Purdue, and hope to integrate their code in the future. We have also had dicussions with the ABAC group about integrating their authorization enforcement mechanisms.

F. Other Contributions

None yet.