Changes between Initial Version and Version 1 of InstrumentationTools-2Q10-status

Timestamp:

06/29/10 17:12:46 (14 years ago)

Author:

Vic Thomas

Comment:

--

InstrumentationTools-2Q10-status

@@ +1 @@
+[[PageOutline]]
+
+= INSTOOLS Project Status Report = 
+
+Period: Q2 2010 - (4/1/10-6/30/10)
+
+== I. Major accomplishments ==
+
+The following highlights our accomplishments during Q2 2010.
+
+=== A. Milestones achieved ===
+
+In the past quarter we continued to make progress toward our milestones.
+Some of our accomplishments include:
+ * continuing to support operation of the Kentucky ProtoGENI prototype,
+   enabling basic experimentation through the ProtoGENI clearinghouse.
+ * released an initial version of our code and helped BBN install/test
+   the code release. 
+ * continued to give input to the security group and discussed ways to
+   integrate the emerging ABAC system. 
+ * attempted using ProtoGENI in our classes.
+
+=== B. Deliverables made ===
+
+ * We released an initial version of our INSTOOLS software.
+
+== II. Description of work performed during last quarter ==
+
+The following provides a description of our activities and results
+from the last quarter.
+
+=== A. Activities and findings ===
+
+During this quarter we made significant progress on several fronts, including
+support for NetFlow data, web interface improvements, the ability to
+instrument an existing slice, the ability to span aggregates, an initial code
+release, and additional testing/evaluation.
+
+Building on the groundwork established last quarter, we completed our
+implementation of support for NetFlow data collected within a user's slice.
+In addition to setting up the instrumentation and measurement infrastructure
+to collect packet data rates, we also set up the NetFlow services needed to
+capture (and categorize) data on a per-flow basis.  Similar to our SNMP
+services, the NetFlow capture services send their data to the measurement
+controller (MC) for processing and viewing.  Currently, the flows to be
+monitored are preconfigured for the user so that they can simply go to the
+web interface to see most common types of flows (e.g., between well known
+ports). Changing what is monitored is a manual task, but we plan to modify
+that in a future release.
+
+The experience of adding a new type of information source (NetFlow) to our
+measurement system caused us to think about the problem of adding new
+information sources in the future.  In particular, we wanted it to be easy
+for users to modify the web interface to display the data they collect on
+each node or link.  To make this possible, we created a web page on the MC
+where a user can enter a parameterized command that is then used to
+(automatically) generate all the web pages needed to view that type of data.
+As a result, it is now relatively easy to incorporate new types of
+(collected) information into the web interface.
+
+To provide authorized access to the data, we have temporarily added user
+accounts and passwords to the web interface.  While we don't expect this to
+be a long-term solution, we needed some way to protect the data being
+collected now that we are beginning to have users of the system.  We have
+been in discussions with the ABAC group and hope to make use of their
+authentication system to regulate access to the data.  We have also looked
+into supporting certificates as a way to regulate access to the data and may
+offer that as well.
+
+Another enhancement we made this past quarter was to make the
+instrumentation and measurement deployment and teardown independent of the
+slice setup and teardown.  Previously, we had modified the Utah slice
+creation scripts to also startup the instrumentation system.
+In our most recent version of the code, slice creation and instrumentation
+are not tied together.  Users can create a slice using any of the ProtoGENI
+approved ways (e.g., via scripts or via the Utah flash GUI).  After the slice
+has been established, our new script will discover the topology of the slice
+and instrument it appropriately. We also created scripts to remove
+instrumentation from a slice when it is no longer needed.
+
+Perhaps the most important advance we made this past quarter is the ability
+to instrument slices that span aggregates.  Our new code identifies all the
+aggregates that comprise a slice, locates the component managers for those
+aggregates, discovers the resources used in each aggregate, and then proceeds
+to set up an MC in each of the aggregates. Finally, our code
+installs and initiates the measurement software on the resources in each of
+the aggregates, with measurement data from each resource being directed to
+the appropriate MC.  Our scripts also allow a user to instrument
+portions of a slice by selecting which aggregates should be instrumented and
+which ones should not be instrumented.  We are currently in the process of
+developing a ''portal'' that will give users the look-and-feel of a single
+interface to their data.  In the meantime, the list of URLs representing the
+various aggregate web servers is returned to the user so they can go to
+individual MCs directly.  Note that getting our code to run across aggregates
+also involved modifying our code to handle differences among the aggregates.
+At this point our code runs across the Utah and Kentucky aggregates and can
+be made to run across any aggregate that support OS loading of Fedora 8.
+
+We also gave an assignment using the ProtoGENI system in our undergraduate 
+networking class. The assignment required students to set up an experiment 
+and collect and view measurement data using the INSTOOLS system. 
+It took much longer than we expected to develop the assignment, and we ended
+up giving out the assignment very late in the semester. 
+As a result, no students were able to finish the assignment.
+Although we did not learn as much as we would have liked, we did
+learn that it took us much longer than we expected to develop the
+step-by-step instructions need by the students.  In short, the interface to
+ProtoGENI is still quite challenging to use.  Presumably the new flash GUI
+will make this much better.
+
+As in previous quarters, we upgraded our ProtGENI software to incorporate the
+latest changes made by the Utah group, making the necessary changes to our
+INSTOOLs software in order to run on the upgraded aggregate.  This is an
+ongoing task.
+
+We also created an initial release of our code which we gave to our
+colleagues at BBN and also released via the Utah GIT repository.
+BBN has installed and tested our software and was able to get it to
+work on our Kentucky aggregate.  They were also able to get it to work
+on the Utah aggregate.  The next step is to try it on their own internal
+aggregate. 
+
+
+=== B. Project participants ===
+
+The following individuals have helped with the project in one way or another
+during the last quarter:
+ * Jim Griffioen - Project PI
+ * Zongming Fei - Project Co-PI
+ * Hussamuddin Nasir - Technician/programmer
+ * Xiongqi Wu - Research Assistant
+ * Jeremy Reed - Research Assistant (half time)
+ * Lowell Pike - Network administrator
+ * Woody Marvel - Assists in Emulab administration
+
+
+=== C. Publications (individual and organizational) ===
+
+None this quarter.
+
+=== D. Outreach activities ===
+
+We released a version of our code to BBN and help them install, configure,
+and test the software.  They were able to successfully use the system and
+provided feedback to improve our documentation.
+
+Jim Griffioen attended the 2nd GENI Instrumentation and Measurement Workshop
+June 8-9, 2010 and participated in the discussions regarding the design of
+the instrumentation and measurement plane.
+
+=== E. Collaborations ===
+
+Most of our collaborations continue to be with the Utah ProtoGENI team, and
+we continue to be actively involved in the bi-weekly meetings of the ProtoGENI
+cluster.
+
+We have also continued our discussions with other measurement groups including 
+the OnTimeMeasure group at Ohio State and the S3 Monitor group at Purdue, and
+hope to integrate their code in the future.  We have also had dicussions with
+the ABAC group about integrating their authorization enforcement mechanisms.
+
+
+=== F. Other Contributions ===
+
+None yet.