| 1 | |
| 2 | [[PageOutline]] |
| 3 | |
| 4 | = INSTOOLS Project Status Report = |
| 5 | |
| 6 | Period: Post-GEC10 Report |
| 7 | |
| 8 | == I. Major accomplishments == |
| 9 | |
| 10 | The following highlights our accomplishments during the last reporting period. |
| 11 | |
| 12 | === A. Milestones achieved === |
| 13 | |
| 14 | During the last reporting period we achieved the following milestones: |
| 15 | |
| 16 | * (milestone s3.c) We gave a demonstration at GEC 10 that illustrated the |
| 17 | use of: |
| 18 | * a portal that provides single point of entry/control across various |
| 19 | aggregates, and |
| 20 | * an archival system with features to archive data for future use and |
| 21 | analysis. |
| 22 | |
| 23 | === B. Deliverables made === |
| 24 | |
| 25 | None. |
| 26 | |
| 27 | == II. Description of work performed during last quarter == |
| 28 | |
| 29 | The following provides a description of our activities and results |
| 30 | during the last report period. |
| 31 | |
| 32 | === A. Activities and findings === |
| 33 | |
| 34 | During this period of time we made significant progress on several fronts. We |
| 35 | developed a portal system to give users the look-and-feel of a single |
| 36 | interface to their data. We also implemented an archival system that can |
| 37 | store the measurement data for later retrieval and viewing. |
| 38 | |
| 39 | In order to attract more experimenters to use the GENI facilities, improving |
| 40 | the usability of the system was one of the priorities. In particular, the |
| 41 | instrumentation and measurement tool needed to collect various information from |
| 42 | different nodes of an experiment, possibly scattered across multiple |
| 43 | aggregates. The interface for presenting the information to the users is as |
| 44 | important as collecting and organizing the data. Earlier versions of the |
| 45 | INSTOOLs provided a web interface to view the measurement data collected at a |
| 46 | measurement controller (MC), which presented the data using the Drupal |
| 47 | content management system through an Apache web server. To minimize the load |
| 48 | imposed by measurement traffic, INSTOOLs allocates one MC per aggregate, |
| 49 | localizing the measurement traffic within an aggregate. This implies that |
| 50 | users must interact with multiple aggregates' MCs in order to "see" all their |
| 51 | measurement data (i.e., data being collected from all aggregages spanned by |
| 52 | the slice/experiment). |
| 53 | |
| 54 | To simplify access to a user's measurement data we developed a "portal" |
| 55 | system. The portal is a one-stop shop that allows a users to access all |
| 56 | their measurement data via a single interface. Instead of requiring the user |
| 57 | to visit multiple URLs, where each URL refers to one MC, the user visits a |
| 58 | single URL representing the portal. The portal presents a visualization of |
| 59 | the topology and allows users to select the measurement data they would like |
| 60 | to see. Given the longitude and latitude information for each resource in the |
| 61 | slice, the portal can show each resource's actual location on a map with |
| 62 | links connecting the resources. If nodes within an aggregate are too close |
| 63 | to be distinguished, the portal provides an expanded view to show the logical |
| 64 | links among them. By clicking on a node or a link, the users can get a list |
| 65 | of measurement information available. They can then choose what data to |
| 66 | view. The portal will then present the corresponding tables and/or graphs on |
| 67 | the screen. The links can also be color-coded to show the current levels of |
| 68 | traffic over the links. |
| 69 | |
| 70 | |
| 71 | Another major development we performed is to implement an archival system for |
| 72 | INSTOOLs. The measurement controller records the most recent measurement |
| 73 | data collected from an experiment. However, as the measurement data streams |
| 74 | in, we need a longer-term storage solution to archive the data for future |
| 75 | reference and analysis. Users may want to take several "snapshots" of |
| 76 | their running experiment over the lifetime of the experiment. As more |
| 77 | experimental data is collected, it is in the public interest to share these |
| 78 | data to avoid repeating experiments. The archival service provides an easy way |
| 79 | for users to store the measurement data for future use and sharing. The |
| 80 | INSTOOLs portal site provides a link to allow users to make an archive at any |
| 81 | time. It can include measurement data from a single MC or from all MCs. The |
| 82 | archival data is uniquely identified by user name, slice name, component |
| 83 | manager and timestamp. Currently our implementation uses a local server at |
| 84 | Kentucky as the archival location. However, we have been in conversation |
| 85 | with the GMOC group and the CNRI group to explore the potential of using |
| 86 | their archival services for long term measurement data storage. |
| 87 | |
| 88 | Rather than store processed measurement data (e.g., graphs and charts), our |
| 89 | system simply records the raw list of measurement numbers and then generates |
| 90 | the graphs on demand. Of course, redrawing (rendering) the graphs requires |
| 91 | reestablishing the environment in which they were originally generated. Our |
| 92 | current implementation stores the raw RRD data for the archival purpose |
| 93 | because of their smaller sizes, but then when the users retrieve these data |
| 94 | from the archival server, our archival system provides a tool to load the RRD |
| 95 | files into a (virtual) environment where the graphs and charts can be |
| 96 | recreated (i.e., by creating the context of the data that existed at the time |
| 97 | the data was archived). |
| 98 | |
| 99 | During this last reporting period we also upgraded our ProtGENI software to |
| 100 | incorporate the latest changes made by the Utah group, making the necessary |
| 101 | changes to our INSTOOLs software in order to run on the upgraded |
| 102 | aggregate. We also worked with the Utah group to support their stitching |
| 103 | effort of setting up a level 2 connection (ION) between the Utah aggregate |
| 104 | and the Kentucky aggregate. |
| 105 | |
| 106 | Finally, we gave a demo of the newly implemented portal and archival service |
| 107 | at GEC-10. |
| 108 | |
| 109 | |
| 110 | === B. Project participants === |
| 111 | |
| 112 | The following individuals have helped with the project in one way or another |
| 113 | during the last quarter: |
| 114 | * Jim Griffioen - Project PI |
| 115 | * Zongming Fei - Project Co-PI |
| 116 | * Hussamuddin Nasir - Technician/programmer |
| 117 | * Xiongqi Wu - Research Assistant |
| 118 | * Jeremy Reed - Research Assistant (half time) |
| 119 | * Lowell Pike - Network administrator |
| 120 | * Woody Marvel - Assists in Emulab administration |
| 121 | |
| 122 | |
| 123 | === C. Publications (individual and organizational) === |
| 124 | |
| 125 | None this quarter. |
| 126 | |
| 127 | === D. Outreach activities === |
| 128 | |
| 129 | * Zongming Fei gave a talk on ``Monitoring GENI Networks" at the Internet2 Joint Techs conference held at Clemson University. |
| 130 | |
| 131 | === E. Collaborations === |
| 132 | |
| 133 | Most of our collaborations continue to be with the Utah ProtoGENI team, and |
| 134 | we continue to be actively involved in the bi-weekly meetings of the ProtoGENI |
| 135 | cluster. |
| 136 | |
| 137 | === F. Other Contributions === |