wiki:netKarma/GEC13_report

NetKarma

Report for the period ending GEC13

Beth Plale, School of Informatics and Computing, Indiana University
Chris Small, InCNTRE, Indiana University

Summary

For the quarter ending with GEC13, our major outcomes are the release of the NetKarma portal, and a successful demonstration of the benefits of provenance for a GENI experiment on DDoS in a WiMAX network. The NetKarma portal is a new web location where experimenters can go to add/retrieve from the archive experiment level data collected by the NetKarma provenance system. The NetKarma Portal is integrated with the NetKarma Provenance System and provides a persistent service on servers at Indiana University to enable GENI experimenters to help understand the conditions of their experiment. NetKarma links the experiment workflow data obtained from tools such as GUSH to representations of the experiment's GENI topology, and GENI Instrumentation and Measurement systems. NetKarma links all of these systems together to simplify the complexity of running an experiment on the GENI framework.

The DDoS analysis on WiMAX experiment carried out by Clemson University is where we focused our attention to demonstrate the value of provenance capture. Since GEC12 we replicated the Clemson experiment at IU, and instrumented N2 to capture provenance. Using the provenance captured from 700 distinct configurations, NetKarma visualizes the throughput and packets dropped and views the provenance at different levels of granularity as shown in our demo and poster at GEC13. The Clemson researchers commented that with NetKarma they can visualize and compare individual runs whereas previously their analysis was based on ANOVA calculations at an aggregate level. We have begun working with a second group to demonstrate provenance, the eXtensible Session Protocol (XSP) experiment, previously run out of UDelaware and now housed at IU.

Milestones Delivered

S4b.1 Demonstration of the provenance system being used with a GENI experiment identified in Milestone 1 Using the WiMAX DDoS experiment from Clemson we show how the provenance of packets successfully transferred and the inferred provenance of packets dropped can be used to provide visualizations of individual experiment runs; providing network researchers with a new approach to analyzing their experiment results. At GEC13 we gave a demo with a poster of the resulting provenance from the WiMAX DDoS experiment. An ancillary outcome of this work is the release of a new NetKarma adaptor that can be used to capture provenance of any experiment run on the NS2 simulator. An enhanced NetKarma plug-in for Cytoscape that contains new visualization algorithms for visualizing the provenance of NS2 experiments was released (version 1.2.1). Both the adaptor and visualization plug-in, along with updated software documentation, are available on the NetKarma wiki.
We have utilized the NetKarma Portal to greatly improve the usability and utility of the experimental data. From the portal the ingestion process just requires the experimenter drag the files related to there experiment and have the experimental data ingested into NetKarma in one easy step. In addition to the portal, the Axis2 and RabbitMQ mechanisms for adapters are supported allowing for experimenters to automate the ingestion process if desired.

S4b.2 Linking workflow provenance information to metadata We developed the NetKarma portal to link metadata such as the metadata object descriptor to the workflow information and provide the experimenter a way to navigate between the metadata and provenance information but also provide a way to visualize the measurement data that any metadata refers to and provide links to additional sources of data such as GEMINI, GIMS or the GMOC. The NetKarma portal supports ingestion of the metadata object directly or reference. For example a reference to a resource in a UNIS or NMWG file could be linked directly to the measurement data collected for that resource on the GENI topology. NetKarma provides an experimenter focused view to the data collected for the entire experiment.

S4b.3 Written plan to work with a second experiment/experimenter for evaluation of the provenance system_ We have identified the eXtensible Session Protocol (XSP) experiment led by Martin Swany at Indiana University as the second GENI project used to evaluate the provenance captured in NetKarma. In comparison to the first project we identified (i.e., WiMAX DDoS attacks based on research at Clemson), the XSP experiment has the challenge of capturing and representing the provenance of events at different network layers. The NetKarma evaluation plan has been updated to include the XSP experiment and is available here on the NetKarma wiki.

Work Performed this Quarter

S4b.4 Work with the GENI I&M community to define a standardized GENI event records

We began to examine some of the issues involved with creation of a standardized GENI event record. While NetKarma captures provenance information including "events" of the infrastructure used for a GENI experiment, NetKarma doesn't produce events but may capture events related to an experiment as part of the provenance capture. These events can then be used to create a provenance graph (e.g., identifying the process that generated a specific output from an experiment). We have worked with ExoGeni, GMOC, I&M projects and the Mesoscale monitoring efforts as possible sources of event information about the creation, resource allocation and possible outages on the experimental infrastucture.

In discussions prior to GEC13, we talked with Harry Mussman and Jeanne Ohren of BBN and Giridhar Manepalli of CNRI regarding the features that are essential in the event record. It is anticipated that a large number of events would be generated and that the event record needs to be lightweight, sharing some similarities to call records in the telecom industry. The event records will capture events in the slice including resource assignment/release, failures, and network events such as resources being up/down. We identified events from Nagios based on thresholds as an example. For some of the events (e.g., assignment of resources) the RSpec contains the information but is too heavyweight.

This also identified a need to revise the NetKarma deliverables planned for GEC14. One of the deliverables is to show NetKarma generating and populating event records. Through discussions in the I&M group, the definition of the GENI event record has evolved. As initially envisioned, the event record could possibly have been created based on provenance captured in the provenance record in NetKarma, but the event records would instead be lightweight messages published as an event occurs and in many cases would be valuable as provenance to be ingested into NetKarma instead of being generated by NetKarma.

NetKarma Portal

The NetKarma Portal provides an easy way for experimenters to:

  • Create a new experiment, pointed to by a universal handle (DOI or ARK)
  • Capture experiment workflow information from log files
  • Integrate GENI topology information to an experiment
  • Integrate measurements from GENI I&M implementations to an experiment
  • Visualize workflow, topology, measurement, status
  • Produce unified experiment data representations such as a KML "movie"
  • Interface to archival services (local and iRODS)
  • Browse the work of other experimenters (given proper permissions)

Experimental Lifecycle using the NetKarma Portal

An experimenter creates an experimental handle to reference from the front page of the NetKarma portal. Any data products from the experiment can be uploaded to NetKarma. All are stored as archives but some supported file types are parsed and additional information gathered.

Supported file types include:

  • Gush log files
  • UNIS/NMWG topology representation (used in PerfSonar and GEMINI)
  • GENI v3 RSpec files

Once the files are uploaded to the NetKarma Provenance store through the portal the workflow can be visualized both on the portal itself and through the NetKarma plugin for the Cytoscape desktop application. Experimenters can visualize the workflow of their experiments as an Open Provenance Model (OPM) graph inside the portal. Filters can be applied against attributes of the processes and artifacts to show, for example, a subset of time of each experimental run. Resources uploaded as Rspec or topology files can also be visualized both in logical or geographically based representations. Data products such as the GoogleEarth KML "tours" allow for combinations of experimental workflow with GMOC status and time series data. Finally a data access to local or remote (such as iRODS) data archive service is provided.

NetKarma Portal Experiment view

NetKarma Portal Experiment creation and artifact upload

The NetKarma Portal development page is available at: http://netkarma.testlab.grnoc.iu.edu/portal/uploader/

A permanent production URL will be available shortly.

Source code for the NetKarma portal is available through GitHub at: https://github.com/InCNTRE/NetKarma_Portal

Poster and Slides explainng the goals and features of the NetKarma Portal.

NetKarma Evaluation Based on WiMAX and Provenance Capture/Visualization Enhancements

Using the WiMAX DDoS attack research from Clemson as a GENI experiment as a basis for evaluation of provenance capture and visualization using NetKarma, we have rerun selected configurations from their research that varied the parameters identified as most significant based on their research while keeping the other parameters constant. Since their experiments utilized the NS2 simulator, we developed a NetKarma adaptor that can ingest provenance based on NS2 trace logs and an NS2 extension we have made available to capture the network topology. This new adaptor can be used with other NS2 experiments to ingest provenance into NetKarma (and is not specific to the Clemson experiments). The adaptor and documentation are available on the NetKarma wiki. To visualize the provenance of packet transfers (and infer packets dropped) in NS2 experiments, the NetKarma visualization plug-in for Cytoscape was extended with new NS2 visualization algorithms and along with updated documentation is available on the NetKarma wiki as version 1.2.1 of the visualization plug-in. The following visualizations from Cytoscape using the NetKarma plug-in illustrate the throughput and dropped packets for three runs based on Clemson's WiMAX DDoS attack research. These three runs differ for the configuration parameter that Clemson's experiments indicated had the most significant impact on throughput while other parameters were held constant.



The visualization plug-in enhancements allow the packet transfers to be viewed at different levels of granularity as illustrated in the following screen capture from Cytoscape using the NetKarma plug-in and zoomed in on a few specific nodes in the topology for the experiment. At this level of granularity, provenance for the individual packets that were successfully transferred or dropped can be viewed as well as additional detailed provenance annotations.


At GEC13 we presented a demo and poster on the visualizations of the WiMAX experiment results based on NS2. A number of GENI participants stopped by the demo and we discussed the purpose of provenance as a record of the experiment and application of provenance as a tool for analyzing and visualizing experiment results. Although reproducibility of experiment results is often cited as a goal for provenance, the experimenters we talked with at GEC13 were more interested in whether it could be used to assist them in comparing experiment runs or visualizing results (as we have done in the WiMAX experiments). In further discussions with the researchers from Clemson, they noted that the visualizations were excellent and such visualizations would be very useful for posters or proposal materials. For the WiMAX experiment used in this evaluation, the researchers are now looking at moving it to the ORBIT testbed. We have been working on extracting provenance from the ORBIT Measurement Library (OML). In the WiMAX tutorial at GEC13 we talked with one of the researchers working on the ORBIT testbed at Rutgers University regarding different approaches that could be used with ORBIT to extract the information we need to construct provenance for WiMAX experiments running on ORBIT; such as through Ruby scripts that ORBIT uses or defining a pre-packaged OML schema that researchers could use.

Project Participants

During this period, active participants in the NetKarma project included: Beth Plale and Chris Small, as well as Scott Jensen, Postdoctoral Fellow in the Data To Insight Center, and students Peng Chen, Devarshi Ghoshal, and Yuan Luo. Robert Ping provided project management for the project.

Collaborations

  • Jon-Paul Herron and Mitch MacCracken - (GMOC) - status of resources and GENI event records
  • Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) - GEMINI/LAMP monitoring integration
  • Ilia Baldine and Shu Huang - RENCI (data and metadata archives using iRODS)
  • Jonathan Mills - RENCI (ExoGENI) (status of ExoGENI cluster and event schema)
  • GPO Monitoring team for GENI MesoScale monitoring
  • Brian Hay - Univ of Alaska (metadata and monitoring retrieval from VM instances)
  • Mike Zink Univ of Massachusetts Amherst (GIMI) data collection and archival mechanisms
  • Richard Brooks and Katherine Cameron of Clemson University - Evaluating NetKarma using WiMAX DDoS experiments
  • Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) - Evaluating NetKarma based on XSP experiments
  • Giridhar Manepalli (CNRI) Measurement Data Object Descriptor (MDOD) and GENI event records.
  • At GEC 13 we discussed the GENI event record and next steps on the MDOD with Harry Mussman (BBN), Jeanne Ohren (BBN), and Giridhar Manepalli (CNRI).

Publications & Documents

Small, C, NetKarma Portal Poster GENI Engineering Conference 13, Los Angeles, CA
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_GEC13.pdf

Small, C, NetKarma Portal Slides
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_Portal.pdf

NetKarma: Role of Provenance in Visualizing Packet Throughput and Packet Loss (Poster at GENI Engineering Conference 13, Los Angeles, CA)
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC%2013-Role%20of%20Provenance%20in%20Visualizing%20Packet%20Throughput%20and%20Packet%20Loss.pdf

Handout at GEC 13 on Provenance for Experimenters
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13-Provenance_Handout.pdf

NetKarma Cytoscape Visualization Plug-In Manual for Version 1.2.0
(Updated manual for NS2 NetKarma visualization enhancements)
http://d2i.indiana.edu/sites/default/files/karmavisualizationusermanual-1.2.0.pdf

NetKarma NS2 Adaptor User Manual
http://d2i.indiana.edu/sites/default/files/netkarma_ns2_adaptor_user_manual_v1.0.0.pdf

Instructions for the NetKarma Provenance Toolkit for NS2 Experiments
http://d2i.indiana.edu/sites/default/files/instructions_for_the_netkarma_provenance_toolkit_for_ns2_experiments.pdf

GENI Documents

None

Last modified 12 years ago Last modified on 04/01/12 19:03:26

Attachments (12)