wiki:netKarma/GEC13_report

Version 16 (modified by scjensen@umail.iu.edu, 12 years ago) (diff)

--

NetKarma

Report for the period ending GEC12

Beth Plale, School of Informatics and Computing, Indiana University
Chris Small, InCNTRE, Indiana University

Summary

For the quarter ending with GEC13, we released the NetKarma portal which allows experimentors to easily add and retrieve experimental data from the NetKarma provenance archive. The NetKarma Portal is integrated with the NetKarma Provenance System and provides a persistent service on servers at Indiana University to enable GENI experimentors to help understand the conditions of their experiment. The NetKarma links the experiment workflow data obtained from tools such as GUSH to representations of the experiment's GENI topology, the GENI Instramentation and Measurement systems, and log term data archives. NetKarma links all of these systems together to simplify the complexity of running an experiment on the GENI framework.

In the plan for evaluating NetKarma we had identified experiments being performed in GENI by researchers at Clemson University on distributed denial of service (DDoS) attacks in WiMAX as a means to evaluate the provenance being captured by Karma. Since GEC12 we have implemented their experiment, and using the standard trace log that is generated and an extension we added to capture the experiment topology, we are able to harvest detailed provenance about their experiment. The DDoS experiments measure the change in throughput as a set of network parameters are varied, resulting in over 700 different configurations being tested. Using the provenance captured from the logs for each packet that was successfully transferred, and the inferred provenance of packets that were dropped, the NetKarma visualization plug-ins can be used to visualize the throughput and packets dropped as shown in Figure below. In addition, the NetKarma visualization plug-n for Cytoscape enables researchers to view the provenance at different levels of granularity as shown in Figure . We shared the visualization plugin and provenance graphs generated with the researchers at Clemson and they noted that NetKarma allows them to visualize and compare individual runs whereas previously their analysis was based on ANOVA calculations at an aggregate level. At GEC13 we presented a demo that visualized the provenance based on our runs of Clemson's experiment to show how provenance can be used to visualize results and is valuable for communicating the results of GENI experiments to other researchers.

Milestones Delivered

S4b.1 Demonstration of the provenance system with WiMAX DDoS Experiments
Using the MiMAX DDoS experiments at Clemson which were identified as a case for evaluating the provenance captured and visualized using NetKarma we showed how the provenance of packets successfully transferred and the inferred provenance of packets dropped can be used to provide visualizations of individual experiment runs; providing network researchers with a new approach to analyzing their experiment results. At GEC13 we presented both a poster and a live demo of the provenance captured from the DDoS experiments we reran based on the research done by GENI researchers at Clemson University. An additional outcome of this evaluation of NetKarma is a new NetKarma adaptor that can be used to capture provenance of experiments run on the NS2 simulator used in the Clemson experiments (and can be applied to any NS2 experiments). An enhanced NetKarma plug-in for Cytoscape that contains new visualization algorithms for visualizing the provenance of NS2 experiments was also released (version 1.2.1). Both the adaptor and visualization plug-in, along with updated software documentation, are available on the NetKarma wiki.
We have utalised the work on the NetKarma Portal to greatly improve the usability and utility of the experimental data. From the portal the ingestion process becomes just requires the experimentor drag the files related to there experiment and have the experimental data ingested into NetKarma in one easy step. In addition to the portal, the AXIS2 and RabbitMQ mechanisms for adapters are supported allowing for experimentors to automated the ingestion process if desired.

S4b.2 Linking workflow provenance information to metadata The NetKarma portal is designed to link metadata such as the metadata object descriptor to the workflow information and provide the experimentor a way to navigate between not only the metadata and provenance information but also provides a way to visualise the measurement data that any metadata refers to and provide links to additional sources of data such as GEMINI, GIMS or the GMOC. The NetKarma portal supports ingestion of the metadata object directly or reference. For example a reference to a resiurce in a UNIS or NMWG file could be linked directly to the measurement data collected for that resource on the GENI topology. NetKarma provides an experimenter focused view to the data collected for the entire experiment.

S4b.3 Written plan to work with a second experiment/experimenter for evaluation of the provenance system_ We have identified the eXtensible Session Protocol (XSP) based on experiments by researchers at Indiana University and the University of Delaware for the second GENI project used to evaluate the provenance captured in NetKarma. In comparison to the first project identified (WiMAX DDoS attacks based on research at Clemson), the XSP research presents a different and diverse set of issues. Whereas the WiMAX experiments presented a high volume of provenance regarding similar data objects (packets being transferred and inferred packet loss), the XSP experiment presents the challenge of capturing and representing the provenance of events at different network layers. The NetKarma evaluation plan has been updated to include the XSP experiment and is available here on the NetKarma wiki.

Work Preformed this Quarter

Sb.4 Work with the GENI I&M community to define a standardized GENI event records We began to examine some of the issues involved with creation of a standardized GENI event record. While NetKarma captures provenance information including "events" of the infrastructure used for a GENI experiment, NetKarma doesn't produce events but may capture events related to an experiment as part of the provenance capture. These events can then be used to create a provenance graph (e.g., identifying the process that generated a specific output from an experiment). We have worked with ExoGeni, GMOC, I&M projects and the Mesoscale monitoring efforts as possible sources of event information about the creation, resource allocation and possible outages on the experimental infrastucture.

In discussions prior to GEC13, we talked with Harry Mussman and Jeanne Ohren of BBN and Giridhar Manepalli of CNRI regarding the features that are essential in the event record. It is anticipated that a large number of events would be generated and that the event record needs to be lightweight, sharing some similarities to call records in the telecom industry. The event records will capture events in the slice including resource assignment/release, failures, and network events such as resources being up/down. We identified events from Nagios based on thresholds as an example. For some of the events (e.g., assignment of resources) the RSpec contains the information but is too heavyweight.

This also identified a need to revise the NetKarma deliverables planned for GEC14. One of the deliverables is to show NetKarma generating and populating event records. Through discussions in the I&M group, the definition of the GENI event record has evolved. As initially envisioned, the event record could possibly have been created based on provenance captured in the provenance record in NetKarma, but the event records would instead be lightweight messages published as an event occurs and in many cases would be valuable as provenance to be ingested into NetKarma instead of being generated by NetKarma.

NetKarma Portal

The NetKarma Portal provides a easy way for experimenters to:

  • Create a new experiment,pointed to by a universial handle (DOI or ARK)
  • Capture experiment workflow information from log files
  • Integrate GENI topology information to an experiment
  • Intigrate measurements from GENI I&M implementations to an experiment
  • Visualise workflow, topology, measurement, status
  • Produce unified experment data representations such as a KML "movie"
  • Interface to archival services (local and iRODS)
  • Browse the work of other experimenters (given proper permissions)

Experimental WorkFlow using the NetKarma Portal

An experimentor creates an experimental handle to reference from the front page of the NetKarma portal. Any data products from the experiment can be uploaded to NetKarma. All are stored as archives but some supported file types are parsed and additional information gathered.

Supported file types include:

  • Gush log files
  • UNIS/NMWG topology representation (used in PerfSonar and GEMINI)
  • GENI v3 RSpec files

Once the files are uploaded to NetKarma Provenance store through the portal the workflow can be visualized both on the portal itself and through the NetKarma plugin for the Cytoscape desktop application.

NetKarma Evaluation Based on WiMAX and Provenance Capture/Visualization Enhancements Using the WiMAX DDoS attack research from Clemson as a GENI experiment as a basis for evaluation of provenance capture and visualization using NetKarma, we have rerun selected configurations from their research that varied the parameters identified as most significant based on their research while keeping the other parameters constant. Since their experiments utilized the NS2 simulator, we developed a NetKarma adaptor that can ingest provenance based on NS2 trace logs and an NS2 extension we have made available to capture the network topology. This new adaptor can be used with other NS2 experiments to ingest provenance into NetKarma (and is not specific to the Clemson experiments). The adaptor and documentation are available on the NetKarma wiki. To visualize the provenance of packet transfers (and infer packets dropped) in NS2 experiments, the NetKarma visualization plug-in for Cytoscape was extended with new NS2 visualization algorithms and along with updated documentation is available on the NetKarma wiki as version 1.2.1 of the visualization plug-in.

Project Participants

During this time period, participants in the NetKarma project included: Beth Plale, PI of School of Informatics and Computing, Chris Small, Co-PI of InCNTRE, Scott Jensen, Postdoctoral Fellow, Devarshi Goshal, PhD student, Peng Chen, PhD student, You-Wei Cheah, PhD student, Yuan Luo, PhD student, Robert Ping, Project Manager in Data to Insight Center that Professor Plale directs.

Collaborations

  • GMOC (status)
  • GEMINI/LAMP (monitoring)
  • iRODS
  • ExoGENI (status)
  • GPO Monitoring team for GENI MesoScale
  • Univ of Alaska (metadata and monitoring retrieval from VM instances)
  • Mike Zink Univ of Massachusetts Amherst on GIMI data collection and archival mechanisms
  • Richard Brooks and Katherine Cameron of Clemson University (Evaluating NetKarma using WiMAX DDoS experiments)
  • Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) Evaluating NetKarma based on XSP experiments
  • Giridhar Manepalli (CNRI) Measurement Data Object Descriptor (MDOD) and GENI event records.

At GEC 13 we discussed the GENI event record and next steps on the MDOD with Harry Mussman (BBN), Jeanne Ohren (BBN), and Giridhar Manepalli (CNRI).

Publications & Documents

Small, C, NetKarma Portal Poster GENI Engineering Conference 13, Los Angeles, CA
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_GEC13.pdf

Small, C, NetKarma Portal Slides
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_Portal.pdf

NetKarma: Role of Provenance in Visualizing Packet Throughput and Packet Loss (Poster at GENI Engineering Conference 13, Los Angeles, CA)
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC%2013-Role%20of%20Provenance%20in%20Visualizing%20Packet%20Throughput%20and%20Packet%20Loss.pdf

Handout at GEC 13 on Provenance for Experimenters
http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13-Provenance_Handout.pdf

NetKarma Cytoscape Visualization Plug-In Manual for Version 1.2.0
(Updated manual for NS2 NetKarma visualization enhancements)
http://d2i.indiana.edu/sites/default/files/karmavisualizationusermanual-1.2.0.pdf

GENI Documents

None

Attachments (12)