Version 3 (modified by, 10 years ago) (diff)



Report for the period ending GEC13

Beth Plale, School of Informatics and Computing, Indiana University

Chris Small, InCNTRE, Indiana University


Milestones Delivered

S4.c Provenance System Demo; I&M Contributions

NetKarma Tutorial and Demo

At GEC14 we presented a tutorial of the NetKarma Portal which is layered on top of the NetKarma server as well as NetKarma plug-ins for visualizing large provenance graphs. After first providing participants with an overview of what provenance is and how it can be beneficial to GENI experimenters, the tutorial continued with three hands-on components that allowed GENI experimenters to capture provenance, ingest and visualize provenance in the NetKarma Portal, and download provenance graphs from the portal for visualization in Cytoscape using the GENI NetKarma plug-ins.

The tutorial first walked the participants through an experiment using Emulab ProtoGENI resources to run an experiment doing a GridFTP transfer based on the eXtensible Session Protocol (XSP). Since the focus was on provenance generation, the resources had already been requested and configured, and the participants ran scripts loaded in the tutorial VM on VirtualBox to run the experiment – generating a log from the XSP experiment, some basic bandwidth measurements using iperf, and downloading the manifest for the resources they are using. The NetKarma adaptor which is incorporated into the NetKarma portal used these two log files and the manifest in the second hands-on portion of the tutorial to extract provenance and load it into NetKarma through the Portal.

<Portal section>

The third hands-on portion of the tutorial showed participants how they can easily download a provenance graph generated by NetKarma from the Portal. The downloaded graph is an XML file based on the Open Provenance Model (OPM) standard for provenance. For very large provenance graphs, the NetKarma plug-in can be used to visualize and explore large provenance graphs using the desktop version of Cytoscape (the web version of cytoscape is used in the Portal). In this section participants downloaded provenance graphs for experiments we had run on both NS2 and ORBIT and loaded into the portal using the corresponding NetKarma adaptors. The provenance graphs for these experiments differ from the XSP experiment in that whereas the adaptor for the XSP experiment uses the manifest to incorporate the topology of the experiment into the provenance graph, the focus of the NS2 and ORBIT adaptors is on capturing the success of each packet transferred. This focus on the packet level results in large provenance graphs, but as participants were able to experience by loading the graphs for both experiments into the copy of Cytoscape included in the tutorial VM, they could quickly see the performance of NS2 and ORBIT experiments as to packets successfully transferred or dropped by using visualization capabilities included in the NetKarma plug-in.

The full instructions, including step-by-step graphics and illustrations of the provenance generation process are included in the tutorial instructions available at:

NetKarma Demo at GEC14

At the Poster and Demo session at GEC14 we presented two posters:

  • NetKarma Reveals the Provenance of WiMAX Experiments on ORBIT PDF
  • GENI Provenance, Instrumentation and Measurement: Integrating Provenance into the GENI Measurement Data Object Descriptor PDF?

We also presented a live demo that followed up on our tutorial to show the latest version of the NetKarma portal and the visualization of an ORBIT experiment using the latest version of the NetKarma visualization plug-in for Cytoscape.

NetKarma and the Measurement Data Object Descriptor (MDOD)

Work Performed this Quarter

Project Participants

During this period, active participants in the NetKarma project included: Beth Plale and Chris Small, as well as Scott Jensen, Postdoctoral Fellow in the Data To Insight Center, and students Peng Chen, Devarshi Ghoshal, and Yuan Luo. Robert Ping provided project management for the project.


  • Katherine Cameron (Clemson University) – ORBIT experiments on DDoS attacks and ORBIT configuration
  • Nilanjan Paul (Rutgers) – ORBIT experiment configuration
  • Fraida Fund (NYU-Poly WITest Lab) ORBIT reservation system
  • Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) – Discussion of measurement metadata capture in the MDOD. XSP-based experiment used in NetKarma tutorial
  • Harry Mussman (BBN) – Metadata capture for measurement data in the MDOD
  • Jeanne Ohren (BBN) – tutorial coordination and MDOD
  • Ahmet Babaoglu (North Carolina State University) capturing NetKarma notifications as events in IMF Messaging

Attachments (22)