= NetKarma = == Report for the period ending GEC12 == Beth Plale, School of Informatics and Computing, Indiana University [[BR]] Chris Small, InCNTRE, Indiana University == Summary == For the quarter ending with GEC13, we released the NetKarma portal which allows experimentors to easily add and retrieve experimental data from the NetKarma provenance archive. The NetKarma Portal is integrated with the NetKarma Provenance System and provides a persistent service on servers at Indiana University to enable GENI experimentors to help understand the conditions of their experiment. The NetKarma links the experiment workflow data obtained from tools such as GUSH to representations of the experiment's GENI topology, the GENI Instramentation and Measurement systems, and log term data archives. NetKarma links all of these systems together to simplify the complexity of running an experiment on the GENI framework. In the plan for evaluating NetKarma we had identified experiments being performed in GENI by researchers at Clemson University on distributed denial of service (DDoS) attacks in WiMAX as a means to evaluate the provenance being captured by Karma. Since GEC12 we have implemented their experiment, and using the standard trace log that is generated and an extension we added to capture the experiment topology, we are able to harvest detailed provenance about their experiment. The DDoS experiments measure the change in throughput as a set of network parameters are varied, resulting in over 700 different configurations being tested. Using the provenance captured from the logs for each packet that was successfully transferred, and the inferred provenance of packets that were dropped, the NetKarma visualization plug-in can be used to visualize the throughput and packets dropped and view the provenance at different levels of granularity as shown in our demo and poster at GEC13. We shared the visualization plugin and provenance graphs generated with the researchers at Clemson and they noted that NetKarma allows them to visualize and compare individual runs whereas previously their analysis was based on ANOVA calculations at an aggregate level. At GEC13 we presented a demo that visualized the provenance based on our runs of Clemson's experiment to show how provenance can be used to visualize results and is valuable for communicating the results of GENI experiments to other researchers. [[BR]] == Milestones Delivered == __S4b.1 Demonstration of the provenance system with WiMAX DDoS Experiments__ [[BR]] Using the MiMAX DDoS experiments at Clemson which were identified as a case for evaluating the provenance captured and visualized using NetKarma we showed how the provenance of packets successfully transferred and the inferred provenance of packets dropped can be used to provide visualizations of individual experiment runs; providing network researchers with a new approach to analyzing their experiment results. At GEC13 we presented both a poster and a live demo of the provenance captured from the DDoS experiments we reran based on the research done by GENI researchers at Clemson University. An additional outcome of this evaluation of NetKarma is a new NetKarma adaptor that can be used to capture provenance of experiments run on the NS2 simulator used in the Clemson experiments (and can be applied to any NS2 experiments). An enhanced NetKarma plug-in for Cytoscape that contains new visualization algorithms for visualizing the provenance of NS2 experiments was also released (version 1.2.1). Both the adaptor and visualization plug-in, along with updated software documentation, are available on the NetKarma wiki. [[BR]] We have utalised the work on the NetKarma Portal to greatly improve the usability and utility of the experimental data. From the portal the ingestion process becomes just requires the experimentor drag the files related to there experiment and have the experimental data ingested into NetKarma in one easy step. In addition to the portal, the AXIS2 and RabbitMQ mechanisms for adapters are supported allowing for experimentors to automated the ingestion process if desired. __S4b.2 Linking workflow provenance information to metadata__ The NetKarma portal is designed to link metadata such as the metadata object descriptor to the workflow information and provide the experimentor a way to navigate between not only the metadata and provenance information but also provides a way to visualise the measurement data that any metadata refers to and provide links to additional sources of data such as GEMINI, GIMS or the GMOC. The NetKarma portal supports ingestion of the metadata object directly or reference. For example a reference to a resiurce in a UNIS or NMWG file could be linked directly to the measurement data collected for that resource on the GENI topology. NetKarma provides an experimenter focused view to the data collected for the entire experiment. __S4b.3 Written plan to work with a second experiment/experimenter for evaluation of the provenance system___ We have identified the eXtensible Session Protocol (XSP) based on experiments by researchers at Indiana University and the University of Delaware for the second GENI project used to evaluate the provenance captured in NetKarma. In comparison to the first project identified (WiMAX DDoS attacks based on research at Clemson), the XSP research presents a different and diverse set of issues. Whereas the WiMAX experiments presented a high volume of provenance regarding similar data objects (packets being transferred and inferred packet loss), the XSP experiment presents the challenge of capturing and representing the provenance of events at different network layers. The NetKarma evaluation plan has been updated to include the XSP experiment and is available [http://groups.geni.net/geni/attachment/wiki/netKarma/GENI%20-%20Provenance%20System%20Evaluation.pdf here] on the NetKarma wiki. == Work Preformed this Quarter == __Sb.4 Work with the GENI I&M community to define a standardized GENI event records__ We began to examine some of the issues involved with creation of a standardized GENI event record. While NetKarma captures provenance information including "events" of the infrastructure used for a GENI experiment, NetKarma doesn't produce events but may capture events related to an experiment as part of the provenance capture. These events can then be used to create a provenance graph (e.g., identifying the process that generated a specific output from an experiment). We have worked with ExoGeni, GMOC, I&M projects and the Mesoscale monitoring efforts as possible sources of event information about the creation, resource allocation and possible outages on the experimental infrastucture. In discussions prior to GEC13, we talked with Harry Mussman and Jeanne Ohren of BBN and Giridhar Manepalli of CNRI regarding the features that are essential in the event record. It is anticipated that a large number of events would be generated and that the event record needs to be lightweight, sharing some similarities to call records in the telecom industry. The event records will capture events in the slice including resource assignment/release, failures, and network events such as resources being up/down. We identified events from Nagios based on thresholds as an example. For some of the events (e.g., assignment of resources) the RSpec contains the information but is too heavyweight. This also identified a need to revise the NetKarma deliverables planned for GEC14. One of the deliverables is to show NetKarma generating and populating event records. Through discussions in the I&M group, the definition of the GENI event record has evolved. As initially envisioned, the event record could possibly have been created based on provenance captured in the provenance record in NetKarma, but the event records would instead be lightweight messages published as an event occurs and in many cases would be valuable as provenance to be ingested into NetKarma instead of being generated by NetKarma. __NetKarma Portal__ The NetKarma Portal provides a easy way for experimenters to: * Create a new experiment,pointed to by a universial handle (DOI or ARK) * Capture experiment workflow information from log files * Integrate GENI topology information to an experiment * Intigrate measurements from GENI I&M implementations to an experiment * Visualise workflow, topology, measurement, status * Produce unified experment data representations such as a KML "movie" * Interface to archival services (local and iRODS) * Browse the work of other experimenters (given proper permissions) __Experimental WorkFlow using the NetKarma Portal__ An experimentor creates an experimental handle to reference from the front page of the NetKarma portal. Any data products from the experiment can be uploaded to NetKarma. All are stored as archives but some supported file types are parsed and additional information gathered. Supported file types include: * Gush log files * UNIS/NMWG topology representation (used in PerfSonar and GEMINI) * GENI v3 RSpec files Once the files are uploaded to NetKarma Provenance store through the portal the workflow can be visualized both on the portal itself and through the NetKarma plugin for the Cytoscape desktop application. Experimenters can visualise the workflow of their experiments as an Open Provenance Model (OPM) graph inside the portal. Filters can be applied against attributes of the processes and artifacts to show for example a subset of time of each experimental run. Resources uploaded as Rspec or topology files can also be visualized both in logical or geographically based representations. Data products such as the GoogleEarth KML "tours" allow for combination of experimental workflow with GMOC status and time seres data. Finally a data access to local or remote (such as iRODS) data archive service is provided. [[Image(NetKarma_experiments.png, 40%)]] [[BR]] NetKarma Portal Experiment creation and artifact upload The NetKarma Portal development page is available at: http://netkarma.testlab.grnoc.iu.edu/portal/uploader/ A permanent production URL will be available shortly. Source code for the NetKarma portal is available through GitHub at: https://github.com/InCNTRE/NetKarma_Portal [http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_GEC13.pdf Poster] and [http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_Portal.pdf Slides] explainng the goals and features of the NetKarma Portal. __NetKarma Evaluation Based on WiMAX and Provenance Capture/Visualization Enhancements__ Using the WiMAX DDoS attack research from Clemson as a GENI experiment as a basis for evaluation of provenance capture and visualization using NetKarma, we have rerun selected configurations from their research that varied the parameters identified as most significant based on their research while keeping the other parameters constant. Since their experiments utilized the NS2 simulator, we developed a NetKarma adaptor that can ingest provenance based on NS2 trace logs and an NS2 extension we have made available to capture the network topology. This new adaptor can be used with other NS2 experiments to ingest provenance into NetKarma (and is not specific to the Clemson experiments). The adaptor and documentation are available on the NetKarma wiki. To visualize the provenance of packet transfers (and infer packets dropped) in NS2 experiments, the NetKarma visualization plug-in for Cytoscape was extended with new NS2 visualization algorithms and along with updated documentation is available on the NetKarma wiki as version 1.2.1 of the visualization plug-in. The visualization plug-in enhancements allow the packet transfers to be viewed at different levels of granularity as illustrated in the demo and [http://groups.geni.net/geni/attachment/wiki/netKarma/GEC%2013-Role%20of%20Provenance%20in%20Visualizing%20Packet%20Throughput%20and%20Packet%20Loss.pdf poster] we presented at GEC13. == Project Participants == During this time period, participants in the NetKarma project included: Beth Plale, PI of School of Informatics and Computing, Chris Small, Co-PI of InCNTRE, Scott Jensen, Postdoctoral Fellow, Devarshi Goshal, PhD student, Peng Chen, PhD student, You-Wei Cheah, PhD student, Yuan Luo, PhD student, Robert Ping, Project Manager in Data to Insight Center that Professor Plale directs. == Collaborations == * Jon-Paul Herron and Mitch MacCracken - (GMOC) - status of resources and GENI event records * Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) - GEMINI/LAMP monitoring integration * Ilia Baldine and Shu Huang - RENCI (data and metadata archives using iRODS) * Jonathan Mills - RENCI (ExoGENI) (status of ExoGENI cluster and event schema) * GPO Monitoring team for GENI MesoScale monitoring * Brian Hay - Univ of Alaska (metadata and monitoring retrieval from VM instances) * Mike Zink Univ of Massachusetts Amherst (GIMI) data collection and archival mechanisms * Richard Brooks and Katherine Cameron of Clemson University - Evaluating NetKarma using WiMAX DDoS experiments * Martin Swany (Indiana University) and Ezra Kissel (University of Delaware) - Evaluating NetKarma based on XSP experiments * Giridhar Manepalli (CNRI) Measurement Data Object Descriptor (MDOD) and GENI event records. * At GEC 13 we discussed the GENI event record and next steps on the MDOD with Harry Mussman (BBN), Jeanne Ohren (BBN), and Giridhar Manepalli (CNRI). == Publications & Documents == Small, C, NetKarma Portal Poster GENI Engineering Conference 13, Los Angeles, CA[[BR]] http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_GEC13.pdf Small, C, NetKarma Portal Slides[[BR]] http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13_report/NetKarma_Portal.pdf NetKarma: Role of Provenance in Visualizing Packet Throughput and Packet Loss (Poster at GENI Engineering Conference 13, Los Angeles, CA)[[BR]] http://groups.geni.net/geni/attachment/wiki/netKarma/GEC%2013-Role%20of%20Provenance%20in%20Visualizing%20Packet%20Throughput%20and%20Packet%20Loss.pdf Handout at GEC 13 on Provenance for Experimenters[[BR]] http://groups.geni.net/geni/attachment/wiki/netKarma/GEC13-Provenance_Handout.pdf NetKarma Cytoscape Visualization Plug-In Manual for Version 1.2.0[[BR]] (Updated manual for NS2 NetKarma visualization enhancements)[[BR]] http://d2i.indiana.edu/sites/default/files/karmavisualizationusermanual-1.2.0.pdf NetKarma NS2 Adaptor User Manual[[BR]] http://d2i.indiana.edu/sites/default/files/netkarma_ns2_adaptor_user_manual_v1.0.0.pdf Instructions for the NetKarma Provenance Toolkit for NS2 Experiments[[BR]] http://d2i.indiana.edu/sites/default/files/instructions_for_the_netkarma_provenance_toolkit_for_ns2_experiments.pdf == GENI Documents == None