| 1 | [[PageOutline]] |
| 2 | |
| 3 | = NetKarma Project Status Report = |
| 4 | |
| 5 | Period: 1Q10 |
| 6 | == I. Major accomplishments == |
| 7 | |
| 8 | === A. Milestones achieved === |
| 9 | ''__Demonstration of NetKarma:__'' Demonstrated a working version of !NetKarma ingesting |
| 10 | provenance information from GUSH. We visualized the provenance graph that is |
| 11 | generated when an experimenter uses Gush to manipulate their experiment. Delivered |
| 12 | at GEC-7 |
| 13 | |
| 14 | !NetKarma Poster used at the demo session can be found here: |
| 15 | http://groups.geni.net/geni/attachment/wiki/netKarma/Netkarma_poster_gec7.pdf |
| 16 | |
| 17 | === B. Deliverables made === |
| 18 | |
| 19 | |
| 20 | == II. Description of work performed during last quarter == |
| 21 | |
| 22 | === A. Activities and findings === |
| 23 | ''__GUSH provenance capture code released:__'' We have released version 1.0 of the |
| 24 | “gush2netkarma” software. This code contains tools to parse log files generated by |
| 25 | Gush. While this software allows injestion of Gush logfiles into the Karma provenance |
| 26 | framework, it also provides a generalized rules framework that can be used to generate |
| 27 | provenance information in other GENI projects. We hope to use this framework in |
| 28 | capturing the status and workflow of GENI experiments. The framework developed |
| 29 | should allow us to capture artifacts and instrument additional GENI components easily. |
| 30 | |
| 31 | The gush2netkarma-1.0 code is linked from the !NetKarma GENI WIKI page and |
| 32 | available at the following location:[[BR]] |
| 33 | https://globalnoc.iu.edu/grnoc-internal/file-bin/system-engineering/softwareprojects/netkarma/gush2netkarma-1.0.tar.zip |
| 34 | |
| 35 | ''__Karma 3.0 Released:__'' The new release of the core Karma system contains |
| 36 | instrumentation using Axis2 handlers, more extensive test clients, and better |
| 37 | documentation. Karma v3.0 supports provenance activities published from services, |
| 38 | workflows and nested workflows. The provenance data is efficiently stored in a |
| 39 | relational database, and supports Open Provenance Model (OPM) v1.0 standard for |
| 40 | interfacing with the tool. |
| 41 | === B. Project participants === |
| 42 | During this time, key participants in the !NetKarma project included: |
| 43 | * Beth Plale, PI |
| 44 | * Chris Small, Co-PI |
| 45 | * Girish Subramanian, PhD student |
| 46 | * Yiming Sun, PhD student |
| 47 | * Devarshi Goshal, PhD student |
| 48 | * Sharanya Chinnusamy, MS student |
| 49 | * David Ripley, technical staff |
| 50 | * Robert Ping, Project and Information Management |
| 51 | === C. Publications (individual and organizational) === |
| 52 | !NetKarma Status Update during GEC7 Planetlab cluster meeting: |
| 53 | |
| 54 | http://groups.geni.net/geni/attachment/wiki/netKarma/NetKarma_PL_cluster_GEC7_slides.pdf?format=raw |
| 55 | |
| 56 | !NetKarma Poster used at the GEC7 demo session: |
| 57 | |
| 58 | http://groups.geni.net/geni/attachment/wiki/netKarma/Netkarma_poster_gec7.pdf |
| 59 | |
| 60 | gush2netkarma-1.0 release: |
| 61 | |
| 62 | https://globalnoc.iu.edu/grnoc-internal/file-bin/system-engineering/softwareprojects/netkarma/gush2netkarma-1.0.tar.zip |
| 63 | === D. Outreach activities === |
| 64 | |
| 65 | === E. Collaborations === |
| 66 | We have worked extensively with Jeannie Albrecht, lead of the Gush team, to ingest |
| 67 | provenance relevant information from log files produced by Gush into the Netkarma |
| 68 | system. Professor Albrecht has been extremely cooperative in providing assistance in |
| 69 | understanding the various data artifacts produced by Gush and in deploying a |
| 70 | demonstration instance for the development of the “gush2netkarma” software. |
| 71 | |
| 72 | We continue to work with the Gush team on refining the provenance data collection |
| 73 | process. This includes adding hooks to the Gush code to facilitate the collection of |
| 74 | artifacts from the Gush processes. |
| 75 | |
| 76 | We have had discussions with John Hartman and Scott Baker of the Raven project to |
| 77 | discuss the workings and workflow of the Raven provisioning service. We are studying |
| 78 | the logs and information presented through the Owl monitoring service for collection |
| 79 | into !NetKarma. |
| 80 | |
| 81 | We have started initial discussion with the GMOC. We are studying the extent to which |
| 82 | operational data collected by the GMOC, especially control plane related data, could be |
| 83 | collected from the GMOC repository instead of the original source. The GMOC |
| 84 | repository would provide a unified, normalized schema to query against compared to |
| 85 | widely differing data and formats used by control plane and other data sources. |
| 86 | Collecting data already obtained from various sources and stored in the GMOC |
| 87 | database schema may speed and simplify the work need for data collection. |
| 88 | |
| 89 | We had a preliminary discussion of how !penFlow state such as flow tables and |
| 90 | controller policy rules can be used. We focused on collection from the Indiana |
| 91 | University !OpenFlow campus trial infrastructure as an initial test but techniques should |
| 92 | be applicable to all !OpenFlow deployments. |
| 93 | |
| 94 | === F. Other Contributions === |
| 95 | |
| 96 | We will work to establish connections with relevant projects in the Control Plane and |
| 97 | Measurement Plane areas. We plan to make decisions of what Control plane |
| 98 | information we will integrate into the !Netkarma data collection during Q3. |
| 99 | |
| 100 | We will continue to develop software to allow the integration of Raven metadata into |
| 101 | !Netkarma. This may include data such the locations, success of the distribution and |
| 102 | time of deployment of software package to all nodes in an experiment. References to |
| 103 | the software itself on the Raven repository would also be incorporated in to the |
| 104 | !Netkarma description of an experiment. The software reference could be very useful if |
| 105 | future creation of the ability to recreate an experiment from information captured in the |
| 106 | !Netkarma provenance trace. |