ࡱ> egd 3bjbj-- 4NOO+~~~~~L$HJJJJJ"l x*,,,,,,,O!$VX~JJX~~JJm~J~J**`A7>\ZB0W$W$J~0XXW$ D:   

NetKarma Report for Period Ending GEC11

PI: Beth Plale, School of Informatics and Computing, Indiana University Bloomington
Co-PI: Chris Small, Global Research Network Operations (GMOC), Indiana University

Summary

For the quarter ending with GEC11, we augmented the existing provenance captured by NetKarma with network topology information from the GMOC Global Research NOC database and network measurement data from PerfSONAR-PS PingER service. These were identified in a GEC10 milestone as an additional source of provenance. Additionally we continue to refine visualization of provenance to make it more useful for experimenters. We built a plug-in for CytoScape and in this quarter enhanced it to display the additional provenance annotations being captured and enable greater configurability of details to display. These developments were highlighted in a poster presented at GEC11.

Through discussion with Harry Mussman at GEC11, we identified two key engagement points of our group with the instrumentation and measurement team: 1.) our team will assist in the definition of the transaction record MDOD schema; 2.) we will also explore the representation of provenance as an object in the instrumentation repository. Through discussion with Vic Thomas and Mark Berman on the benefit of using one of the current GENI experiments from which to capture provenance, we agreed to explore a WiMAX DoS application being carried out by Lu Yu under direction of professor Richard Brooks, both at Clemson.

Quarter Deliverables Discussed

S3.d Updated plan for making provenance information available to experimenters:

The updated plan for users to submit their GUSH logs and get back a visualizable provenance graph documents details of the submission process; including the URL of the persistent Axis2 web service where donators of GUSH logs can programmatically download their provenance graph and then visualize it using the CytoScape visualization tool.

S3.e GEC11 demonstration and outreach :

At GEC11 we presented Experiment Provenance: Towards Links to Network Measurement Data poster that showed annotations to a NetKarma provenance graph using network topology from the GMOC database and measurement data from PerfSONAR PingER service. Due to the change in venue for GEC11, the demo session was cancelled, so a poster session was the only venue available to us to satisfy the deliverable.

S3.f Deliver software and documentation (Due 08/26/11):

The upcoming end of August release of NetKarma will include 1.) the Karma provenance system, 2.) visualization plugin to cytoscape, 3.) adaptor for extracting provenance from GUSH, 4.) adaptor for extracting network topology data from the GMOC database, and 5.) adaptor for capturing PingER data and adding it to NetKarma.

2. Description of Work Performed Prior Quarter

At GEC10 we identified the GMOC database as a source of additional metadata on the network topology used to run GENI experiments on PlanetLab slices. During this quarter, the NetKarma plug-in to the CytoScape visualization tool was enhanced to display these metadata as a pop-up dialog when a user clicks on a process node in the provenance graph. Additionally, instrument and measurement data was identified as another potential source of metadata annotations. As a prototype of capturing measurement data, we configured GUSH experiments that performed a distributed experiment on a PlanetLab slice to capture network measurement data using the PerfSONAR-PS PingER module and then harvested that metadata using a NetKarma adaptor to annotate the edges between processes in the provenance graph. As with the GMOC data, the NetKarma plug-in for CytoScape was enhanced to query the Karma database when a user clicks on an edge in the provenance graph and display the measurement data annotations in a pop-up dialog.

The additional annotations added to the provenance graph are captured by a harvesting module added to NetKarma that runs periodically during a GUSH experiment and determines which new edges have been added to the provenance graph since the last iteration. The NetKarma adaptors for the GMOC database and measurement archive are then triggered to retrieve metadata and submit provenance notifications to Karma which are then added to the OPM graph generated by NetKarma.

The NetKarma adaptor that processes the GUSH logs to generate Karma provenance notifications was enhanced to capture provenance between child processes in GUSH experiments. The NetKarma software was also enhanced to enable metadata annotations to be added for a wider array of data (artifacts in OPM) so that not only files could be annotated, but also data blocks which capture data written to standard output by the programs executed in a GUSH experiment. The Karma software was also extended to allow annotations on the edges between processes. The NetKarma plug-in for CytoScape was enhanced to provide greater configurability by scientists as to the annotations and details that would be displayed for each node in the provenance graph displayed in CytoScape.

At GEC11 we presented a poster that illustrated the enhancements made to NetKarma during this quarter to harvest additional metadata regarding the network topology of an experiments nodes from the GMOC database and periodically retrieve measurement data during the running of an experiment to annotate the provenance graph with data captured using PingER. The poster also presented screen captures of the enhanced Karma CytoScape plug-in that illustrated the new annotations of the provenance graph nodes and edges.

B. Project Participants

During this time period, participants in the NetKarma project included: Beth Plale, PI of School of Informatics and Computing, Chris Small, Co-PI of GMOC, Mehmet Aktas, Postdoctoral Fellow, Scott Jensen, Postdoctoral Fellow, Devarshi Goshal, PhD student, Peng Chen, PhD student, You-Wei Cheah, PhD student, Yuan Luo, PhD student, Robert Ping, Project Manager in Data to Insight Center that Professor Plale directs.

Mehmet Aktas is beginning a faculty position in Turkey August 2011. His commitment to the project will be missed. Scott Jensen, PhD is taking his place.

Publications & Documents

Karma Adaptor Tool V 2.0 User Manual.

GENI Provenance Registry. Updated plan for making provenance information available to experimenters.

Experiment Provenance: Towards Links to Network Measurement Data, Poster presented at the 11th GENI Engineering Conference.

Unmanaged Workflows: Their Provenance and Use, Mehmet Aktas, Beth Plale, David Leake, Nirmal K. Mukhi, book chapter under review, July 2011

Instrumenting Earth Science Applications for OPM-Driven Provenance, Mehmet Aktas, Beth Plale, Helen Conover, Prajakta Purohit, Technical Report, Data to Insight Center, Indiana University, July 2011

Collaborations and Planned Activities

At GEC11 we participated in discussions with the Instrumentation & Measurement team on their development of a Measurement Data Object Descriptor (MDOD) to contain the metadata describing measurement data. Following up on these discussions, we are collaborating with them on the prototyping of the section of the MDOD that captures provenance and distribution of the measurement data. In the I&M working session at GEC11 we presented a lightning talk on NetKarma and goals for annotating provenance with additional measurement data.

At the I&M working session we discussed the capture of provenance in distributed systems and NetKarmas approach to provenance capture with Max Ott of NICTA in that identifying the completeness of provenance is an issue in his research.

Following discussions at GEC10, in the GEC11 Workshop on Attribution in GENI, NetKarma was mentioned as one of a handful of projects in GENI that address attribution. Following the workshop we also talked with Jeffrey Hunker about provenance, attribution, and their efforts to create an ontology of attribution.

At GEC11 we discussed possible projects in GENI that would benefit the most from provenance capture with Mark Berman of the GPO and following up on those initial discussions we initiated discussions with Richard Brooks regarding the work his team is doing on researching DDoS attacks using WiMAX. As another avenue for making the GENI community aware of NetKarma, Heidi Dempsey of the GPO suggested the possibility of incorporating it into a GUSH tutorial at a future GEC conference.

14:;ELORS_acfgk " $ & ʳ hFFhFFCJOJQJ^JaJ hFFh CJOJQJ^JaJhFFh 0JhFFh 0JjhFFh UjhFFh UhFFh0JhFFh 0JhFFh 0JhFFh 0J hFFh 2& '  " x8gdgd  2( Px 4 #\'*.25@9gd gd  P f g   Zwx/EF˜|k|k|[[LhFFhCJOJQJaJhFFh 6CJOJQJaJ hFFhFFCJOJQJ^JaJ hFFh CJOJQJ^JaJhFFhFFCJOJQJaJ,hFFh>*B*CJOJQJ^JaJph/j8hFFhCJOJQJU^JaJ)jhFFhCJOJQJU^JaJ hFFhCJOJQJ^JaJhFFh CJOJQJaJF-CD«~n~V«~/jhFFhCJOJQJU^JaJhFFh6CJOJQJaJhFFhCJOJQJaJhFFhFFCJOJQJaJhFFh CJOJQJaJ,hFFh>*B*CJOJQJ^JaJph)jhFFhCJOJQJU^JaJ/jphFFhCJOJQJU^JaJ hFFhCJOJQJ^JaJ&'58:;?B;<BCJNPQһҪyyri`rW`rLrjhFFhUhFFh0JhFFh0JhFFh0J hFFh hFFh CJOJQJ^JaJ hFFhtXCJOJQJ^JaJhFFhCJOJQJaJ hFFhCJOJQJ^JaJ,hFFh>*B*CJOJQJ^JaJph)jhFFhCJOJQJU^JaJ/jhFFhCJOJQJU^JaJ89>?r E M $$5';'<'|''' 2( Px 4 #\'*.25@9gdgdgd 2( Px 4 #\'*.25@9gd 9:;NOKLMaboprֿ֩֗։ziT)jhFFhCJOJQJU^JaJ hFFhCJOJQJ^JaJhFFhCJOJQJaJjhFFhUhFFh0JhFFh0JhFFh0JhFFh>*B*phjShFFhUhFFh0J hFFhhFFh0JjhFFhUjhFFhU EFGZ[i)*+>?Mr««u]«/j6 hFFhCJOJQJU^JaJhFFhtXCJOJQJaJ/j hFFhCJOJQJU^JaJhFFhCJOJQJaJ,hFFh>*B*CJOJQJ^JaJph)jhFFhCJOJQJU^JaJ/j hFFhCJOJQJU^JaJ hFFhCJOJQJ^JaJ ! 4 5 C E I Q g h u!!!!һҪҪһҪtҪ\һҪҪ/jhFFhCJOJQJU^JaJhFFhtXCJOJQJaJ/jhFFhCJOJQJU^JaJhFFhCJOJQJaJ hFFhCJOJQJ^JaJ,hFFh>*B*CJOJQJ^JaJph)jhFFhCJOJQJU^JaJ/jn hFFhCJOJQJU^JaJ!!!!!!f#|#}#######$$%%%\%]%^%q%r%%5':'<'һҪҪһҪtҪ\һҪK hFFh CJOJQJ^JaJ/jhFFhCJOJQJU^JaJhFFhtXCJOJQJaJ/jNhFFhCJOJQJU^JaJhFFhCJOJQJaJ hFFhCJOJQJ^JaJ,hFFh>*B*CJOJQJ^JaJph)jhFFhCJOJQJU^JaJ/jhFFhCJOJQJU^JaJ<'?'\''''' ( ((!("(0(e))v******++F-v-x----333{jY hFFhIlCJOJQJ^JaJ hFFhFFCJOJQJ^JaJhFFhtXCJH*OJQJaJ,hFFhtX>*B*CJOJQJ^JaJph/jhFFhtXCJOJQJU^JaJ)jhFFhtXCJOJQJU^JaJhFFhtXCJOJQJaJ hFFh CJOJQJ^JaJ hFFhtXCJOJQJ^JaJ'))v*{*|******_+g+++w,,F-K-L--- 2( Px 4 #\'*.25@9gd  2( Px 4 #\'*.25@9gdtXgdtX-//001233 2( Px 4 #\'*.25@9gdFFgdtX21h:p2B/ =!"#$% 8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma;DyK F8view-source:\http\--groups.geni.net-geni-wiki-PlanetLabtnview-source:http://groups.geni.net/geni/wiki/PlanetLabview-source:http://groups.geni.net/geni/wiki/PlanetLab8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma;DyK F8view-source:\http\--groups.geni.net-geni-wiki-PlanetLabtnview-source:http://groups.geni.net/geni/wiki/PlanetLabview-source:http://groups.geni.net/geni/wiki/PlanetLab8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma8DyK F7view-source:\http\--groups.geni.net-geni-wiki-NetKarmarlview-source:http://groups.geni.net/geni/wiki/NetKarmaview-source:http://groups.geni.net/geni/wiki/NetKarma666666666vvvvvvvvv666666>6666666666666666666666666666666666666666666666666hH6666666666666666666666666666666666666666666666666666666666666666666666666662 0@P`p2( 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p8XV~ OJPJQJ_HmH nH sH tH @`@ NormalCJ_HaJmH sH tH DA D Default Paragraph FontRiR 0 Table Normal4 l4a (k ( 0No List e`  0HTML Preformatted7 2( Px 4 #\'*.25@9CJOJQJ^JaJXoX  0HTML Preformatted CharCJOJQJ^JaJ*o*  start-tag&o!& end-tag4o14 attribute-name6oA6 attribute-value4U`Q4 0 Hyperlink >*ph$/a$ entityPK!pO[Content_Types].xmlj0Eжr(΢]yl#!MB;.n̨̽\A1&ҫ QWKvUbOX#&1`RT9<l#$>r `С-;c=1g~'}xPiB$IO1Êk9IcLHY<;*v7'aE\h>=^,*8q;^*4?Wq{nԉogAߤ>8f2*<")QHxK |]Zz)ӁMSm@\&>!7;wP3[EBU`1OC5VD Xa?p S4[NS28;Y[꫙,T1|n;+/ʕj\\,E:! t4.T̡ e1 }; [z^pl@ok0e g@GGHPXNT,مde|*YdT\Y䀰+(T7$ow2缂#G֛ʥ?q NK-/M,WgxFV/FQⷶO&ecx\QLW@H!+{[|{!KAi `cm2iU|Y+ ި [[vxrNE3pmR =Y04,!&0+WC܃@oOS2'Sٮ05$ɤ]pm3Ft GɄ-!y"ӉV . `עv,O.%вKasSƭvMz`3{9+e@eՔLy7W_XtlPK! ѐ'theme/theme/_rels/themeManager.xml.relsM 0wooӺ&݈Э5 6?$Q ,.aic21h:qm@RN;d`o7gK(M&$R(.1r'JЊT8V"AȻHu}|$b{P8g/]QAsم(#L[PK-!pO[Content_Types].xmlPK-!֧6 -_rels/.relsPK-!kytheme/theme/themeManager.xmlPK-!!Z!theme/theme/theme1.xmlPK-! ѐ'( theme/theme/_rels/themeManager.xml.relsPK]# +N F!<'3 !"#$8'-3%&fE C &P:NLaFZ*> 4g|]q ! +XXXXXXXXXXXXXXXXXX _Hlt175121574 +@ +<D5:`d>GKQ1:@G#$'htZ c ? C     = A &)1JN<@FNQYmvNRXadmHLRZ]e,06>AIZc"&,47?aevz_ciqt|G\   ! $ , H M !!3!6!y!}!!!!!""!$&$-$2$:$?$A$G$K$P$$$$$$$$$W%v%t'|'((2([(^())++$+)+e+m++()  7 > @Asx=>}"~""""##^#M%N%0'5'))+::::::::::::::::::::::::::?<G6j 2BFFtXIl++@####+@UnknownG*Ax Times New Roman5Symbol3 *Cx Arial7Courier7@Cambria3N-3 fgA$BCambria Math qhYFFm%Om%O!20++@HP $P' 2!xxj , Robert Ping Robert Ping Oh+'0*  < H T`hpx' Robert Ping Normal.dotm Robert Ping1Microsoft Macintosh Word@q@m9\@">\m%GD)PICT)<b HHb bHHbb !! Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ#{{{{yo{Vg9ckZVkZw^^c^Z^s{kZc^Z^kZ{ZkZ^kZwo{g9Zg9g9wkZ^ZZcsg9^^g9ZZ^^wkZ}{{{w{{ww{{{{swwww{s{wssw{ww{w o{^g9ckZZkZV^5c^Zccwo{o{co{kZZkZZ^^cZcVZZV^^g9cckZo{g9o{g9Zso{swkZg9o{kZg9cZ^kZZkZckZ{o{sskZwwwswowos{k_ook_ssowwwkZsskZwo{sw{ss{o{skZsskZso{o{so{sso{kZo{o{kZo{sco{wwg9o{so{wo{cg9o{sg9so{g9o{swckZg9kZkZo{g9{g9co{kZo{o{g9g9kZkZo{g9wo{kZkZ{skZZg9g9so{s{ssw {ssws{wo{o{sso{sw{s o{ww{sswsswso{kZsws{wwso{sws3g9cVkZcc^Zcg9^^g9cckZww{wo{sZkZ{޲ Ƣ3so{kZo{o{g9o{g9kZo{kZo{wg9o{kZ{3{g9{{ss{o{o{kZo{o{ssw{o{w Ƣ Ƣo{o{^g9kZ^kZww Ƣ'o{ssso{wswwo{sw{{sso{wwwwso{o{kZsww{w{{wwo{s wo{so{wswwo{w{wswswsswo{skZg9cg9g9skZg9{skZg9g9o{^g9o{skZg9cs wscwskZcg9^kZsg9g9sg9kZwkZkZg9kZso{csg9kZkZo{g9kZg9kZskZo{ g9g9g9o{{o{swkZskZo{kZsg9swg9kZkZg9kZwk_g?k_k_wok_k_ss^g?ock_ osg9cg9g9kZo{kZg9o{skZo{wo{g9wswso{o{sww{sso{ssswscV^Zc^g?^^c^Z^Zg?{{swsso{so{{{o{{ cg9g9^c^cZc^g9sg9^kZ^^{kZcg9Z^g9c^o{w^g9VskZ^^{ZRZ^ZcZ^co{kZZ^cg9ZVZ^Z{{{sws{{{{{sw{wsscVg9Z^cZVZk_Zcswswwwswo{wswwso{wsw{kZo{s{wswo{kZo{kZso{skZ^g9skZ^g9o{kZg9o{wo{g9ssg9*g9g9sg9g9cckZ{ckZso{cg9ckZkZg9cckZsg9g9kZkZkZo{g9o{o{kZg9sg9kZccg9kZo{g9ckZkZo{so{o{sso{kZ{o{so{kZkZo{kZ#o{o{wwg9kZo{kZkZo{o{g9co{sswo{kZswcg9o{{o{kZo{o{so{o{7wssg9o{ssswwo{sw{wso{wswsssso{sg9wo{{{wso{ssswo{o{swws{ss{wo{sg9^cg9cg9g9co{c^g9{scso{wkZc^g9ckZcg9ccso{ccskZcc{g9g9o{g9g9ckZwcg9ckZkZ^ccg9cg9cv{{{w{s{{w{{ww{s{{ww{{{{{{{w{{{{{{Cg9c^^g9kZ^cg9{Z^kZo{V^kZg9Z^^ZkZZw!{w{ww{޹ss^o{sZs Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ Ƣ w ՜.+,D՜.+,< hp  'Indiana UniversityO+  Title 8@ _PID_HLINKS'A l*/3Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/0Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/-Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/*Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/'Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/$Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/!Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarmaF%Hfile://localhost/view-source/http/::groups.geni.net:geni:wiki:PlanetLab*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarmaF%Hfile://localhost/view-source/http/::groups.geni.net:geni:wiki:PlanetLab*/ Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/ Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma*/Gfile://localhost/view-source/http/::groups.geni.net:geni:wiki:NetKarma  !"#$%&')*+,-./012456789:;<=>?@ABCDEGHIJKLMNOPQRSTUVWXYZ[]^_`abcfRoot Entry Ft7>\hData (1Table3W$WordDocument4NSummaryInformation(F*DocumentSummaryInformation8\CompObj` F Microsoft Word 97-2004 DocumentNB6WWord.Document.8