Changes between Initial Version and Version 1 of GIR2.1_NetKarma

Timestamp:

03/14/11 22:00:54 (13 years ago)

Author:

lnevers@bbn.com

Comment:

--

GIR2.1_NetKarma

@@ +1 @@
+== !NetKarma Evaluation ==
+
+The gush2netkarma package deliverable is a script that uses a set of rules to read a log file and generate appropriate provenance information.   The initial version was deemed too old (GEC7 version), a new version [https://www.slashtmp.iu.edu/public/download.php?FILE=dghoshal/44577wbJ1BL geni_adaptor.tar.gz ](GEC8 version) was made available on Aug 3rd and requires a download password. Time frame for GEC8 version: This evaluation took place August 25-28, 2010.
+
+A second version was made available on September 13, which delivered a new ruleset to handle log issues discovered in the previous evaluation. 
+
+== !NetKarma Findings ==
+
+Requested account to get access to code that allows experiment provenance capture using GUSH. The [http://globalnoc.iu.edu/grnoc-internal/file-bin/system-engineering/software-projects/netkarma/gush2netkarma-1.0.tar.zip gush2netkarma-1.0.tar.zip] file is available Global Research NOC. 
+
+ * Pre-requisite of openjdk-6-jre and sun-java6-jdk not documented. 
+ * Installation instructions need clarification about the expected location of the various files.  
+ * Initial make fails with what seems like a missing file error.
+ * env.sh sets CLASSPATH for java-1.6.0-open-jdk but does not set PATH for /usr/lib/jvm/java-1.6.0-openjdk/bin
+
+Found from PI that version available is the deliverable of the first generation adaptor and that the version 
+posted is suited to the purposes of the GEC7 demo only.  Developer is working on cleaning that up and posting 
+a version that has a clear separation of the front end adaptor, and demo-only back end graph generation.  
+
+The GEC8 version was downloaded on August 3. Upon use, it was found that some prerequisites where not captured:
+    * graphviz: rich set of graph drawing tools
+    * libgtkglext1: OpenGL Extension to GTK+ (shared libraries)
+    * libgts-dev: GNU Triangulated Surface Library
+Only the graphviz tool was added to the list of pre-requisistes.
+
+Once all pre-requisites were installed, using logs from gush testing revealed an issue with the ruleset.xml delivered. 
+Tried to verify that new ruleset addresses issues in September 3rd version.
+
+== !NetKarma How-to ==
+
+'''!NetKarma GEC7 Version How-to'''
+
+The GEC7 version of the netkarma tgz files deliver a Readme.txt which describes installation and run time:
+{{{
+Installation
+-------------
+The package has 4 parts:
+i) src: contains the java files for parsing the logfiles.
+ii) lib: contains jar files required for successful execution of the parser.
+iii) toolbox-1.1.1-release.zip: the visualization tool for OPM graphs.
+iv) config-files and shell-scripts: required for executing theend-to-end process for generating OPM graphs.
+
+Configuration-info:
+i) Copy the src/*.java files in a directory where the config-files & shell-scripts have been kept.
+ii) Copy the lib directory.
+iii) Unzip the OPM-toolbox
+iv) Modify the config-parameters ($CLASSPATH & $JAVA_HOME) in env.sh and provenance_collector.sh, if required.
+
+Compilation:
+Compile the source-code using the following commands:-
+i) source env.sh
+ii) make
+
+Additional-Info:
+The package also contains the following files which are required for the programs to execute:-
+i) ruleset.xml: the rule file to parse the geni logfiles.
+ii) ruleset.properties: some decode-info for the ruleset.
+
+Execution
+----------
+To generate the OPM graph, run the shell-script as:
+./provenance_collector.sh <rule.xml file> <logfile> <out_opm_file> <.dot file> <.pdf file> <opmtoolbox-path>
+}}}
+
+1. Unpacked the gush2netkarma-1.0.tar.zip which created directory named !NetKarmaFiles. 
+
+2. Executed the "Configuration-info" steps:
+
+   * Not sure what is the target of step i. Some configuration files are delivered at the top level of the !NetKarmaFiles
+directory, so I created links in the !NetKarmaFiles to the "src*.java" files.
+
+   * The file "env.sh" and "provenance_collector.sh" files set CLASSPATH to a directory named ~/geni_parser, based on this, the step ii copies the '''lib''' to '''~/geni_parser''' directory.
+
+   * Unzip a files named "toolbox-1.1.1-release.zip" which created a directory named "OPM-toolbox".
+
+   * Did not modify configuration parameters as files were placed in expected locations.
+
+3. Executed "Compilation" steps, ran into undocumented requirements:
+  * Two packages had to be installed: openjdk-6-jre and sun-java6-jdk.
+  * The env.sh sets the CLASSPATH, but also PATH needs to be set to point to /usr/lib/jvm/java-1.6.0-openjdk/bin
+
+Once the above were completed, was able to execute the compilation steps:
+{{{
+ $ source env.sh
+ $ make
+ javac -g PropertyReader.java
+ javac -g Decoder.java
+ javac -g ProvenanceSummary.java
+ javac -g ProvenanceInformation.java
+ javac -g Comparators.java
+ javac -g Constants.java
+ javac -g SelectionQueries.java
+ javac -g GeniLogParser.java
+ javac -g OPMMapper.java
+ javac -g OPMGraphGenerator.java
+}}}
+
+Tried to use Provenance collector script, but not enough information is available, could not get details
+about parameters and their meaning:
+{{{
+ $ ./provenance_collector.sh 
+Usage: ./provenance_collector.sh <rule.xml file> <logfile> <out_opm_file> <.dot file> <.pdf file> <opmtoolbox-path>
+}}}
+
+'''!NetKarma GEC8 Version How-to'''
+
+1. Downloaded geni_adaptor.tar.gz from https://www.slashtmp.iu.edu/public/download.php?FILE=dghoshal/44577wbJ1BL
+
+2. Unpacked the tgz bundle which delivers a directory named '''geni_parser''' containing a Readme.txt:
+
+{{{
+#
+# Copyright 2010 The Trustees of Indiana University
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# -----------------------------------------------------------------
+#
+# Project:  NetKarma Provenance Service Adaptor
+# File:  Readme.txt
+# Description:  
+#    This prototype provenance collection adaptor mines a log file against a 
+#    set of rules and generates provenance events.  The provenance events would
+#    normally be sent to the Karma service, but in this GEC7 demo (March 2010), the provenance
+#    events are written to a file.  The file is then fed as input to a visualization
+#    client that reads the file and generates a visual representation of the provenance 
+#    graph.
+# -----------------------------------------------------------------
+# 
+
+This module can be built from the included source.
+
+
+Building from Source:
+---------------------
+Prerequisites:
+i)  Apache-Ant  (the directory where this is installed is referred to as ANT_HOME)
+ii) JDK 1.6     (the directory where this is installed is referred to as JAVA_HOME)
+
+Edit the Configuration File:
+In the env.sh file, set the paths for the GENI_HOME, JAVA_HOME, and ANT_HOME 
+environment variables to match the directory where you unzipped the tarball (GENI_HOME)
+and where you installed java and ant respectively.
+
+Aside from the above prerequisites, all of the necessary source, configuration files, 
+and jars are included in the tarball named geni_adaptor.tar.gz.  Unzip this tarball and it will 
+create a directory named geni_parser (containing this readme file) that will be 
+your build and installation directory.
+This installation directory is referred to as <GENI_HOME>.
+
+Also, unzip the toolbox-1.1.1-release.zip.
+
+This module is built and run using a set of scripts.  The scripts in the <GENI_HOME>
+directory are configured for the bash shell.  If you are using the C shell, please 
+use the scripts in the <GENI_HOME>/tcsh_scripts instead.
+
+To build the parser, run the following script in the <GENI_HOME> directory.
+./compile.sh
+
+If you encounter an error, please be sure that both ANT and JDK 1.6 are on 
+your path. 
+
+Running the Geni Parser:
+------------------------
+Edit the Configuration File:
+Similar to compiling, please edit the settings for GENI_HOME and JAVA_HOME in the
+geni_stdenvs.cfg file used to set the environment for running the parser.
+
+Specifying the Rules Files:
+When the parser is run it expects the following two files which are used to specify 
+the rules used to map log entries into notifications. Defaults are included for these
+in the <GENI_HOME> directory:
+ i) ruleset.xml: the rule file to parse the geni logfiles.
+ ii) ruleset.properties: some decode-info for the ruleset.
+
+Execution:
+-----------
+To generate the OPM graph, run the shell-script as:
+provenance_collector.sh <logfile>
+where <logfile> is the log-file which is to be parsed
+
+There is a sample log file named "logfile-ubuntu-15555-1267499048.txt" in the tarball.
+
+Troubleshooting:
+----------------
+In case, you get any java-runtime linkage error for jaxb-api, put the
+lib/jaxb-api-2.1.jar in <JAVA_HOME>/lib/endorsed directory.
+}}}
+
+3. Install Apache Ant and JDK 1.6:
+{{{
+  $ sudo apt-get install ant
+  $ sudo apt-get install sun-java6-jdk 
+}}}
+4. Edit the configuration file env.sh to set GENI_HOME, JAVA_HOME, and ANT_HOME:
+
+{{{
+  export GENI_HOME=/home/lnevers/NetKarma/geni_parser
+  export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
+  export ANT_HOME=/usr/share/ant
+  export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$PATH
+
+  export CLASSPATH=$GENI_HOME/lib/provInfo.jar:$GENI_HOME/lib/commons-lang-2.3.jar:$GENI_HOME/lib/jaxb-impl.jar:$GENI_HOME/lib/jsr173_1.0_api.jar:$GENI_HOME/lib/opm-core-1.1.1.jar:$GENI_HOME/lib/opm-elmo-1.1.1.jar:$GENI_HOME/lib/Ruleset.jar:$GENI_HOME/lib/runtime-0.4.1.5.jar:$GENI_HOME/lib/xbean.jar:$GENI_HOME/lib/xml-apis-1.0.b2.jar:.
+
+}}}
+
+5. Unzipped toolbox-1.1.1-release.zip, which created a directory named opmToolbox:
+{{{
+  $ unzip toolbox-1.1.1-release.zip 
+}}}
+
+6. Compiled the parser tool:
+{{{
+ lnevers@sendaria:~/NetKarma/geni_parser$ ./compile.sh 
+ Buildfile: build.xml
+
+ build-subprojects:
+
+ init:
+
+ build-project:
+     [echo] Geni-Update-1.0: /home/lnevers/NetKarma/geni_parser/build.xml
+    [javac] Compiling 11 source files to /home/lnevers/NetKarma/geni_parser/bin
+
+ build:
+
+ BUILD SUCCESSFUL
+ Total time: 1 second
+}}}
+
+7. Running the parser tool requires the following:
+
+ * Edit the geni_stdenvs.cfg gile to set GENI_HOME and JAVA_HOME
+ * Check rule files ruleset.xml and ruleset.properties.
+
+Note: Initial attempt to run the parser showed a failure which resulted in the following packages being identified as required:
+{{{
+ $ sudo apt-get install graphviz
+ $ sudo apt-get install libgtkglext1
+ $ sudo apt-get install libgts-dev
+}}}
+
+Once the prerequisites are install, the parser can be run:
+{{{
+ $ ./provenance_collector.sh logfile-ubuntu-15555-1267499048.txt 
+}}}
+
+The provenance_collector.sh script generates 4 output files:
+
+ * opm_out.dot
+ * opm_out.pdf
+ * opm_out.rdf
+ * opm_out.xml
+
+The opm_out.pdf for logfile-ubuntu-15555-1267499048.txt shows the following:
+[[Image(2010-09-15_netkarma-pdf.jpg)]]
+
+The opm_out.rdf for logfile-ubuntu-15555-1267499048.txt when viewed with suggested RDF tools Welkin:
+[[Image(2010-09-15_netkarma-rdf.jpg)]]
+
+A "Play" button on the drawing tab for welkin tools moved the nodes in the upper-right-hand quadrant:
+[[Image(2010-09-15_netkarma-rdf2.jpg)]]
+
+The opm_out.dot for logfile-ubuntu-15555-1267499048.txt shows the following:
+[[Image(2010-09-15_netkarma-dot.jpg)]]