wiki:GIR3.1_GushProto

Version 8 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

GUSH Evaluation

GENI User Shell (GUSH) provides an execution management system that allows users to describe an experiment in an XML file, and Gush uses the xml file to locate, and prepare resources through interactions with GENI Clearinghouses. Gush tools run and clean-up an experiment. Gush software can be checked out from the GUSH SVN Repository using username guest and password is provided out of band. Software revisions used is 120.

Evaluation Time frame:

  • First Evaluation took place between March 15, 28, 2011. Software revisions 120 & 126.
  • Second Evaluation took place on April 11, 2011. Software version 136.
  • Third Evaluation took place April 19,2011. Software version 141.

Gush Findings

The first and second evaluation completed found environment and packaging issues, which were resolved by the third evaluation. The third version delivered a version that build without environment issues as documented.

GUSH How-to

Third Evaluation (rev 141)

Using an ubuntu 10.04.01 VM was able to SVN check out and build gush as instructed in the enclosed README.txt. Also, a pre-build binary was installed and configured as part of the evaluation. Both approaches are captured in this section.

GUSH code

Checked out GUSH repository:

$ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest

Note: password availaliable upon request.

GUSH Binary Package

Downloaded the 32 bit Ubuntu Linux Binaries named gush-bin-32bit.tgz. Note the package delivers several file at the top level, so created a directory to contain all files delivered as part of the package. Following procedure was used:

    $ mkdir gush-bin
    $ cd gush-bin
    $ wget http://gush.cs.williams.edu/gush-bin-32bit.tgz
    $ tar xvzf gush-bin-32bit.tgz 

Configurations were modified for define user specific details. The following directory.xml was put in place to access GENI resources:

<?xml version="1.0" encoding="UTF-8"?>
<gush>
    <resource_manager type="geni">
        <user>lnevers@bbn.com</user>
      <port_map slice="bbn_gusheval" port="61414"/>
    </resource_manager>
</gush>

The following omni_configuration was used:

[omni]
default_cf = pg
users = lnevers

# ---------- Users ----------
[lnevers]
urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
keys = ~/.ssh/id_rsa.pub

# ---------- Frameworks ----------
[pg]
type = pg
ch = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/ch
sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa
cert = ~/.ssl/encrypted-cleartext.pem
key = ~/.ssl/encrypted-cleartext.pem
verbose=false

To Be captured: Running gush on PlanetLab slice and on PG Slice.

    $  ./gush -P 15555
      gush> Gush has learned about the slice XXX.

First (rev 126) and Second (rev 136) Evaluations

Using a newly build Ubuntu 10.04.2 VM for this evaluation in order to verify all prerequisites. For first evaluation (rev 126), gush gush code was evaluated. During the evaluation found that a binary pages had been added. Binary package evaluation was completed with the latter version (rev 136).

Had to installed SVN package (Subversion version 1.6.6) not explicitly stated as a prerequisite and then checked out the GUSH repo:

$ sudo apt-get install subversion
$ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest

No changes have been made to the README.txt since the last GUSH Evaluation:

Gush is written in a mix of C++ and Perl, and we intend for it to be runnable 
on any UNIX-ish OS. That said, we have been using it primarily with Linux 2.6.

Compiling Gush can be challenging, so please contact us if you experience
too many problems.  Gush requires the following libraries:

    * xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, 
      libxmlrpc_server_abyss++, libxmlrpc_server++, 
      libxmlrpc_server_abyss, libxmlrpc_server, 
      libxmlrpc_abyss, libxmlrpc++, libxmlrpc, 
      libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok
      
      Note: lbxmlrpc-c3 will not work. xmlrpc-c must be 
      installed, and to my knowledge, there is no debian package available.

    * curl: libcurl
    * xml2: libxml2
    * zlib compression/decompression: libz
    * math: libm
    * openssl (version 0.9.8e): libssl, libcrypto
    * dynamic linking: libdl
    * readline: libreadline
    * curses: libcurses
    * intl domain names: libidn
    * pthreads: libpthread (NOTE: NPTL recommended)

    The ares library (libares) for asynchronous DNS lookups is optional, 
    though recommended for performance on PlanetLab. The boost 
    software package is also required, as well as lex and yacc 
    (we use flex and bison).

    Gush uses some Perl helper scripts. If you want Gush to automatically 
    interface with the PlanetLab Central (PLC) database for you, you will 
    need to have Frontier::Client and Crypt::SSLeay properly installed. 
    We'll get to that in the next step.

Getting and building the code

    After downloading the code from SVN, follow the instructions below 
    from a Linux host. NOTE: This statically compiles gush and client. Remove 
    the --with-static compile flags to compile dynamically (which may be 
    easier to configure).

    Compiling Gush
      $ cd gush
      $ autoconf
      $ ./configure --with-static-client --with-static-gush
      $ make dep
      $ make

    Setting up Gush
      $ perl helper-scripts/setup.pl

That should be it!  Let us know if you have problems.  This is still a work 
in progress!

Following is a capture of all software installed to meet GUSH pre-requisites:

$ sudo apt-get install gcc
$ sudo apt-get install g++
$ wget https://launchpad.net/ubuntu/+archive/primary/+files/xmlrpc-c_1.06.27.orig.tar.gz
$ tar xvzf xmlrpc-c_1.06.27.orig.tar.gz 
$ cd xmlrpc-c-1.06.27/
$ ./configure
$ make
$ sudo make install ; cd ..
$ sudo apt-get install curl
$ sudo apt-get install xml2
$ wget http://www.zlib.net/zlib-1.2.5.tar.gz
$ tar -xvzf zlib-1.2.5.tar.gz
$ cd zlib-1.2.5/
$ ./configure --prefix=/usr/local/zlib
$ make 
$ sudo make install ; cd ..

Notes: The libm and pthreads libraries and OpenSSL were already on systems.
      
$ sudo apt-get install libreadline6
$ sudo apt-get install libncurses5
$ sudo apt-get install libidn11

Found that Perl library Crypt::SSLeay required the libssl-dev package:

$ sudo apt-get install libssl-dev
$ sudo cpan
cpan[1]> install Frontier::Client           
cpan[2]> install Crypt::SSLeay
cpan[3]> quit

Now onto the GUSH code:

$ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest
$ cd trunk
$ sudo apt-get install autoconf
$ ./configure --with-static-client --with-static-gush

Configure and make had "numerous failures" which were addressed by installing the following additional packages:

$ sudo apt-get install pkg-config libxml2-dev
$ sudo apt-get install libboost-dev makedepend  
$ sudo apt-get install bison flex
$ sudo apt-get install libreadline6 libreadline6-dev libexpat1-dev
$ sudo apt-get install libcurl4-openssl-dev

Note that various changes had to be made for the "make" to work. Added the following to main.cc file:

 #include <limits.h>
 #include <strings.h>
 #include <stdlib.h>
 #include <string.h>
 #include <cstring>

Also the gcc-4.4 and g++-4.4 had to both be downgraded to avoid the following build issue:

g++ -g -Wall -Wno-unused -I. -I/usr/include/libxml2 -I/usr/local/include     -c version.cc -o version.o
version.cc: In constructor 'FileVersion::FileVersion(const char*)':
version.cc:56: error: invalid conversion from 'const char*' to 'char*'
version.cc:63: error: invalid conversion from 'const char*' to 'char*'
version.cc:69: error: invalid conversion from 'const char*' to 'char*'
version.cc:75: error: invalid conversion from 'const char*' to 'char*'
version.cc:81: error: invalid conversion from 'const char*' to 'char*'
make: *** [version.o] Error 1

With the additional packages, environment settings and downgraded versions was able to complete:

$ ./configure  --with-static-client --with-static-gush
$ make dep
$ make

Also found the following are required for LD_LIBRARY_PATH: /usr/local/lib:/usr/lib:/usr/local/include/

Was able to eventually compile without the "--with-static-client --with-static-gush" flags.

Initial Binary package evaluation

Was not able to use the first version of the Binary package due to missing files, had to install binary on top of svn checkout of gush repo. With the combination of the two was able to run all new "./helper-scripts/handle-geni.py" features.