Changes between Initial Version and Version 1 of GIR2.1_GushProto

Timestamp:

03/14/11 21:56:55 (13 years ago)

Author:

lnevers@bbn.com

Comment:

--

GIR2.1_GushProto

@@ +1 @@
+== GUSH Evaluation ==
+
+GENI User Shell (GUSH) provides an execution management system that allows users to describe 
+an experiment in an XML file, and Gush uses the xml file to locate, and prepare resources 
+through interactions with GENI Clearinghouses. Gush will run and clean-up the experiment.
+
+Software can be checked out from the [http://gush.cs.williams.edu/svn/gush/trunk GUSH SVN Repository]
+using username '''guest''' and password is provided out of band. Software revisions used 
+were 106, 107 and 108. The update from 106 to 107 to 108 modified the following files:
+{{{
+Update from 106 to 107:
+U    gush.prefs
+U    tests/simple.xml
+U    Makefile.in
+G    main.cc
+G    Makefile
+Updates from 107 to 108:
+U    helper-scripts/setup.pl
+}}}
+
+Successful compilation was achieved with both version 107 and 108, but not 106.
+
+Time frame: Evaluation took place between June 1-30, 2010.
+
+== Gush Findings ==
+
+List of pre-requisites is fairly extensive. Versioning information is not captured in the instructions.
+Several attempts to compile tools on Ubuntu 9.10 (karmic) failed, due to gcc 4.4.1 differences with 
+gcc 4.2.4.  Although developer has run tools on the following Operating Systems:
+ * Linux 2.6 (Fedora, Debian, Ubuntu)
+ * Mac OS X
+ * FreeBSD
+
+Successful compilation On Ubuntu 9.10 was achieved with upgrade of GUSH to revision 107 and the following:
+ * downgraded gcc 4.4.1 to gcc 4.2.4
+ * downgraded g++ 4.3 to g++ 4.2 
+ * Added '#include <limits.h>' to main.cc
+ * Added '#include <strings.h>' to main.cc
+ * Could not configure static configuration to simplify compilation.
+
+Before using Gush, you must obtain an account and slice on !PlanetLab. Once you have the slice, the following must be considered for each experiment:
+ * The gush software installs a client process on each node that is part of the experiment.  
+ * A port must be reserved at !PlanetLab for GUSH clients to communicate with the GUSH Server.
+
+''' Reserving a port with !PlanetLab '''
+
+To reserve a port, sign up for a port at [http://planetflow.planet-lab.org/#port port registration site].
+Once reserved, send an email to support@planet-lab.org requesting the following slice changes:
+{{{
+ 1. the 'goodfences' vsys privilege.
+ 2. the slice tag: vsys_portreservation, with the value set to the port number reserved.
+}}}
+
+Once support has made the changes to the slice, they can be seen in its definition:
+
+[[Image(2010-07-14_bbn_gusheval.jpg)]]
+
+== GUSH How-to ==
+
+A README.txt is enclosed which captures all pre-requisites and compilation steps
+required to build GUSH.  README.txt content:
+
+{{{
+Gush is written in a mix of C++ and Perl, and we intend for it to be runnable on
+any UNIX-ish OS. That said, we have been using it primarily with Linux 2.6.
+
+Compiling Gush can be challenging, so please contact us if you experience
+too many problems.  Gush requires the following libraries:
+
+    * xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, 
+      libxmlrpc_server_abyss++, libxmlrpc_server++, 
+      libxmlrpc_server_abyss, libxmlrpc_server, 
+      libxmlrpc_abyss, libxmlrpc++, libxmlrpc, 
+      libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok
+      
+      Note: lbxmlrpc-c3 will not work. xmlrpc-c must be 
+      installed, and to my knowledge, there is no debian package available.
+
+    * curl: libcurl
+    * xml2: libxml2
+    * zlib compression/decompression: libz
+    * math: libm
+    * openssl (version 0.9.8e): libssl, libcrypto
+    * dynamic linking: libdl
+    * readline: libreadline
+    * curses: libcurses
+    * intl domain names: libidn
+    * pthreads: libpthread (NOTE: NPTL recommended)
+
+    The ares library (libares) for asynchronous DNS lookups is optional, 
+    though recommended for performance on PlanetLab. The boost 
+    software package is also required, as well as lex and yacc 
+    (we use flex and bison).
+
+    Gush uses some Perl helper scripts. If you want Gush to automatically 
+    interface with the PlanetLab Central (PLC) database for you, you will 
+    need to have Frontier::Client and Crypt::SSLeay properly installed. 
+    We'll get to that in the next step.
+
+Getting and building the code
+
+    After downloading the code from SVN, follow the instructions below 
+    from a Linux host. NOTE: This statically compiles gush and client. Remove 
+    the --with-static compile flags to compile dynamically (which may be 
+    easier to configure).
+
+    Compiling Gush
+      $ cd gush
+      $ autoconf
+      $ ./configure --with-static-client --with-static-gush
+      $ make dep
+      $ make
+
+    Setting up Gush
+      $ perl helper-scripts/setup.pl
+
+That should be it!  Let us know if you have problems.  This is still a work 
+in progress!
+
+For more info, please visit http://gush.cs.williams.edu
+}}}
+
+Pre-requisite packages and libraries installed and the origin of each download:
+
+ * xmlrpc-c-1.06.40.tgz from http://sourceforge.net/projects/xmlrpc-c/
+ * curl-7.20.1.tar.gz from http://www.softpedia.com/get/Programming/File-Editors/cURL.shtml
+ * libxml2-git-snapshot.tar.gz from http://www.xmlsoft.org/downloads.html 
+ * zlib-1.2.5.tar.gz from http://www.zlib.net/
+ * libdl.1.dylib from http://cogent.psyc.bbk.ac.uk/libdl/
+ * libcureses from http://packages.ubuntu.com/en/jaunty/i386/libcurses-ocaml-dev/download
+ * libidn  from http://www.gnu.org/software/libidn/#downloading
+
+Additionally had to install Perl modules Frontier::Client and Crypt::SSLeay.
+
+''' Gush Setup '''
+
+Instructions can be found at the [http://gush.cs.williams.edu/trac/gush/wiki/UsingGush Using Gush] page. Once the gush software is compiled, a setup script is provided to define several settings.  Below is capture for the settings used in this evaluation effort:
+{{{
+$ perl helper-scripts/setup.pl 
+Please select from the following options:
+  1. Configure known hosts
+  2. Configure Gush preferences
+  3. Quit
+> 1
+Gush uses SSH and TCP to authenticate and run executables on remote
+hosts. To do this, Gush must know (a) which hosts it can use,
+and (b) how to authenticate to them. We will first handle the
+big cases, like PlanetLab, and then you will get the chance to
+add any other hosts you would like to use.
+
+Do you have a PlanetLab account (Y/N) [N]? Y
+
+Gush can automatically query the PlanetLab database for
+your slice information, renew your slices, and change the
+hosts assigned to your slice. To do this, it must have
+your planet-lab.org login (probably your email address)
+and password. Your password is stored in the clear in a
+separate file on the local disk.
+
+Please enter your planet-lab.org login: lnevers@bbn.com
+Please enter your planet-lab.org password: 
+Please confirm your planet-lab.org password: 
+Please wait while we fetch a recent version of the
+PlanetLab host set ...
+
+
+All done configuring PlanetLab. Writing files...
+Please enter your PlanetLab slice: bbn_gusheval
+Please choose a port for this slice (use a different port for each slice): 61414 
+Please enter your PlanetLab slice: 
+Now we will configure any other hosts that you wish Gush to use.
+You can select any host that can be reached via SSH without password
+prompting (e.g. pubkey, kerberos tickets, ...).
+
+Enter a hostname of the form host:port (empty to finish): 
+
+No more questions! Writing directory file...
+Done.
+
+Please select from the following options:
+  1. Configure known hosts
+  2. Configure Gush preferences
+  3. Quit
+> 2
+To skip a preference, just hit enter.
+
+Enter your preferred ClientPath (i.e., /home/slice_name/): ./
+Enter your preferred ClientPrefix (i.e., ./): ./
+Do you plan to use PlanetLab? (Y/N) [Y]: y
+Done.
+
+Please select from the following options:
+  1. Configure known hosts
+  2. Configure Gush preferences
+  3. Quit
+> 3
+}}}
+
+The above script generates 2 files:
+
+ * gush.prefs
+{{{
+<preferences>
+    <pref key="ClientPath">./</pref>
+    <pref key="ClientPrefix">./</pref>
+    <pref key="UsePlanetLab">true</pref>
+</preferences>
+}}}
+ * directory.xml
+{{{
+<?xml version="1.0" encoding="UTF-8"?>
+<gush>
+    <resource_manager type="planetlab">
+        <user>lnevers@bbn.com</user>
+        <allsites>allsites.xml</allsites>
+        <port_map slice="bbn_gusheval" port="61414"/>
+    </resource_manager>
+    <resource_manager type="ssh">
+    </resource_manager>
+</gush>
+}}}
+
+
+'''Using Gush'''
+
+First, make sure that you have nodes assigned to you and a slice defined in the Planetlab site.
+
+You can now run gush and issue any of the supported commands.  The following example shows a 
+connection to PlanetLab using the preferences defined above to all hosts in the slice. Mesh 
+information is being requested as the experiment is set up:
+
+{{{
+$ ./gush -P 15555 
+here
+gush> Gush has learned about the slice bbn_gusheval.
+gush> connect slice bbn_gusheval
+Found 12 hosts
+Initiated connections to 12 of 12 hosts.
+gush> bbn_gusheval@plnode-03.gpolab.bbn.com:61414 has joined the mesh.
+bbn_gusheval@plnode-04.gpolab.bbn.com:61414 has joined the mesh.
+bbn_gusheval@node2.planetlab.albany.edu:61414 has joined the mesh.
+bbn_gusheval@node-2.mcgillplanetlab.org:61414 has joined the mesh.
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414 has joined the mesh.
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414 has joined the mesh.
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414 has joined the mesh.
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414 has joined the mesh.
+bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414 has joined the mesh.
+bbn_gusheval@nis-planet2.doshisha.ac.jp:61414 has joined the mesh.
+For an unknown reason, we were unable to invite bbn_gusheval@dplanet2.uoc.edu:61414.
+
+gush> info mesh
+Mesh:
+bbn_gusheval@plnode-03.gpolab.bbn.com:61414: connected; (fds=6,6 last_heard=1279137801)
+bbn_gusheval@plnode-04.gpolab.bbn.com:61414: connected; (fds=7,7 last_heard=1279137801)
+bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=9,9 last_heard=1279137801)
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=12,12 last_heard=1279137802)
+bbn_gusheval@pl2.planet.cs.kent.edu:61414: connecting via TCP; (fds=-1,-1 last_heard=0)
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=14,14 last_heard=1279137802)
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1279137801)
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=16,16 last_heard=1279137801)
+bbn_gusheval@dplanet2.uoc.edu:61414: failed; (fds=22,21 last_heard=0)
+bbn_gusheval@nis-planet2.doshisha.ac.jp:61414: connected; (fds=17,17 last_heard=1279137804)
+bbn_gusheval@node-2.mcgillplanetlab.org:61414: connected; (fds=19,19 last_heard=1279137801)
+bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414: connected; (fds=20,20 last_heard=1279137803)
+Summary: 10 mesh members; 1 nodes connecting.
+}}}
+
+The connect command can also use a "pat" argument (where pat=pattern) to request a subset of nodes.
+The next example requests 10 nodes that have the patten "planetlab" in their hostname, but only 6 
+nodes in the slice match the specified pattern:
+{{{
+lnevers@sendaria:~/gush/trunk$ ./gush -P 15555
+here
+gush> Gush has learned about the slice bbn_gusheval.
+      
+gush> connect pat @planetlab 10
+Initiated connections to 6 of 6 hosts.
+gush> For an unknown reason, we were unable to invite bbn_gusheval@node2.planetlab.albany.edu:61414.
+bbn_gusheval@node2.planetlab.albany.edu:61414 has joined the mesh.
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414 has joined the mesh.
+
+gush> info mesh
+Mesh:
+bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689)
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0)
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695)
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0)
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0)
+bbn_gusheval@node-2.mcgillplanetlab.org:61414: bootstrapping; (fds=-1,-1 last_heard=0)
+Summary: 2 mesh members; 4 nodes connecting.
+
+gush> bbn_gusheval@miranda.planetlab.cs.umd.edu:61414 has joined the mesh.
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414 has joined the mesh.
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414 has joined the mesh.
+
+gush> info mesh
+Mesh:
+bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689)
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715)
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695)
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700)
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701)
+bbn_gusheval@node-2.mcgillplanetlab.org:61414: bootstrapping; (fds=-1,-1 last_heard=0)
+Summary: 5 mesh members; 1 nodes connecting.
+gush> info mesh
+Mesh:
+bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689)
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715)
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695)
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700)
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701)
+bbn_gusheval@node-2.mcgillplanetlab.org:61414: handshake (fds=16,13 last_heard=0)
+Summary: 5 mesh members; 0 nodes connecting.
+
+gush> info mesh
+Mesh:
+bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689)
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715)
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695)
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700)
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701)
+bbn_gusheval@node-2.mcgillplanetlab.org:61414: handshake (fds=16,13 last_heard=0)
+Summary: 5 mesh members; 0 nodes connecting.
+gush> For an unknown reason, we were unable to invite bbn_gusheval@node-2.mcgillplanetlab.org:61414.
+
+}}}
+
+Other Gush commands:
+Command: Show slice information
+{{{
+gush> info slice bbn_gusheval
+Slice bbn_gusheval #hosts=12 exp=1279808609 and options _ _
+}}}
+Command: Renew slice
+{{{
+gush> slice renew bbn_gusheval
+gush> The slice bbn_gusheval has been renewed.
+}}}
+Command: Execute command on each node, this example executes the hostname command.
+{{{
+gush> shell "hostname"
+gush> 
+bbn_gusheval@node2.planetlab.albany.edu:61414,14011: node2.planetlab.albany.edu
+bbn_gusheval@miranda.planetlab.cs.umd.edu:61414,21420: miranda.planetlab.cs.umd.edu
+bbn_gusheval@osiris.planetlab.cs.umd.edu:61414,28140: osiris.planetlab.cs.umd.edu
+bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414,3352: node2.planetlab.mathcs.emory.edu
+bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414,21140: mtuplanetlab2.cs.mtu.edu
+}}}
+
+Command: Show Node Status
+{{{
+gush> info nodes
+There are 12 known nodes:
+[ P         ] bbn_gusheval@plnode-03.gpolab.bbn.com:61414(pref=0) (Disconnected.)
+[ P         ] bbn_gusheval@plnode-04.gpolab.bbn.com:61414(pref=0) (Disconnected.)
+[ P   C     ] bbn_gusheval@node2.planetlab.albany.edu:61414(pref=10) (Idle.)
+[ P   C     ] bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414(pref=10) (Idle.)
+[ P         ] bbn_gusheval@pl2.planet.cs.kent.edu:61414(pref=0) (Disconnected.)
+[ P   C     ] bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414(pref=10) (Idle.)
+[ P   C     ] bbn_gusheval@miranda.planetlab.cs.umd.edu:61414(pref=10) (Idle.)
+[ P   C     ] bbn_gusheval@osiris.planetlab.cs.umd.edu:61414(pref=10) (Idle.)
+[ P         ] bbn_gusheval@dplanet2.uoc.edu:61414(pref=0) (Disconnected.)
+[ P         ] bbn_gusheval@nis-planet2.doshisha.ac.jp:61414(pref=0) (Disconnected.)
+[ P       F ] bbn_gusheval@node-2.mcgillplanetlab.org:61414(pref=-25) (Failed.)
+[ P         ] bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414(pref=0) (Disconnected.)
+gush> 
+}}}
+
+The gush software installs a client process on each node.  In an example where
+the host sendaria was used to run gush, and the client was configured for port 61414,
+the following process will run on each of the nodes:
+{{{
+ ./client -c ./ -d 5 -P 61414 -b lnevers@sendaria:15555
+}}}