== GUSH Evaluation == GENI User Shell (GUSH) provides an execution management system that allows users to describe an experiment in an XML file, and Gush uses the xml file to locate, and prepare resources through interactions with GENI Clearinghouses. Gush will run and clean-up the experiment. Software can be checked out from the [http://gush.cs.williams.edu/svn/gush/trunk GUSH SVN Repository] using username '''guest''' and password is provided out of band. Software revisions used were 106, 107 and 108. The update from 106 to 107 to 108 modified the following files: {{{ Update from 106 to 107: U gush.prefs U tests/simple.xml U Makefile.in G main.cc G Makefile Updates from 107 to 108: U helper-scripts/setup.pl }}} Successful compilation was achieved with both version 107 and 108, but not 106. Time frame: Evaluation took place between June 1-30, 2010. == Gush Findings == List of pre-requisites is fairly extensive. Versioning information is not captured in the instructions. Several attempts to compile tools on Ubuntu 9.10 (karmic) failed, due to gcc 4.4.1 differences with gcc 4.2.4. Although developer has run tools on the following Operating Systems: * Linux 2.6 (Fedora, Debian, Ubuntu) * Mac OS X * FreeBSD Successful compilation On Ubuntu 9.10 was achieved with upgrade of GUSH to revision 107 and the following: * downgraded gcc 4.4.1 to gcc 4.2.4 * downgraded g++ 4.3 to g++ 4.2 * Added '#include ' to main.cc * Added '#include ' to main.cc * Could not configure static configuration to simplify compilation. Before using Gush, you must obtain an account and slice on !PlanetLab. Once you have the slice, the following must be considered for each experiment: * The gush software installs a client process on each node that is part of the experiment. * A port must be reserved at !PlanetLab for GUSH clients to communicate with the GUSH Server. ''' Reserving a port with !PlanetLab ''' To reserve a port, sign up for a port at [http://planetflow.planet-lab.org/#port port registration site]. Once reserved, send an email to support@planet-lab.org requesting the following slice changes: {{{ 1. the 'goodfences' vsys privilege. 2. the slice tag: vsys_portreservation, with the value set to the port number reserved. }}} Once support has made the changes to the slice, they can be seen in its definition: [[Image(2010-07-14_bbn_gusheval.jpg)]] == GUSH How-to == A README.txt is enclosed which captures all pre-requisites and compilation steps required to build GUSH. README.txt content: {{{ Gush is written in a mix of C++ and Perl, and we intend for it to be runnable on any UNIX-ish OS. That said, we have been using it primarily with Linux 2.6. Compiling Gush can be challenging, so please contact us if you experience too many problems. Gush requires the following libraries: * xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, libxmlrpc_server_abyss++, libxmlrpc_server++, libxmlrpc_server_abyss, libxmlrpc_server, libxmlrpc_abyss, libxmlrpc++, libxmlrpc, libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok Note: lbxmlrpc-c3 will not work. xmlrpc-c must be installed, and to my knowledge, there is no debian package available. * curl: libcurl * xml2: libxml2 * zlib compression/decompression: libz * math: libm * openssl (version 0.9.8e): libssl, libcrypto * dynamic linking: libdl * readline: libreadline * curses: libcurses * intl domain names: libidn * pthreads: libpthread (NOTE: NPTL recommended) The ares library (libares) for asynchronous DNS lookups is optional, though recommended for performance on PlanetLab. The boost software package is also required, as well as lex and yacc (we use flex and bison). Gush uses some Perl helper scripts. If you want Gush to automatically interface with the PlanetLab Central (PLC) database for you, you will need to have Frontier::Client and Crypt::SSLeay properly installed. We'll get to that in the next step. Getting and building the code After downloading the code from SVN, follow the instructions below from a Linux host. NOTE: This statically compiles gush and client. Remove the --with-static compile flags to compile dynamically (which may be easier to configure). Compiling Gush $ cd gush $ autoconf $ ./configure --with-static-client --with-static-gush $ make dep $ make Setting up Gush $ perl helper-scripts/setup.pl That should be it! Let us know if you have problems. This is still a work in progress! For more info, please visit http://gush.cs.williams.edu }}} Pre-requisite packages and libraries installed and the origin of each download: * xmlrpc-c-1.06.40.tgz from http://sourceforge.net/projects/xmlrpc-c/ * curl-7.20.1.tar.gz from http://www.softpedia.com/get/Programming/File-Editors/cURL.shtml * libxml2-git-snapshot.tar.gz from http://www.xmlsoft.org/downloads.html * zlib-1.2.5.tar.gz from http://www.zlib.net/ * libdl.1.dylib from http://cogent.psyc.bbk.ac.uk/libdl/ * libcureses from http://packages.ubuntu.com/en/jaunty/i386/libcurses-ocaml-dev/download * libidn from http://www.gnu.org/software/libidn/#downloading Additionally had to install Perl modules Frontier::Client and Crypt::SSLeay. ''' Gush Setup ''' Instructions can be found at the [http://gush.cs.williams.edu/trac/gush/wiki/UsingGush Using Gush] page. Once the gush software is compiled, a setup script is provided to define several settings. Below is capture for the settings used in this evaluation effort: {{{ $ perl helper-scripts/setup.pl Please select from the following options: 1. Configure known hosts 2. Configure Gush preferences 3. Quit > 1 Gush uses SSH and TCP to authenticate and run executables on remote hosts. To do this, Gush must know (a) which hosts it can use, and (b) how to authenticate to them. We will first handle the big cases, like PlanetLab, and then you will get the chance to add any other hosts you would like to use. Do you have a PlanetLab account (Y/N) [N]? Y Gush can automatically query the PlanetLab database for your slice information, renew your slices, and change the hosts assigned to your slice. To do this, it must have your planet-lab.org login (probably your email address) and password. Your password is stored in the clear in a separate file on the local disk. Please enter your planet-lab.org login: lnevers@bbn.com Please enter your planet-lab.org password: Please confirm your planet-lab.org password: Please wait while we fetch a recent version of the PlanetLab host set ... All done configuring PlanetLab. Writing files... Please enter your PlanetLab slice: bbn_gusheval Please choose a port for this slice (use a different port for each slice): 61414 Please enter your PlanetLab slice: Now we will configure any other hosts that you wish Gush to use. You can select any host that can be reached via SSH without password prompting (e.g. pubkey, kerberos tickets, ...). Enter a hostname of the form host:port (empty to finish): No more questions! Writing directory file... Done. Please select from the following options: 1. Configure known hosts 2. Configure Gush preferences 3. Quit > 2 To skip a preference, just hit enter. Enter your preferred ClientPath (i.e., /home/slice_name/): ./ Enter your preferred ClientPrefix (i.e., ./): ./ Do you plan to use PlanetLab? (Y/N) [Y]: y Done. Please select from the following options: 1. Configure known hosts 2. Configure Gush preferences 3. Quit > 3 }}} The above script generates 2 files: * gush.prefs {{{ ./ ./ true }}} * directory.xml {{{ lnevers@bbn.com allsites.xml }}} '''Using Gush''' First, make sure that you have nodes assigned to you and a slice defined in the Planetlab site. You can now run gush and issue any of the supported commands. The following example shows a connection to PlanetLab using the preferences defined above to all hosts in the slice. Mesh information is being requested as the experiment is set up: {{{ $ ./gush -P 15555 here gush> Gush has learned about the slice bbn_gusheval. gush> connect slice bbn_gusheval Found 12 hosts Initiated connections to 12 of 12 hosts. gush> bbn_gusheval@plnode-03.gpolab.bbn.com:61414 has joined the mesh. bbn_gusheval@plnode-04.gpolab.bbn.com:61414 has joined the mesh. bbn_gusheval@node2.planetlab.albany.edu:61414 has joined the mesh. bbn_gusheval@node-2.mcgillplanetlab.org:61414 has joined the mesh. bbn_gusheval@miranda.planetlab.cs.umd.edu:61414 has joined the mesh. bbn_gusheval@osiris.planetlab.cs.umd.edu:61414 has joined the mesh. bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414 has joined the mesh. bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414 has joined the mesh. bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414 has joined the mesh. bbn_gusheval@nis-planet2.doshisha.ac.jp:61414 has joined the mesh. For an unknown reason, we were unable to invite bbn_gusheval@dplanet2.uoc.edu:61414. gush> info mesh Mesh: bbn_gusheval@plnode-03.gpolab.bbn.com:61414: connected; (fds=6,6 last_heard=1279137801) bbn_gusheval@plnode-04.gpolab.bbn.com:61414: connected; (fds=7,7 last_heard=1279137801) bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=9,9 last_heard=1279137801) bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=12,12 last_heard=1279137802) bbn_gusheval@pl2.planet.cs.kent.edu:61414: connecting via TCP; (fds=-1,-1 last_heard=0) bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=14,14 last_heard=1279137802) bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1279137801) bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=16,16 last_heard=1279137801) bbn_gusheval@dplanet2.uoc.edu:61414: failed; (fds=22,21 last_heard=0) bbn_gusheval@nis-planet2.doshisha.ac.jp:61414: connected; (fds=17,17 last_heard=1279137804) bbn_gusheval@node-2.mcgillplanetlab.org:61414: connected; (fds=19,19 last_heard=1279137801) bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414: connected; (fds=20,20 last_heard=1279137803) Summary: 10 mesh members; 1 nodes connecting. }}} The connect command can also use a "pat" argument (where pat=pattern) to request a subset of nodes. The next example requests 10 nodes that have the patten "planetlab" in their hostname, but only 6 nodes in the slice match the specified pattern: {{{ lnevers@sendaria:~/gush/trunk$ ./gush -P 15555 here gush> Gush has learned about the slice bbn_gusheval. gush> connect pat @planetlab 10 Initiated connections to 6 of 6 hosts. gush> For an unknown reason, we were unable to invite bbn_gusheval@node2.planetlab.albany.edu:61414. bbn_gusheval@node2.planetlab.albany.edu:61414 has joined the mesh. bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414 has joined the mesh. gush> info mesh Mesh: bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689) bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0) bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695) bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0) bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: bootstrapping; (fds=-1,-1 last_heard=0) bbn_gusheval@node-2.mcgillplanetlab.org:61414: bootstrapping; (fds=-1,-1 last_heard=0) Summary: 2 mesh members; 4 nodes connecting. gush> bbn_gusheval@miranda.planetlab.cs.umd.edu:61414 has joined the mesh. bbn_gusheval@osiris.planetlab.cs.umd.edu:61414 has joined the mesh. bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414 has joined the mesh. gush> info mesh Mesh: bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689) bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715) bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695) bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700) bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701) bbn_gusheval@node-2.mcgillplanetlab.org:61414: bootstrapping; (fds=-1,-1 last_heard=0) Summary: 5 mesh members; 1 nodes connecting. gush> info mesh Mesh: bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689) bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715) bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695) bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700) bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701) bbn_gusheval@node-2.mcgillplanetlab.org:61414: handshake (fds=16,13 last_heard=0) Summary: 5 mesh members; 0 nodes connecting. gush> info mesh Mesh: bbn_gusheval@node2.planetlab.albany.edu:61414: connected; (fds=6,6 last_heard=1278010689) bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414: connected; (fds=7,7 last_heard=1278010715) bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414: connected; (fds=12,12 last_heard=1278010695) bbn_gusheval@miranda.planetlab.cs.umd.edu:61414: connected; (fds=14,14 last_heard=1278010700) bbn_gusheval@osiris.planetlab.cs.umd.edu:61414: connected; (fds=15,15 last_heard=1278010701) bbn_gusheval@node-2.mcgillplanetlab.org:61414: handshake (fds=16,13 last_heard=0) Summary: 5 mesh members; 0 nodes connecting. gush> For an unknown reason, we were unable to invite bbn_gusheval@node-2.mcgillplanetlab.org:61414. }}} Other Gush commands: Command: Show slice information {{{ gush> info slice bbn_gusheval Slice bbn_gusheval #hosts=12 exp=1279808609 and options _ _ }}} Command: Renew slice {{{ gush> slice renew bbn_gusheval gush> The slice bbn_gusheval has been renewed. }}} Command: Execute command on each node, this example executes the hostname command. {{{ gush> shell "hostname" gush> bbn_gusheval@node2.planetlab.albany.edu:61414,14011: node2.planetlab.albany.edu bbn_gusheval@miranda.planetlab.cs.umd.edu:61414,21420: miranda.planetlab.cs.umd.edu bbn_gusheval@osiris.planetlab.cs.umd.edu:61414,28140: osiris.planetlab.cs.umd.edu bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414,3352: node2.planetlab.mathcs.emory.edu bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414,21140: mtuplanetlab2.cs.mtu.edu }}} Command: Show Node Status {{{ gush> info nodes There are 12 known nodes: [ P ] bbn_gusheval@plnode-03.gpolab.bbn.com:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@plnode-04.gpolab.bbn.com:61414(pref=0) (Disconnected.) [ P C ] bbn_gusheval@node2.planetlab.albany.edu:61414(pref=10) (Idle.) [ P C ] bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414(pref=10) (Idle.) [ P ] bbn_gusheval@pl2.planet.cs.kent.edu:61414(pref=0) (Disconnected.) [ P C ] bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414(pref=10) (Idle.) [ P C ] bbn_gusheval@miranda.planetlab.cs.umd.edu:61414(pref=10) (Idle.) [ P C ] bbn_gusheval@osiris.planetlab.cs.umd.edu:61414(pref=10) (Idle.) [ P ] bbn_gusheval@dplanet2.uoc.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@nis-planet2.doshisha.ac.jp:61414(pref=0) (Disconnected.) [ P F ] bbn_gusheval@node-2.mcgillplanetlab.org:61414(pref=-25) (Failed.) [ P ] bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414(pref=0) (Disconnected.) gush> }}} The gush software installs a client process on each node. In an example where the host sendaria was used to run gush, and the client was configured for port 61414, the following process will run on each of the nodes: {{{ ./client -c ./ -d 5 -P 61414 -b lnevers@sendaria:15555 }}} '''GENI Integration Release 2.1 Guidelines''' Spiral 2 Release 1 [[BR]] {{{ #!html Email us with any questions and feedback on this page! }}}