[[PageOutline]] = GUSH Evaluation = GENI User Shell ([http://gush.cs.williams.edu/trac/gush GUSH]) provides an execution management system that allows users to describe an experiment in an XML file, and Gush uses the xml file to locate, and prepare resources through interactions with GENI Clearinghouses. Gush tools run and clean-up an experiment. Gush software can be checked out from the [http://gush.cs.williams.edu/svn/gush/trunk GUSH SVN Repository] using username '''guest''' and password is provided out of band. Software revisions used is 120. Evaluation Time frame: * First Evaluation took place between March 15, 28, 2011. Software revisions 120 & 126. * Second Evaluation took place on April 11, 2011. Software version 136. * Third Evaluation took place April 19,2011. Software version 141. = Gush Findings = The first and second evaluation completed found environment and packaging issues, which were resolved by the third evaluation. The third version delivered a version that build without environment issues as documented. Additionally the third version (rev 141) was used to manage both PL and PG resources. = GUSH How-to = == Third Evaluation (rev 141) == Using an ubuntu 10.04.01 VM was able to SVN check out and build gush as instructed in the enclosed README.txt. Also, a pre-build binary was installed and configured as part of the evaluation. Both approaches are captured in this section. === GUSH code === Checked out GUSH repository: {{{ $ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest $ cd trunk }}} Using the newly updated README.tx was able to complete all steps as instructed: {{{ * Prerequisites This page describes the setup process required to use Gush with PlanetLab. If you want to use Gush with ProtoGENI, please check out the ProtoGeniExample page. Important: Before using Gush with PlanetLab, you must obtain an account/slice on PlanetLab. In addition, Gush requires the ability to SSH to PlanetLab hosts without typing a passphrase. You should set up an ssh-agent to provide password-free ssh access to PlanetLab hosts. To handle the case where connecting to a new node brings up a fingerprint verification question, add "StrictHostKeyChecking no" to your ~/.ssh/config. We have found that PlanetLab keys change periodically, and by not requiring StrictHostKeyChecking we encounter less errors. ** Library Dependencies Gush is written in a mix of C++ and Python, and we intend for it to be runnable on any UNIX-ish OS. That said, we have been using it primarily with Linux and Mac OS. Compiling Gush can be challenging, so please contact us if you experience too many problems. Gush requires the following libraries: * xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, libxmlrpc_server_abyss++, libxmlrpc_server++, libxmlrpc_server_abyss, libxmlrpc_server, libxmlrpc_abyss, libxmlrpc++, libxmlrpc, libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok Note: lbxmlrpc-c3 will not work. xmlrpc-c must be installed, and to our knowledge, there is no debian package available. * curl: libcurl * xml2: libxml2 * zlib compression/decompression: libz * math: libm * openssl: libssl, libcrypto * dynamic linking: libdl * readline: libreadline * curses: libcurses * intl domain names: libidn * pthreads: libpthread The boost software package is also required, as well as lex and yacc (we use flex and bison). Gush supports the latest version of the GENI API. It uses a modified version of the Omni client for communication with GENI aggregate managers. More information about Omni can be found on the OmniOverview page and on the QuickStart guide. Since Omni is written in Python, you must install a few Python libraries in addition to the C++ libraries mentioned above. More details about configuring Python and Omni are explained below. For reference, the following commands were used to install the prerequisite software for Gush (including Omni) on a new machine with a clean installation of Ubuntu 10.04 Desktop 64bit. $ sudo apt-get install openssh-server subversion keychain g++-4.1 autoconf2.59 makedepend emacs23 $ sudo ln -s /usr/bin/g++-4.1 /usr/bin/g++ $ sudo apt-get install curl libcurl3 libcurl4-openssl-dev libboost-dev xutils-dev flex bison libreadline-dev $ sudo apt-get install python-m2crypto python-dateutil python-pyopenssl libxmlsec1 xmlsec1 libxmlsec1-openssl libxmlsec1-dev $ wget http://gush.cs.williams.edu/xmlrpc-c-1.16.34.tgz $ tar xzf xmlrpc-c-1.16.34.tgz $ cd xmlrpc-c-1.16.34 $ ./configure $ make $ sudo make install * Using Gush with PlanetLab After installing the required libraries mentioned above, the first step to using Gush is to download and compile the code. The code can be obtained from the SVN repository linked to this site. You must request an account before obtaining the code. Please email jeannie AT cs dot williams dot edu for access to the repository. After obtaining the password, check out Gush using the following command: $ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest Once the code has been downloaded, follow the instructions found in README.txt to compile and install Gush. These instructions are included here for your reference. NOTE: We have statically compiled versions of both the Gush and Gush client binaries available. If you would like to skip the compilation step (which can be tedious!), most users can simply use the statically compiled binaries. We currently have binaries for 64 bit Ubuntu Linux and an older 32 bit Ubuntu Linux. Other Linux distributions may be able to run these binaries, but this has not been tested fully. To use these binaries, download the appropriate tarball, untar it, and skip ahead to the "Configuring Python for Omni" section below. 64 bit Ubuntu Linux: gush-bin-64bit.tgz 32 bit Ubuntu Linux: gush-bin-32bit.tgz ** Compiling Gush After downloading the code from SVN, follow the instructions below to compile Gush. NOTE: This dynamically compiles Gush and the Gush client. You can compile statically using the --with-static-gush and --with-static-client compile flags. Statically compiling is usually a bit more error prone because of some issues with curl. If static compilation is required, we recommend installing curl directly from source and using --disable-ldap while compiling curl. For most installations, dynamically compiling should be sufficient. $ cd gush/trunk $ autoconf (Make sure you use autoconf version 2.50 or higher.) $ ./configure $ make dep $ make ** Configuring Python for Omni Python 2.6 is required. In addition, the following Python packages must be installed: M2Crypto, dateutil, OpenSSL, and xmlsec1. According to the Omni QuickStart page, the following installation methods may be helpful: * Debian / Ubuntu $ sudo apt-get install python-m2crypto python-dateutil python-pyopenssl libxmlsec1 xmlsec1 libxmlsec1-openssl libxmlsec1-dev * Mac OS X (using MacPorts) $ sudo port install py26-m2crypto py26-dateutil py26-openssl xmlsec $ cd /opt/local/lib $ sudo ln -s libxmlsec1-openssl.dylib libxmlsec1-openssl.so You must configure the omni_config file before using Omni to connect to any control frameworks. Details about omni_config can be found on the OmniOverview page and OmniConfiguration page, but a sample omni_config is included for reference in the Gush SVN repository. In the sample omni_config file, your PlanetLab private key is stored in the ~/.gcf directory. Note that this key cannot require a passphrase to work with Gush. You can remove the passphrase of a private key called keyin.pem, for example, by running: $ openssl rsa -in keyin.pem -out keyout.pem In this case, your omni_config file should include a reference to keyout.pem. You can use the handle-geni.py script included with Gush (in the helper-scripts directory) to automatically download your certificate. This is also a good way to make sure you have Omni correctly configured to talk to the PLC aggregate manager. To do this, insert your PlanetLab slice name into the following command: $ helper-scripts/handle-geni.py -f plc -a http://www.planet-lab.org:12346 sliverstatus If everything is working correctly, you should see something like this: $ helper-scripts/handle-geni.py -f plc -a http://www.planet-lab.org:12346 sliverstatus williams_gush INFO:omni:Loading config file omni_config INFO:omni:Using control framework plc Your certificate file (/home/jeannie/.gcf/plc.williams.jeannie.gid) was not found, would you like me to download it for you to /home/jeannie/.gcf/plc.williams.jeannie.gid? (Y/n)Y INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345 INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347 INFO:omni:Sliver at http://www.planet-lab.org:12346: 2011-04-28 13:41:35 Please double check and make sure you have a valid key and certificate, and that Omni is correctly configured before moving on! * Setting up Gush You need three configuration files to use Gush: directory.xml, gush.prefs, and omni_config. Details about omni_config are described above. For most installations, gush.prefs should not need to be modified. In directory.xml, you should modify the SVN version of the file to include your slice name instead of williams_gush2. For the port number, choose a port greater than 61000. Please change the port! You probably should reserve a port on the PlanetLab port reservation page. To summarize, you must modify directory.xml and omni_config before Gush will work! That should be it! Let us know if you have problems. This is still a work in progress! * Gush Commands You can now start Gush with "./gush -P ". Using the default command-line user interface, Gush tries to act like a shell (well, almost -- unfortunately many of the commands are asynchronous, so you don't have the nice background/foreground distinction present in normal shells). A very-partial list of those commands is listed below; the official source is the "terminal_parser.y" YACC file. For instant gratification, try the following commands: $ ./gush -P 15555 Start Gush on port 15555. Wait until Gush prints out two messages with your slice in it. gush> connect williams_gush@planetlab1.williams.edu Connect to planetlab1.williams.edu. This basically installs/starts the Gush client. Wait until you see a message about the host joining the mesh. gush> info mesh Print out the current connection status of various hosts. gush> shell "ps aux" Run "ps aux" on every connected host, and print the results locally. gush> disconnect Disconnect hosts from the mesh--this kills remote clients. gush> quit There are several example application description XML files in the "tests" directory that can be used to experiment with different aspects of Gush. * Additional Commands gush> load path: Read a project file. gush> run: Start executing the first experiment in the active project. gush> connect hostname: Start a Gush client on a remote host. gush> connect slice slice_name: Connect to all nodes assigned a slice. gush> connect pat : Connect to hosts that match the regular expression pattern . gush> debug fail node : Mark node as failed and find replacement (only during experiment). gush> disconnect: Close all open connections. gush> disconnect hostname: Disconnect from a host. gush> info control: Print controller state information. gush> info nodes: Print summary information on all known nodes. gush> info node hostname: Print information about a node. gush> info slices: Print summary information on all known slices. gush> info slice slice_name: Print information about a slice. gush> info mesh: Print the mesh status (membership). gush> prefer : Tell Gush to increase preference value for nodes matching regex. Nodes with higher preference values are used first. gush> shell : Run the string as a shell command on all connected nodes. gush> slice update slice_name: Get updated information for a slice from the PLC Database. gush> slice list slice_name: Show information on a slice. gush> slice renew slice_name: Renew the given slice. gush> slice add slice_name regex: Assign hosts that match regex (regular expression) to the given slice. gush> slice remove slice_name regex: Unassign hosts that match regex from the slice. gush> quit: Quit Gush. }}} Since the installation is as documented in the README.txt it is not captured here. Following are captures showing the use of GUSH within PlanetLab and ProtoGENI. All captured commands were completed using the following omni_config: {{{ [omni] default_cf = pg users = lnevers # ---------- Users ---------- [lnevers] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers keys = ~/.ssh/id_rsa.pub # ---------- Frameworks ---------- [plc] type=sfa authority=plc.bbn user=plc.bbn.lnevers cert=~/.gcf/plc.bbn.lnevers.gid key=~/.gcf/lnevers.pkey registry=http://www.planet-lab.org:12345 slicemgr=http://www.planet-lab.org:12347 [pg] type = pg ch = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/ch sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa cert = ~/.ssl/encrypted-cleartext.pem key = ~/.ssl/encrypted-cleartext.pem }}} Request sliver status for an existing PlanetLab slice: {{{ $ helper-scripts/handle-geni.py -f plc -a http://www.planet-lab.org:12346 sliverstatus bbn_gusheval INFO:omni:Loading config file omni_config INFO:omni:Using control framework plc INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345 INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347 INFO:omni:Sliver at http://www.planet-lab.org:12346: 2011-05-29 11:01:47 }}} Create a slice which was being authenticated by the pgeni.gpolab.bbn.com clearing house with compute resources from both PlanetLab and Emulab. 1. Get a list of resources available: {{{ $ helper-scripts/handle-geni.py -n -f pg -a https://www.emulab.net/protogeni/xmlrpc/am listresources >>emulab.txt INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni:Resources at https://www.emulab.net/protogeni/xmlrpc/am: $ helper-scripts/handle-geni.py -n -f pg -a http://www.planet-lab.org:12346 listresources >>planetlab-resources.txt INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni:Resources at http://www.planet-lab.org:12346: }}} 2. Create rpsec based on available resources. 3. Create a slice and get resources for both PlanetLab and Emulab: {{{ $ helper-scripts/handle-geni.py -f pg createslice myslice INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg Created slice with Name myslice, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+myslice $ helper-scripts/handle-geni.py -n -f pg -a https://www.emulab.net/protogeni/xmlrpc/am createsliver myslice emulab.rspec INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result: $ helper-scripts/handle-geni.py -n -f pg -a http://www.planet-lab.org:12346 createsliver myslice planet-rspec.txt INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg Asked http://www.planet-lab.org:12346 to reserve resources. Result: RIT "planet2.cs.rit.edu Williams College planetlab3.williams.edu BBN Technologies plnode-03.gpolab.bbn.com INFO:omni:Please run the omni sliverstatus call on your slice to determine your login name to PL resources }}} 4. Get slicer status and adjust slice expiration is possible: {{{ $ helper-scripts/handle-geni.py -f pg -a http://www.planet-lab.org:12346 sliverstatus (syntax problem unresolved) $ helper-scripts/handle-geni.py -f pg -a https://www.emulab.net/protogeni/xmlrpc/am sliverstatus myslice INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni:Sliver at https://www.emulab.net/protogeni/xmlrpc/am: 2011-04-29 16:09:36 $ helper-scripts/handle-geni.py -f pg -a https://www.emulab.net/protogeni/xmlrpc/am renewslice myslice "2011-04-30 16:09:36" INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni.protogeni:requesting new expiration '2011-04-30T16:09:36' Slice myslice now expires at 2011-04-30 16:09:36 $ helper-scripts/handle-geni.py -f pg -a http://www.planet-lab.org:12346 renewslice myslice "2011-04-30 16:09:36" INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni.protogeni:requesting new expiration '2011-04-30T16:09:36' Slice myslice now expires at 2011-04-30 16:09:36 }}} Using hte following directory.xml: {{{ }}} Note: "geni"=PlanetLab and "geni-pg"=ProtoGENI 5. Used gush to manage nodes: {{{ $ ./gush -P 15555 gush> Gush has learned about the slice bbn_gusheval. Gush has learned about the slice myslice. Updated information on the slice bbn_gusheval is available. Updated information on the slice myslice is available. gush> info nodes There are 14 known nodes: [ P ] bbn_gusheval@plnode-03.gpolab.bbn.com:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@plnode-04.gpolab.bbn.com:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@node2.planetlab.albany.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@node2.planetlab.mathcs.emory.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@pl2.planet.cs.kent.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@mtuplanetlab2.cs.mtu.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@miranda.planetlab.cs.umd.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@osiris.planetlab.cs.umd.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@dplanet2.uoc.edu:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@nis-planet2.doshisha.ac.jp:61414(pref=0) (Disconnected.) [ P ] myslice@pc485.emulab.net:61666(pref=0) (Disconnected.) [ P ] myslice@pc499.emulab.net:61666(pref=0) (Disconnected.) [ P ] bbn_gusheval@node-2.mcgillplanetlab.org:61414(pref=0) (Disconnected.) [ P ] bbn_gusheval@node2.lbnl.nodes.planet-lab.org:61414(pref=0) (Disconnected.) gush> connect pc485.emulab.net Connecting to host myslice@pc485.emulab.net:61666. gush> connect plnode-04.gpolab.bbn.com Connecting to host bbn_gusheval@plnode-04.gpolab.bbn.com:61414. gush> bbn_gusheval@plnode-04.gpolab.bbn.com:61414 has joined the mesh. connect pc499.emulab.net Connecting to host myslice@pc499.emulab.net:61666. gush> info For an unknown reason, we were unable to invite myslice@pc485.emulab.net:61666. }}} === GUSH Binary Package === Downloaded the 32 bit Ubuntu Linux Binaries named [http://gush.cs.williams.edu/gush-bin-32bit.tgz gush-bin-32bit.tgz]. Note the package delivers several file at the top level, so created a directory to contain all files delivered as part of the package. Following procedure was used: {{{ $ mkdir gush-bin $ cd gush-bin $ wget http://gush.cs.williams.edu/gush-bin-32bit.tgz $ tar xvzf gush-bin-32bit.tgz }}} Configurations were modified for define user specific details. The following directory.xml was put in place to access GENI resources: {{{ lnevers@bbn.com }}} The following omni_configuration was used: {{{ [omni] default_cf = pg users = lnevers # ---------- Users ---------- [lnevers] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers keys = ~/.ssh/id_rsa.pub # ---------- Frameworks ---------- [pg] type = pg ch = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/ch sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa cert = ~/.ssl/encrypted-cleartext.pem key = ~/.ssl/encrypted-cleartext.pem verbose=false }}} To Be captured: Running gush on PlanetLab slice and on PG Slice. {{{ $ ./gush -P 15555 gush> Gush has learned about the slice XXX. }}} == First (rev 126) and Second (rev 136) Evaluations == Using a newly build Ubuntu 10.04.2 VM for this evaluation in order to verify ''all'' prerequisites. For first evaluation (rev 126), gush gush code was evaluated. During the evaluation found that a binary pages had been added. Binary package evaluation was completed with the latter version (rev 136). === GUSH Code === Had to installed SVN package (Subversion version 1.6.6) not explicitly stated as a prerequisite and then checked out the GUSH repo: {{{ $ sudo apt-get install subversion $ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest }}} No changes have been made to the README.txt since the last [wiki:GIR2.1_GushProto GUSH Evaluation]: {{{ Gush is written in a mix of C++ and Perl, and we intend for it to be runnable on any UNIX-ish OS. That said, we have been using it primarily with Linux 2.6. Compiling Gush can be challenging, so please contact us if you experience too many problems. Gush requires the following libraries: * xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, libxmlrpc_server_abyss++, libxmlrpc_server++, libxmlrpc_server_abyss, libxmlrpc_server, libxmlrpc_abyss, libxmlrpc++, libxmlrpc, libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok Note: lbxmlrpc-c3 will not work. xmlrpc-c must be installed, and to my knowledge, there is no debian package available. * curl: libcurl * xml2: libxml2 * zlib compression/decompression: libz * math: libm * openssl (version 0.9.8e): libssl, libcrypto * dynamic linking: libdl * readline: libreadline * curses: libcurses * intl domain names: libidn * pthreads: libpthread (NOTE: NPTL recommended) The ares library (libares) for asynchronous DNS lookups is optional, though recommended for performance on PlanetLab. The boost software package is also required, as well as lex and yacc (we use flex and bison). Gush uses some Perl helper scripts. If you want Gush to automatically interface with the PlanetLab Central (PLC) database for you, you will need to have Frontier::Client and Crypt::SSLeay properly installed. We'll get to that in the next step. Getting and building the code After downloading the code from SVN, follow the instructions below from a Linux host. NOTE: This statically compiles gush and client. Remove the --with-static compile flags to compile dynamically (which may be easier to configure). Compiling Gush $ cd gush $ autoconf $ ./configure --with-static-client --with-static-gush $ make dep $ make Setting up Gush $ perl helper-scripts/setup.pl That should be it! Let us know if you have problems. This is still a work in progress! }}} Following is a capture of all software installed to meet GUSH pre-requisites: {{{ $ sudo apt-get install gcc $ sudo apt-get install g++ $ wget https://launchpad.net/ubuntu/+archive/primary/+files/xmlrpc-c_1.06.27.orig.tar.gz $ tar xvzf xmlrpc-c_1.06.27.orig.tar.gz $ cd xmlrpc-c-1.06.27/ $ ./configure $ make $ sudo make install ; cd .. $ sudo apt-get install curl $ sudo apt-get install xml2 $ wget http://www.zlib.net/zlib-1.2.5.tar.gz $ tar -xvzf zlib-1.2.5.tar.gz $ cd zlib-1.2.5/ $ ./configure --prefix=/usr/local/zlib $ make $ sudo make install ; cd .. Notes: The libm and pthreads libraries and OpenSSL were already on systems. $ sudo apt-get install libreadline6 $ sudo apt-get install libncurses5 $ sudo apt-get install libidn11 }}} Found that Perl library Crypt::SSLeay required the libssl-dev package: {{{ $ sudo apt-get install libssl-dev $ sudo cpan cpan[1]> install Frontier::Client cpan[2]> install Crypt::SSLeay cpan[3]> quit }}} Now onto the GUSH code: {{{ $ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest $ cd trunk $ sudo apt-get install autoconf $ ./configure --with-static-client --with-static-gush }}} Configure and make had "numerous failures" which were addressed by installing the following additional packages: {{{ $ sudo apt-get install pkg-config libxml2-dev $ sudo apt-get install libboost-dev makedepend $ sudo apt-get install bison flex $ sudo apt-get install libreadline6 libreadline6-dev libexpat1-dev $ sudo apt-get install libcurl4-openssl-dev }}} Note that various changes had to be made for the "make" to work. Added the following to main.cc file: {{{ #include #include #include #include #include }}} Also the gcc-4.4 and g++-4.4 had to both be downgraded to avoid the following build issue: {{{ g++ -g -Wall -Wno-unused -I. -I/usr/include/libxml2 -I/usr/local/include -c version.cc -o version.o version.cc: In constructor 'FileVersion::FileVersion(const char*)': version.cc:56: error: invalid conversion from 'const char*' to 'char*' version.cc:63: error: invalid conversion from 'const char*' to 'char*' version.cc:69: error: invalid conversion from 'const char*' to 'char*' version.cc:75: error: invalid conversion from 'const char*' to 'char*' version.cc:81: error: invalid conversion from 'const char*' to 'char*' make: *** [version.o] Error 1 }}} With the additional packages, environment settings and downgraded versions was able to complete: {{{ $ ./configure --with-static-client --with-static-gush $ make dep $ make }}} Also found the following are required for LD_LIBRARY_PATH: ''/usr/local/lib:/usr/lib:/usr/local/include/'' Was able to eventually compile without the "--with-static-client --with-static-gush" flags. === Initial GUSH Binary Package === Was not able to use the first version of the Binary package due to missing files, had to install binary on top of svn checkout of gush repo. With this "binary+code" combination was able to run the new features delivered by "./helper-scripts/handle-geni.py". This evaluation of "binary+code" was not captured because it is not a likely scenario in the field.