Changes between Version 25 and Version 26 of JOhrenSandbox/GIMINotes8.1.2012


Ignore:
Timestamp:
08/07/12 16:43:57 (8 years ago)
Author:
Jeanne Ohren
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • JOhrenSandbox/GIMINotes8.1.2012

    v25 v26  
    1010
    1111
    12 ||  '''Step'''  ||   '''Status'''  || '''Open Tickets''' ||
    13 || 1.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a1.Establishtestexperimentenvironment  Establish test/experiment environment] || [[Color(green,Success)]] ||  ||
    14 || 2.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a2.ObtainsliceofGENIresourcesinstallandconfigureGIMIIMtools Obtain slice of GENI resources, install and configure GIMI I&M tools] ||  [[Color(green,Success)]] ||  ||
    15 || 3.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a3.RunandorchestrateGIMIIMtoolsandreferenceactualexperiment Run and orchestrate GIMI I&M tools and reference/actual experiment] || [[Color(green,Success)]]  ||  ||
    16 || 4.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a4.PushIMmeasurementresultstoiRODSarchiveservice Push I&M measurement results to iRODS archive service] || [[Color(green,Success)]] ||  ||
    17 || 5.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a5.PullIMmeasurementresultsfromiRODSarchiveservicetoUWiREEL Pull I&M measurement results from iRODS archive service to UW for processing] || [[Color(red,Failure)]] || [http://groups.geni.net/gimi/ticket/2#preview #2]||
    18 || 6.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a6.ObserveIMmeasurementresultsonGIMIpresentationservice Observe I&M measurement results on GIMI presentation service] || [[Color(orange,Blocked)]] ||  ||
    19 || 7.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a7.PushAnalysisResultstoiRODSarchiveservice Push Analysis Results to iRODS archive service] || [[Color(orange,Blocked)]] ||  ||
    20 || 8.  [wiki:JOhrenSandbox/GIMINotes8.1.2012#a8.Releaseresourcesandcleanup Release resources, and cleanup] || [[Color(green,Success)]] ||  ||
    21 
    22 == 1.  Establish test/experiment environment ==
    23 
    24 [[Image(wiki:JOhrenSandbox/GIMINotes8.1.2012:GIMIEnvironment.png)]]
    25 
    26 === User Workspace ===
    27   * Using the GEC14 Tutorial VM
    28   * Inputs: 
    29      * GENI certificate
    30          * installed in $HOME/.ssl
    31          * location configured in omni_config
    32          * stripped the passphrase for automation
    33      * SSH key pair
    34          * installed in $HOME/.ssh
    35          * location configured in omni_config
    36          * key added to ssh-agent
    37      * iRODS account
    38          * username configured in $HOME/.irods/.irodsEnv
    39          * password stored using iinit
    40    * All credentials configured using credconfig.sh script
    41    * Outputs: 
    42        * omni.py ready to use
    43        * i-commands ready to use
    44        * ready to log into nodes in the slice
    45 
    46 === iRODS server ===
    47   * iRODS server on emmy8 was down so I set up my own on pc41.emulab.net
    48 
    49 === OML server ===
    50   * Set up my own OML server on pc41.emulab.net
    51   * I was not able to get the oml2-server to use the hook script when started as a service (/etc/init.d/oml2-server start) - only when started on the command line:
    52 {{{
    53 $ oml2-server --listen=3003 --data-dir=/users/johren/gimi --logfile=/users/johren/oml_server.log -H /users/johren/oml2-server-hook.sh
    54 }}}
    55   * Configured hook script to use my iRODS server
    56 {{{
    57 ***************
    58 *** 23,31 ****
    59   # THE SOFTWARE.
    60   #
    61   irodsUserName=rods
    62 ! irodsHost=emmy8.casa.umass.edu
    63   irodsPort=1247
    64 ! irodsZone=geniRenci
    65   export irodsUserName irodsHost irodsPort irodsZone
    66  
    67   # XXX: You might need to initialise the iRODS password for the UNIX user
    68 --- 23,31 ----
    69   # THE SOFTWARE.
    70   #
    71   irodsUserName=rods
    72 ! irodsHost=pc41.emulab.net
    73   irodsPort=1247
    74 ! irodsZone=bbnZone
    75   export irodsUserName irodsHost irodsPort irodsZone
    76  
    77   # XXX: You might need to initialise the iRODS password for the UNIX user
    78 }}}
    79   * Configured /etc/omf-expctl-5.4/omf-expctl.yaml to point to my OML server (can also be specified on the command line)
    80 {{{
    81       # URI to the OML server to use for this EC
    82       # (can be overwritten on the EC command line)
    83       :omluri: 'tcp:pc41.emulab.net:3003'
    84 }}}
    85 
    86 
    87 == 2.  Obtain slice of GENI resources, install and configure GIMI I&M tools ==
    88   * Used omni to create the slice and the sliver on ExoSM aggregate
    89   * Rspec includes an execute service for each node that downloads [https://github.com/johren/GENI-UserWorkspace/blob/master/GIMI/gimi-postboot.sh gimi-postboot.sh] from the web server (on pc41.emulab.net) and runs it with the slicename and nodeid parameters
    90 {{{
    91     <services>
    92       <execute command="wget -q -P /tmp http://pc41.emulab.net/gimi-postboot.sh ;chmod +x /tmp/gimi-postboot.sh;/tmp/gimi-postboot.sh %SLICENAME% NodeB > /tmp/gimi-postboot.log" shell="/bin/sh"/>
    93     </services>
    94 }}}
    95   * Inputs:
    96     * [https://github.com/johren/GENI-UserWorkspace/blob/master/GIMI/gimi-template.rspec rspec template] - %SLICENAME% must be replaced by the slice name when using this script
    97   * Output:
    98     * slice manifest
    99     * slice login information
    100     * all nodes are ready to run the experiment
    101        * hostname is set
    102        * oml2-iperf installed
    103        * RC is configured and running
    104 
    105 == 3.  Run and orchestrate GIMI I&M tools and reference/actual experiment  ==
    106   * Run the tcp_iperf.rb script using the EC on the UW VM
    107 {{{
    108 $ omf-5.4 exec --no-cmc -S johGIM1208011452 tcp_iperf.rb -- --source1 johGIM1208011452-NodeA --sink johGIM1208011452-NodeC
    109 }}}
    110   * Inputs:
    111     * slicename
    112     * tcp_iperf.rb
    113   * Outputs:
    114     * sq3 database file on the OML server
    115 
    116 == 4.  Push I&M measurement results to iRODS archive service  ==
    117   * This is done by the oml2-server-hook.sh script
    118   * I had to do a lot of modification to this script because it is very tailored to the OML server configuration
    119      * Pulls filename and username from the path of the file so it is very dependent upon the filesystem configuration of the server (e.g. home directories are in /users instead of /home)
    120 {{{
    121 !                                         NAME=${DBFILE:12:6};
    122 !                                         FILE=${DBFILE:12};
    123 -----------
    124 !                                         NAME=${DBFILE:7:6};
    125 !                                         FILE=${DBFILE:19};
    126 }}}
    127      * Had to make some changes to the ichmod commands to get it to work with my iRODS server
    128 {{{
    129 !                                             ichmod -M own $NAME /geniRenci/home/$NAME/$FILE
    130 !                                             iput -f ${DBFILE} /geniRenci/home/$NAME/$FILE;
    131 !                                             imeta add -d /geniRenci/home/$NAME/$FILE Date ${DATE}
    132 !                                             imeta add -d /geniRenci/home/$NAME/$FILE UserName $NAME
    133 !                                             imeta add -d /geniRenci/home/$NAME/$FILE SliceName $SLICE
    134 --------
    135 !                                             ichmod -M own rods /bbnZone/home/$NAME
    136 !                                             iput -f ${DBFILE} /bbnZone/home/$NAME/$FILE;
    137 !                                             ichmod -M own $NAME /bbnZone/home/$NAME/$FILE
    138 !                                             imeta add -d /bbnZone/home/$NAME/$FILE Date ${DATE}
    139 !                                             imeta add -d /bbnZone/home/$NAME/$FILE UserName $NAME
    140 !                                             imeta add -d /bbnZone/home/$NAME/$FILE SliceName $SLICE
    141 }}}
    142   * The hook script currently expects the usernames to begin with "gimi".  I had to modify my script to look specifically for my username.  '''This will not currently work on emmy9 when experimenters use their own credentials'''
    143 {{{
    144 !                                           if [[ $NAME =~ ^gimi[0-9] ]]; then
    145 --------
    146 !                                           if [[ $NAME =~ ^johren ]]; then
    147 }}}
    148   * Inputs:
    149     * oml2-server-hook script running on the OML server
    150     * sq3 database file on the OML server
    151   * Outputs:
    152     * sq3 database file in experimenter's home location on the iRODS server
    153     * experimenter is able to iget the sq3 file
    154 
    155 == 5.  Pull I&M measurement results from iRODS archive service to UW/iREEL ==
    156   * Ran the tutorial_viz.sh command to pull the results from iRODS and run the R script on it to create the images
    157   * The R script is currently failing and, therefore, not producing the graphs:
    158 {{{
    159 > mydata1 <- dbGetQuery(con, "select oml_sender_id,begin_interval,size,end_interval from iperf_transfer where oml_sender_id=1")
    160 > intervals1 <- mydata1$end_interval - mydata1$begin_interval
    161 > throughput1 <- abs(mydata1$size)/intervals1/1024/1024*8
    162 Error in abs(mydata1$size) :
    163   Non-numeric argument to mathematical function
    164 Execution halted
    165 spawn sudo cp johGIM1208011452.png /var/www/
    166 [sudo] password for geniuser:
    167 cp: cannot stat `johGIM1208011452.png': No such file or directory
    168 }}}
    169   * Inputs:
    170      * sq3 file is accessible in the experimenter's home location on the iRODS server
    171   * Outputs:
    172      * visualization images available on the web server on the UW
    173 
    174 == 6.  Observe I&M measurement results on GIMI presentation service  ==
    175 
    176   * Currently blocked on failure in step 5.
    177   * Inputs:
    178     * None
    179   * Outputs:
    180     * visualization images viewable in the web server on the UW
    181 
    182 == 7.  Push Analysis Results to iRODS archive service  ==
    183 
    184   * Currently blocked on failure in step 5.
    185   * Inputs:
    186     * visualization images in UW
    187   * Outputs:
    188     * visualization images viewable in iRODS
    189 
    190 == 8.  Release resources, and cleanup  ==
    191   * Used omni to delete the sliver
    192   * When should the data on the OML server be cleaned up?
    193   * Inputs:
    194       * slice name
    195   * Outputs:
    196       * sliver has been deleted
     128/1/2012 Test execution notes have been moved to [wiki:GIMIAcceptanceTests/GIMITest8.1.2012]
    19713
    19814= Release Management concerns =