Notes from Post-GEC14 testing of GIMI

Status of All Tests

Step	Status	Open Tickets
1. Establish test/experiment environment	Color(green,Success)?
2. Obtain slice of GENI resources, install and configure GIMI I&M tools	Color(green,Success)?
3. Run and orchestrate GIMI I&M tools and reference/actual experiment	Color(green,Success)?
4. Push I&M measurement results to iRODS archive service	Color(green,Success)?
5. Pull I&M measurement results from iRODS archive service to UW for processing	Color(red,Failure)?	​#2
6. Observe I&M measurement results on GIMI presentation service	Color(orange,Blocked)?
7. Push Analysis Results to iRODS archive service	Color(orange,Blocked)?
8. Release resources, and cleanup	Color(green,Success)?

1. Establish test/experiment environment

User Workspace

• Using the GEC14 Tutorial VM
• Inputs:
  • GENI certificate
    • installed in $HOME/.ssl
    • location configured in omni_config
    • stripped the passphrase for automation
  • SSH key pair
    • installed in $HOME/.ssh
    • location configured in omni_config
    • key added to ssh-agent
  • iRODS account
    • username configured in $HOME/.irods/.irodsEnv
    • password stored using iinit
• All credentials configured using credconfig.sh script
• Outputs:
  • omni.py ready to use
  • i-commands ready to use
  • ready to log into nodes in the slice

iRODS server

• iRODS server on emmy8 was down so I set up my own on pc41.emulab.net

OML server

• Set up my own OML server on pc41.emulab.net
• I was not able to get the oml2-server to use the hook script when started as a service (/etc/init.d/oml2-server start) - only when started on the command line:

  $ oml2-server --listen=3003 --data-dir=/users/johren/gimi --logfile=/users/johren/oml_server.log -H /users/johren/oml2-server-hook.sh
  

• Configured hook script to use my iRODS server

  ***************
  *** 23,31 ****
    # THE SOFTWARE.
    #
    irodsUserName=rods
  ! irodsHost=emmy8.casa.umass.edu
    irodsPort=1247
  ! irodsZone=geniRenci
    export irodsUserName irodsHost irodsPort irodsZone
    
    # XXX: You might need to initialise the iRODS password for the UNIX user
  --- 23,31 ----
    # THE SOFTWARE.
    #
    irodsUserName=rods
  ! irodsHost=pc41.emulab.net
    irodsPort=1247
  ! irodsZone=bbnZone
    export irodsUserName irodsHost irodsPort irodsZone
    
    # XXX: You might need to initialise the iRODS password for the UNIX user
  

• Configured /etc/omf-expctl-5.4/omf-expctl.yaml to point to my OML server (can also be specified on the command line)

        # URI to the OML server to use for this EC 
        # (can be overwritten on the EC command line)
        :omluri: 'tcp:pc41.emulab.net:3003'
  

2. Obtain slice of GENI resources, install and configure GIMI I&M tools

• Used omni to create the slice and the sliver on ExoSM aggregate
• Rspec includes an execute service for each node that downloads ​gimi-postboot.sh from the web server (on pc41.emulab.net) and runs it with the slicename and nodeid parameters

      <services>
        <execute command="wget -q -P /tmp http://pc41.emulab.net/gimi-postboot.sh ;chmod +x /tmp/gimi-postboot.sh;/tmp/gimi-postboot.sh %SLICENAME% NodeB > /tmp/gimi-postboot.log" shell="/bin/sh"/>
      </services>
  

• Inputs:
  • ​rspec template - %SLICENAME% must be replaced by the slice name when using this script
• Output:
  • slice manifest
  • slice login information
  • all nodes are ready to run the experiment
    • hostname is set
    • oml2-iperf installed
    • RC is configured and running

3. Run and orchestrate GIMI I&M tools and reference/actual experiment

• Run the tcp_iperf.rb script using the EC on the UW VM

  $ omf-5.4 exec --no-cmc -S johGIM1208011452 tcp_iperf.rb -- --source1 johGIM1208011452-NodeA --sink johGIM1208011452-NodeC
  

• Inputs:
  • slicename
  • tcp_iperf.rb
• Outputs:
  • sq3 database file on the OML server

4. Push I&M measurement results to iRODS archive service

• This is done by the oml2-server-hook.sh script
• I had to do a lot of modification to this script because it is very tailored to the OML server configuration
  • Pulls filename and username from the path of the file so it is very dependent upon the filesystem configuration of the server (e.g. home directories are in /users instead of /home)

    !                                         NAME=${DBFILE:12:6};
    !                                         FILE=${DBFILE:12};
    -----------
    !                                         NAME=${DBFILE:7:6};
    !                                         FILE=${DBFILE:19};
    

  • Had to make some changes to the ichmod commands to get it to work with my iRODS server

    !                                             ichmod -M own $NAME /geniRenci/home/$NAME/$FILE
    !                                             iput -f ${DBFILE} /geniRenci/home/$NAME/$FILE;
    !                                             imeta add -d /geniRenci/home/$NAME/$FILE Date ${DATE}
    !                                             imeta add -d /geniRenci/home/$NAME/$FILE UserName $NAME
    !                                             imeta add -d /geniRenci/home/$NAME/$FILE SliceName $SLICE
    --------
    !                                             ichmod -M own rods /bbnZone/home/$NAME 
    !                                             iput -f ${DBFILE} /bbnZone/home/$NAME/$FILE;
    !                                             ichmod -M own $NAME /bbnZone/home/$NAME/$FILE 
    !                                             imeta add -d /bbnZone/home/$NAME/$FILE Date ${DATE}
    !                                             imeta add -d /bbnZone/home/$NAME/$FILE UserName $NAME
    !                                             imeta add -d /bbnZone/home/$NAME/$FILE SliceName $SLICE
    

• The hook script currently expects the usernames to begin with "gimi". I had to modify my script to look specifically for my username. This will not currently work on emmy9 when experimenters use their own credentials

  !                                           if [[ $NAME =~ ^gimi[0-9] ]]; then
  --------
  !                                           if [[ $NAME =~ ^johren ]]; then
  

• Inputs:
  • oml2-server-hook script running on the OML server
  • sq3 database file on the OML server
• Outputs:
  • sq3 database file in experimenter's home location on the iRODS server
  • experimenter is able to iget the sq3 file

5. Pull I&M measurement results from iRODS archive service to UW/iREEL

• Ran the tutorial_viz.sh command to pull the results from iRODS and run the R script on it to create the images
• The R script is currently failing and, therefore, not producing the graphs:

  > mydata1 <- dbGetQuery(con, "select oml_sender_id,begin_interval,size,end_interval from iperf_transfer where oml_sender_id=1")
  > intervals1 <- mydata1$end_interval - mydata1$begin_interval
  > throughput1 <- abs(mydata1$size)/intervals1/1024/1024*8
  Error in abs(mydata1$size) :
    Non-numeric argument to mathematical function
  Execution halted
  spawn sudo cp johGIM1208011452.png /var/www/
  [sudo] password for geniuser:
  cp: cannot stat `johGIM1208011452.png': No such file or directory 
  

• Inputs:
  • sq3 file is accessible in the experimenter's home location on the iRODS server
• Outputs:
  • visualization images available on the web server on the UW

6. Observe I&M measurement results on GIMI presentation service

• Currently blocked on failure in step 5.
• Inputs:
  • None
• Outputs:
  • visualization images viewable in the web server on the UW

7. Push Analysis Results to iRODS archive service

• Currently blocked on failure in step 5.
• Inputs:
  • visualization images in UW
• Outputs:
  • visualization images viewable in iRODS

8. Release resources, and cleanup

• Used omni to delete the sliver
• When should the data on the OML server be cleaned up?
• Inputs:
  • slice name
• Outputs:
  • sliver has been deleted