| 12 | | || '''Step''' || '''Status''' || '''Open Tickets''' || |
| 13 | | || 1. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a1.Establishtestexperimentenvironment Establish test/experiment environment] || [[Color(green,Success)]] || || |
| 14 | | || 2. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a2.ObtainsliceofGENIresourcesinstallandconfigureGIMIIMtools Obtain slice of GENI resources, install and configure GIMI I&M tools] || [[Color(green,Success)]] || || |
| 15 | | || 3. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a3.RunandorchestrateGIMIIMtoolsandreferenceactualexperiment Run and orchestrate GIMI I&M tools and reference/actual experiment] || [[Color(green,Success)]] || || |
| 16 | | || 4. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a4.PushIMmeasurementresultstoiRODSarchiveservice Push I&M measurement results to iRODS archive service] || [[Color(green,Success)]] || || |
| 17 | | || 5. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a5.PullIMmeasurementresultsfromiRODSarchiveservicetoUWiREEL Pull I&M measurement results from iRODS archive service to UW for processing] || [[Color(red,Failure)]] || [http://groups.geni.net/gimi/ticket/2#preview #2]|| |
| 18 | | || 6. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a6.ObserveIMmeasurementresultsonGIMIpresentationservice Observe I&M measurement results on GIMI presentation service] || [[Color(orange,Blocked)]] || || |
| 19 | | || 7. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a7.PushAnalysisResultstoiRODSarchiveservice Push Analysis Results to iRODS archive service] || [[Color(orange,Blocked)]] || || |
| 20 | | || 8. [wiki:JOhrenSandbox/GIMINotes8.1.2012#a8.Releaseresourcesandcleanup Release resources, and cleanup] || [[Color(green,Success)]] || || |
| 21 | | |
| 22 | | == 1. Establish test/experiment environment == |
| 23 | | |
| 24 | | [[Image(wiki:JOhrenSandbox/GIMINotes8.1.2012:GIMIEnvironment.png)]] |
| 25 | | |
| 26 | | === User Workspace === |
| 27 | | * Using the GEC14 Tutorial VM |
| 28 | | * Inputs: |
| 29 | | * GENI certificate |
| 30 | | * installed in $HOME/.ssl |
| 31 | | * location configured in omni_config |
| 32 | | * stripped the passphrase for automation |
| 33 | | * SSH key pair |
| 34 | | * installed in $HOME/.ssh |
| 35 | | * location configured in omni_config |
| 36 | | * key added to ssh-agent |
| 37 | | * iRODS account |
| 38 | | * username configured in $HOME/.irods/.irodsEnv |
| 39 | | * password stored using iinit |
| 40 | | * All credentials configured using credconfig.sh script |
| 41 | | * Outputs: |
| 42 | | * omni.py ready to use |
| 43 | | * i-commands ready to use |
| 44 | | * ready to log into nodes in the slice |
| 45 | | |
| 46 | | === iRODS server === |
| 47 | | * iRODS server on emmy8 was down so I set up my own on pc41.emulab.net |
| 48 | | |
| 49 | | === OML server === |
| 50 | | * Set up my own OML server on pc41.emulab.net |
| 51 | | * I was not able to get the oml2-server to use the hook script when started as a service (/etc/init.d/oml2-server start) - only when started on the command line: |
| 52 | | {{{ |
| 53 | | $ oml2-server --listen=3003 --data-dir=/users/johren/gimi --logfile=/users/johren/oml_server.log -H /users/johren/oml2-server-hook.sh |
| 54 | | }}} |
| 55 | | * Configured hook script to use my iRODS server |
| 56 | | {{{ |
| 57 | | *************** |
| 58 | | *** 23,31 **** |
| 59 | | # THE SOFTWARE. |
| 60 | | # |
| 61 | | irodsUserName=rods |
| 62 | | ! irodsHost=emmy8.casa.umass.edu |
| 63 | | irodsPort=1247 |
| 64 | | ! irodsZone=geniRenci |
| 65 | | export irodsUserName irodsHost irodsPort irodsZone |
| 66 | | |
| 67 | | # XXX: You might need to initialise the iRODS password for the UNIX user |
| 68 | | --- 23,31 ---- |
| 69 | | # THE SOFTWARE. |
| 70 | | # |
| 71 | | irodsUserName=rods |
| 72 | | ! irodsHost=pc41.emulab.net |
| 73 | | irodsPort=1247 |
| 74 | | ! irodsZone=bbnZone |
| 75 | | export irodsUserName irodsHost irodsPort irodsZone |
| 76 | | |
| 77 | | # XXX: You might need to initialise the iRODS password for the UNIX user |
| 78 | | }}} |
| 79 | | * Configured /etc/omf-expctl-5.4/omf-expctl.yaml to point to my OML server (can also be specified on the command line) |
| 80 | | {{{ |
| 81 | | # URI to the OML server to use for this EC |
| 82 | | # (can be overwritten on the EC command line) |
| 83 | | :omluri: 'tcp:pc41.emulab.net:3003' |
| 84 | | }}} |
| 85 | | |
| 86 | | |
| 87 | | == 2. Obtain slice of GENI resources, install and configure GIMI I&M tools == |
| 88 | | * Used omni to create the slice and the sliver on ExoSM aggregate |
| 89 | | * Rspec includes an execute service for each node that downloads [https://github.com/johren/GENI-UserWorkspace/blob/master/GIMI/gimi-postboot.sh gimi-postboot.sh] from the web server (on pc41.emulab.net) and runs it with the slicename and nodeid parameters |
| 90 | | {{{ |
| 91 | | <services> |
| 92 | | <execute command="wget -q -P /tmp http://pc41.emulab.net/gimi-postboot.sh ;chmod +x /tmp/gimi-postboot.sh;/tmp/gimi-postboot.sh %SLICENAME% NodeB > /tmp/gimi-postboot.log" shell="/bin/sh"/> |
| 93 | | </services> |
| 94 | | }}} |
| 95 | | * Inputs: |
| 96 | | * [https://github.com/johren/GENI-UserWorkspace/blob/master/GIMI/gimi-template.rspec rspec template] - %SLICENAME% must be replaced by the slice name when using this script |
| 97 | | * Output: |
| 98 | | * slice manifest |
| 99 | | * slice login information |
| 100 | | * all nodes are ready to run the experiment |
| 101 | | * hostname is set |
| 102 | | * oml2-iperf installed |
| 103 | | * RC is configured and running |
| 104 | | |
| 105 | | == 3. Run and orchestrate GIMI I&M tools and reference/actual experiment == |
| 106 | | * Run the tcp_iperf.rb script using the EC on the UW VM |
| 107 | | {{{ |
| 108 | | $ omf-5.4 exec --no-cmc -S johGIM1208011452 tcp_iperf.rb -- --source1 johGIM1208011452-NodeA --sink johGIM1208011452-NodeC |
| 109 | | }}} |
| 110 | | * Inputs: |
| 111 | | * slicename |
| 112 | | * tcp_iperf.rb |
| 113 | | * Outputs: |
| 114 | | * sq3 database file on the OML server |
| 115 | | |
| 116 | | == 4. Push I&M measurement results to iRODS archive service == |
| 117 | | * This is done by the oml2-server-hook.sh script |
| 118 | | * I had to do a lot of modification to this script because it is very tailored to the OML server configuration |
| 119 | | * Pulls filename and username from the path of the file so it is very dependent upon the filesystem configuration of the server (e.g. home directories are in /users instead of /home) |
| 120 | | {{{ |
| 121 | | ! NAME=${DBFILE:12:6}; |
| 122 | | ! FILE=${DBFILE:12}; |
| 123 | | ----------- |
| 124 | | ! NAME=${DBFILE:7:6}; |
| 125 | | ! FILE=${DBFILE:19}; |
| 126 | | }}} |
| 127 | | * Had to make some changes to the ichmod commands to get it to work with my iRODS server |
| 128 | | {{{ |
| 129 | | ! ichmod -M own $NAME /geniRenci/home/$NAME/$FILE |
| 130 | | ! iput -f ${DBFILE} /geniRenci/home/$NAME/$FILE; |
| 131 | | ! imeta add -d /geniRenci/home/$NAME/$FILE Date ${DATE} |
| 132 | | ! imeta add -d /geniRenci/home/$NAME/$FILE UserName $NAME |
| 133 | | ! imeta add -d /geniRenci/home/$NAME/$FILE SliceName $SLICE |
| 134 | | -------- |
| 135 | | ! ichmod -M own rods /bbnZone/home/$NAME |
| 136 | | ! iput -f ${DBFILE} /bbnZone/home/$NAME/$FILE; |
| 137 | | ! ichmod -M own $NAME /bbnZone/home/$NAME/$FILE |
| 138 | | ! imeta add -d /bbnZone/home/$NAME/$FILE Date ${DATE} |
| 139 | | ! imeta add -d /bbnZone/home/$NAME/$FILE UserName $NAME |
| 140 | | ! imeta add -d /bbnZone/home/$NAME/$FILE SliceName $SLICE |
| 141 | | }}} |
| 142 | | * The hook script currently expects the usernames to begin with "gimi". I had to modify my script to look specifically for my username. '''This will not currently work on emmy9 when experimenters use their own credentials''' |
| 143 | | {{{ |
| 144 | | ! if [[ $NAME =~ ^gimi[0-9] ]]; then |
| 145 | | -------- |
| 146 | | ! if [[ $NAME =~ ^johren ]]; then |
| 147 | | }}} |
| 148 | | * Inputs: |
| 149 | | * oml2-server-hook script running on the OML server |
| 150 | | * sq3 database file on the OML server |
| 151 | | * Outputs: |
| 152 | | * sq3 database file in experimenter's home location on the iRODS server |
| 153 | | * experimenter is able to iget the sq3 file |
| 154 | | |
| 155 | | == 5. Pull I&M measurement results from iRODS archive service to UW/iREEL == |
| 156 | | * Ran the tutorial_viz.sh command to pull the results from iRODS and run the R script on it to create the images |
| 157 | | * The R script is currently failing and, therefore, not producing the graphs: |
| 158 | | {{{ |
| 159 | | > mydata1 <- dbGetQuery(con, "select oml_sender_id,begin_interval,size,end_interval from iperf_transfer where oml_sender_id=1") |
| 160 | | > intervals1 <- mydata1$end_interval - mydata1$begin_interval |
| 161 | | > throughput1 <- abs(mydata1$size)/intervals1/1024/1024*8 |
| 162 | | Error in abs(mydata1$size) : |
| 163 | | Non-numeric argument to mathematical function |
| 164 | | Execution halted |
| 165 | | spawn sudo cp johGIM1208011452.png /var/www/ |
| 166 | | [sudo] password for geniuser: |
| 167 | | cp: cannot stat `johGIM1208011452.png': No such file or directory |
| 168 | | }}} |
| 169 | | * Inputs: |
| 170 | | * sq3 file is accessible in the experimenter's home location on the iRODS server |
| 171 | | * Outputs: |
| 172 | | * visualization images available on the web server on the UW |
| 173 | | |
| 174 | | == 6. Observe I&M measurement results on GIMI presentation service == |
| 175 | | |
| 176 | | * Currently blocked on failure in step 5. |
| 177 | | * Inputs: |
| 178 | | * None |
| 179 | | * Outputs: |
| 180 | | * visualization images viewable in the web server on the UW |
| 181 | | |
| 182 | | == 7. Push Analysis Results to iRODS archive service == |
| 183 | | |
| 184 | | * Currently blocked on failure in step 5. |
| 185 | | * Inputs: |
| 186 | | * visualization images in UW |
| 187 | | * Outputs: |
| 188 | | * visualization images viewable in iRODS |
| 189 | | |
| 190 | | == 8. Release resources, and cleanup == |
| 191 | | * Used omni to delete the sliver |
| 192 | | * When should the data on the OML server be cleaned up? |
| 193 | | * Inputs: |
| 194 | | * slice name |
| 195 | | * Outputs: |
| 196 | | * sliver has been deleted |
| | 12 | 8/1/2012 Test execution notes have been moved to [wiki:GIMIAcceptanceTests/GIMITest8.1.2012] |
