Changes between Version 1 and Version 2 of OpenFlow/FOAM/Testing


Ignore:
Timestamp:
03/07/12 14:58:27 (7 years ago)
Author:
Josh Smift
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OpenFlow/FOAM/Testing

    v1 v2  
    55This page describes a couple of ways to test FOAM:
    66
    7  * Testing that the software complies with the GENI AM API, using the test suite that comes with GCF.
    87 * Testing that slivers can be created, and then managed in various ways, including:
    98   * Simple tests that slivers appear and disappear in FlowVisor.
    109   * More complex tests for other sliver management operations.
    1110   * Tests for sliver expiration.
     11 * Testing that the software complies with the GENI AM API, using the test suite that comes with GCF.
     12
     13= Sliver creation and management =
     14
     15This set of tests includes a Setup section to set some variables, a Creation section to create a sliver, and then additional sections to do various tests with it.
     16
     17You only need to do the Setup section once.
     18
     19You need to repeat the Creation section before doing any of the other sections, because the Creation section sets some variables that you'll need for the other sections, and each of the other sections concludes by deleting the sliver.
     20
     21Pre-requisites/assumptions:
     22
     23 * You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file, etc).
     24 * You have a valid request rspec file to test with. We plan to make some sample rspecs available.
     25 * You can log in non-interactively to the FOAM and FV servers, and have admin privs there, and have the FOAM and fvadmin password stored in files (/opt/foam/etc/passwd and /etc/flowvisor/fvpasswd). (This generally can't be used as-is to test FOAM at other sites.)
     26 * You know what output to expect when common things succeed. (Expected output is described here only in cases where an error is the expected result, or where it otherwise might not be obvious; if you think the expected output isn't obvious for any given step, let us know.)
     27 * The URL of the FOAM server is also a name that you can use to log in to it. (Very likely, but might be untrue with virtualhosts or some other odd thing.)
     28 * On the FOAM server you're testing, when you create a sliver, it's marked as Pending, and isn't automatically approved.
     29
     30You can run most of these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server, except as noted below.
     31
     32== Setup ==
     33
     34Set the FOAM server name (modify this to use the DNS name of the server that you want to test):
     35
     36{{{
     37foamserver=foam.example.edu
     38}}}
     39
     40Set the name of the FlowVisor server to be the same (if it's the same, or something else if it isn't):
     41
     42{{{
     43fvserver=$foamserver
     44}}}
     45
     46Set the AM API URL:
     47
     48{{{
     49am=https://${foamserver}:3626/foam/gapi/1
     50}}}
     51
     52Identify the rspec you plan to use (modify this if you want to use a different rspec):
     53
     54{{{
     55rspec=~/rspecs/request/misc/openflow-any-acedcodefood.rspec
     56}}}
     57
     58Set the slicename to something with your username and "foamtest":
     59
     60{{{
     61slicename=${USER}foamtest
     62}}}
     63
     64Set the full slice URN (modify this if you're not using pgeni.gpolab.bbn.com as your Slice Authority), and a version of it for handing off to egrep later:
     65
     66{{{
     67slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
     68slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
     69}}}
     70
     71Make sure FOAM is listening at all:
     72
     73{{{
     74omni -a $am getversion
     75omni -a $am listresources
     76}}}
     77
     78Now you're ready to create a sliver.
     79
     80== Creation ==
     81
     82Create a slice:
     83
     84{{{
     85omni createslice $slicename
     86}}}
     87
     88Create a sliver:
     89
     90{{{
     91omni -a $am createsliver $slicename $rspec
     92}}}
     93
     94(That should generate a "sliver created" e-mail message, one to the experimenter and one to the FOAM admin.)
     95
     96Confirm that it looks right via Omni:
     97
     98{{{
     99omni -a $am listresources $slicename
     100omni -a $am sliverstatus $slicename
     101}}}
     102
     103Confirm that it shows up in FOAM:
     104
     105{{{
     106ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep slice_urn.+$slice_urn_egrep
     107}}}
     108
     109Get the sliver URN and FlowVisor slicename from FOAM, and put them into $sliver_urn and $fv_slicename:
     110
     111{{{
     112sliver_urn=$(ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
     113fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
     114}}}
     115
     116Now you're ready to do additional tests with this sliver.
     117
     118== Simple management ==
     119
     120NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
     121
     122The steps in this section look at a newly-created sliver, approve it, confirm that it shows up in the FlowVisor, delete it, and confirm that it vanishes from the FlowVisor.
     123
     124Show the sliver, and confirm that 'status' is "Pending":
     125
     126{{{
     127ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     128}}}
     129
     130Confirm that the sliver doesn't yet show up in FlowVisor:
     131
     132{{{
     133ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     134}}}
     135
     136(Expect a "slice does not exist" error.)
     137
     138Show the sliver's rspec:
     139
     140{{{
     141ssh $foamserver foamctl show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     142}}}
     143
     144Show the sliver's flowspec:
     145
     146{{{
     147ssh $foamserver foamctl show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     148}}}
     149
     150Show the sliver's flowspace rules:
     151
     152{{{
     153ssh $foamserver foamctl show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     154}}}
     155
     156Approve the sliver, and confirm that 'status' is now "Approved":
     157
     158{{{
     159ssh $foamserver foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     160ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     161}}}
     162
     163(That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
     164
     165Confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     166
     167{{{
     168ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     169ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     170}}}
     171
     172(Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't.)
     173
     174Delete the sliver:
     175
     176{{{
     177omni -a $am deletesliver $slicename
     178}}}
     179
     180(That should generate a "sliver deleted" e-mail message, one to the experimenter and one to the FOAM admin.)
     181
     182Confirm that Omni thinks it's gone:
     183
     184{{{
     185omni -a $am listresources $slicename
     186omni -a $am sliverstatus $slicename
     187}}}
     188
     189(Expect an "unknown slice" error for each of those.)
     190
     191Confirm that the sliver no longer shows up in FOAM:
     192
     193{{{
     194ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | grep $slice_urn
     195ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     196}}}
     197
     198(Expect no output for the first, and 'deleted' to be "True" for the second; note that 'status' will still be "Approved", which is fine, it's just tracking what the status was when it was deleted.)
     199
     200Confirm that the sliver no longer shows up in FlowVisor:
     201
     202{{{
     203ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     204ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     205}}}
     206
     207(Expect a "slice does not exist" error for the first, and no output for the second.)
     208
     209== Complex management ==
     210
     211NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
     212
     213The steps in this section look at a newly-created sliver, run through a sequence of approve/disable/reject actions to confirm that it can transition from any of those states to the others, and check the FlowVisor after each change; then delete the sliver, and confirm that it vanishes from the FlowVisor.
     214
     215NOTE: These steps are run directly on the FOAM/FlowVisor server, rather than on your Omni client system. If those aren't the same server, do the 'foamctl' commands on the FOAM server, and the 'fvctl' commands on the FV server.
     216
     217Repeat the setup steps from above, but on the server, so you've got the right variables set there:
     218
     219{{{
     220slicename=${USER}foamtest
     221slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
     222slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
     223sliver_urn=$(foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
     224fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
     225}}}
     226
     227Show the sliver, and confirm that 'status' is "Pending":
     228
     229{{{
     230foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     231}}}
     232
     233Confirm that the sliver doesn't yet show up in FlowVisor:
     234
     235{{{
     236fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     237}}}
     238
     239(Expect a "slice does not exist" error; and likewise every time you do a "confirm that the sliver doesn't show up in FlowVisor" step below.)
     240
     241Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:
     242
     243{{{
     244sudo sh -c "echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest"
     245}}}
     246
     247(Within a minute (when the cron job fires), that should generate e-mail to the FOAM admin with a list of pending slivers, including your test sliver; and a message to gpo-ops saying that the cron job ran.)
     248
     249Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     250
     251{{{
     252foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     253foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     254fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     255fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     256}}}
     257
     258(Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't. Likewise every time you confirm that the sliver shows up in Flowvisor below.)
     259
     260(That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
     261
     262Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:
     263
     264{{{
     265sudo sh -c "echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest"
     266}}}
     267
     268(Within a minute (when the cron job fires), either (a) if there were other slivers in the previous mail, that should generate a message to the FOAM admin listing only those slivers, and not your test sliver, or otherwise (b) not send any message to the FOAM admin; and a message to gpo-ops saying that the cron job ran.)
     269
     270Clean up the cron job:
     271
     272{{{
     273sudo rm /etc/cron.d/foamtest
     274}}}
     275
     276This next set of tests will generate a bunch of e-mail; check at the end that it all showed up as expected.
     277
     278Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
     279
     280{{{
     281foamctl disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     282foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     283fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     284fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     285}}}
     286
     287Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
     288
     289{{{
     290foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     291foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     292fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     293fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     294}}}
     295
     296Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     297
     298{{{
     299foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     300foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     301fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     302fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     303}}}
     304
     305Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
     306
     307{{{
     308foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     309foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     310fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     311fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     312}}}
     313
     314Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
     315
     316{{{
     317foamctl disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     318foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     319fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     320fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     321}}}
     322
     323Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     324
     325{{{
     326foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     327foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     328fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     329fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     330}}}
     331
     332Delete the sliver, confirm that 'deleted' is now "True", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
     333
     334{{{
     335foamctl delete-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     336foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     337fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     338fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     339}}}
     340
     341That set of tests should generate seven e-mail messages to the experimenter and seven to the FOAM admin: There should be a "sliver disabled" message, then "sliver rejected", then approved, rejected, disabled, approved, and deleted. Probably in that order, although e-mail is sometimes delivered out of sequence, so don't worry too much if they're not in order.
     342
     343== Expiration ==
     344
     345NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
     346
     347The steps in this section look at a newly-created sliver, and test expiration in FOAM, by changing its expiration date and, then running the scripts that expire slivers and notify about soon-to-expire ones.
     348
     349First, review and approve the sliver.
     350
     351Show the sliver, and confirm that 'status' is "Pending":
     352
     353{{{
     354ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     355}}}
     356
     357Confirm that the sliver doesn't yet show up in FlowVisor:
     358
     359{{{
     360ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     361}}}
     362
     363(Expect a "slice does not exist" error.)
     364
     365Show the sliver's rspec:
     366
     367{{{
     368ssh $foamserver foamctl show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     369}}}
     370
     371Show the sliver's flowspec:
     372
     373{{{
     374ssh $foamserver foamctl show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     375}}}
     376
     377Show the sliver's flowspace rules:
     378
     379{{{
     380ssh $foamserver foamctl show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     381}}}
     382
     383Approve the sliver, and confirm that 'status' is now "Approved":
     384
     385{{{
     386ssh $foamserver foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     387ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     388}}}
     389
     390(That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
     391
     392Confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     393
     394{{{
     395ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     396ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     397}}}
     398
     399(Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't.)
     400
     401Now, do some tests to make sure FOAM notifies you about a sliver that will expire within a day, but only once.
     402
     403Change the expiration date of your slice to be nine days away, since otherwise you can't extend the sliver expiration date very far:
     404
     405{{{
     406omni renewslice $slicename "$(date -d 'now + 9 days')" -f gpolab
     407}}}
     408
     409Change the expiration date of your sliver to be twelve hours away, and confirm that the sliver has the new expiration date:
     410
     411{{{
     412omni -a $am renewsliver $slicename "$(date -d 'now + 12 hours')"
     413omni -a $am sliverstatus $slicename
     414}}}
     415
     416(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
     417
     418Add a temporary cron job to tell FOAM to send mail about slivers that are expiring soon and remove any expired slivers:
     419
     420{{{
     421ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     422}}}
     423
     424(Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     425
     426Add the cron job again, to confirm that it doesn't send a second notification:
     427
     428{{{
     429ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     430}}}
     431
     432(Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     433
     434Next, test to make sure FOAM notifies you again if you change the expiration date.
     435
     436Change the expiration date of your sliver to be one day away, and confirm that the sliver has the new expiration date:
     437
     438{{{
     439omni -a $am renewsliver $slicename "$(date -d 'now + 1 day')"
     440omni -a $am sliverstatus $slicename
     441}}}
     442
     443(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
     444
     445Add the cron job again, to confirm that it does notify you again now that the expiration date has changed:
     446
     447{{{
     448ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     449}}}
     450
     451(Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     452
     453Next, do some tests to make sure that FOAM notifies you about a sliver that will expire within a week, but only once.
     454
     455Change the expiration date of your sliver to be two days away, and confirm that the sliver has the new expiration date:
     456
     457{{{
     458omni -a $am renewsliver $slicename "$(date -d 'now + 2 days')"
     459omni -a $am sliverstatus $slicename
     460}}}
     461
     462(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
     463
     464Add the cron job again, to confirm that it now sends a "within a week" message:
     465
     466{{{
     467ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     468}}}
     469
     470(Within a minute (when the cron job fires), that should generate a "sliver expiring soon" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     471
     472Add the cron job again, to confirm that it doesn't send a second notification:
     473
     474{{{
     475ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     476}}}
     477
     478(Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     479
     480Next, test to make sure FOAM doesn't notify you at all if your sliver is expiring more than a week from now.
     481
     482Change the expiration date of your sliver to be eight days away, and confirm that the sliver has the new expiration date:
     483
     484{{{
     485omni -a $am renewsliver $slicename "$(date -d 'now + 8 days')"
     486omni -a $am sliverstatus $slicename
     487}}}
     488
     489(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
     490
     491Add the cron job again, to confirm that it doesn't send a notification:
     492
     493{{{
     494ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     495}}}
     496
     497(Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
     498
     499Finally, test to make sure FOAM deletes a sliver if its expiration date is in the past.
     500
     501Change the expiration date of your sliver to be right now, and confirm that the sliver has the new expiration date:
     502
     503{{{
     504omni -a $am renewsliver $slicename "$(date -d 'now')"
     505omni -a $am sliverstatus $slicename
     506}}}
     507
     508(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
     509
     510Add the cron job again, to confirm that it deletes the sliver:
     511
     512{{{
     513ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
     514}}}
     515
     516(Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; a "sliver expired" message, one to the experimenter and one to the FOAM admin; and one message to gpo-ops saying that the cron job ran.)
     517
     518Confirm that the sliver no longer shows up in FOAM:
     519
     520{{{
     521ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | grep $slice_urn
     522ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
     523}}}
     524
     525(Expect no output for the first, and 'deleted' to be "True" for the second.)
     526
     527Confirm that the sliver no longer shows up in FlowVisor:
     528
     529{{{
     530ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
     531ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     532}}}
     533
     534(Expect a "slice does not exist" error for the first, and no output for the second.)
     535
     536Clean up the last cron job:
     537
     538{{{
     539ssh -t $foamserver sudo rm /etc/cron.d/foamtest
     540}}}
    12541
    13542= GENI AM API =
    14543
    15 This tests whether a given FOAM server is compliant with the GENI AM API, using Omni's built-in acceptance test suite.
     544This tests whether a given FOAM server is compliant with the GENI AM API, using Omni's built-in acceptance test suite. Stanford/ONLab and the GPO generally run these tests when new versions of FOAM are released, but you can run them yourself too, if you want.
    16545
    17546Pre-requisites/assumptions:
     
    90619
    91620The test suite should also generate sixteen e-mail messages to the experimenter and sixteen to the FOAM admin. There should be a "sliver created" message, two "sliver renewed" messages, and a "sliver deleted" message, for each of four slices; the slices have names like accjbs, accjbs0, accjbs1, and accjbs2, replacing 'jbs' with your Unix username on the system where you're running the tests.
    92 
    93 = Sliver creation and management =
    94 
    95 This set of tests includes a Setup section to set some variables, a Creation section to create a sliver, and then additional sections to do various tests with it.
    96 
    97 You only need to do the Setup section once.
    98 
    99 You need to repeat the Creation section before doing any of the other sections, because the Creation section sets some variables that you'll need for the other sections, and each of the other sections concludes by deleting the sliver.
    100 
    101 Pre-requisites/assumptions:
    102 
    103  * You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file, etc).
    104  * You have a valid request rspec file to test with. We plan to make some sample rspecs available.
    105  * You can log in non-interactively to the FOAM and FV servers, and have admin privs there, and have the FOAM and fvadmin password stored in files (/opt/foam/etc/passwd and /etc/flowvisor/fvpasswd). (This generally can't be used as-is to test FOAM at other sites.)
    106  * You know what output to expect when common things succeed. (Expected output is described here only in cases where an error is the expected result, or where it otherwise might not be obvious; if you think the expected output isn't obvious for any given step, let us know.)
    107  * The URL of the FOAM server is also a name that you can use to log in to it. (Very likely, but might be untrue with virtualhosts or some other odd thing.)
    108  * On the FOAM server you're testing, when you create a sliver, it's marked as Pending, and isn't automatically approved.
    109 
    110 You can run most of these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server, except as noted below.
    111 
    112 == Setup ==
    113 
    114 Set the FOAM server name (modify this to use the DNS name of the server that you want to test):
    115 
    116 {{{
    117 foamserver=foam.example.edu
    118 }}}
    119 
    120 Set the name of the FlowVisor server to be the same (if it's the same, or something else if it isn't):
    121 
    122 {{{
    123 fvserver=$foamserver
    124 }}}
    125 
    126 Set the AM API URL:
    127 
    128 {{{
    129 am=https://${foamserver}:3626/foam/gapi/1
    130 }}}
    131 
    132 Identify the rspec you plan to use (modify this if you want to use a different rspec):
    133 
    134 {{{
    135 rspec=~/rspecs/request/misc/openflow-any-acedcodefood.rspec
    136 }}}
    137 
    138 Set the slicename to something with your username and "foamtest":
    139 
    140 {{{
    141 slicename=${USER}foamtest
    142 }}}
    143 
    144 Set the full slice URN (modify this if you're not using pgeni.gpolab.bbn.com as your Slice Authority), and a version of it for handing off to egrep later:
    145 
    146 {{{
    147 slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
    148 slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
    149 }}}
    150 
    151 Make sure FOAM is listening at all:
    152 
    153 {{{
    154 omni -a $am getversion
    155 omni -a $am listresources
    156 }}}
    157 
    158 Now you're ready to create a sliver.
    159 
    160 == Creation ==
    161 
    162 Create a slice:
    163 
    164 {{{
    165 omni createslice $slicename
    166 }}}
    167 
    168 Create a sliver:
    169 
    170 {{{
    171 omni -a $am createsliver $slicename $rspec
    172 }}}
    173 
    174 (That should generate a "sliver created" e-mail message, one to the experimenter and one to the FOAM admin.)
    175 
    176 Confirm that it looks right via Omni:
    177 
    178 {{{
    179 omni -a $am listresources $slicename
    180 omni -a $am sliverstatus $slicename
    181 }}}
    182 
    183 Confirm that it shows up in FOAM:
    184 
    185 {{{
    186 ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep slice_urn.+$slice_urn_egrep
    187 }}}
    188 
    189 Get the sliver URN and FlowVisor slicename from FOAM, and put them into $sliver_urn and $fv_slicename:
    190 
    191 {{{
    192 sliver_urn=$(ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
    193 fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
    194 }}}
    195 
    196 Now you're ready to do additional tests with this sliver.
    197 
    198 == Simple management ==
    199 
    200 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    201 
    202 The steps in this section look at a newly-created sliver, approve it, confirm that it shows up in the FlowVisor, delete it, and confirm that it vanishes from the FlowVisor.
    203 
    204 Show the sliver, and confirm that 'status' is "Pending":
    205 
    206 {{{
    207 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    208 }}}
    209 
    210 Confirm that the sliver doesn't yet show up in FlowVisor:
    211 
    212 {{{
    213 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    214 }}}
    215 
    216 (Expect a "slice does not exist" error.)
    217 
    218 Show the sliver's rspec:
    219 
    220 {{{
    221 ssh $foamserver foamctl show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    222 }}}
    223 
    224 Show the sliver's flowspec:
    225 
    226 {{{
    227 ssh $foamserver foamctl show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    228 }}}
    229 
    230 Show the sliver's flowspace rules:
    231 
    232 {{{
    233 ssh $foamserver foamctl show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    234 }}}
    235 
    236 Approve the sliver, and confirm that 'status' is now "Approved":
    237 
    238 {{{
    239 ssh $foamserver foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    240 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    241 }}}
    242 
    243 (That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
    244 
    245 Confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
    246 
    247 {{{
    248 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    249 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    250 }}}
    251 
    252 (Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't.)
    253 
    254 Delete the sliver:
    255 
    256 {{{
    257 omni -a $am deletesliver $slicename
    258 }}}
    259 
    260 (That should generate a "sliver deleted" e-mail message, one to the experimenter and one to the FOAM admin.)
    261 
    262 Confirm that Omni thinks it's gone:
    263 
    264 {{{
    265 omni -a $am listresources $slicename
    266 omni -a $am sliverstatus $slicename
    267 }}}
    268 
    269 (Expect an "unknown slice" error for each of those.)
    270 
    271 Confirm that the sliver no longer shows up in FOAM:
    272 
    273 {{{
    274 ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | grep $slice_urn
    275 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    276 }}}
    277 
    278 (Expect no output for the first, and 'deleted' to be "True" for the second; note that 'status' will still be "Approved", which is fine, it's just tracking what the status was when it was deleted.)
    279 
    280 Confirm that the sliver no longer shows up in FlowVisor:
    281 
    282 {{{
    283 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    284 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    285 }}}
    286 
    287 (Expect a "slice does not exist" error for the first, and no output for the second.)
    288 
    289 == Complex management ==
    290 
    291 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    292 
    293 The steps in this section look at a newly-created sliver, run through a sequence of approve/disable/reject actions to confirm that it can transition from any of those states to the others, and check the FlowVisor after each change; then delete the sliver, and confirm that it vanishes from the FlowVisor.
    294 
    295 NOTE: These steps are run directly on the FOAM/FlowVisor server, rather than on your Omni client system. If those aren't the same server, do the 'foamctl' commands on the FOAM server, and the 'fvctl' commands on the FV server.
    296 
    297 Repeat the setup steps from above, but on the server, so you've got the right variables set there:
    298 
    299 {{{
    300 slicename=${USER}foamtest
    301 slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
    302 slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
    303 sliver_urn=$(foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
    304 fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
    305 }}}
    306 
    307 Show the sliver, and confirm that 'status' is "Pending":
    308 
    309 {{{
    310 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    311 }}}
    312 
    313 Confirm that the sliver doesn't yet show up in FlowVisor:
    314 
    315 {{{
    316 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    317 }}}
    318 
    319 (Expect a "slice does not exist" error; and likewise every time you do a "confirm that the sliver doesn't show up in FlowVisor" step below.)
    320 
    321 Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:
    322 
    323 {{{
    324 sudo sh -c "echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest"
    325 }}}
    326 
    327 (Within a minute (when the cron job fires), that should generate e-mail to the FOAM admin with a list of pending slivers, including your test sliver; and a message to gpo-ops saying that the cron job ran.)
    328 
    329 Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
    330 
    331 {{{
    332 foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    333 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    334 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    335 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    336 }}}
    337 
    338 (Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't. Likewise every time you confirm that the sliver shows up in Flowvisor below.)
    339 
    340 (That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
    341 
    342 Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:
    343 
    344 {{{
    345 sudo sh -c "echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest"
    346 }}}
    347 
    348 (Within a minute (when the cron job fires), either (a) if there were other slivers in the previous mail, that should generate a message to the FOAM admin listing only those slivers, and not your test sliver, or otherwise (b) not send any message to the FOAM admin; and a message to gpo-ops saying that the cron job ran.)
    349 
    350 Clean up the cron job:
    351 
    352 {{{
    353 sudo rm /etc/cron.d/foamtest
    354 }}}
    355 
    356 This next set of tests will generate a bunch of e-mail; check at the end that it all showed up as expected.
    357 
    358 Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
    359 
    360 {{{
    361 foamctl disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    362 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    363 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    364 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    365 }}}
    366 
    367 Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
    368 
    369 {{{
    370 foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    371 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    372 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    373 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    374 }}}
    375 
    376 Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
    377 
    378 {{{
    379 foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    380 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    381 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    382 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    383 }}}
    384 
    385 Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
    386 
    387 {{{
    388 foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    389 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    390 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    391 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    392 }}}
    393 
    394 Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
    395 
    396 {{{
    397 foamctl disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    398 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    399 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    400 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    401 }}}
    402 
    403 Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
    404 
    405 {{{
    406 foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    407 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    408 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    409 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    410 }}}
    411 
    412 Delete the sliver, confirm that 'deleted' is now "True", and confirm that the sliver now does not show up in FlowVisor, and has no flowspace rules:
    413 
    414 {{{
    415 foamctl delete-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    416 foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    417 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    418 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    419 }}}
    420 
    421 That set of tests should generate seven e-mail messages to the experimenter and seven to the FOAM admin: There should be a "sliver disabled" message, then "sliver rejected", then approved, rejected, disabled, approved, and deleted. Probably in that order, although e-mail is sometimes delivered out of sequence, so don't worry too much if they're not in order.
    422 
    423 == Expiration ==
    424 
    425 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    426 
    427 The steps in this section look at a newly-created sliver, and test expiration in FOAM, by changing its expiration date and, then running the scripts that expire slivers and notify about soon-to-expire ones.
    428 
    429 First, review and approve the sliver.
    430 
    431 Show the sliver, and confirm that 'status' is "Pending":
    432 
    433 {{{
    434 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    435 }}}
    436 
    437 Confirm that the sliver doesn't yet show up in FlowVisor:
    438 
    439 {{{
    440 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    441 }}}
    442 
    443 (Expect a "slice does not exist" error.)
    444 
    445 Show the sliver's rspec:
    446 
    447 {{{
    448 ssh $foamserver foamctl show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    449 }}}
    450 
    451 Show the sliver's flowspec:
    452 
    453 {{{
    454 ssh $foamserver foamctl show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    455 }}}
    456 
    457 Show the sliver's flowspace rules:
    458 
    459 {{{
    460 ssh $foamserver foamctl show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    461 }}}
    462 
    463 Approve the sliver, and confirm that 'status' is now "Approved":
    464 
    465 {{{
    466 ssh $foamserver foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    467 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    468 }}}
    469 
    470 (That should generate a "sliver approved" e-mail message, one to the experimenter and one to the FOAM admin.)
    471 
    472 Confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
    473 
    474 {{{
    475 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    476 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    477 }}}
    478 
    479 (Don't worry about the ouptut saying that it can't connect to the controller; this is expected if your rspec didn't include a controller that's actually running, which the example one doesn't.)
    480 
    481 Now, do some tests to make sure FOAM notifies you about a sliver that will expire within a day, but only once.
    482 
    483 Change the expiration date of your slice to be nine days away, since otherwise you can't extend the sliver expiration date very far:
    484 
    485 {{{
    486 omni renewslice $slicename "$(date -d 'now + 9 days')" -f gpolab
    487 }}}
    488 
    489 Change the expiration date of your sliver to be twelve hours away, and confirm that the sliver has the new expiration date:
    490 
    491 {{{
    492 omni -a $am renewsliver $slicename "$(date -d 'now + 12 hours')"
    493 omni -a $am sliverstatus $slicename
    494 }}}
    495 
    496 (That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
    497 
    498 Add a temporary cron job to tell FOAM to send mail about slivers that are expiring soon and remove any expired slivers:
    499 
    500 {{{
    501 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    502 }}}
    503 
    504 (Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    505 
    506 Add the cron job again, to confirm that it doesn't send a second notification:
    507 
    508 {{{
    509 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    510 }}}
    511 
    512 (Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    513 
    514 Next, test to make sure FOAM notifies you again if you change the expiration date.
    515 
    516 Change the expiration date of your sliver to be one day away, and confirm that the sliver has the new expiration date:
    517 
    518 {{{
    519 omni -a $am renewsliver $slicename "$(date -d 'now + 1 day')"
    520 omni -a $am sliverstatus $slicename
    521 }}}
    522 
    523 (That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
    524 
    525 Add the cron job again, to confirm that it does notify you again now that the expiration date has changed:
    526 
    527 {{{
    528 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    529 }}}
    530 
    531 (Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    532 
    533 Next, do some tests to make sure that FOAM notifies you about a sliver that will expire within a week, but only once.
    534 
    535 Change the expiration date of your sliver to be two days away, and confirm that the sliver has the new expiration date:
    536 
    537 {{{
    538 omni -a $am renewsliver $slicename "$(date -d 'now + 2 days')"
    539 omni -a $am sliverstatus $slicename
    540 }}}
    541 
    542 (That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
    543 
    544 Add the cron job again, to confirm that it now sends a "within a week" message:
    545 
    546 {{{
    547 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    548 }}}
    549 
    550 (Within a minute (when the cron job fires), that should generate a "sliver expiring soon" e-mail message, one to the experimenter and one to the FOAM admin; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    551 
    552 Add the cron job again, to confirm that it doesn't send a second notification:
    553 
    554 {{{
    555 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    556 }}}
    557 
    558 (Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    559 
    560 Next, test to make sure FOAM doesn't notify you at all if your sliver is expiring more than a week from now.
    561 
    562 Change the expiration date of your sliver to be eight days away, and confirm that the sliver has the new expiration date:
    563 
    564 {{{
    565 omni -a $am renewsliver $slicename "$(date -d 'now + 8 days')"
    566 omni -a $am sliverstatus $slicename
    567 }}}
    568 
    569 (That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
    570 
    571 Add the cron job again, to confirm that it doesn't send a notification:
    572 
    573 {{{
    574 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    575 }}}
    576 
    577 (Within a minute (when the cron job fires), that should generate no messages about slivers expiring in the future; no messages about expired slivers being deleted; and one message to gpo-ops saying that the cron job ran.)
    578 
    579 Finally, test to make sure FOAM deletes a sliver if its expiration date is in the past.
    580 
    581 Change the expiration date of your sliver to be right now, and confirm that the sliver has the new expiration date:
    582 
    583 {{{
    584 omni -a $am renewsliver $slicename "$(date -d 'now')"
    585 omni -a $am sliverstatus $slicename
    586 }}}
    587 
    588 (That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)
    589 
    590 Add the cron job again, to confirm that it deletes the sliver:
    591 
    592 {{{
    593 ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""
    594 }}}
    595 
    596 (Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; a "sliver expired" message, one to the experimenter and one to the FOAM admin; and one message to gpo-ops saying that the cron job ran.)
    597 
    598 Confirm that the sliver no longer shows up in FOAM:
    599 
    600 {{{
    601 ssh $foamserver foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | grep $slice_urn
    602 ssh $foamserver foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    603 }}}
    604 
    605 (Expect no output for the first, and 'deleted' to be "True" for the second.)
    606 
    607 Confirm that the sliver no longer shows up in FlowVisor:
    608 
    609 {{{
    610 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    611 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    612 }}}
    613 
    614 (Expect a "slice does not exist" error for the first, and no output for the second.)
    615 
    616 Clean up the last cron job:
    617 
    618 {{{
    619 ssh -t $foamserver sudo rm /etc/cron.d/foamtest
    620 }}}