Changes between Version 12 and Version 13 of OpenFlow/FOAM/Testing


Ignore:
Timestamp:
05/03/12 16:28:29 (12 years ago)
Author:
Josh Smift
Comment:

Replaced some manual commands with some wrapper scripts

Legend:

Unmodified
Added
Removed
Modified
  • OpenFlow/FOAM/Testing

    v12 v13  
    1515= Sliver creation and management =
    1616
    17 This set of tests includes a Setup section to set some variables, a Creation section to create a sliver, and then additional sections to do various tests with it.
    18 
    19 You only need to do the Setup section once.
    20 
    21 You need to repeat the Creation section before doing any of the other sections, because the Creation section sets some variables that you'll need for the other sections, and each of the other sections concludes by deleting the sliver.
     17This section contains multiple tests, and includes a Setup subsection to set some variables, a Creation subsection to create a sliver, and then additional subsections to do various tests with it.
     18
     19You only need to do the Setup subsection once.
     20
     21You need to repeat the Creation subsection before doing any of the other subsections, because the Creation subsection sets some variables that you'll need for the other subsections, and each of the other subsections concludes by deleting the sliver.
    2222
    2323Pre-requisites/assumptions:
     
    3030 * On the FOAM server you're testing, when you create a sliver, it will be automatically approved. (More specifically, that geni.openflow.analysis-engine = True (analyze slivers for overlap), and geni.approval.approve-on-creation = 2 (approve slivers at creation time if they don't fail any overlap-analysis tests), and the test sliver's rspec describes a flowspace that will pass overlap analysis.)
    3131 * On the FOAM server you're testing, when the 'www-data' user runs a cron job that has output, it sends mail to 'www-data', and you can receive this mail.
     32 * On the FOAM server you're testing, you have the foam-confirm-sliver, flowvisor-confirm-fvslice-exists, and flowvisor-confirm-flowspace-rule-count scripts. (These are currently GPO-only; GPO folks, they're in the syseng repo, under gposw.)
    3233
    3334You can run most of these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server, except as noted below.
     
    121122== Simple management ==
    122123
    123 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    124 
    125 NOTE: This section needs to be revamped to match the new assumption that when you create a sliver, it will be automatically approved.
    126 
    127 The steps in this section look at a newly-created sliver, approve it, confirm that it shows up in the FlowVisor, delete it, and confirm that it vanishes from the FlowVisor.
     124NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] subsection immediately before starting this subsection.
     125
     126NOTE: This subsection needs to be revamped to match the new assumption that when you create a sliver, it will be automatically approved.
     127
     128The steps in this subsection look at a newly-created sliver, approve it, confirm that it shows up in the FlowVisor, delete it, and confirm that it vanishes from the FlowVisor.
    128129
    129130Show the sliver, and confirm that 'status' is "Pending":
     
    214215== Complex management ==
    215216
    216 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    217 
    218 The steps in this section look at a newly-created sliver, run through a sequence of approve/disable/reject actions to confirm that it can transition from any of those states to the others, and check the FlowVisor after each change; then delete the sliver, and confirm that it vanishes from the FlowVisor.
    219 
    220 NOTE: These steps are run directly on the FOAM/FlowVisor server, rather than on your Omni client system. If those aren't the same server, do the 'foamctl' commands on the FOAM server, and the 'fvctl' commands on the FV server.
     217NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] subsection immediately before starting this subsection.
     218
     219The steps in this subsection look at a newly-created sliver, run through a sequence of approve/disable/reject actions to confirm that it can transition from any of those states to the others, and check the FlowVisor after each change; then delete the sliver, and confirm that it vanishes from the FlowVisor.
     220
     221NOTE: These steps are run directly on the FOAM/FlowVisor server, rather than on your Omni client system. If those aren't the same server, do the FOAM commands on the FOAM server, and the FlowVisor commands on the FV server.
    221222
    222223Repeat the setup steps from above, but on the server, so you've got the right variables set there:
     
    233234
    234235{{{
    235 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    236 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    237 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    238 }}}
    239 
    240 (Ignore warnings saying that it can't connect to the sliver's controller, if any; and likewise every time you confirm that the sliver shows up in Flowvisor below.)
     236foam-confirm-sliver $sliver_urn status approved
     237flowvisor-confirm-fvslice-exists $fv_slicename true
     238flowvisor-confirm-flowspace-rule-count $fv_slicename 2
     239}}}
    241240
    242241Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:
     
    259258
    260259{{{
    261 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    262 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    263 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     260foam-confirm-sliver $sliver_urn status pending
     261flowvisor-confirm-fvslice-exists $fv_slicename false
     262flowvisor-confirm-flowspace-rule-count $fv_slicename 0
    264263}}}
    265264
     
    286285{{{
    287286foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    288 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    289 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    290 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     287foam-confirm-sliver $sliver_urn status approved
     288flowvisor-confirm-fvslice-exists $fv_slicename true
     289flowvisor-confirm-flowspace-rule-count $fv_slicename 2
    291290}}}
    292291
     
    295294{{{
    296295foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    297 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    298 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    299 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     296foam-confirm-sliver $sliver_urn status rejected
     297flowvisor-confirm-fvslice-exists $fv_slicename false
     298flowvisor-confirm-flowspace-rule-count $fv_slicename 0
    300299}}}
    301300
     
    304303{{{
    305304foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    306 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    307 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    308 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     305foam-confirm-sliver $sliver_urn status pending
     306flowvisor-confirm-fvslice-exists $fv_slicename false
     307flowvisor-confirm-flowspace-rule-count $fv_slicename 0
    309308}}}
    310309
     
    313312{{{
    314313foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    315 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    316 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    317 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     314foam-confirm-sliver $sliver_urn status rejected
     315flowvisor-confirm-fvslice-exists $fv_slicename false
     316flowvisor-confirm-flowspace-rule-count $fv_slicename 0
    318317}}}
    319318
     
    322321{{{
    323322foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    324 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    325 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    326 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
     323foam-confirm-sliver $sliver_urn status approved
     324flowvisor-confirm-fvslice-exists $fv_slicename true
     325flowvisor-confirm-flowspace-rule-count $fv_slicename 2
    327326}}}
    328327
     
    331330{{{
    332331foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    333 foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    334 fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    335 fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    336 }}}
    337 
    338 That set of tests should generate six e-mail messages to the experimenter and eight to the FOAM admin: There should be a "sliver approved" message, then "sliver rejected", then "sliver disabled", then rejected, approved, and deleted. Probably in that order, although e-mail is sometimes delivered out of sequence, so don't worry too much if they're not in order.
     332foam-confirm-sliver $sliver_urn deleted true
     333flowvisor-confirm-fvslice-exists $fv_slicename false
     334flowvisor-confirm-flowspace-rule-count $fv_slicename 0
     335}}}
     336
     337That set of tests should generate six e-mail messages to the experimenter and six to the FOAM admin: There should be a "sliver approved" message, then "sliver rejected", then "sliver disabled", then rejected, approved, and deleted. Probably in that order, although e-mail is sometimes delivered out of sequence, so don't worry too much if they're not in order.
    339338
    340339== Expiration ==
    341340
    342 NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] section immediately before starting this section.
    343 
    344 The steps in this section look at a newly-created sliver, and test expiration in FOAM, by changing its expiration date and, then running the scripts that expire slivers and notify about soon-to-expire ones.
     341NOTE: Do the [wiki:OpenFlow/FOAM/Testing#Creation Creation] subsection immediately before starting this subsection.
     342
     343The steps in this subsection look at a newly-created sliver, and test expiration in FOAM, by changing its expiration date and, then running the scripts that expire slivers and notify about soon-to-expire ones.
    345344
    346345Confirm that the sliver was approved at creation time: Check that 'status' is "Approved", and that the sliver shows up in FlowVisor, with two flowspace rules:
    347346
    348347{{{
    349 ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    350 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    351 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    352 }}}
    353 
    354 (Ignore warnings saying that it can't connect to the sliver's controller, if any; and likewise every time you confirm that the sliver shows up in Flowvisor below.)
     348foam-confirm-sliver $sliver_urn status approved
     349flowvisor-confirm-fvslice-exists $fv_slicename true
     350flowvisor-confirm-flowspace-rule-count $fv_slicename 2
     351}}}
    355352
    356353Now, do some tests to make sure FOAM notifies you about a sliver that will expire within a day, but only once.
     
    471468(Within a minute (when the cron job fires), that should generate a "sliver expiring shortly" e-mail message, one to the experimenter and one to the FOAM admin; a "sliver expired" message, one to the experimenter and one to the FOAM admin; and one message to www-data saying that the cron job ran.)
    472469
    473 Confirm that the sliver no longer shows up in FOAM:
    474 
    475 {{{
    476 ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    477 ssh $foamserver foamctl geni:list-slivers --passwd-file=/opt/foam/etc/foampasswd | grep $slice_urn
    478 }}}
    479 
    480 (Expect 'deleted' to be "True" for the first, and no output for the second. For the first, note that 'status' will still be "Approved", which is fine, it's just tracking what the status was when it was deleted.)
    481 
    482 Confirm that the sliver no longer shows up in FlowVisor:
    483 
    484 {{{
    485 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo $fv_slicename
    486 ssh $fvserver fvctl --passwd-file=/etc/flowvisor/fvpasswd listFlowSpace | grep $fv_slicename
    487 }}}
    488 
    489 (Expect a "slice does not exist" error for the first, and no output for the second.)
     470Confirm that the sliver was deleted, does not show up in FlowVisor, and has no flowspace rules::
     471
     472{{{
     473foam-confirm-sliver $sliver_urn deleted true
     474flowvisor-confirm-fvslice-exists $fv_slicename false
     475flowvisor-confirm-flowspace-rule-count $fv_slicename 0
     476}}}
    490477
    491478Clean up the last cron job:
     
    497484= GENI AM API =
    498485
    499 This tests whether a given FOAM server is compliant with the GENI AM API, using Omni's built-in acceptance test suite. Stanford/ONLab and the GPO generally run these tests when new versions of FOAM are released, but you can run them yourself too, if you want.
     486This section tests whether a given FOAM server is compliant with the GENI AM API, using Omni's built-in acceptance test suite. Stanford/ONLab and the GPO generally run these tests when new versions of FOAM are released, but you can run them yourself too, if you want.
    500487
    501488Pre-requisites/assumptions: