Changes between Version 42 and Version 43 of OpenFlow/FOAM/Testing


Ignore:
Timestamp:
06/22/13 01:41:30 (11 years ago)
Author:
Josh Smift
Comment:

Replaced some automation with some actual commands, to make this more accessible

Legend:

Unmodified
Added
Removed
Modified

  • OpenFlow/FOAM/Testing

    v42 v43  
    22
    33= Testing FOAM =
    4 
    5 NOTE that all the foamctl commands on this page now use the new 0.8 syntax. If you want an older version, with the 0.6 syntax, try http://groups.geni.net/geni/wiki/OpenFlow/FOAM/Testing?version=6 instead.
    6 
    7 NOTE that we've steadily increased the extent to which these tests are automated, by scripts that are currently GPO-only. We're working on making them public, but in the meanwhile, this page may not be as useful for non-BBN sites.
    84
    95This page describes a couple of ways to test FOAM:
    … …  
    2319Pre-requisites/assumptions:
    2420
    25  * You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file, etc).
    26  * You have a valid request rspec file to test with. We plan to make some sample rspecs available.
    27  * You can log in non-interactively to the FOAM and FV servers, and have admin privs there, and have the FOAM and fvadmin password stored in files (/etc/foam.passwd and /etc/flowvisor.passwd). (This generally can't be used as-is to test FOAM at other sites.)
    28  * You know what output to expect when common things succeed. (Expected output is described here only in cases where an error is the expected result, or where it otherwise might not be obvious; if you think the expected output isn't obvious for any given step, let us know.)
    29  * The URL of the FOAM server is also a name that you can use to log in to it. (Very likely, but might be untrue with virtualhosts or some other odd thing.)
    30  * On the FOAM server you're testing, when you create a sliver, it will be automatically approved. (More specifically, that geni.openflow.analysis-engine = True (analyze slivers for overlap), and geni.approval.approve-on-creation = 2 (approve slivers at creation time if they don't fail any overlap-analysis tests), and the test sliver's rspec describes a flowspace that will pass overlap analysis.)
     21 * You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file which sets a default project, etc).
     22 * You have a valid request rspec file to test with, which will be auto-approved on your FOAM server, and result in two !FlowVisor flowspace rules. We plan to make some sample rspecs available.
     23 * You can log in non-interactively to the FOAM and FV servers, and have admin privs there, and have the FOAM and fvadmin password stored in files (/etc/foam.passwd and /etc/flowvisor.passwd). (This requirement means that these tests generally can't be used as-is to test FOAM at sites other than your own.)
     24 * You know what output to expect when common things succeed. (Expected output is described here only in cases where an error is the expected result, or where it otherwise might not be obvious; if you think the expected output isn't obvious for any given step, let us know so we can update the docs.)
     25 * The hostname part of the URL of the FOAM server is also a name that you can use to log in to it. (Very likely, but might be untrue with virtualhosts or some other odd thing.)
    3126 * On the FOAM server you're testing, when the 'www-data' user runs a cron job that has output, it sends mail to 'www-data', and you can receive this mail.
    32  * On the FOAM server you're testing, you have the foam-confirm-sliver, flowvisor-confirm-fvslice-exists, and flowvisor-confirm-flowspace-rule-count scripts. (These are currently GPO-only; GPO folks, they're in the syseng repo, under gposw.)
    33  * On the system where you're running commands, you have the foam-create-and-verify-sliver script.
    34 
    35 Note that the scripts mentioned in the last two items are currently GPO-only. (GPO folks, they're in the syseng repo, under gposw.)
    3627
    3728You can run most of these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server, except as noted below.
    … …  
    6051
    6152{{{
    62 rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/openflow-mac-acedcodefood.rspec
    63 }}}
    64 
    65 Set the slicename to something with your username and "foamtest":
     53rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-acedcodefood-vlan-none.rspec
     54}}}
     55
     56Set the slicename to be your username plus "foamtest":
    6657
    6758{{{
    … …  
    6960}}}
    7061
    71 Set the full slice URN (modify this if you're not using pgeni.gpolab.bbn.com as your Slice Authority), and a version of it for handing off to egrep later:
    72 
    73 {{{
    74 slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
     62Set the full slice URN (modify this if you're not using ch.geni.net as your Slice Authority), and a version of it for handing off to egrep later:
     63
     64{{{
     65default_project=$(grep '^default_project' ~/.gcf/omni_config | head -1 | awk '{print $NF}')
     66slice_urn=urn:publicid:IDN+ch.geni.net:${default_project}+slice+$slicename
    7567slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
    7668}}}
    … …  
    9082fvserver=$foamserver
    9183am=https://${foamserver}:3626/foam/gapi/1
    92 rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/openflow-mac-acedcodefood.rspec
     84rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-acedcodefood-vlan-none.rspec
    9385slicename=${USER}foamtest
    94 slice_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+$slicename
     86default_project=$(grep '^default_project' ~/.gcf/omni_config | head -1 | awk '{print $NF}')
     87slice_urn=urn:publicid:IDN+ch.geni.net:${default_project}+slice+$slicename
    9588slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
    9689omni -a $am getversion
    … …  
    10295This subsection runs creates a sliver, and runs through a sequence of approve/disable/reject/delete actions to confirm that it can transition from any of those states to the others, and checks the FlowVisor after each change to confirm that the sliver is or isn't enabled (as appropriate).
    10396
    104 Create and verify the sliver:
    105 
    106 {{{
    107 foam-create-and-verify-sliver -q $foamserver $fvserver $am $slicename $rspec approved true 2
     97Create the slice and sliver:
     98
     99{{{
     100omni createslice $slicename >& /dev/null
     101omni -a $am createsliver $slicename $rspec >& /dev/null
    108102}}}
    109103
    110104(That should generate a "sliver created" and a "sliver approved" e-mail message, one of each to the experimenter and one of each to the FOAM admin.)
    111105
    112 Get the sliver URN from FOAM, and put it into $sliver_urn:
     106Get the sliver URN from FOAM, and put it into $sliver_urn; and deduce the FV slicename from that:
    113107
    114108{{{
    115109sliver_urn=$(ssh $foamserver foamctl geni:list-slivers --passwd-file=/etc/foam.passwd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
     110fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
     111}}}
     112
     113Confirm that the FOAM sliver's 'status' is now "approved", and confirm that the sliver shows up in !FlowVisor, and has two flowspace rules:
     114
     115{{{
     116ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
     117ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
     118[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
    116119}}}
    117120
    … …  
    124127(Within a minute (when the cron job fires), that should generate a message to www-data saying that the cron job ran, and one of two other things will happen: (a) If there are *other* pending slivers, it should generate a message to the FOAM admin listing only those slivers, and not your test sliver; OR (b) If there are no other pending slivers, it should not send any message to the FOAM admin about pending slivers.)
    125128
    126 Disable the sliver (to put it back into the pending queue), confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:
     129Disable the sliver (to put it back into the pending queue), confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in !FlowVisor, and has zero flowspace rules:
    127130
    128131{{{
    129132ssh $foamserver foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    130 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status pending false 0
     133ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "pending"' || echo "ERROR: Wrong FOAM status"
     134ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     135[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
    131136}}}
    132137
    … …  
    149154This next set of tests will generate a bunch of e-mail; check at the end that it all showed up as expected.
    150155
    151 Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     156Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in !FlowVisor, with two flowspace rules:
    152157
    153158{{{
    154159ssh $foamserver foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    155 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status approved true 2
    156 }}}
    157 
    158 Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:
     160ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
     161ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
     162[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
     163}}}
     164
     165Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in !FlowVisor, and has zero flowspace rules:
    159166
    160167{{{
    161168ssh $foamserver foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    162 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status rejected false 0
    163 }}}
    164 
    165 Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:
     169ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "rejected"' || echo "ERROR: Wrong FOAM status"
     170ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     171[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
     172}}}
     173
     174Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in !FlowVisor, and has zero flowspace rules:
    166175
    167176{{{
    168177ssh $foamserver foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    169 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status pending false 0
    170 }}}
    171 
    172 Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:
     178ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "pending"' || echo "ERROR: Wrong FOAM status"
     179ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     180[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
     181}}}
     182
     183Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in !FlowVisor, and has zero flowspace rules:
    173184
    174185{{{
    175186ssh $foamserver foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    176 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status rejected false 0
    177 }}}
    178 
    179 Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:
     187ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "rejected"' || echo "ERROR: Wrong FOAM status"
     188ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     189[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
     190}}}
     191
     192Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in !FlowVisor, with two flowspace rules:
    180193
    181194{{{
    182195ssh $foamserver foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    183 foam-verify-sliver -q $foamserver $fvserver $sliver_urn status approved true 2
    184 }}}
    185 
    186 Delete the sliver, confirm that 'deleted' is now "True", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:
     196ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
     197ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
     198[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
     199}}}
     200
     201Delete the sliver, confirm that 'deleted' is now "true", and confirm that the sliver now does not show up in !FlowVisor, and has zero flowspace rules:
    187202
    188203{{{
    189204ssh $foamserver foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
    190 foam-verify-sliver -q $foamserver $fvserver $sliver_urn deleted true false 0
     205ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"deleted": "true"' || echo "ERROR: Wrong FOAM status"
     206ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     207[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
    191208}}}
    192209
    … …  
    199216In this subsection, we explicitly identify how many e-mail messages of each type we expect to receive after each run of the expiration scripts, since those notifications are what we're testing here, not just a side effect. The sliver-related ones from FOAM should be sent to the sliver owner and to the FOAM admin; the cron job confirmations should be sent to the user that the cron job runs as ('www-data' in these examples).
    200217
    201 Create and verify the sliver:
    202 
    203 {{{
    204 foam-create-and-verify-sliver -q $foamserver $fvserver $am $slicename $rspec approved true 2
     218Create the slice and sliver:
     219
     220{{{
     221omni createslice $slicename >& /dev/null
     222omni -a $am createsliver $slicename $rspec >& /dev/null
    205223}}}
    206224
    207225(That should generate a "sliver created" and a "sliver approved" e-mail message, one of each to the experimenter and one of each to the FOAM admin.)
    208226
    209 Get the sliver URN from FOAM, and put it into $sliver_urn:
     227Get the sliver URN from FOAM, and put it into $sliver_urn; and deduce the FV slicename from that:
    210228
    211229{{{
    212230sliver_urn=$(ssh $foamserver foamctl geni:list-slivers --passwd-file=/etc/foam.passwd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
     231fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')
     232}}}
     233
     234Confirm that the FOAM sliver's 'status' is now "approved", and confirm that the sliver shows up in !FlowVisor, and has two flowspace rules:
     235
     236{{{
     237ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
     238ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
     239[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
    213240}}}
    214241
    … …  
    330357(Within a minute (when the cron job fires), expected e-mail: 0 "sliver expiring", 1 "sliver expired", and confirmation that the cron job ran.)
    331358
    332 Confirm that the sliver was deleted, does not show up in FlowVisor, and has zero flowspace rules::
    333 
    334 {{{
    335 foam-verify-sliver -q $foamserver $fvserver $sliver_urn deleted true false 0
     359Confirm that the sliver was deleted, does not show up in !FlowVisor, and has zero flowspace rules::
     360
     361{{{
     362ssh $foamserver foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
     363ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"deleted": "true"' || echo "ERROR: Wrong FOAM status"
     364ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
     365[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"
    336366}}}
    337367