wiki:OpenFlow/FOAM/Testing

Version 47 (modified by Josh Smift, 10 years ago) (diff)

--

Testing FOAM

This page describes a couple of ways to test FOAM:

  • Testing sliver administration, including:
    • Test for various sliver management operations.
    • Tests for sliver expiration.
  • Testing auto-approval, in each of the three approve-on-creation modes.
  • Testing that the software complies with the GENI AM API, using the test suite that comes with GCF.

Sliver administration

This section contains multiple tests, and includes a "General setup" subsection to set some variables, and then additional subsections to do various tests.

You only need to do the "General setup" subsection once.

Pre-requisites/assumptions:

  • You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file which sets a default project, etc).
  • You have a valid request rspec file to test with, which will be auto-approved on your FOAM server, and result in two FlowVisor flowspace rules. We plan to make some sample rspecs available.
  • You can log in non-interactively to the FOAM and FV servers, and have admin privs there, and have the FOAM and fvadmin password stored in files (/etc/foam.passwd and /etc/flowvisor.passwd). (This requirement means that these tests generally can't be used as-is to test FOAM at sites other than your own.)
  • You know what output to expect when common things succeed. (Expected output is described here only in cases where an error is the expected result, or where it otherwise might not be obvious; if you think the expected output isn't obvious for any given step, let us know so we can update the docs.)
  • The hostname part of the URL of the FOAM server is also a name that you can use to log in to it. (Very likely, but might be untrue with virtualhosts or some other odd thing.)
  • On the FOAM server you're testing, when the 'www-data' user runs a cron job that has output, it sends mail to 'www-data', and you can receive this mail.

You can run most of these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server, except as noted below.

General setup

Set the FOAM server name (modify this to use the DNS name of the server that you want to test):

foamserver=foam.example.edu

Set the name of the FlowVisor server to be the same (if it's the same, or something else if it isn't):

fvserver=$foamserver

Set the AM API URL:

am=https://${foamserver}:3626/foam/gapi/2

Identify the rspec you plan to use (modify this if you want to use a different rspec):

rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-acedcodefood-vlan-none.rspec

Set the slicename to be your username plus "foamtest":

slicename=${USER}foamtest

Set the full slice URN (modify this if you're not using ch.geni.net as your Slice Authority), and a version of it for handing off to egrep later:

default_project=$(grep '^default_project' ~/.gcf/omni_config | head -1 | awk '{print $NF}')
slice_urn=urn:publicid:IDN+ch.geni.net:${default_project}+slice+$slicename
slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)

Make sure FOAM is listening at all:

omni -a $am getversion
omni -a $am listresources

Here's all of those commands in one block for ease of copy-and-paste:

foamserver=foam.example.edu

fvserver=$foamserver
am=https://${foamserver}:3626/foam/gapi/2
rspec=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-acedcodefood-vlan-none.rspec
slicename=${USER}foamtest
default_project=$(grep '^default_project' ~/.gcf/omni_config | head -1 | awk '{print $NF}')
slice_urn=urn:publicid:IDN+ch.geni.net:${default_project}+slice+$slicename
slice_urn_egrep=$(echo $slice_urn | sed -e 's/+/\\+'/g)
omni -a $am getversion
omni -a $am listresources

Management

This subsection runs creates a sliver, and runs through a sequence of approve/disable/reject/delete actions to confirm that it can transition from any of those states to the others, and checks the FlowVisor after each change to confirm that the sliver is or isn't enabled (as appropriate).

Create the slice and sliver:

omni createslice $slicename >& /dev/null
omni -a $am createsliver $slicename $rspec >& /dev/null

(That should generate a "sliver created" and a "sliver approved" e-mail message, one of each to the experimenter and one of each to the FOAM admin.)

Get the sliver URN from FOAM, and put it into $sliver_urn; and deduce the FV slicename from that:

sliver_urn=$(ssh $foamserver foamctl geni:list-slivers --passwd-file=/etc/foam.passwd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')

Confirm that the FOAM sliver's 'status' is now "approved", and confirm that the sliver shows up in FlowVisor, and has two flowspace rules:

ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), that should generate a message to www-data saying that the cron job ran, and one of two other things will happen: (a) If there are *other* pending slivers, it should generate a message to the FOAM admin listing only those slivers, and not your test sliver; OR (b) If there are no other pending slivers, it should not send any message to the FOAM admin about pending slivers.)

Disable the sliver (to put it back into the pending queue), confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:

ssh $foamserver foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "pending"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

(That should generate a "sliver disabled" e-mail message, one to the experimenter and one to the FOAM admin.)

Add a temporary cron job to tell FOAM to generate e-mail with a list of pending slivers:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/daily-queue ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), that should generate a message to www-data saying that the cron job ran; and e-mail to the FOAM admin with a list of pending slivers, including your test sliver.)

Clean up the cron job:

ssh -t $foamserver sudo rm /etc/cron.d/foamtest

This next set of tests will generate a bunch of e-mail; check at the end that it all showed up as expected.

Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:

ssh $foamserver foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:

ssh $foamserver foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "rejected"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Disable the sliver, confirm that 'status' is now "Pending", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:

ssh $foamserver foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "pending"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Reject the sliver, confirm that 'status' is now "Rejected", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:

ssh $foamserver foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "rejected"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Approve the sliver, confirm that 'status' is now "Approved", and confirm that the sliver now shows up in FlowVisor, with two flowspace rules:

ssh $foamserver foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Delete the sliver, confirm that 'deleted' is now "true", and confirm that the sliver now does not show up in FlowVisor, and has zero flowspace rules:

ssh $foamserver foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"deleted": "true"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

That set of tests should generate six e-mail messages to the experimenter and six to the FOAM admin: There should be a "sliver approved" message, then "sliver rejected", then "sliver disabled", then "rejected", then "approved", and then "deleted". Probably in that order, although e-mail is sometimes delivered out of sequence, so don't worry too much if they're not in order.

Expiration

This subsection runs creates a sliver, and tests the sliver expiration process in FOAM, by changing the sliver's expiration date, and then running the scripts that expire slivers and notify about soon-to-expire ones, for a variety of expiration dates.

In this subsection, we explicitly identify how many e-mail messages of each type we expect to receive after each run of the expiration scripts, since those notifications are what we're testing here, not just a side effect. The sliver-related ones from FOAM should be sent to the sliver owner and to the FOAM admin; the cron job confirmations should be sent to the user that the cron job runs as ('www-data' in these examples).

Create the slice and sliver:

omni createslice $slicename >& /dev/null
omni -a $am createsliver $slicename $rspec >& /dev/null

(That should generate a "sliver created" and a "sliver approved" e-mail message, one of each to the experimenter and one of each to the FOAM admin.)

Get the sliver URN from FOAM, and put it into $sliver_urn; and deduce the FV slicename from that:

sliver_urn=$(ssh $foamserver foamctl geni:list-slivers --passwd-file=/etc/foam.passwd | egrep sliver_urn.+$slice_urn_egrep | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/')
fv_slicename=$(echo $sliver_urn | awk -F : '{print $NF}')

Confirm that the FOAM sliver's 'status' is now "approved", and confirm that the sliver shows up in FlowVisor, and has two flowspace rules:

ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"status": "approved"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null || echo "ERROR: fvctl exited with non-zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "2" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

First, test that FOAM notifies you about a sliver that will expire within a day, but only once.

Change the expiration date of your slice to be nine days away, since otherwise you can't extend the sliver expiration date very far:

omni renewslice $slicename "$(date -d 'now + 9 days')"

Change the expiration date of your sliver to be twelve hours away, and confirm that the sliver has the new expiration date:

omni -a $am renewsliver $slicename "$(date -d 'now + 12 hours')" >& /dev/null
omni -a $am sliverstatus $slicename |& grep foam_expires

(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)

Add a temporary cron job to tell FOAM to send mail about slivers that are expiring in the near future, and remove any expired slivers:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 1 "sliver expiring shortly", 0 "sliver expired", and confirmation that the cron job ran.)

Add the cron job again, to confirm that it doesn't send a second notification:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 0 "sliver expiring", 0 "sliver expired", only confirmation that the cron job ran.)

Next, test to make sure FOAM notifies you again if you change the expiration date.

Change the expiration date of your sliver to be one day away, and confirm that the sliver has the new expiration date:

omni -a $am renewsliver $slicename "$(date -d 'now + 1 day')" >& /dev/null
omni -a $am sliverstatus $slicename |& grep foam_expires

(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)

Add the cron job again, to confirm that it does notify you again now that the expiration date has changed:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 1 "sliver expiring shortly", 0 "sliver expired", and confirmation that the cron job ran.)

Next, do some tests to make sure that FOAM notifies you about a sliver that will expire within a week, but only once.

Change the expiration date of your sliver to be two days away, and confirm that the sliver has the new expiration date:

omni -a $am renewsliver $slicename "$(date -d 'now + 2 days')" >& /dev/null
omni -a $am sliverstatus $slicename |& grep foam_expires

(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)

Add the cron job again, to confirm that it now sends a "within a week" message:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 1 "sliver expiring within a week", 0 "sliver expired", and confirmation that the cron job ran.)

Add the cron job again, to confirm that it doesn't send a second notification:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 0 "sliver expiring", 0 "sliver expired", only confirmation that the cron job ran.)

Next, test to make sure FOAM doesn't notify you at all if your sliver is expiring more than a week from now.

Change the expiration date of your sliver to be eight days away, and confirm that the sliver has the new expiration date:

omni -a $am renewsliver $slicename "$(date -d 'now + 8 days')" >& /dev/null
omni -a $am sliverstatus $slicename |& grep foam_expires

(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)

Add the cron job again, to confirm that it doesn't send a notification:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 0 "sliver expiring", 0 "sliver expired", only confirmation that the cron job ran.)

Finally, test to make sure FOAM deletes a sliver if its expiration date is in the past.

Change the expiration date of your sliver to be right now, and confirm that the sliver has the new expiration date:

omni -a $am renewsliver $slicename "$(date -d 'now')" >& /dev/null
omni -a $am sliverstatus $slicename |& grep foam_expires

(That should generate a "sliver renewed" e-mail message, one to the experimenter and one to the FOAM admin.)

Add the cron job again, to confirm that it deletes the sliver:

ssh -t $foamserver "sudo sh -c \"echo $(date +'%M %H' -d 'now + 1 min') '* * *  www-data  /opt/foam/bin/expire ; /opt/foam/bin/expire-emails ; echo foamtest cron job ran at $(date)' > /etc/cron.d/foamtest\""

(Within a minute (when the cron job fires), expected e-mail: 0 "sliver expiring", 1 "sliver expired", and confirmation that the cron job ran.)

Confirm that the sliver was deleted, does not show up in FlowVisor, and has zero flowspace rules::

ssh $foamserver foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd
ssh $foamserver foamctl geni:show-sliver -u $sliver_urn --passwd-file=/etc/foam.passwd | grep -q -i '"deleted": "true"' || echo "ERROR: Wrong FOAM status"
ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd getSliceInfo $fv_slicename >& /dev/null && echo "ERROR: fvctl exited with zero status"
[ $(ssh $fvserver fvctl --passwd-file=/etc/flowvisor.passwd listFlowSpace | grep $fv_slicename | wc -l) == "0" ] || echo "ERROR: Didn't get the expected number of FV flowspace rules"

Clean up the last cron job:

ssh -t $foamserver sudo rm /etc/cron.d/foamtest

Auto-approval

Since we first started testing auto-approval, our test suite has gotten increasingly more complicated, and more automated. At this point, we can't just post the script that we use to do the testing on the wiki for people to copy and use directly, but we do want people to be able to see what we do, so this section now contains pseudocode that describes what the script does, but which can't actually be run without a fair bit of additional setup (which is also described).

These days, we only bother testing auto-approval mode 2 ("analysis"), because that's what we recommend that GENI sites use.

Setup

The script has a general setup section, which defines:

  • the FOAM and FV server hostnames ($foamserver and $fvserver)
  • the AM URL ($am)
  • a command to run the foam-create-and-verify-sliver script ($fcavs)
  • four slice names ($slicename1, $slicename2, $slicename3, $slicename4)
  • a filename for the saved slice credential for the four slices ($slicecred1, $slicecred2, $slicecred3, $slicecred4)

It then:

  • creates the slices
  • makes sure there aren't any leftover slivers in those slices on the FOAM server you're testing
  • turns off admin e-mail for creation, approval, and deletion, so you're not swamped with admin mail

It then sets up a bunch of rspec variables, referring to rspecs defining a variety of slivers. For testing shared VLANs, it defines a VLAN ID, or 'none', and there's a variant of each rspec for each VLAN ID we want to test.

The rspecs are:

  • Four that specify subnets: The first two don't overlap, the third overlaps with one of the first two, the fourth overlaps with all of the first three.
  • Four that specify MAC addresses: The first two don't overlap, the third overlaps with one of the first two, the fourth overlaps with all of the first three.
  • Four that specify ethertypes: The first two don't overlap, the third overlaps with one of the first two, the fourth overlaps with all of the first three.
  • Two that specify an ethertype and a VLAN: One is a restricted VLAN, one is an exclusive VLAN.
  • One that specifies a subnet, a MAC address, and an ethertype, each of which overlaps with one of the rspecs in the corresponding one of the three groups above.
  • One that specifies a port on a shared VLAN, but no packet match.
  • Twenty-seven that test various combinations of port groups.
  • Two that test restricted VLANs: One that specifies only a restricted VLAN, and one that specifies a restricted VLAN and an exclusive VLAN.
  • Four that specify combinations of exclusive VLANs: The first two don't overlap, the third overlaps with one of the first two, the fourth overlaps with all of the first three.
  • Three that specify combinations of a VLAN and a DPID: VLAN 5 and DPID 1, VLAN 5 and DPID 2, and VLAN 6 and DPID 1.
  • Three that specify combinations of a VLAN and an ethertype: VLAN 5 and ethertype 3171, VLAN 5 and ethertype 3172, and VLAN 6 and ethertype 3171.

Those are used in various combinations in the tests below.

Tests

Overview

The script first runs a set of shared VLAN tests, twice: Once with the VLAN ID unspecified (as on a VLAN-hybrid switch), and once with a specified shared VLAN (which must be configured as a shared VLAN on the FOAM server of course). The test_shared_vlan function below has more details.

It then runs three sets of exclusive VLAN tests with four slivers (ABCD):

  • One with four slivers which just specify VLANs, where A and B don't overlap, C overlaps with A, and D overlaps with all of them.
  • One with four slivers which specify VLAN IDs and DPIDs, where A, B, and C all overlap, and D doesn't overlap with any of them.
  • One with four slivers which specify VLAN IDs and ethertypes, where A, B, and C all overlap, and D doesn't overlap with any of them.

Finally, it runs two simple tests using slivers with restricted VLANs.

Four-sliver tests

The "four-sliver" tests use four rspecs that share a feature, where some overlap with respect to that feature, and some don't. Each test creates the slivers in a different order, to confirm that the anlysis engine detects overlap regardless of whether a more specific or less specific sliver is created first.

There are two "styles" of four-sliver tests. In "style 1", the first two rspecs (A and B) don't overlap, the third (C) overlaps with one of the first two (A), and the fourth (D) overlaps with all of the first three. The ensuing tests verify these principles:

  • Whichever of A and C gets created first should be approved, and will then block the other.
  • B should always be approved as long as it's created before D, since it isn't blocked by either A or C.
  • D should only be approved if it's created first, because otherwise it's blocked by any of A, B, or C; and if it's created first, it blocks all of them.
  • If something would block other things if it was approved, it doesn't block those things if it's pending. (For example, D normally blocks B, but in A D B C, B should not be blocked, because D should not be approved, because A blocks it; the pending D should not block B, even though an approved B would block D.)

In "style 2", the first three slivers all overlap, and the fourth doesn't overlap with any of them. The ensuing tests verify these principles:

  • Whichever of A, B, or C gets created first should be approved, and block the other two.
  • D should always be approved, no matter when it's created.
  • If D is approved before A, B, or C, it should have no effect on whether the others are approved.

Each stle is used at appropriate points in the script.

Three-feature tests

The "three-feature" tests use one rspec that reserves three features on a shared VLAN (an IP subnet, a MAC address, and an Ethertype), and tests situations where the three-way rspec does and doesn't overlap with three other rspecs that only reserve one of those three things each. Each test creates the slivers in a different order, to confirm that the anlysis engine detects overlap regardless of whether a more specific or less specific sliver is created first.

The ensuing tests verify these principles:

  • If the three-way sliver doesn't overlap with any of the one-way slivers, they all get approved.
  • If the three-way sliver overlaps with the others:
    • It should be approved if it's created first, and the others should be pending.
    • It should be pending if it's created last, and the others should be approved.
    • If any one-way sliver is created first, and the three-way sliver is created second, the three-way sliver should be pending, and the others should all be approved.

We don't bother to test what happens if the there-way sliver is created third, since it should be the same as when it's created second.

Port group tests

Note that the port group tests assumes that you have certain datapaths and port groups defined in your FOAM server, which correspond to the datapaths and port groups in the rspecs. In our rspecs, we test combinations that reflect the network managed by our production FOAM server, with fake datapaths on the pre-production systems.

Pseudocode

All the tests, in pseudocode, follow. DO NOT just copy and paste this; it shouldn't do much, but it definitely won't do anything good.

Main loop

This mostly calls the other functions defined below, but also does the restricted VLAN test.

## test_approval_mode_2
#
# This function tests auto-approval mode 2.

function test_approval_mode_2
{

  echo "Checking the current approval-on-creation mode..."

  ssh $foamserver foamctl config:get-value --key="geni.approval.approve-on-creation" --passwd-file=/etc/foam.passwd

  echo "If that value was '2', any errors after this point are unexpected."

  # Test shared VLAN stuff twice: Once for the 'none' VLAN, and once for
  # shared VLAN 1750.

  do_rspec_setup none
  test_shared_vlans

  do_rspec_setup 1750
  test_shared_vlans

  # Run the exclusive VLAN tests.

  test_four_slivers_style_1 $rspec_vlan_5 $rspec_vlan_6 $rspec_vlan_7_and_5 $rspec_vlan_7_and_6_and_5 1 1 2 3

  test_four_slivers_style_2 $rspec_vlan_5 $rspec_vlan_5_dpid1 $rspec_vlan_5_dpid2 $rspec_vlan_6_dpid1 1 1 1 1

  test_four_slivers_style_2 $rspec_vlan_5 $rspec_vlan_5_ethertype_3171 $rspec_vlan_5_ethertype_3172 $rspec_vlan_6_ethertype_3171 1 1 1 1

  # Run the restricted VLAN tests.

  $fcavs $slicename1 $rspec_vlan_1 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_vlan_5_and_1 pending false 0 $slicecred1
  $fcavs $slicename2 $rspec_vlan_5 approved true 1 $slicecred2
  for slicename in $slicename1 $slicename2 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

}

test_shared_vlans

## test_shared_vlans
#
# Any tests that relate to shared VLANs need to live here, so they can be
# repeated with all shared VLANs (either VLAN-hybrid ones or ones with
# dl_vlan that are marked as shared).

function test_shared_vlans
{  

  ## Four-sliver tests

  # Run these three times: Once for IP subnets, once for MAC addresses, and once for ethertypes.

  test_four_slivers_style_1 $rspec_172_17_16_0_24 $rspec_172_17_32_0_24 $rspec_172_17_00_0_19 $rspec_172_17_00_0_18 2 2 2 2

  test_four_slivers_style_1 $rspec_cafebeefface $rspec_deadbeadfeed $rspec_aced_and_cafe $rspec_aced_and_cafe_and_dead 2 2 4 6

  test_four_slivers_style_1 $rspec_ethertype_3171 $rspec_ethertype_3172 $rspec_ethertype_3173_and_3171 $rspec_ethertype_3173_and_3172_and_3171 1 1 2 3

  ## Three-feature tests

  # No overlap, three-way first.

  $fcavs $slicename1 $rspec_one_of_each approved true 5 $slicecred1
  $fcavs $slicename2 $rspec_172_17_32_0_24 approved true 2 $slicecred2
  $fcavs $slicename3 $rspec_deadbeadfeed approved true 2 $slicecred3
  $fcavs $slicename4 $rspec_ethertype_3172 approved true 1 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # No overlap, three-way last.

  $fcavs $slicename1 $rspec_172_17_32_0_24 approved true 2 $slicecred1
  $fcavs $slicename2 $rspec_deadbeadfeed approved true 2 $slicecred2
  $fcavs $slicename3 $rspec_ethertype_3172 approved true 1 $slicecred3
  $fcavs $slicename4 $rspec_one_of_each approved true 5 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # Overlap, three-way first.

  $fcavs $slicename1 $rspec_one_of_each approved true 5 $slicecred1
  $fcavs $slicename2 $rspec_172_17_16_0_24 pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_cafebeefface pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_ethertype_3171 pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # Overlap, three-way last.

  $fcavs $slicename1 $rspec_172_17_16_0_24 approved true 2 $slicecred1
  $fcavs $slicename2 $rspec_cafebeefface approved true 2 $slicecred2
  $fcavs $slicename3 $rspec_ethertype_3171 approved true 1 $slicecred3
  $fcavs $slicename4 $rspec_one_of_each pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # Overlap, three-way second, IP first.

  $fcavs $slicename1 $rspec_172_17_16_0_24 approved true 2 $slicecred1
  $fcavs $slicename2 $rspec_one_of_each pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_cafebeefface approved true 2 $slicecred3
  $fcavs $slicename4 $rspec_ethertype_3171 approved true 1 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # Overlap, three-way second, MAC first.

  $fcavs $slicename1 $rspec_cafebeefface approved true 2 $slicecred1
  $fcavs $slicename2 $rspec_one_of_each pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_172_17_16_0_24 approved true 2 $slicecred3
  $fcavs $slicename4 $rspec_ethertype_3171 approved true 1 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # Overlap, three-way second, ethertype first.

  $fcavs $slicename1 $rspec_ethertype_3171 approved true 1 $slicecred1
  $fcavs $slicename2 $rspec_one_of_each pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_172_17_16_0_24 approved true 2 $slicecred3
  $fcavs $slicename4 $rspec_cafebeefface approved true 2 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  ## Forbidden slivers

  $fcavs $slicename1 $rspec_no_packet_match pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  ## Port groups

  $fcavs $slicename1 $rspec_port_groups_03_05 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_03_09 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_03_11 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_05_09 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_05_11 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_09_11 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_20_03 approved true 4 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_ganel_03 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_ganel_05 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_ganel_09 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_ganel_11 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_gardil_03 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_gardil_05 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_gardil_09 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_gardil_11 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_sardis_03 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_sardis_05 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_sardis_09 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_sardis_11 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_bain_03 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_bain_05 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_bain_09 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_bain_11 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_navis_03 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_navis_05 approved true 8 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_navis_09 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_port_groups_navis_11 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  ## Exclusive VLANs

  $fcavs $slicename1 $rspec_vlan_5_ethertype_3172 approved true 1 $slicecred1
  $fcavs $slicename2 $rspec_ethertype_3173_and_3172_and_3171 approved true 3 $slicecred2
  $fcavs $slicename3 $rspec_vlan_6_ethertype_3171 approved true 1 $slicecred3
  for slicename in $slicename1 $slicename2 $slicename3 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  ## Restricted VLANs

  $fcavs $slicename1 $rspec_ethertype_3173_and_vlan_1 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  $fcavs $slicename1 $rspec_ethertype_3173_and_vlan_1_and_5 pending false 0 $slicecred1
  for slicename in $slicename1 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

}

Four-sliver tests

### Four-sliver tests
#
# There are multiple styles of this test; see the comments before each
# function for more info.

## test_four_slivers_style_1
#
# There are multiple styles of "the four-sliver test"; see the comments
# before each function for more info.
#
# This one tests various combinations of four slivers (A B C D), assuming
# that A and B don't overlap, C overlaps with A, and D overlaps with all
# of them.

function test_four_slivers_style_1
{

  ## Variables

  rspec_A=$1
  rspec_B=$2
  rspec_C=$3
  rspec_D=$4

  rules_A=$5
  rules_B=$6
  rules_C=$7
  rules_D=$8

  ## Tests

  # ABCD - A and B should be approved, C and D should be pending.

  $fcavs $slicename1 $rspec_A approved true $rules_A $slicecred1
  $fcavs $slicename2 $rspec_B approved true $rules_B $slicecred2
  $fcavs $slicename3 $rspec_C pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # ACBD - A should be approved, C pending, B approved, and D pending.

  $fcavs $slicename1 $rspec_A approved true $rules_A $slicecred1
  $fcavs $slicename2 $rspec_C pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_B approved true $rules_B $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # BACD - B and A should be approved, C and D should be pending.

  $fcavs $slicename1 $rspec_B approved true $rules_B $slicecred1
  $fcavs $slicename2 $rspec_A approved true $rules_A $slicecred2
  $fcavs $slicename3 $rspec_C pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # BCAD - B and C should be approved, A and D should be pending.

  $fcavs $slicename1 $rspec_B approved true $rules_B $slicecred1
  $fcavs $slicename2 $rspec_C approved true $rules_C $slicecred2
  $fcavs $slicename3 $rspec_A pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # CABD - C should be approved, A pending, B approved, and D pending.

  $fcavs $slicename1 $rspec_C approved true $rules_C $slicecred1
  $fcavs $slicename2 $rspec_A pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_B approved true $rules_B $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # CBAD - C and B should be approved, A and D should be pending.

  $fcavs $slicename1 $rspec_C approved true $rules_C $slicecred1
  $fcavs $slicename2 $rspec_B approved true $rules_B $slicecred2
  $fcavs $slicename3 $rspec_A pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # DABC - D should be approved, A and B and C should be pending.

  $fcavs $slicename1 $rspec_D approved true $rules_D $slicecred1
  $fcavs $slicename2 $rspec_A pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_B pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_C pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # ADBC - A should be approved, D pending, B approved, and C pending.

  $fcavs $slicename4 $rspec_A approved true $rules_A $slicecred4
  $fcavs $slicename3 $rspec_D pending false 0 $slicecred3
  $fcavs $slicename2 $rspec_B approved true $rules_B $slicecred2
  $fcavs $slicename1 $rspec_C pending false 0 $slicecred1
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # ADCB - A should be approved, D pending, C pending, and B approved.

  $fcavs $slicename4 $rspec_A approved true $rules_A $slicecred4
  $fcavs $slicename3 $rspec_D pending false 0 $slicecred3
  $fcavs $slicename2 $rspec_C pending false 0 $slicecred2
  $fcavs $slicename1 $rspec_B approved true $rules_B $slicecred1
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

}

## test_four_slivers_style_2
#
# There are multiple styles of "the four-sliver test"; see the comments
# before each function for more info.
#
# This one tests various combinations of four slivers (A B C D), in which
# A, B, and C all overlap, and D doesn't overlap with any of them. Thus,
# it tests these six combinations: ABCD BACD CABD DABC DBAC DCAB; those
# are sufficient to confirm that the overlap is handled correctly.

function test_four_slivers_style_2
{

  ## Variables

  rspec_A=$1
  rspec_B=$2
  rspec_C=$3
  rspec_D=$4

  rules_A=$5
  rules_B=$6
  rules_C=$7
  rules_D=$8

  ## Tests

  # ABCD - A should be approved, B pending, C pending, D approved.

  $fcavs $slicename1 $rspec_A approved true $rules_A $slicecred1
  $fcavs $slicename2 $rspec_B pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_C pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D approved true $rules_D $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # BACD - B should be approved, A pending, C pending, D approved.

  $fcavs $slicename1 $rspec_B approved true $rules_B $slicecred1
  $fcavs $slicename2 $rspec_A pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_C pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D approved true $rules_D $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # CABD - C should be approved, A pending, B pending, D approved.

  $fcavs $slicename1 $rspec_C approved true $rules_C $slicecred1
  $fcavs $slicename2 $rspec_A pending false 0 $slicecred2
  $fcavs $slicename3 $rspec_B pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_D approved true $rules_D $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # DABC - D should be approved, A approved, B pending, C pending.

  $fcavs $slicename1 $rspec_D approved true $rules_D $slicecred1
  $fcavs $slicename2 $rspec_A approved true $rules_A $slicecred2
  $fcavs $slicename3 $rspec_B pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_C pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # DACB - D should be approved, A approved, C pending, B pending.

  $fcavs $slicename1 $rspec_D approved true $rules_D $slicecred1
  $fcavs $slicename2 $rspec_B approved true $rules_B $slicecred2
  $fcavs $slicename3 $rspec_A pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_C pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

  # DCAB - D should be approved, C approved, A pending, B pending.

  $fcavs $slicename1 $rspec_D approved true $rules_D $slicecred1
  $fcavs $slicename2 $rspec_C approved true $rules_C $slicecred2
  $fcavs $slicename3 $rspec_A pending false 0 $slicecred3
  $fcavs $slicename4 $rspec_B pending false 0 $slicecred4
  for slicename in $slicename1 $slicename2 $slicename3 $slicename4 ; do omni --slicecredfile=$slicename-cred.xml -a $am deletesliver $slicename ; done

}

Cleanup

Finally, the script does some general cleanup: Right now, it just turns back on admin e-mail for creation, approval, and deletion.

GENI AM API

This section tests whether a given FOAM server is compliant with the GENI AM API, using Omni's built-in acceptance test suite. The GPO generally runs these tests when new versions of FOAM are released, but you can run them yourself too, if you want.

Pre-requisites/assumptions:

  • You're set up to use Omni in general, and have it in your path as 'omni', i.e. you've successfully used Omni to reserve GENI resources (and have the latest version of the software, a user credential signed by a CA that's trusted by the FOAM server you intend to test, a working omni_config file, etc).
  • You have four rspec files needed to test with (three valid request rspecs, and one valid manifest rspec). We plan to make some sample rspecs available.
  • You have rspeclint in your path.

You can run these tests wherever you usually use Omni; they don't need to run on the FOAM/FV server.

As of this writing, the current version of GCF is 2.4.1; some of these steps may need to be different if you have a more recent version than that.

Note that this is a streamlined summary of the full docs for running the GCF GENI AM API Acceptance Test Suite. The full docs for that suite are in .../gcf-2.4.1/acceptance_tests/AM_API/README-accept-AMAPI.txt; there's also more information at http://trac.gpolab.bbn.com/gcf/wiki/AmApiAcceptanceTests.

Setup

Set the FOAM server name (modify this to use the DNS name of the server that you want to test):

foamserver=foam.example.edu

Identify the rspecs you plan to use (modify this to use your actual rspecs):

rspec_request1=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-acedcodefood-vlan-1750.rspec
rspec_request2=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-cafebeefface-vlan-1750.rspec
rspec_request3=~/rspecs/request/foam-testing/rspecs/$foamserver/mac-deadbeadfeed-vlan-1750.rspec
rspec_manifest1=~/rspecs/manifest/jbs-gpolab-cafebeefface.rspec

Run the tests

If you don't already have the latest version of Omni, you can get it from http://www.gpolab.bbn.com/local-sw/.

The rest of these docs assume that you have GCF 2.4.1 in ~/src/gcf-2.4.1.

Change into that directory, set your Python path, and copy in your rspec files:

cd ~/src/gcf-2.4.1/acceptance_tests/AM_API
export PYTHONPATH=~/src/gcf-2.4.1/src
cp $rspec_request1 request1.xml
cp $rspec_request3 request2.xml
cp $rspec_request2 request3.xml
cp $rspec_manifest1 bad.xml
cp request1.xml request.xml

Set the V1 AM API URL, and run the V1 tests:

am=https://${foamserver}:3626/foam/gapi/1
rm acceptance.log
./am_api_accept.py -V1 --sleep-time 1 -a $am

Set the V2 AM API URL, and run the V2 tests:

am=https://${foamserver}:3626/foam/gapi/2
rm acceptance.log
./am_api_accept.py -V2 --sleep-time 1 -a $am

If you'd like more verbose details about what tests it's running, you can add '--v'.

When not in verbose mode, each of those will run for a few minutes, and should produce output like

.............
----------------------------------------------------------------------
Ran 13 tests in 195.402s

OK

This may generate e-mail with harmless "slice is busy" errors to the admins of the Slice Authority that you're using; this can happen if something tries to use a slice too quickly after it's been created. Omni should re-try in those circumstances, and you won't see any errors yourself (unless you happen to be an admin for the Slice Authority you're using).

Any other errors or failures are unexpected and should be investigated.

The test suite should also generate sixteen or twenty e-mail messages to the experimenter, and the same number to the FOAM admin. There should be a "sliver created" message, two "sliver renewed" messages, and a "sliver deleted" message, and might be a "sliver approved" message if the sliver is automatically approved, for each of four slices; the slices have names like acc${user}-${time}, where $user is your Unix username on the system where you're running the tests, and $time is a six-digit timestamp for uniqueness.