Version 27 (modified by 12 years ago) (diff) | ,
---|
FOAM
FOAM is an OpenFlow aggregate manager, which sites in GENI use to allow experimenters to allocate OpenFlow resources.
https://openflow.stanford.edu/display/FOAM has more information about FOAM (from the official site at Stanford), including installation instructions, a FAQ with common error messages (for both experimenters and admins), etc.
Info for experimenters
The following sections are mostly of interest to GENI experimenters.
Rspecs
FOAM uses GENI v3 rspecs, with the OpenFlow v3 extensions. We have a page about how to write OF v3 rspecs, including some examples, information about differences from previous versions, etc. If you need a hand, just drop a note to help@geni.net.
Getting your sliver approved
If you allocate a shared resource that connects to an OpenFlow aggregate (e.g. a MyPLC plnode or ProtoGENI host), you'll typically also need to reserve some OpenFlow resources. When you do this, your reservation request may be held for approval, in which case a local FOAM admin will need to approve your request before your sliver actually becomes live. You should get e-mail from FOAM when your sliver is created, and another message when it's been approved; if you don't hear back, you may be able to reach a FOAM admin by replying to that message (or to the contact information in the body of the message).
As of FOAM 0.8, FOAM can read your e-mail address from your slice credential. If that address is valid, then you don't need to include an e-mail address in your rspec, but make sure it is! If it isn't, make sure that you also provide a valid email address in your rspec so that you can get the notifications about status changes of your OpenFlow sliver.
If you're setting up a multi-campus topology, note that your sliver will need to be approved separately at each FOAM aggregate.
Info for admins
The following sections are mostly of interest to FOAM admins.
Version
The GPO currently recommends version 0.8.1 (the latest stable release) for GENI sites. There are many changes from 0.8 to 0.6, and we recommend working with the GPO to help the upgrade process go smoothly. Contact us at gpo-infra@geni.net and we'll be happy to help!
NOTE that much of this page is FOAM 0.8 specific. http://groups.geni.net/geni/wiki/OpenFlow/FOAM?version=22 has the last 0.6 specific version.
Advisories
Here are some things that FOAM admins should be aware of and watch out for.
Rspecs that include both cross-connect ports
In the current standard campus topology, a local OpenFlow-controlled VLAN (often VLAN 1750) is cross-connected to multiple inter-campus VLANs, and a given sliver typically only includes one cross-connect port. FOAM admins should be careful not to approve rspecs that include both cross-connect ports (either explicitly or implicitly, e.g. by requesting the entire local DPID for VLAN 1750).
Some experimenters may have good reasons for wanting to use both cross-connects; if they do, make sure they understand that this is a risky topology before you approve their sliver.
Rspecs that include match rules with no protocol specified
Rspecs that include match rules at one layer typically need to include match rules specifying the protocol at a lower layer. For example:
- An rspec that specifies a layer 3 match, like nw_src=A.B.C.D/N, also needs to specify dl_type to indicate what the layer 3 protocol is (e.g. dl_type=0x800 for IP and/or 0x806 for ARP).
- An rspec that specifies a layer 4 match, like tp_port=N, also needs to specify nw_proto to indicate what the layer 4 protocol is (e.g. nw_proto=1 for ICMP, or 6 for TCP, or 17 for UDP).
The important thing is that if the protocol isn't specified, some switches will end up with flowtable entries that match more traffic than they should, interfering with other experiments at your site. FOAM admins should be careful not to approve rspecs that include a match at one layer without specifying the protocol at a lower layer.
We don't currently think that there's a valid use case for an experimenter wanting to specify a match at one layer without specifying the protocol at a lower layer, but will amend this advisory if we encounter one.
Administrative list-resources blocks other FOAM operations
In FOAM 0.6, an administrative list-resources call locks the FOAM database, and thus blocks other FOAM operations. If an experimenter tries to use FOAM while this is happening, they'll get a "database is locked" error; if they retry, it should work. (If the error persists, something else is probably wrong.)
The FOAM monitoring code (from the 'tango-monitor-foam' package) does an administrative list-slivers whenever it runs, and it runs once per minute by default, so there's small window each minute (a second or two) when this can happen.
This issue is fixed in FOAM 0.8.
Auto-approval
FOAM 0.8 adds a simple analysis engine which can be used to inform auto-approval decisions. The analysis is done by default, but can be disabled if you really want to for some reason.
The action taken on a newly created sliver depends on the geni.approval.approve-on-creation configuration option. It has three possible values:
- Mode 0, "never": Newly created slivers are never automatically approved.
- Mode 1, "always": Newly created slivers are always automatically approved.
- Mode 2, "analysis": Newly created slivers are automatically approved if they pass all the checks in the analysis engine.
The default value is "0", for safety, so no slivers will be auto-approved.
The rest of this section is only really relevant if you use mode 2. If you've read the rest of this section and want to do that, you can do it with foamctl:
foamctl config:set-value --key="geni.approval.approve-on-creation" --value="2" --passwd-file=/opt/foam/etc/foampasswd
The analysis engine performs three tests related to the sliver's flowspace:
- If the sliver's flowspace includes IP space: That IP space must not overlap with the IP space of any already-approved slivers.
- If the sliver's flowspace includes MAC addresses: Those MAC addresses must not overlap with the MAC addresses of any already-approved slivers.
- If the sliver's flowspace includes ethertypes (other than IP and ARP): Those ethertypes must not overlap with the ethertypes of any already-approved slivers.
For each of those tests, the result could be "pass", "fail", or "n/a" (e.g. if the sliver's flowspace doesn't include IP space, or there are no port groups defined).
It also performs three tests related to datapaths, ports, and VLANs:
- The sliver's flowspace must include at least one datapath.
- The sliver's flowspace must not include more than one of the ports in any port group.
- The sliver's flowspace must not include any flowspace defined by VLAN. (no 'dl_vlan')
In mode 2, the sliver is automatically approved if at least one test in the first group passes, and no test fails (in either group).
In practice, this implements a policy somewhat like: "Auto-approve a sliver that includes IP, MAC, or ethertype flowspace, doesn't overlap with anything else, and doesn't include more than one port in these port groups". Port groups might include things like cross-connects, or any other collection of ports where you only want a given sliver to have one port in the group.
If a sliver is not auto-approved, the 'pend_reason' field (visible in 'foamctl geni:show-sliver') should explain why. (More precisely, it'll include one test that the sliver didn't pass; if it failed multiple tests, only one will be listed.)
A more flexible policy is expected in future versions of FOAM.
Sliver approval workflow
This section describes our workflow for approving slivers at BBN (those which aren't auto-approved).
FOAM sends e-mail about new slivers to the FOAM admin e-mail address that you configured when you set up FOAM. If further communication about a sliver request is needed, we copy that address on the e-mail, so that everyone will see it. We also send mail to to that address when we approve or reject the sliver (or if we review the request and we're not sure whether to approve or reject it), so everyone knows who did it.
Using the commands below, decide whether to approve it:
- Get a list of pending slivers, and look for the new sliver in that list.
- Get the sliver URN from the slicename.
- Show the sliver's basic info, to confirm that we've got the right sliver URN:
- Verify that the email field is valid, so that we and FOAM can contact the experimenter later about the sliver.
- Show the sliver's rspec, to confirm that it matches the owner's description of what they're asking for:
- Public information about common requests is on the GPO Lab OpenFlow aggregate info page.
- For more complicated requests:
- Look up the DPID in our inventory and find out what switch/VLAN it is.
- Look up the hosts the experimenter asked for in our inventory, and make sure the ports the experimenter requested make sense.
- Show the sliver's flowspace, and confirm that it matches the rspec, and doesn't contain anything dangerous, such as:
- Look for flowspace rules that match any packet -- the third field in each rule -- as these might indicate a subtle error in the rspec.
- Check to make sure the rspec and flowspace don't include multiple cross-connects, unless the experimenter has convinced us that they understand the risks and will be careful.
- Check to make sure the rspec and flowspace don't include I2 plnodes (ganel, gardil, sardis) and NLR VLANs/cross-connects, or NLR plnodes (bain, navis) and I2 VLANs/cross-connects.
If we conclude that the sliver is ok, approve it:
foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
If we conclude that we should definitely not approve the sliver, reject it:
foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
If we're not sure, do nothing, which will leave the sliver as Pending while we get more information.
Regardless, send e-mail to the admin address saying what we did, so everyone's in the loop. (One convenient way is to reply to the notification message about the sliver.)
Managing FOAM slivers
https://openflow.stanford.edu/display/FOAM/foamctl+Guide is the official guide to foamctl, and describes in detail everything that it can do. (NOTE: As of 2012-07-17, this guide was FOAM 0.6 specific, and marked as a "legacy document". FOAM-160 is tracking the task of updating it.)
Here are some specific commands that we've found useful for performing common tasks.
These commands all assume that you're running them on the FOAM server, and that you have a file /opt/foam/etc/foampasswd, containing the FOAM admin password.
Get a list of slivers
Pending ones (this is essentially "the queue" of slivers that are awaiting admin action):
foamctl geni:list-slivers -s Pending --passwd-file=/opt/foam/etc/foampasswd
All active ones:
foamctl geni:list-slivers --passwd-file=/opt/foam/etc/foampasswd
Either of these will give you a sliver URN; if you do
sliver_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbsstghosts:678fc69b-76e1-4a50-9fb2-ab5c4a5298d6
(with the actual URN of course), the rest of these commands will then work as-is -- or you can find a sliver from a slice name, see below.
Deleted ones:
foamctl geni:list-slivers --deleted --passwd-file=/opt/foam/etc/foampasswd
Find a sliver from a slice name
If you know a user's slice name, you can grep for it:
foamctl geni:list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+exampleslice
You can use this to get a sliver URN and/or an FV slice name from a GENI slice name, assigned to $sliver_urn and $flowvisor_slice:
slicename=exampleslice ; sliver_urn=$(foamctl geni:list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slicename | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/') ; flowvisor_slice=$(echo $sliver_urn | awk -F : '{print $NF}')
Replace "exampleslice" with a string to egrep for. Choose something that's unique to the slice; note that if the string contains egrep regexp type characters, like * or +, you'll need to backslash those.
The rest of these commands assume that you've used that (or something similar) to set $sliver_urn.
Show a sliver's basic info
foamctl geni:show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Show a sliver's rspec
foamctl geni:show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Show a sliver's flowspec
foamctl geni:show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Show a sliver's flowspace
foamctl geni:show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Approve a sliver
This marks a sliver in FOAM as Pending, and adds its FV slice and flowspace rules for it to the FlowVisor (if it wasn't already).
foamctl geni:approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Disable a sliver
This marks a sliver in FOAM as Pending, and removes its FV slice and flowspace rules for it from the FlowVisor (if there were any).
foamctl geni:disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Reject a sliver
This marks a sliver in FOAM as Rejected, and removes its FV slice and flowspace rules for it from the FlowVisor (if there were any).
foamctl geni:reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
Delete a sliver
This disables a sliver, and marks it as deleted, just like the GENI AM API DeleteSliver call, and removes its FV slice and flowspace rules for it from the FlowVisor (if there were any).
foamctl geni:delete-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
You should generally only do this with the experimenter's permission, and if the experimenter can't delete their own sliver for some reason, so they're not confused about where their sliver went. (If you disable or reject it, they can still see it; if you delete it, it's essentially gone forever from their point of view.)
Compare two slivers
This compares two slivers (called "old" and "new" here, since a common use case is to compare a new request from an experimenter to an existing old request).
rm -f old.txt new.txt oldsliver=<old sliver URN> newsliver=<new sliver URN> foamctl geni:show-sliver -r -u $oldsliver --passwd-file=/opt/foam/etc/foampasswd > old.txt foamctl geni:show-sliver -r -u $newsliver --passwd-file=/opt/foam/etc/foampasswd > new.txt diff -u old.txt new.txt
In the case of someone who deleted and then re-created a sliver, you could get the sliver URNs from the e-mail from FOAM, for example.
You can also use 'geni:show-sliver -s' to show flowspaces instead, which have the advantage of not including any comments or other formatting in the experimenter's rspec, but are a less familiar format for most admins.
Slice Authority trust configuration
You may want to configure FOAM to trust user certificates signed by additional Slice Authorities. To do that, install the CA cert for the Slice Authority in a file in /opt/foam/etc/gcf-ca-certs, and then rebuild the nginx CA cert bundle and restart FOAM and nginx:
sudo foamctl geni:bundle-certs sudo service foam restart sudo service nginx restart
In particular, GENI mesoscale deployments should trust the pgeni.gpolab.bbn.com SA; the official FOAM installation guide includes this step, or you can get the cert from http://www.pgeni.gpolab.bbn.com/ca-cert/pgeni.gpolab.bbn.com.pem if you need it.
Changing the FlowVisor password
FOAM's database stores information about the FlowVisor it manages, including its hostname and password. If you want to change that FlowVisor's password, you need to change it both in FV itself, and also in FOAM.
- Change the password in FlowVisor: this assumes
/etc/flowvisor/config.xml
FlowVisor's config, and is owned by useropenflow
:- Change the password in FlowVisor's config:
$ sudo service flowvisor stop $ sudo -u openflow fvconfig chpasswd /etc/flowvisor/config.xml fvadmin Enter password for account 'fvadmin' on the flowvisor: Enter password again $ sudo service flowvisor start
- If you store the password in
/etc/flowvisor/fvpasswd
, change it there too by hand:$ sudo vi /etc/flowvisor/fvpasswd
- Change the password in FlowVisor's config:
- Change the password in FOAM:
foamctl config:set-value --key="flowvisor.passwd" --value="$(cat /etc/flowvisor/fvpasswd)" --passwd-file=/opt/foam/etc/foampasswd
Testing FOAM
We have a separate page describing our procedure for testing FOAM, e.g. after an upgrade.