wiki:OpenFlow/FOAM

Version 21 (modified by Josh Smift, 12 years ago) (diff)

--

FOAM

FOAM is an OpenFlow aggregate manager, which sites in GENI use to allow experimenters to allocate OpenFlow resources.

https://openflow.stanford.edu/display/FOAM has more information about FOAM (from the official site at Stanford), including installation instructions, a FAQ with common error messages (for both experimenters and admins), etc.

Info for experimenters

The following sections are mostly of interest to GENI experimenters.

Rspecs

FOAM uses GENI v3 rspecs, with the OpenFlow v3 extensions. We have a page about how to write OF v3 rspecs, including some examples, information about differences from previous versions, etc. If you need a hand, just drop a note to help@geni.net.

Getting your sliver approved

If you allocate a shared resource that connects to an OpenFlow aggregate (e.g. a MyPLC plnode or ProtoGENI host), you'll typically also need to reserve some OpenFlow resources. When you do this, your reservation request may be held for approval, and a local FOAM admin needs to approve your request before your sliver actually becomes live. You should get e-mail from FOAM when your sliver is created, and another message when it's been approved; if you don't hear back, you may be able to reach a FOAM admin by replying to that message.

Make sure that you also provide a valid email address in your rspec so that you can get the notifications about status changes of your OpenFlow sliver.

If you're setting up a multi-campus topology, note that your sliver will need to be approved separately at each FOAM aggregate.

Info for admins

The following sections are mostly of interest to FOAM admins.

Version

The GPO currently recommends version 0.6.4 (the latest stable release) for GENI sites.

Advisories

Here are some things that FOAM admins should be aware of and watch out for.

Administrative list-resources blocks other FOAM operations

In FOAM 0.6, an administrative list-resources call locks the FOAM database, and thus blocks other FOAM operations. If an experimenter tries to use FOAM while this is happening, they'll get a "database is locked" error; if they retry, it should work. (If the error persists, something else is probably wrong.)

The FOAM monitoring code (from the 'tango-monitor-foam' package) does an administrative list-slivers whenever it runs, and it runs once per minute by default, so there's small window each minute (a second or two) when this can happen.

This issue is being tracked in JIRA ticket FOAM-227, and should be fixed in FOAM 0.8.

Rspecs that include both cross-connect ports

In the current standard campus topology, a local OpenFlow-controlled VLAN (often VLAN 1750) is cross-connected to multiple inter-campus VLANs, and a given sliver typically only includes one cross-connect port. FOAM admins should be careful not to approve rspecs that include both cross-connect ports (either explicitly or implicitly, e.g. by requesting the entire local DPID for VLAN 1750).

Some experimenters may have good reasons for wanting to use both cross-connects; if they do, make sure they understand that this is a risky topology before you approve their sliver.

Rspecs that include match rules with no protocol specified

Rspecs that include match rules at one layer typically need to include match rules specifying the protocol at a lower layer. For example:

  • An rspec that specifies a layer 3 match, like nw_src=A.B.C.D/N, also needs to specify dl_type to indicate what the layer 3 protocol is (e.g. dl_type=0x800 for IP and/or 0x806 for ARP).
  • An rspec that specifies a layer 4 match, like tp_port=N, also needs to specify nw_proto to indicate what the layer 4 protocol is (e.g. nw_proto=1 for ICMP, or 6 for TCP, or 17 for UDP).

The important thing is that if the protocol isn't specified, some switches will end up with flowtable entries that match more traffic than they should, interfering with other experiments at your site. FOAM admins should be careful not to approve rspecs that include a match at one layer without specifying the protocol at a lower layer.

We don't currently think that there's a valid use case for an experimenter wanting to specify a match at one layer without specifying the protocol at a lower layer, but will amend this advisory if we encounter one.

Sliver approval workflow

This section describes our workflow for approving slivers at BBN.

FOAM sends e-mail about new slivers to the FOAM admin e-mail address that you configured when you set up FOAM. If further communication about a sliver request is needed, we copy that address on the e-mail, so that everyone will see it. We also send mail to to that address when we approve or reject the sliver (or if we review the request and we're not sure whether to approve or reject it), so everyone knows who did it.

Using the commands below, decide whether to approve it:

  • Get a list of pending slivers, and look for the new sliver in that list.
  • Get the sliver URN from the slicename.
  • Show the sliver's basic info, to confirm that we've got the right sliver URN:
    • Verify that the email field is valid, so that we and FOAM can contact the experimenter later about the sliver.
  • Show the sliver's rspec, to confirm that it matches the owner's description of what they're asking for:
    • Public information about common requests is on the GPO Lab OpenFlow aggregate info page.
    • For more complicated requests:
      • Look up the DPID in our inventory and find out what switch/VLAN it is.
      • Look up the hosts the experimenter asked for in our inventory, and make sure the ports the experimenter requested make sense.
  • Show the sliver's flowspace, and confirm that it matches the rspec, and doesn't contain anything dangerous, such as:
    • Look for flowspace rules that match any packet -- the third field in each rule -- as these might indicate a subtle error in the rspec.
    • Check to make sure the rspec and flowspace don't include multiple cross-connects, unless the experimenter has convinced us that they understand the risks and will be careful.
    • Check to make sure the rspec and flowspace don't include I2 plnodes (ganel, gardil, sardis) and NLR VLANs/cross-connects, or NLR plnodes (bain, navis) and I2 VLANs/cross-connects.

If we conclude that the sliver is ok, approve it:

foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

If we conclude that we should definitely not approve the sliver, reject it:

foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

If we're not sure, do nothing, which will leave the sliver as Pending while we get more information.

Regardless, send e-mail to the admin address saying what we did, so everyone's in the loop. (One convenient way is to reply to the notification message about the sliver.)

Managing FOAM slivers

https://openflow.stanford.edu/display/FOAM/foamctl+Guide is the official guide to foamctl, and describes in detail everything that it can do. Here are some specific commands that we've found useful for performing common tasks.

These commands all assume that you're running them on the FOAM server, and that you have a file /opt/foam/etc/foampasswd, containing the FOAM admin password.

Get a list of slivers

Pending ones:

foamctl list-slivers -s Pending --passwd-file=/opt/foam/etc/foampasswd

All active ones:

foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd

Either of these will give you a sliver URN; if you do

sliver_urn=urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbsstghosts:678fc69b-76e1-4a50-9fb2-ab5c4a5298d6

(with the actual URN of course), the rest of these commands will then work as-is.

Deleted ones:

foamctl list-slivers -d --passwd-file=/opt/foam/etc/foampasswd

Find a sliver from a slice name

If you know a user's slice name, you can grep for it:

foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+exampleslice

You can use this to get a sliver URN and/or an FV slice name from a GENI slice name, assigned to $sliver_urn and $flowvisor_slice:

slicename=exampleslice ; sliver_urn=$(foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slicename | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/') ; flowvisor_slice=$(echo $sliver_urn | awk -F : '{print $NF}')

The rest of these commands assume that you've used that (or something similar) to set $sliver_urn.

Show a sliver's basic info

foamctl show-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Show a sliver's rspec

foamctl show-sliver -r -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Show a sliver's flowspec

foamctl show-sliver -s -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Show a sliver's flowspace

foamctl show-sliver -f -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Approve a sliver

This marks a sliver in FOAM as Approved, and adds a FV slice and flowspace rules for it to the FlowVisor.

foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Disable a sliver

This marks a sliver in FOAM as Pending, and removes a FV slice and flowspace rules for it from the FlowVisor.

foamctl disable-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Reject a sliver

This marks a sliver in FOAM as Rejected, and removes a FV slice and flowspace rules for it from the FlowVisor.

foamctl reject-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

Delete a sliver

This disables a sliver, and marks it as deleted, just like the GENI AM API DeleteSliver call.

foamctl delete-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd

You should generally only do this with the experimenter's permission, and if the experimenter can't delete their own sliver for some reason, so they're not confused about where their sliver went. (If you disable or reject it, they can still see it; if you delete it, it's essentially gone forever from their point of view.)

Compare two slivers

This compares two slivers (called "old" and "new" here, since a common use case is to compare a new request from an experimenter to an existing old request).

rm -f old.txt new.txt
oldsliver=<old sliver URN>
newsliver=<new sliver URN>
foamctl show-sliver -s -u $oldsliver --passwd-file=/opt/foam/etc/foampasswd > old.txt
foamctl show-sliver -s -u $newsliver --passwd-file=/opt/foam/etc/foampasswd > new.txt
diff -u old.txt new.txt

In the case of someone who deleted and then re-created a sliver, you could get the sliver URNs from the e-mail from FOAM, for example.

We use 'show-sliver -s' because '-r' doesn't work at all on deleted slivers; and '-f' includes a priority value for approved slivers but not non-approved slivers, which clutters up the diff.

Slice Authority trust configuration

You may want to configure FOAM to trust user certificates signed by additional Slice Authorities. To do that, install the CA cert for the Slice Authority in a file in /opt/foam/etc/gcf-ca-certs, and then rebuild the nginx CA cert bundle and restart FOAM and nginx:

sudo foamctl bundle-certs
sudo service foam restart
sudo service nginx restart

In particular, GENI mesoscale deployments should trust the pgeni.gpolab.bbn.com SA; the official FOAM installation guide includes this step, or you can get the cert from http://www.pgeni.gpolab.bbn.com/ca-cert/pgeni.gpolab.bbn.com.pem if you need it.

Changing FlowVisor password in FOAM

FOAM's database stores information about FV, including its hostname and password. If you want to change FV's password, you need to change it both in FV itself, and also in FOAM.

  • Change the password in FlowVisor: this assumes /etc/flowvisor/config.xml FlowVisor's config, and is owned by user openflow:
    • Change the password in flowvisor's config:
      $ sudo service flowvisor stop
      $ sudo -u openflow fvconfig chpasswd /etc/flowvisor/config.xml fvadmin
      Enter password for account 'fvadmin' on the flowvisor:
      Enter password again
      $ sudo service flowvisor start  
      
    • If you store the password in /etc/flowvisor/fvpasswd, change it there too by hand:
      $ sudo vi /etc/flowvisor/fvpasswd
      
  • Change the password in FOAM. Future versions of foamctl will support the config:set-flowvisor-info command. If you are running foam-0.6.3, that command doesn't exist yet, so you'll need to use a helper script which invokes curl to talk to FOAM:
    • Copy the script change_foam_fvinfo.py from https://openflow.stanford.edu/display/FOAM/Scripts+for+admins on your foam
    • Ask FOAM for the current FlowVisor information, so you can set any fields you do not want to change to the same values again:
      $ foamctl get-config --key="flowvisor_info" --passwd-file=/opt/foam/etc/foampasswd
      {
       "value": {
        "passwd": "<old_fv_passwd>",
        "xmlrpc_port": 8080, 
        "host": "<old_fv_host>",
        "json_port": 8081
       }
      }
      
    • Invoke that script to set FOAM's FlowVisor info the same way you would on an initial install:
      $ python change_foam_fvinfo.py --passwd-file=/opt/foam/etc/foampasswd
      FlowVisor Hostname: <old_fv_host>
      FlowVisor XMLRPC Port [8080]: 
      FlowVisor JSON RPC Port [8081]: 
      fvadmin user password: 
      {
       "status": "success"
      }
      

Testing FOAM

We have a separate page describing our procedure for testing FOAM, e.g. after an upgrade.