Changes between Version 12 and Version 13 of OpenFlow/FOAM


Ignore:
Timestamp:
03/07/12 14:56:39 (12 years ago)
Author:
Josh Smift
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OpenFlow/FOAM

    v12 v13  
    55FOAM is an OpenFlow aggregate manager, which sites in GENI use to allow experimenters to allocate !OpenFlow resources.
    66
    7 !OpenFlow aggregates in GENI are currently in the process of migrating to FOAM from Expedient.
    8 
    9 https://openflow.stanford.edu/display/FOAM/FAQ has more information, including common error messages (for both experimenters and admins).
     7https://openflow.stanford.edu/display/FOAM/FAQ has more information about FOAM (from the official site at Stanford), including common error messages (for both experimenters and admins).
    108
    119= Info for experimenters =
     
    1311The following sections are mostly of interest to GENI experimenters.
    1412
    15 == Rspec changes ==
    16 
    17 Expedient used !OpenFlow v1 and v2 rspecs; FOAM uses GENI v3 with the !OpenFlow v3 extensions. We have a page about [wiki:HowTo/WriteOFv3Rspecs how to write OF v3 rspecs], including some examples, information about differences from OF v2, etc. If you need a hand, just drop a note to help@geni.net.
    18 
    19 == Aggregate Manager URL changes ==
    20 
    21 The Aggregate Manager URL for FOAM is generally different from the one for Expedient. FOAM URL typically look like this:
    22 
    23 {{{
    24 https://foam.gpolab.bbn.com:3626/foam/gapi/1
    25 }}}
    26 
    27 In particular, note port 3626, "/foam/gapi/1" as the pathname (the one on the end indicates GENI AM API v1), and no trailing slash.
    28 
    29 Expedient URL typically looked like this:
    30 
    31 {{{
    32 https://expedient.gpolab.bbn.com:1443/openflow/gapi/
    33 }}}
    34 
    35 In particular, note port 1443, "/openflow/gapi/" as the pathname, and a trailing slash.
     13== Rspecs ==
     14
     15FOAM uses GENI v3 rspecs, with the !OpenFlow v3 extensions. We have a page about [wiki:HowTo/WriteOFv3Rspecs how to write OF v3 rspecs], including some examples, information about differences from previous versions, etc. If you need a hand, just drop a note to help@geni.net.
    3616
    3717== Getting your sliver approved ==
     
    3919If you allocate a shared resource that connects to an !OpenFlow aggregate (e.g. a MyPLC plnode or ProtoGENI host), you'll typically also need to reserve some !OpenFlow resources. When you do this, your reservation request may be held for approval, and a local FOAM admin needs to approve your request before your sliver actually becomes live. You should get e-mail from FOAM when your sliver is created, and another message when it's been approved; if you don't hear back, you may be able to reach a FOAM admin by replying to that message.
    4020
    41 Note that you don't need to send a separate email to the administrator, like you did for Expedient, FOAM will automatically send the email to the administrator of FOAM. Make sure that you also provide a '''valid email address''' in your rspec so that you can get the notifications about status changes of your !OpenFlow sliver.   
     21Make sure that you also provide a '''valid email address''' in your rspec so that you can get the notifications about status changes of your !OpenFlow sliver.   
    4222
    4323If you're setting up a multi-campus topology, note that your sliver will need to be approved separately at each FOAM aggregate.
     
    6444 * An rspec that specifies a layer 4 match, like tp_port=N, also needs to specify nw_proto to indicate what the layer 4 protocol is (e.g. nw_proto=1 for ICMP, or 6 for TCP, or 17 for UDP).
    6545
    66 The important thing is that if the protocol ''isn't'' specified, some switches will end up with flowtable entries that match more traffic than they should, interfering with other experiments at your site. FOAM admins should be careful not to approve rspecs that include a match at one layer without specifying the protocol.
     46The important thing is that if the protocol ''isn't'' specified, some switches will end up with flowtable entries that match more traffic than they should, interfering with other experiments at your site. FOAM admins should be careful not to approve rspecs that include a match at one layer without specifying the protocol at a lower layer.
    6747
    6848We don't currently think that there's a valid use case for an experimenter wanting to specify a match at one layer without specifying the protocol at a lower layer, but will amend this advisory if we encounter one.
     
    226206In particular, GENI mesoscale deployments should trust the pgeni.gpolab.bbn.com SA; the official FOAM installation guide includes this step, or you can get the cert from http://www.pgeni.gpolab.bbn.com/ca-cert/pgeni.gpolab.bbn.com.pem if you need it.
    227207
    228 == Switching from Expedient to FOAM ==
    229 
    230 https://openflow.stanford.edu/display/FOAM/Install+Guide is the official FOAM installation guide, and following those steps will generally work. Before you get started, here are some specific notes for mesoscale campuses who are migrating from Expedient.
    231 
    232 As an overview, this will involve:
    233 
    234  * identifying where you want to run FOAM, and its !FlowVisor
    235  * choosing a site tag
    236  * configuring e-mail
    237  * disabling Expedient
    238  * informing experimenters@geni.net and response-team@geni.net about the change
    239  * helping experimenters migrate orphaned Expedient slivers
    240 
    241 You should be able to run FOAM on the same system where you're currently running Expedient, if you'd like. Conversely, if you'd like to switch to a different system, the migration is an opportune time to do that.
    242 
    243 We recommend having a dedicated FOAM-controlled !FlowVisor; this isn't strictly necessary, but it keeps things pleasantly simple. We recommend using the same !FlowVisor that Expedient was using, disabling Expedient once FOAM is running, and migrating orphaned Expedient slivers to FOAM, which allows experiments with existing slivers to keep running while you upgrade.
    244 
    245 !FlowVisor and FOAM can run on the same host, which should ideally have at least 3 GB of RAM and at least two CPUs. Fast disks also help, as !FlowVisor (as of 0.8.1) can be I/O intensive. These requirements may increase for larger scale deployments.
    246 
    247 The binary package works well with the current Long Term Stable release of Ubuntu (10.04), and we recommend that if you don't have another preference. We're working on creating RPMs, for sites who prefer Red Hat type distributions.
    248 
    249 You can choose whatever you'd like as your site tag; we recommend something that looks more like a hostname (e.g. "foam.gpolab.bbn.com") than like a single word (e.g. "BBN"). Using the fully-qualified domain name of your FOAM server is convenient, as it allows you to use a different site tag on different servers, if you have more than one (e.g. one for testing purposes). You can also use a CNAME; for example, the GPO Lab FOAM server's hostname is tulum.gpolab.bbn.com, but we use foam.gpolab.bbn.com as our site tag (and then other tags for staging and testing instances).
    250 
    251 FOAM will send e-mail to experimenters and to admins when various things happen: When a sliver is created, approved, disabled, rejected, or deleted; and when a sliver is within a week of expiring, a day of expiring, and actually expires. You'll also get a message once a day with a list of slivers that are currently in the pending queue.
    252 
    253 We recommend using an e-mail address for the "From" setting, at installation time, which actually receives mail, so that experimenters can reach you by replying to messages from FOAM, if they need help. You don't need to set Reply-to (just hit return when prompted) unless you specifically want replies from experimenters to go somewhere other than the address in the From line. We use foam-admin@gpolab.bbn.com for both "From" and "Admin email", and don't set reply-to.
    254 
    255 You can customize the text of the messages that FOAM sends: The default templates are in /opt/foam/etc/templates/default, and if you create a file in /opt/foam/etc/templates/custom with the same name as one of the files in .../default, the one in .../custom will take precedence. The files in .../default will be replaced when you install new versions of FOAM; anything you create in .../custom will persist through upgrades.
    256 
    257 We recommend disabling Expedient once FOAM is up and running, to avoid confusion. This is how we removed it in the GPO Lab:
    258 
    259 {{{
    260 sudo apt-get remove python-optin-manager python-expedient apache2
    261 sudo apt-get autoremove
    262 sudo dpkg --purge apache2.2-common python-expedient
    263 sudo rm -rf /usr/lib/python2.6/dist-packages/{openflow,geni,expedient,sfa,expedient_geni} /etc/{expedient,optin_manager} /etc/apt/sources.list.d/expedient.list
    264 }}}
    265 
    266 You should drop a note to experimenters@geni.net and response-team@geni.net once you're done, to inform others of the change.
    267 
    268 If you had any existing Expedient slivers, you should contact those sliver owners about migrating to FOAM; see [wiki:OpenFlow/FOAM#MovingorphanedExpedient-createdFVslicestoFOAMslivers below] for more about that.
    269 
    270 If you have any trouble migrating, or suggestions for how to improve these docs, just drop a note to gpo-infra@geni.net, or stop by the #geni-openflow IRC channel on Freenode.
    271 
    272 If you encounter any bugs, or have improvement or feature requests, https://openflow.stanford.edu/bugs/browse/FOAM is the JIRA issue tracker for FOAM. (There's also one for !FlowVisor, and other Stanford !OpenFlow projects.)
    273 
    274 == Moving orphaned Expedient-created FV slices to FOAM slivers ==
    275 
    276 If you've shut down Expedient and brought up FOAM, you may have !FlowVisor slices that were created by Expedient, which are now essentially orphaned. For each of those FV slices, its owner should create an !OpenFlow v3 rspec (the format FOAM uses), and create a new sliver in FOAM. You can then verify that the new sliver looks right, manually delete their old sliver, and approve the new one. (If a sliver owner needs help writing their new rspec, feel free to encourage them to contact help@geni.net.)
    277 
    278 Here's an example, showing how we migrated the jbs15 and jbs16 slivers at BBN, after the owner had been contacted about the change, and created their new slivers in FOAM.
    279 
    280 Check to see that the new slivers are there in FOAM, awaiting approval:
    281 
    282 {{{
    283 +$ foamctl list-slivers -s Pending --passwd-file=/opt/foam/etc/foampasswd
    284 {
    285  "slivers": [
    286   {
    287    "status": "Pending",
    288    "flowvisor_slice": "8d32974c-5a1b-4ebf-8c5f-097c9c64cf8d",
    289    "deleted": "False",
    290    "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15",
    291    "enabled": false,
    292    "id": 1,
    293    "expiration": "2011-10-19 02:56:32.798032",
    294    "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15:8d32974c-5a1b-4ebf-8c5f-097c9c64cf8d",
    295    "email": "jbs@bbn.com",
    296    "desc": "The controller on naxos:33015, for jbs15."
    297   },
    298   {
    299    "status": "Pending",
    300    "flowvisor_slice": "d82dae58-5de5-4caa-b458-46ee130462d0",
    301    "deleted": "False",
    302    "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16",
    303    "enabled": false,
    304    "id": 2,
    305    "expiration": "2011-10-19 02:56:51.263455",
    306    "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16:d82dae58-5de5-4caa-b458-46ee130462d0",
    307    "email": "jbs@bbn.com",
    308    "desc": "The controller on naxos:33016, for jbs16."
    309   }
    310  ]
    311 }
    312 }}}
    313 
    314 Identify the Expedient-created FV slices:
    315 
    316 {{{
    317 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd listSlices | grep jbs15
    318 Slice 8: jbs15-naxos-33015_ID__tulum_gpolab_bbn_com_133
    319 
    320 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo jbs15-naxos-33015_ID__tulum_gpolab_bbn_com_133
    321 Got reply:
    322 connection_2=06:d6:00:24:a8:c4:b9:00-->/192.1.249.23:45621-->naxos.gpolab.bbn.com/192.1.249.133:33015
    323 connection_1=06:d6:00:12:e2:b8:a5:d0-->/192.1.249.23:45563-->naxos.gpolab.bbn.com/192.1.249.133:33015
    324 contact_email=jbs@bbn.com
    325 controller_hostname=naxos.gpolab.bbn.com
    326 controller_port=33015
    327 creator=fvadmin
    328 
    329 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd listSlices | grep jbs16
    330 Slice 16: jbs16-naxos-33016_ID__tulum_gpolab_bbn_com_106
    331 
    332 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo jbs16-naxos-33016_ID__tulum_gpolab_bbn_com_106
    333 Got reply:
    334 connection_2=06:d6:00:24:a8:c4:b9:00-->/192.1.249.23:36076-->naxos.gpolab.bbn.com/192.1.249.133:33016
    335 connection_1=06:d6:00:12:e2:b8:a5:d0-->/192.1.249.23:36018-->naxos.gpolab.bbn.com/192.1.249.133:33016
    336 contact_email=jbs@bbn.com
    337 controller_hostname=naxos.gpolab.bbn.com
    338 controller_port=33016
    339 creator=fvadmin
    340 }}}
    341 
    342 Delete those:
    343 
    344 {{{
    345 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd deleteSlice jbs16-naxos-33016_ID__tulum_gpolab_bbn_com_106
    346 success!
    347 
    348 +$ fvctl --passwd-file=/etc/flowvisor/fvpasswd deleteSlice jbs15-naxos-33015_ID__tulum_gpolab_bbn_com_133
    349 success!
    350 }}}
    351 
    352 Approve the new FOAM slivers:
    353 
    354 {{{
    355 +$ slicename=jbs15 ; sliver_urn=$(foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slicename | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/') ; flowvisor_slice=$(echo $sliver_urn | awk -F : '{print $NF}')
    356 +$ foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    357 
    358 +$ slicename=jbs16 ; sliver_urn=$(foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd | egrep sliver_urn.+$slicename | sed -e 's/ *"sliver_urn": "\(.*\)".*/\1/') ; flowvisor_slice=$(echo $sliver_urn | awk -F : '{print $NF}')
    359 +$ foamctl approve-sliver -u $sliver_urn --passwd-file=/opt/foam/etc/foampasswd
    360 }}}
    361 
    362 And voila.
     208== Testing FOAM ==
     209
     210We have a separate page describing our procedure for [wiki:OpenFlow/FOAM/Testing testing FOAM], e.g. after an upgrade.