wiki:GENIExperimenter/ExperimentExample-OF-FOAM

Version 6 (modified by lnevers@bbn.com, 7 years ago) (diff)

--

GENI OpenFlow Experiment Example 2

This page illustrates an OpenFlow (OF) example for the Omni How-to page. OpenFlow is used for the control of the network resources in the GENI backbone and at meso-scale campus locations. Experimenters can use these GENI resources without learning OpenFlow. The GPO help@geni.net can help arrange connectivity across your GENI slice, including IP connectivity and/or layer 2 VLANs.

For experimenters who desire control over the network switching fabric in their slices, OpenFlow provides increased flexibility. Your experiment will incorporate an OpenFlow controller. Many experimenters use existing controller code as-is, some use controllers with relatively simple modifications, and some experimenters have written their own controller. The GPO is available to help design your controller and plan your experiment configuration. Feel free to contact us at help@geni.net.

The example captured in this page uses credentials from pgeni.gpolab.bbn.com, the GPO ProtoGENI Clearinghouse, to set up an experiment that includes 3 meso-scale OpenFlow sites that include the following compute resources:

  • 1 PG compute resource at BBN running as a web server
  • 1 MyPLC compute resource at Wisconsin running as a web client
  • 1 MyPLC compute resource at Clemson running as a web client.

This example also shows a NOX OpenFlow Controller running on a slice resource that is available and accessible to the FOAM instances.

No image "OF-Experiment-example.jpg" attached to GENIExperimenter/ExperimentExample-OF-FOAM

Omni tools are used to reserve resources in each of the site's FlowVisor OpenFlow Aggregate Manager (FOAM) and GENI Aggregate Managers. All steps required to set up the Compute and Network Resources are outlined in this page:

  • Configure Omni tools
  • Define OpenFlow RSpecs for sites
  • Define OpenFlow RSpecs for GENI Core.
  • Define Compute Resources RSpecs for MyPLC and PG sites.
  • Request flowspace resources for sites and core.
  • Request compute resources from sites
  • Confirm connection
  • Run the experiment

Note: FOAM sites must be federated in order to trust a Clearinghouse. This is an OF administrator task that must take place before any of the user OF operations can take place.

Authentication and Omni tools configuration

This example assumes that GENI credentials have been set up as detailed in the GENI Credentials section of the Omni how-to page.

The following omni_config was used for this experiment along with the credentials from the GPO Clearinghouse to access the GENI Aggregate Managers in this experiment:

[omni]
default_cf = pgeni
users = lnevers

# ---------- Users ----------
[lnevers]
urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
keys = ~/.ssh/id_rsa.pub

# ---------- Frameworks ----------
[pgeni]
type = pg
ch = https://www.emulab.net:443/protogeni/xmlrpc/ch
sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa
cert = ~/.ssl/pgeni/encrypted-cleartext.pem
key = ~/.ssl/pgeni/encrypted-cleartext.pem
verbose=false

[pg]
type=pg
ch = https://www.emulab.net:443/protogeni/xmlrpc/ch
sa = https://www.emulab.net:443/protogeni/xmlrpc/sa
cert = ~/.ssl/protogeni/encrypted-cleartext.pem
key = ~/.ssl/protogeni/encrypted-cleartext.pem

A list of Aggregate managers is compiled in the GENI Aggregate page, if your site is not listed on this page you should contact your site administrator, or help@geni.net.

Determine Resources

Each of the 3 sites will requires an OpenFlow RSpec and a compute resource RSpec. To find out what resources are available use Omni tools to get a list of available resources. FOllowing is a list of available GENI Aggregate Managers:

Site ProtoGENI Aggregate URL MyPLC Aggregate URL FOAM Aggregate URL
BBN https://pgeni.gpolab.bbn.com/protogeni/xmlrpc/am http://myplc.gpolab.bbn.com:12346/ https://foam.gpolab.bbn.com:3626/foam/gapi/1
Clemson http://myplc.clemson.edu:12346/ https://foam.clemson.edu:3626/foam/gapi/1
GA Tech https://myplc.cip.gatech.edu:12346/ https://nox.orbit-lab.org:3626/foam/gapi/1
Indiana https://myplc.grnoc.iu.edu:12346/ https://foam.noc.iu.edu:3626/foam/gapi/1
Intenet2 https://foam.net.internet2.edu:3626/foam/gapi/1
NLR https://foam.nlr.net:3626/foam/gapi/1
PLC https://www.planet-lab.org:12346
Rutgers https://plc.orbit-lab.org:12346/ https://foam.oflow.cip.gatech.edu:3626/foam/gapi/1
Stanford https://myplc.stanford.edu:12346/ https://openflow4.stanford.edu:3626/foam/gapi/1
UKY https://www.uky.emulab.net/protogeni/xmlrpc/am
Utah https://www.emulab.net/protogeni/xmlrpc/am
Washington https://of.cs.washington.edu:12346/
Wisconsin https://wings-openflow-1.wail.wisc.edu:12346/https://foam.wail.wisc.edu:3626/foam/gapi/1

<<TO REMOVE ????>> For a list of available aggregates see the GENI Aggregates page.

To set up the 3 site experiment on this page, the following Aggregate Managers were queried with omni to get a list of available resources:

  1. Clemson FOAM and MyPLC aggregates:
     ./src/omni.py -a https://foam.clemson.edu:3626/foam/gapi/1 listresources -o   # generates output file rspec-foam-clemson-edu-3626-foam-gapi-1.xml
     ./src/omni.py -a  http://myplc.clemson.edu:12346/ listresources -o            # generates rspec-myplc-clemson-edu-12346-.xml
    
  2. Wisconsin FOAM and MyPLC aggregates:
     ./src/omni.py -a https://foam.wail.wisc.edu:3626/foam/gapi/1 listresources -o    # generates rspec-foam-wail-wisc-edu-3626-foam-gapi-1.xml
     ./src/omni.py -a https://wings-openflow-1.wail.wisc.edu:12346/  listresources -o # rspec-wings-openflow-1-wail-wisc-edu-12346-.xml
    
  3. BBN FOAM and PG aggregates:
     ./src/omni.py -a  http://www.pgeni.gpolab.bbn.com/protogeni/xmlrpc/am listresources -o # generates rspec-www-pgeni-gpolab-bbn-com-protogeni.xml
     ./src/omni.py -a https://foam.gpolab.bbn.com:3626/foam/gapi/1 listresources -o         # generates rspec-foam-gpolab-bbn-com-3626-foam-gapi-1.xml
    

Note: Each of the omni command used the -o argument to get the listresources output to a file. Each of the listresources output files are attached to this page.

The content of the OF RSpec depends on the type OF resources that you may need. You may need to define an RSpec using any of the following:

  • IP Addresses
  • MAC Address
  • Data Path Identifier (DPID).

Here is an example of listing resources advertised by each of the sites in this experiment and in the GENI Core:

NOTE: This example uses Omni V1.6.1

$ ./src/omni.py -a https://ofam1.site1.com:1443/openflow/gapi/ -o listresources
<<add query for BBN FOAM, XXX FOAM, YYY FOAM, NLR FOAM and INTERNET2 FOAM>>>
<<<attach output from list resources>>>

The resulting Advertisement RSpec looks like this:

<?xml version="1.0" ?>
<!-- Resources at AM:
        URN: unspecified_AM_URN
        URL: https://ofam1.site1.com:1443/openflow/gapi/
 -->

<rspec type="openflow">
  <network location="Somewhere USA " name="local-oim ">
    <switches>
      <switch urn="urn:publicid:IDN+ofam1.site1.com:expedient:openflow+switch
:06:bb:00:24:88:cc:bb:00">
        <port urn="urn:publicid:IDN+ofam1.site1.com:expedient:openflow+switch:044:d0:02:6f:13:f3:b0:0+port:26"/>
      </switch>
    </switches>
    <links/>
  </network>
</rspec>

The above provides the switch and the port URN information to be used in making the OF flowspace reservation. The above results will give you some insight into what is available, but it is recommended that you coordinate with the site contact to ensure RSpec accuracy for each of the OF Aggregate Managers.

Request Resources

The OF RSpecs are used to request resources at each site's OpenFlow Aggregate Manager. This will result in your resource request being imported into the Opt-In manager, where an Administrator must approve your request for resources. Currently requesting OF resources for the sites in the experiment does not require OF setup for the Network Core path; the default handling will allow OF traffic. This is expected to change.

The OF RSpec defines a flowspace for each OF rule for your traffic sliver. Following are example OF RSpecs that could be used to reserve the flowspaces needed for each of the generic 3 node scenario in this page. The scenario uses 3 OF Aggregate Managers (one per site) and one OF Controller at Site 1 for all sites. The example RSpecs are for each of the 3 OF AM and assume that the flowspace is based on traffic type and port (TCP port 9080 and 9082).

RSpec Example for Site 1 OpenFlow Aggregate Manager

<resv_rspec type="openflow">
  <user affiliation="GPO" email="help@geni.net" firstname="GPO" lastname="Ops" password="xxxxxxx"/>
  <project description="Project description" name=" Project name"/>
  <slice controller_url="tcp:controller.site1.com:9933" description="description" name="name"/>
<flowspace>
    <switch urn="urn:publicid:IDN+ofam1.site1.com:expedient:openflow+switch:044:d0:02:6f:13:f3:b0:0"/>
    <tp_dst from="9080" to="9082"/>
  </flowspace>
  <flowspace>
    <switch urn="urn:publicid:IDN+ofam1.site1.com:expedient:openflow+switch:044:d0:02:6f:13:f3:b0:0"/>
    <tp_src from="9080" to="9082"/>
  </flowspace>
</resv_rspec>

RSpec Example for Site 2 OpenFlow Aggregate Manager

<resv_rspec type="openflow">
  <user affiliation="GPO" email="help@geni.net" firstname="GPO" lastname="Ops" password="xxxxxxx"/>
  <project description="Project description" name=" Project name"/>
  <slice controller_url="tcp:controller.site1.com:9933" description="description" name="name"/>
<flowspace>
    <switch urn="urn:publicid:IDN+ofam2.site2.com:expedient:openflow+switch:00:00:00:13:65:f4:21:4f"/>
    <tp_dst from="9080" to="9082"/>
  </flowspace>
  <flowspace>
    <switch urn="urn:publicid:IDN+ofam2.site2.com:expedient:openflow+switch:00:00:00:13:65:f4:21:4f"/>
    <tp_src from="9080" to="9082"/>
  </flowspace>
</resv_rspec>

RSpec Example for Site 3 OpenFlow Aggregate Manager

<resv_rspec type="openflow">
  <user affiliation="GPO" email="help@geni.net" firstname="GPO" lastname="Ops" password="xxxxxxx"/>
  <project description="Project description" name=" Project name"/>
  <slice controller_url="tcp:controller.site1.com:9933" description="description" name="name"/>
<flowspace>
    <switch urn="urn:publicid:IDN+ofam3.site3.com:expedient:openflow+switch:00:00:00:21:6f:b8:f3:d0"/>
    <tp_dst from="9080" to="9082"/>
  </flowspace>
  <flowspace>
    <switch urn="urn:publicid:IDN+ofam3.site3.com:expedient:openflow+switch:00:00:00:21:6f:b8:f3:d0"/>
    <tp_src from="9080" to="9082"/>
  </flowspace>
</resv_rspec>

Create Slice and Request Resources

Using the RSpec that defines the required OF flowspace, you can now request OF resources with Omni. First create a slice using your GPO Clearinghouse credentials as defined in the omni_config. Note that the slice name should be meaningful to you:

$ ./src/omni.py createslice ln_OF_slice
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pgeni
INFO:omni:Created slice with Name ln_OF_slice, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+ln_OF_slice, Expiration 2011-06-16 19:46:49
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createslice:

  Options as run:
		configfile: omni_config
		framework: pgeni
		native: True

  Args: createslice ln_OF_slice

  Result Summary: Created slice with Name ln_OF_slice, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+ln_OF_slice, Expiration 2011-06-16 19:46:49
 
INFO:omni: ============================================================

Then create a sliver for each resource aggregate. In this example there are 3 OF aggregates (one per site) and 3 compute resource aggregates (1 PG & 2 MyPLC), which maps to creating 6 slivers, and the following commands are executed:

 $ ./src/omni.py -a https://ofam1.site1.com:1443/openflow/gapi/ -o createsliver ln_OF_slice OF-site1.txt.rspec
 $ ./src/omni.py -a https://ofam2.site2.com:1443/openflow/gapi/ -o createsliver ln_OF_slice OF-site2.rspec
 $ ./src/omni.py -a https://ofam3.site3.com:1443/openflow/gapi/ -o createsliver ln_OF_slice OF-site3.rspec
 $ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am -o createsliver ln_OF_slice emulab-site2.rspec
 $ ./src/omni.py -a http://myplc.site1.com:12346 -o createsliver ln_OF_slice myplc-site1.rspec
 $ ./src/omni.py -a http://myplc.site3.com:12346 -o createsliver ln_OF_slice myplc-site3.rspec

Note: Examples that detailed rspec creation examples for the PG and MyPLC resource aggregates can be found in the other non-OF experiment example.

Opt-in Manager Administrative Approval

The OF Aggregate Administrator has to allow traffic from various nodes to be routed as decided by your OpenFlow controller. In this case, you are asking the administrator to route traffic on the PG and PL nodes you have just reserved. So you will need to give the site OF Aggregate Administrator:

  • your name
  • slice name and clearinghouse
  • PI name
  • duration of your experiment
  • IP address and names of the nodes whose traffic you need opted in (see the result of listresources and/or sliverstatus)
  • your flowspace request in plain text - what were you looking for

The OF Aggregate Administrator that opts-in your flows is responsible for resolving any conflicts between the traffic flows that you are requesting and the traffic flows requested by other experiments. If your flows are too generic (ex. all port 80 traffic) then they may overlap with other experiments; the administrator will have control over whether the conflict is resolved in your favor or someone else's.

Once the OF Aggregate Administrator has opted-in your traffic, you may start using the OF resources.

Run Experiment

Once your flowspace has been opted in you should be able to exchange traffic over the approved flows between the hosts that have been allocated to your experiment. Simply SSH into one of the hosts and ping any of the other hosts to verify connectivity. If you have connectivity between the allocated hosts, you may start your experiment, if you do not have connectivity, contact help@geni.net.

Remember to check the expiration time for the flowspace and for each aggregate with the Omni sliverstatus and to renewslice and renewsliver to expand the duration of the resource allocation to match your experiment needs. For an example on how to check sliverstatus and renewslice see the checking status section of the other non-OF experiment.

Release Resources

When the experiment is completed you should release the resources by deleting the sliver for each of the Resource Aggregates used in the Experiment. In the example experiment used in this page, the following commands should be issued to release resources:

 $ ./src/omni.py -a https://ofam1.site1.com:1443/openflow/gapi/ deletesliver ln_OF_slice 
 $ ./src/omni.py -a https://ofam2.site2.com:1443/openflow/gapi/ deletesliver ln_OF_slice 
 $ ./src/omni.py -a https://ofam3.site3.com:1443/openflow/gapi/ deletesliver ln_OF_slice 
 $ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am deletesliver ln_OF_slice 
 $ ./src/omni.py -a http://myplc.site1.com:12346 deletesliver ln_OF_slice 
 $ ./src/omni.py -a http://myplc.site3.com:12346 deletesliver ln_OF_slice 

Email us with any questions and feedback on this page!