[[PageOutline]] = GENI !OpenFlow Experiment Example 2 = This page illustrates an !OpenFlow (OF) example for the [wiki:HowToUseOmni Omni How-to] page. OpenFlow is used for the control of the network resources in the GENI backbone and at meso-scale campus locations. Experimenters can use GENI resources without learning !OpenFlow. Experimenters can choose to control the network switching fabric in their slices by running an !OpenFlow controller. Many experimenters use existing !OpenFlow controller code as-is, some use controllers with relatively simple modifications, and some experimenters have written their own controller. The GPO is available to help design your controller and plan your experiment configuration. Feel free to contact us at [mailto:help@geni.net]. The experiment in this page uses credentials from the GPO ProtoGENI Clearinghouse (pgeni.gpolab.bbn.com) to set up a 3 Meso-scale sites !OpenFlow experiment. The sites are BBN, Wisconsin and Clemson each with a MyPLC compute resource. If you are interested in using PG resources, see the [http://groups.geni.net/geni/wiki/HelloGENI HelloGENI] experiment page for an !OpenFlow experiment using PG nodes. [[Image(OF-Experiment-example.jpg)]] Omni tools are used to reserve resources in each of the site's !FlowVisor !OpenFlow Aggregate Manager (FOAM) and GENI Aggregate Managers. Steps required to set up this !OpenFlow experiment are detailed in this page: * Authentication and Omni tools configuration * Determine Resources Availability * Define Compute Resources RSpecs * Define !OpenFlow Network Resources Rspecs * Create Slice and Request Resources * Run Experiment * Release Resources Note: FOAM sites must be federated in order to trust a Clearinghouse. This is an !OpenFlow administrator task that must take place before any of the user !OpenFlow operations can take place. == Authentication and Omni tools configuration == This example assumes that GENI credentials have been set up as detailed in the [wiki:HowToUseOmni#a1.GetGENICredentials GENI Credentials] section of the [wiki:HowToUseOmni Omni how-to] page. The following omni_config was used for this experiment along with the credentials from the GPO Clearinghouse to access the GENI Aggregate Managers in this experiment: {{{ [omni] default_cf = pgeni users = lnevers # ---------- Users ---------- [lnevers] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers keys = ~/.ssh/id_rsa.pub # ---------- Frameworks ---------- [pgeni] type = pg ch = https://www.emulab.net:443/protogeni/xmlrpc/ch sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa cert = ~/.ssl/pgeni/encrypted-cleartext.pem key = ~/.ssl/pgeni/encrypted-cleartext.pem verbose=false [pg] type=pg ch = https://www.emulab.net:443/protogeni/xmlrpc/ch sa = https://www.emulab.net:443/protogeni/xmlrpc/sa cert = ~/.ssl/protogeni/encrypted-cleartext.pem key = ~/.ssl/protogeni/encrypted-cleartext.pem }}} A complete list of Aggregate managers is compiled in the [http://groups.geni.net/geni/wiki/GeniAggregate GENI Aggregate] page, if your site is not listed on this page you should contact your site administrator, or help@geni.net. == Reserve GENI IP subnet == In the Meso-scale network a layer 3 experiment requires its own separate IP subnet in order to control all traffic without interfering with traffic from other experiments. Before you start this experiment, you need to request an IP subnet from the gpo-infra@geni.net mail list along with a duration for the subnet reservation. To view the current list of subnets and their current allocation see the [http://groups.geni.net/geni/wiki/NetworkCore/SubnetReservations GENI Network Core Subnet] page. == Determine Resources Availability == Each of the 3 sites requires an !OpenFlow RSpec and a compute resource RSpec. Also, an !OpenFlow RSpec is required for the GENI Core (NLR in this example). To find out what resources are available use Omni tools to get a list of available resources from the available aggregates. Following is a list of MyPLC and FOAM aggregates available in the meso-scale network: ||'''Site'''|| '''MyPLC Aggregate URL''' || ''' FOAM Aggregate URL ''' || || || || BBN || http://myplc.gpolab.bbn.com:12346/ ||https://foam.gpolab.bbn.com:3626/foam/gapi/1 || || Clemson || http://myplc.clemson.edu:12346/ ||https://foam.clemson.edu:3626/foam/gapi/1 || || GA Tech || https://myplc.cip.gatech.edu:12346/ ||https://nox.orbit-lab.org:3626/foam/gapi/1 || ||Indiana || https://myplc.grnoc.iu.edu:12346/ ||https://foam.noc.iu.edu:3626/foam/gapi/1 || || Intenet2 || ||https://foam.net.internet2.edu:3626/foam/gapi/1 || || NLR || ||https://foam.nlr.net:3626/foam/gapi/1 || || Rutgers ||https://plc.orbit-lab.org:12346/ ||https://foam.oflow.cip.gatech.edu:3626/foam/gapi/1 || || Stanford ||https://myplc.stanford.edu:12346/ ||https://openflow4.stanford.edu:3626/foam/gapi/1 || ||Washington||https://of.cs.washington.edu:12346/ ||https://of.cs.washington.edu:3626/foam/gapi/1 || ||Wisconsin ||https://wings-openflow-1.wail.wisc.edu:12346/||https://foam.wail.wisc.edu:3626/foam/gapi/1 || The experiment in this page uses the BBN, Clemson, Wisconsin and NLR Aggregates. For complete list of available aggregates see the available [http://groups.geni.net/geni/wiki/GeniAggregate GENI Aggregates] page. == Define Compute Resources RSpecs == This experiment uses Omni V1.6.1 to set up the experiment. As a first step, the experimenter determines the available compute resources at each site. Following are the omni commands issued to get the manifests for available compute resources from the BBN, Clemson and Wisconsin MyPLC aggregates: {{{ ./src/omni.py -a http://myplc.gpolab.bbn.com:12346/ listresources -o # generates rspec-myplc-gpolab-bbn-com-12346-.xml ./src/omni.py -a http://myplc.clemson.edu:12346/ listresources -o # generates rspec-myplc-clemson-edu-12346-.xml ./src/omni.py -a https://wings-openflow-1.wail.wisc.edu:12346/ listresources -o # generates rspec-wings-openflow-1-wail-wisc-edu-12346-.xml }}} Each of the omni command above used the -o argument to get the listresources output to a file, all output advertisement RSpec files are attached to this page as a reference. The advertisement RSpecs are used to generate the request RSpecs to reserve the resources. Following are the compute resources RSpec defined based on the output of the listresources commands above. '''BBN MyPLC Compute Resource RSpec ''' {{{ navis.gpolab.bbn.com }}} ''' Clemson MyPLC Compute Resources RSpec''' {{{ #!xml planetlab4.clemson.edu }}} '''Wisconsin MyPLC Compute Resources RSpec''' {{{ #!xml wings-openflow-2.wail.wisc.edu }}} The MyPLC plnodes are pre-configured to use pre-defined addresses and MyPLC doesn't allow experimenters to change that configuration. When the experiment is running you will need to choose the appropriate interface for your experiment traffic based on the IP subnet assigned by the GPO infrastructure group when you requested one before starting the experiment. == Define !OpenFlow Network Resources Rspecs == To write an OF RSpec you must determine the end-to-end OF path that will be used for the experiment. This section defines how to define the OF RSpecs by following the process outlined below to determine the content of the OF RSpec: Step 1. Collect listresources from all FOAM Aggregate. The manifest RSpec details all resources at the aggregate, but does not provide any insight into whether they are available. Step 2. Review [http://groups.geni.net/geni/wiki/GeniAggregate GeniAggregate] to determine how each MyPLC host maps to the Site's !OpenFlow switch. Determine addresses and ports, and define site specific OF Group rules. Step 3. Using guidelines from the [http://groups.geni.net/geni/wiki/HowTo/WriteOFv3Rspecs#WritingOpenFlowv3requestrspecs Writing OF V3 RSpec] page merge the information from step 1 and 2 to generate the request RSpecs. Each of the above steps is covered in detail below: === Step 1: Collect listresources from all FOAM Aggregate === 1. Clemson FOAM and MyPLC aggregates: {{{ ./src/omni.py -a https://foam.clemson.edu:3626/foam/gapi/1 listresources -o # generates output file rspec-foam-clemson-edu-3626-foam-gapi-1.xml }}} 2. Wisconsin FOAM and MyPLC aggregates: {{{ ./src/omni.py -a https://foam.wail.wisc.edu:3626/foam/gapi/1 listresources -o # generates rspec-foam-wail-wisc-edu-3626-foam-gapi-1.xml }}} 3. BBN FOAM and MyPLC aggregates: {{{ ./src/omni.py -a https://foam.gpolab.bbn.com:3626/foam/gapi/1 listresources -o # generates rspec-foam-gpolab-bbn-com-3626-foam-gapi-1.xml }}} 4. NLR !OpenFlow Core FOAM server: {{{ $ ./src/omni.py -a https://foam.nlr.net:3626/foam/gapi/1 -o listresources # generates rspec-foam-nlr-net-3626-foam-gapi-1.xml }}} Information in the output manifest files provides the switch and the port information to be used in making the OF flowspace reservation. === Step 2: Determine site specific OpenFlow details === Define the site specific !OpenFlow details based on [http://groups.geni.net/geni/wiki/GeniAggregate GeniAggregate] and MyPLC hosts mapping to the Site's !OpenFlow switch. This step walk through how the site details are defined. In the [http://groups.geni.net/geni/wiki/GeniAggregate GeniAggregate] page, select the !OpenFlow Aggregate link for BBN, Clemson and Wisconsin and use the OF site diagrams to determine which ports and DPID should be used in the OF RSpec. In addition OF groups rules will be defined for each of the site to handle traffic for the hosts. Each site Rspec will include a group definition that looks as follows: {{{ }}} ''Note:''' The ''group-name'' above will be replace by the actual group name values defined in flowspec. [[BR]] ''Note: The address "10.42.11.0/24" was assigned as described in the section XXXX above. Now, let map the 3 nodes from the Site's Compute Resource RSpecs to the appropriate OF Switch port and determine how they connect to the GENI CORE network VLAN 3715. ==== BBN !OpenFlow Aggregate ==== To define the OF RSpec for the BBN site we need to consider the compute resource used for the site which is the node navis.gpolab.bbn.com and determine how it is connected on the OF switch at the site, and then determine how that connection connects to the network core OF VLAN 3715. To determine the ports and Data Path IDentifier (DPID) review the [http://groups.geni.net/geni/wiki/GeniAggregate/GpoLabOpenFlow BBN OF Aggregate] diagram and locate where the node is connected. The following diagram show the information from the BBN OF Aggregate diagram that is needed for the BBN !OpenFlow request RSpec: [[Image(BBNOFAggregateResources.jpg)]] The [http://groups.geni.net/geni/wiki/GeniAggregate/GpoLabOpenFlow BBN OF Aggregate] diagran shows the information required for the !OpenFlow RSpec. The image shows: * Host navis.bbn.com is connected to port 32 of the OF switch named ''habanero'' * OF switch ''habanero'' has a DPID of ''06:d6:00:24:a8:c4:b9:00'' for VLAN 1750. * OF switch ''habanero'' uses port 47 to connect to port 20 on the OF switch ''poblano''. * OF switch ''poblano'' has a DPID of ''06:d6:00:12:e2:b8:a5:d0'' for VLAN 1750. * VLAN 1750 is mapped to the Core VLAN 3715 on ''poblano'' port 3. The VLAN 1750 is used at every meso-scale site for MyPLC resources to access the back bone !OpenFlow VLANs. Also, for the BBN ''habanero'' OF switch *only*, you must add 24 to the physical port number to determine the port to use in the rspec. The host navis.bbn.com is on port 32 (+24=56), which means your rspec will use port ''56'' and port 47 (+24=71) maps to ''71'' to get to poblano. The syntax for each of these can be "borrowed" from the listresources associated to each DPID. Now we have all the information needed to define the BBN OF resource request RSpec. These are lines that define the ports on habanero: {{{ }}} These are the RSpec lines that define the ports on habanero: {{{ }}} In addition each of the groups must be allowed to share traffic: {{{ }}} To complete the BBN OF RSpec see ''Step 3'' ==== Clemson !OpenFlow Aggregate ==== To define the OF RSpec for the Clemson site review the information in the [http://groups.geni.net/geni/wiki/GeniAggregate/ClemsonOpenFlow Clemson OF Aggregate] diagram. The compute resource used for the site which is the node for planetlab4.clemson.edu. The following diagram show the information required from the Clemson OF Aggregate that is needed for the Clemson !OpenFlow request RSpec: [[Image(ClemsonOFAggregateResources.jpg)]] These are the lines from the Advertisement that should be used in the request RSpec: {{{ }}} In addition each of the groups must be allowed to share traffic: {{{ }}} To complete the Clemson OF RSpec see ''Step 3'' ==== Wisconsin !OpenFlow Aggregate ==== To define the OF RSpec for the Wisconsin site review the [http://groups.geni.net/geni/wiki/GeniAggregate/WisconsinOpenFlow Winsconsin OF Aggregate] diagram. The compute resource used at the Wisconsin site is the node wings-openflow-2.wail.wisc.edu. The Wisconsin OF aggregate diagram only shows the MAC address for the OF switch to determine the DPID simply combine the hexadecimal value for 1750 (06d6) and the MAC address (00:23:47:CC:44:00) to create the DPID (06:d6:00:23:47:CC:44:00). [[Image(WisconsinOFAggregateResources.jpg)]] These are the lines from the Wisconsin OF Advertisement that should be used in the OF request RSpec: {{{ }}} In addition each of the groups must be allowed to share traffic: {{{ }}} To complete the Wisconsin OF RSpec see ''Step 3'' ==== NLR !OpenFlow Aggregate ==== To define the NLR OF RSpec review the [http://groups.geni.net/geni/wiki/SiteNlr#OpenFlow OpenFlow DPIDs] table for the core in the [http://groups.geni.net/geni/wiki/SiteNlr National LambaRail] GENI page. From the table you can see the DPIDs for OF VLAN 3715 for NLR. You may choose to use only the two core sites (Chicago and Atlanta) associated with the three sites in this experiment, but for simplicity you may choose to define all DPIDs in the NLR Core for VLAN 3715, which is how this example is shown: {{{ }}} To complete the NLR OF RSpec see ''Step 3'' === Step 3: Generate OpenFlow V3 RSpec === Use the request template for OF V3 Request RSpecs shown below to create the RSpecs for the BBN, Clemson and Wisconsin OF aggregates: {{{ #xml }}} For the above template: * Modify the "openflow:sliver" line to replace the "user@geni.net" with your valid email address, and update the sliver description. * Update the "openflow:controller" line to include your controller information. Note: If you do not have a controller, you may set one up as shown [http://groups.geni.net/geni/wiki/OpenFlow/GeniBackboneTest#SetupyourOpenFlowcontroller here] You can now combine the modified template with each of the Site specific !OpenFlow details from Step 2. This will result in the following final OF RSpecs: ==== BBN OpenFlow RSpec ==== {{{ #xml }}} ==== Clemson OpenFlow RSpec ==== {{{ #xml }}} ==== Wisconsin OpenFlow RSpec ==== {{{ #xml }}} ==== NLR OpenFlow RSpec ==== {{{ #xml }}} == Create Slice and Request Resources == Using the RSpecs that define the required OF flowspace, you can now request OF resources with Omni. First create a slice using your GPO Clearinghouse credentials as defined in the omni_config. Note that the slice name should be meaningful to you and then create a sliver for each resource aggregate. In this example there are 4 OF aggregates (one per site plus one for the GENI core) and 3 compute resource aggregates (3 MyPLC), which maps to creating 7 slivers, and the following commands are executed: {{{ $ cd gcf-1.6.1 $ ./src/omni.py createslice 3sites-OF-Exp $ ./src/omni.py -a https://foam.gpolab.bbn.com:3626/foam/gapi/1 createsliver 3sites-OF-Exp openflow-bbn.rspec $ ./src/omni.py -a https://foam.clemson.edu:3626/foam/gapi/1 createsliver 3sites-OF-Exp openflow-clemson.rspec $ ./src/omni.py -a https://foam.wail.wisc.edu:3626/foam/gapi/1 createsliver 3sites-OF-Exp openflow-wisconsin.rspec $ ./src/omni.py -a https://foam.nlr.net:3626/foam/gapi/1 createsliver 3sites-OF-Exp openflow-nlr.rspec $ ./src/omni.py -a http://myplc.gpolab.bbn.com:12346/ createsliver 3sites-OF-Exp myplc-bbn.rspec $ ./src/omni.py -a http://myplc.clemson.edu:12346/ createsliver 3sites-OF-Exp myplc-clemson.rspec $ ./src/omni.py -a https://wings-openflow-1.wail.wisc.edu:12346/ createsliver 3sites-OF-Exp myplc-wisc.rspec }}} The output for running each of the above commands is attached [http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/ExperimentExample-OF-FOAM/openflow-foam-example.txt here]. Remember, to renew the slice and slivers reservation time, as the default time is 8 hours. Following are the command to extend the reservation for the resources: {{{ $ ./src/omni.py renewslice 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a https://foam.gpolab.bbn.com:3626/foam/gapi/1 renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a https://foam.clemson.edu:3626/foam/gapi/1 renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a https://foam.wail.wisc.edu:3626/foam/gapi/1 renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a https://foam.nlr.net:3626/foam/gapi/1renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a http://myplc.clemson.edu:12346/ renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a https://wings-openflow-1.wail.wisc.edu:12346/ renewsliver 3sites-OF-Exp 2012-04-27 $ ./src/omni.py -a http://myplc.gpolab.bbn.com:12346/ renewsliver 3sites-OF-Exp 2012-04-27 }}} The output for running each of the above renew commands is attached [http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/ExperimentExample-OF-FOAM/openflow-foam-renew.txt here]. === Setup an OpenFlow Controller for the Experiment === Once you have set up your resources you need to install and run an !OpenFlow Controller to manage your experiment. See the following page for instructions to [http://groups.geni.net/geni/wiki/OpenFlow/GeniBackboneTest#SetupyourOpenFlowcontroller Setup an OpenFlow Controller]. Also, attached is a [http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/ExperimentExample-OF-FOAM/nox-install.txt capture] of the !OpenFlow Controller installation executed for this example. === FOAM sliver approval === Each of the !OpenFlow slivers must be approved by an administrator at each of the !OpenFlow Aggregates. You do not need to take any action, an email notification will be sent for each of the FOAM aggregates that looks as follows: [[Image(wiki:GENIExperimenter/ExperimentExample-OF-FOAM/FOAM-sliver-approve.jpg)]] In the scenario captured in this page, there are 4 notification one from each of the FOAM aggregates (BBN, Clemson, NLR and Wisconsin. == Run Experiment == Once your flowspace has been opted in you should be able to exchange traffic over the approved flows between the hosts that have been allocated to your experiment. Simply SSH into one of the hosts and ping any of the other hosts to verify connectivity. If you have connectivity between the allocated hosts, you may start your experiment, if you do not have connectivity, contact [mailto:help@geni.net]. Remember to check the expiration time for the flowspace and for each aggregate with the Omni ''sliverstatus'' and to ''renewslice'' and ''renewsliver'' to expand the duration of the resource allocation to match your experiment needs. For an example on how to check ''sliverstatus'' and ''renewslice'' see the [wiki:GENIExperimenter/ExperimentExample#Checkingsliverresourcestatus checking status] section of the other non-OF experiment. == Release Resources == When the experiment is completed you should release the resources by deleting the sliver for each of the Resource Aggregates used in the Experiment. In the example experiment used in this page, the following commands should be issued to release resources: {{{ ./src/omni.py -a https://foam.gpolab.bbn.com:3626/foam/gapi/1 deletesliver 3sites-OF-Exp ./src/omni.py -a https://foam.noc.iu.edu:3626/foam/gapi/1 deletesliver 3sites-OF-Exp ./src/omni.py -a https://foam.wail.wisc.edu:3626/foam/gapi/1 deletesliver 3sites-OF-Exp ./src/omni.py -a https://foam.nlr.net:3626/foam/gapi/1 deletesliver 3sites-OF-Exp ./src/omni.py -a http://myplc.clemson.edu:12346/ deletesliver 3sites-OF-Exp ./src/omni.py -a https://wings-openflow-1.wail.wisc.edu:12346/ deletesliver 3sites-OF-Exp ./src/omni.py -a http://myplc.gpolab.bbn.com:12346/ deletesliver 3sites-OF-Exp }}} ---- {{{ #!html Email us with any questions and feedback on this page! }}}