wiki:GIR3.2_EnterpriseGENI

Version 54 (modified by lnevers@bbn.com, 13 years ago) (diff)

--

  1. Enterprise GENI Evaluation
  2. Enterprise GENI Findings
  3. Enterprise GENI How-to
    1. FlowVIsor Installation and Configuration
    2. FOAM Installation and Configuration
    3. NOX Installation
    4. Expedient Installation

Enterprise GENI Evaluation

The Enterprise-GENI solution is composed of four main entities:

  • OpenFlow-based Substrate Switches that communicate using the OpenFlow protocol to the OpenFlow controller. Software version used was ​Indigo-2011.04.07. Other version available but no used in this evaluation includes Pica8.
  • FlowVisor Version 0.8.1.2 : Custom-build OpenFlow controller that isolates experiment control traffic.
  • Opt In Manager Version 0.2.5 : Administrative application for OpenFlow-based network.
  • Expedient Version 4.0.2: Portal for defining experiment resources.
  • NOX Zaku branch: An OpenFlow Controller providing flow level control of network traffic.
  • ​SNAC An OpenFlow Controller which is a NOX module. Not covered here.
  • FOAM Version 0.2.x will replace the Expedient Aggregate Manager and Opt In Manager. Software is available as package or code. The FOAM ubuntu package evaluated, which is available as described in the [​http://groups.geni.net/geni/wiki/GIR3.2_EnterpriseGENI#FOAMInstallationandConfiguration FOAM Installation and Configuration} section.

Evaluation Time Frame: September 26, 2011 through October 5, 2011.

Enterprise GENI Findings

Initial review of the ​FlowVisor deployment page and FlowVisor 0.8.1.2 found the following issues​. The evaluation found one blocking issue which causes the FlowVisor package installation not to work. Issue ​FLOWVISOR-137 captures that two required steps (generating a FlowVisor password, and defining a user account for the process) are not documented in the package install instructions.

Reviewed several versions of the FOAM package (0.2.9, 0.2.11, and 0.2.51). Ran into several issues:

  • Advertisement rspecs do not have the component_manager_id in the openflow:datapath element
  • Advertisement rspecs have dpid attribute in the datapath element.
  • Foam package was corrupted in re-install due to process not being stopped before uninstall.

Issues are addressed in fixes and documentation, no major issues remain.

The expedient review ran into to blocking issues that could not be resolved and evaluation was stopped.

Enterprise GENI How-to

FlowVIsor Installation and Configuration

Using instructions from ​https://openflow.stanford.edu/display/flowvisor/fv_deploy, installed the Ubuntu package FlowVisor Version 0.8.1.2. First, add the following to /etc/apt/sources.list file: 

   deb http://updates.flowvisor.org/openflow/downloads/GENI/DEB unstable/binary-$(ARCH)/

Install package: 

   $ sudo apt-get update && sudo apt-get install flowvisor

Before proceeding to configuration step, should modify the SSL_KEYPASSWD in the file /usr/etc/flowvisor/envs.sh to define the password to be used by the fv-admin profile:

You can now start the FlowVisor, note if this is a new FlowVisor installation, the SSL keystor must be generated, below is a capture of the process: 

  $ sudo /usr/sbin/flowvisor /usr/etc/flowvisor/config.xml 
  Starting FlowVisor
  Running with FV_JVM_ARGS=-server -Xms100M -Xmx1000M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC
  --- Setting logging level to NOTE
  java.lang.RuntimeException: SSL Key Store file not found: '/usr/etc/flowvisor/mySSLKeyStore'
  Please generate with `fvconfig generateCert`
        at org.flowvisor.api.APIServer.spawn(APIServer.java:85)
        at org.flowvisor.FlowVisor.run(FlowVisor.java:143)
        at org.flowvisor.FlowVisor.main(FlowVisor.java:191)

  $ sudo fvconfig generateCert
  Trying to generate SSL Server Key with passwd from scripts/envs.sh
  Generating cert with common name == mallorea.gpolab.bbn.com

  $ sudo /usr/sbin/flowvisor /usr/etc/flowvisor/config.xml &
  [1] 1283
  Starting FlowVisor
  Running with FV_JVM_ARGS=-server -Xms100M -Xmx1000M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC
  --- Setting logging level to NOTE

The FlowVisor configuration that is delivered includes sample slices that should be deleted: 

  fvctl --passwd-file=/root/.fvp  deleteSlice alice
  fvctl --passwd-file=/root/.fvp  deleteSlice bob

Issues found, captured in email​. (and bugs?)

FOAM Installation and Configuration

Followed the install from package instruction found at the [ ​https://openflow.stanford.edu/display/FOAM/install FOAM install] page. Created a file foam.list in the directory /etc/apt/sources.list.d/ containing the following: 

   deb http://updates.flowvisor.org/openflow/downloads/GENI/DEB unstable/all/

Installed the foam package and initialize the FOAM database: 

   $ sudo apt-get update 
   $ sudo apt-get install foam
   $ sudo -u www-data foamctl init-db
   Certificates Directory [/opt/foam/etc/gcf-ca-certs]: 
   Site Tag [mallorea.gpolab.bbn.com]:
   FlowVisor Hostname: mallorea.gpolab.bbn.com
   FlowVisor XMLRPC Port [8080]: 
   FlowVisor JSON RPC Port [8081]: 
   fvadmin user password:

Set up the FOAM administrative account password: 

  $ sudo foamctl set-password
    FOAM Admin Password: 
  $

Once the password is defined, you may store-it in a file to avoid being prompted for a password when running foam commands. In this example the password was stored in a file named /opt/foam/etc/foampasswd.

Removed the default nginx site: 

sudo rm /etc/nginx/sites-enabled/default

Installed a trusted root certificate from pgeni.gpolab.bbn.com, bundled the certificate and started the FOAM services: 

  $ sudo wget http://www.pgeni.gpolab.bbn.com/ca-cert/pgeni.gpolab.bbn.com.pem -O /opt/foam/etc/gcf-ca-certs/pgeni.gpolab.bbn.com.pem
  $ sudo wget https://emulab.net/genica.bundle -O /opt/foam/etc/gcf-ca-certs/emulab.pem --no-check-certificate
  $ sudo foamctl bundle-certs
  $ sudo service nginx start
  $ sudo service foam start

Once the FOAM services are started was able to query for a list of resources: 

$  ./src/omni.py -n -a  https://mallorea.gpolab.bbn.com:3626/foam/gapi listresources
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pgeni
INFO:omni:Listed resources on 1 out of 1 possible aggregates.
INFO:omni:<?xml version="1.0" ?>
INFO:omni:<!-- Resources at AM:
        URN: unspecified_AM_URN
        URL: https://mallorea.gpolab.bbn.com:3626/foam/gapi
 -->
INFO:omni:
<rspec type="advertisement" xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:openflow="http://www.protogeni.net/resources/rspec/ext/openflow/2.1" xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xs:schemaLocation="http://www.protogeni.net/resources/rspec/2 http://www.protogeni.net/resources/rspec/2/ad.xsd http://www.protogeni.net/resources/rspec/ext/openflow/2.1 http://www.protogeni.net/resources/rspec/ext/openflow/2.1/of-ad.xsd">
  <node component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:12:e2:b8:a5:d0" component_manager_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+authority+am" component_name="04:4d:00:12:e2:b8:a5:d0" exclusive="false">
    <hardware_type name="openflow-switch"/>
    <available now="true"/>
  </node>
  <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:12:e2:b8:a5:d0" dpid="04:4d:00:12:e2:b8:a5:d0">
    <openflow:port name="GBE0/16" num="16"/>
    <openflow:port name="GBE0/19" num="19"/>
    <openflow:port name="GBE0/21" num="21"/>
    <openflow:port name="GBE0/22" num="22"/>
  </openflow:datapath>
  <node component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:26:f1:3f:3b:00" component_manager_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+authority+am" component_name="04:4d:00:26:f1:3f:3b:00" exclusive="false">
    <hardware_type name="openflow-switch"/>
    <available now="true"/>
  </node>
  <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:26:f1:3f:3b:00" dpid="04:4d:00:26:f1:3f:3b:00">
    <openflow:port name="43" num="43"/>
    <openflow:port name="44" num="44"/>
    <openflow:port name="45" num="45"/>
    <openflow:port name="46" num="46"/>
    <openflow:port name="47" num="47"/>
    <openflow:port name="local" num="65534"/>
  </openflow:datapath>
</rspec>

INFO:omni: ------------------------------------------------------------
INFO:omni: Completed listresources:

  Options as run:
                aggregate: https://mallorea.gpolab.bbn.com:3626/foam/gapi
                framework: pgeni
                native: True

  Args: listresources
  Result Summary: Retrieved resources from 1 aggregates.
Wrote rspecs from 1 aggregates. 
INFO:omni: ============================================================

Generated a simple RSPEC using instructions from the ​Example RSPEC and ​Convert Expedient to FOAM RSPEC page: 

<?xml version="1.1" encoding="UTF-8"?>
<rspec  xmlns="http://www.protogeni.net/resources/rspec/2"
        xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:openflow="http://www.geni.net/resources/rspec/ext/openflow/2"
        xs:schemaLocation="http://www.protogeni.net/resources/rspec/2
              http://www.protogeni.net/resources/rspec/2/request.xsd
              http://www.geni.net/resources/rspec/ext/openflow/2
              http://www.geni.net/resources/rspec/ext/openflow/2/of-resv.xsd"
    type="request">

    <openflow:sliver description="Luisa sliver description"
                     email="lnevers@bbn.com"
                     ref="http://www.bbn.com">

        <openflow:controller url="tcp:mallorea.gpolab.bbn.com:9933" type="primary" />
        <openflow:group name="mygroup">

            <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:12:e2:b8:a5:d0" component_manager_id="urn:publi
cid:IDN+openflow:foam:mallorea.gpolab.bbn.com+authority+am" > 
                <openflow:port name="GBE0/22" num="22"/>
  </openflow:datapath>
        </openflow:group>

        <openflow:match>
              <openflow:use-group name="mygroup" />
              <openflow:packet>
                     <openflow:dl_dst value="00:0c:29:9a:28:00,00:0c:29:1f:75:b0"/>
              </openflow:packet>
        </openflow:match>

        <openflow:match>
              <openflow:use-group name="mygroup" />
              <openflow:packet>
                 <openflow:dl_src value="00:0c:29:9a:28:00,00:0c:29:1f:75:b0"/> 
              </openflow:packet>
       </openflow:match>
    </openflow:sliver>
</rspec>

Using omni tools created a slice: 

$  ./src/omni.py -n -a  https://mallorea.gpolab.bbn.com:3626/foam/gapi createslice foamslice1
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pgeni
INFO:omni:Created slice with Name foamslice1, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1, Expiration 2011-09-30 00:26:18
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createslice:

  Options as run:
                aggregate: https://mallorea.gpolab.bbn.com:3626/foam/gapi
                framework: pgeni
                native: True

  Args: createslice foamslice1

  Result Summary: Created slice with Name foamslice1, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1, Expiration 2011-09-30 00:26:18
 
INFO:omni: ============================================================

Then created a sliver: 

$ ./src/omni.py -n -a  https://mallorea.gpolab.bbn.com:3626/foam/gapi createsliver foamslice1 FOAM.rspec      
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pgeni
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1 expires within 1 day on 2011-09-30 00:26:18 UTC
INFO:omni:Creating sliver(s) from rspec file FOAM.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1
INFO:omni:Asked https://mallorea.gpolab.bbn.com:3626/foam/gapi to reserve resources. Result:
INFO:omni:<?xml version="1.0" ?>
INFO:omni:<!-- Reserved resources for:
        Slice: foamslice1
        At AM:
        URL: https://mallorea.gpolab.bbn.com:3626/foam/gapi
 -->
INFO:omni:
<rspec type="manifest" xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:openflow="http://www.geni.net/resources/rspec/ext/openflow/2" xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xs:schemaLocation="http://www.protogeni.net/resources/rspec/2               http://www.protogeni.net/resources/rspec/2/request.xsd               http://www.geni.net/resources/rspec/ext/openflow/2               http://www.geni.net/resources/rspec/ext/openflow/2/of-resv.xsd">  

      <openflow:sliver description="Luisa sliver description" email="lnevers@bbn.com" ref="http://www.bbn.com">    

            <openflow:controller type="primary" url="tcp:mallorea.gpolab.bbn.com:9933"/>    
            <openflow:group name="mygroup">      

                  <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+datapath:04:4d:00:12:e2:b8:a5:d0" component_manager_id="urn:publicid:IDN+openflow:foam:mallorea.gpolab.bbn.com+authority+am">         
                        <openflow:port name="GBE0/22" num="22"/>        
        </openflow:datapath>      
           </openflow:group>    

            <openflow:match>      
                    <openflow:use-group name="mygroup"/>      
                    <openflow:packet>        
                             <openflow:dl_dst value="00:0c:29:9a:28:00,00:0c:29:1f:75:b0"/>        
                    </openflow:packet>      
            </openflow:match>    

            <openflow:match>      
                    <openflow:use-group name="mygroup"/>      
                    <openflow:packet>        
                         <openflow:dl_src value="00:0c:29:9a:28:00,00:0c:29:1f:75:b0"/>         
                    </openflow:packet>      
           </openflow:match>    

      </openflow:sliver>  
</rspec>
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:

  Options as run:
                aggregate: https://mallorea.gpolab.bbn.com:3626/foam/gapi
                framework: pgeni
                native: True
  Args: createsliver foamslice1 FOAM.rspec

  Result Summary: Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1 expires within 1 day(s) on 2011-09-30 00:26:18 UTC
Reserved resources on https://mallorea.gpolab.bbn.com:3626/foam/gapi.  
INFO:omni: ============================================================

On the FOAM server verified that the sliver was created: 

$ foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd
{
 "slivers": [
  {
   "status": "Pending", 
   "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1", 
   "enabled": false, 
   "id": 3, 
   "flowvisor_slice": "2140384d-48fa-447c-930d-13c2066777bc", 
   "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1:2140384d-48fa-447c-930d-13c2066777bc", 
   "email": "lnevers@bbn.com", 
   "desc": "Luisa sliver description"
  }
 ]
}

Approved the sliver: 

$ foamctl approve-sliver -u urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1:2140384d-48fa-447c-930d-13c2066777bc --passwd-file=/opt/foam/etc/foampasswd
true

Confirmed the approval: 

$ foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd
{
 "slivers": [
  {
   "status": "Approved", 
   "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1", 
   "enabled": true, 
   "id": 3, 
   "flowvisor_slice": "2140384d-48fa-447c-930d-13c2066777bc", 
   "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1:2140384d-48fa-447c-930d-13c2066777bc", 
   "email": "lnevers@bbn.com", 
   "desc": "Luisa sliver description"
  }
 ]
}

Following is an example where a sliver is disabled: 

$ foamctl disable-sliver --passwd-file=/opt/foam/etc/foampasswd disable-sliver -u urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1:2140384d-48fa-447c-930d-13c2066777bc
""

Disabling puts the sliver back in pending state: 

$ foamctl list-slivers --passwd-file=/opt/foam/etc/foampasswd
{
 "slivers": [
  {
   "status": "Pending", 
   "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1", 
   "enabled": false, 
   "id": 3, 
   "flowvisor_slice": "2140384d-48fa-447c-930d-13c2066777bc", 
   "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+foamslice1:2140384d-48fa-447c-930d-13c2066777bc", 
   "email": "lnevers@bbn.com", 
   "desc": "Luisa sliver description"
  }
 ]
}

NOX Installation

The NOX controller was install before the FOAM package. To install this controller followed the ​NOX Installation instructions from the noxrepo.org web site. Only evaluated Ubuntu instructions. Install dependencies: 

$ cd /etc/apt/sources.list.d
$ sudo wget http://openflowswitch.org/downloads/debian/nox.list
$ sudo apt-get update
$ sudo apt-get install nox-dependencies

Downloaded the NOX source code, but had to install the following undocumented packages for the instruction to work: 

$ sudo apt-get install git-core

NOX installation 

$ git clone git://noxrepo.org/nox
$ cd nox
$ git checkout zaku 
$ ./boot.sh 
$ mkdir build/
$ cd build/
$ ../configure
$ make -j 5

Followed the ​Using NOX instructions from the noxrepo.org web site to start the NOX controller. 

$ cd nox/build/src
$ ./nox_core -v -i ptcp:9933

Expedient Installation

Installed Expedient using the git repository instructions found in the ​Checkout Repository section of the ​Installing Expedient page, but first had to install pre-requisites: 

  $ sudo apt-get install python-2.6
  $ sudo apt-get install xmlsec1
  $ sudo apt-get install libxmlsec1-devel
  $ sudo apt-get install python-setuptools 
  $ sudo apt-get install python-django
  $ sudo apt-get install python-django-registration
  $ sudo apt-get install python-decorator
  $ sudo apt-get install m2crypto
  $ sudo apt-get install python-imaging (PIL)
  $ sudo apt-get install python-dateutil
  $ sudo apt-get install python-crypto
  $ sudo apt-get install python-paramiko
  $ sudo apt-get install python-webob            
  $ sudo apt-get install python-OpenSSL
  $ sudo apt-get install python-pyquery
  $ sudo apt-get install sphinxsearch
  $ sudo apt-get install python-pygments
  $ sudo apt-get install libxslt-dev   
  $ sudo apt-get install python-zsi 
  $ sudo apt-get install python-mysqldb
  $ sudo easy_install django-autoslug
  & sudo easy_install django-renderform

Checked out the code: 

  $ git clone git://openflow.org/expedient
  $ cd expedient/src/python
  $ PYTHONPATH=. python expedient/clearinghouse/bootstrap_local_settings.py expedient/clearinghouse/
Traceback (most recent call last):
  File "expedient/clearinghouse/bootstrap_local_settings.py", line 10, in <module>
    from expedient.clearinghouse.commands.utils import bootstrap_local_settings
  File "/home/lnevers/expedient/src/python/expedient/clearinghouse/commands/utils.py", line 12, in <module>
    from expedient.clearinghouse.defaultsettings.django import CONF_DIR
  File "/home/lnevers/expedient/src/python/expedient/clearinghouse/defaultsettings/django.py", line 104, in <module>
    "TEMPLATE_LOADERS", TEMPLATE_LOADERS, globals())
  File "/home/lnevers/expedient/src/python/expedient/clearinghouse/defaultsettings/utils.py", line 31, in append_to_local_setting
    import localsettings
ImportError: No module named localsettings

After running into several issues, stopped evaluation due to the documentation and source not being maintained.

Attachments (1)

Download all attachments as: .zip