Changes between Version 18 and Version 19 of Omni


Ignore:
Timestamp:
09/21/10 22:09:27 (14 years ago)
Author:
tmitchel@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Omni

    v18 v19  
    1 [[PageOutline]]
    2 
    3 = The Omni GENI Client =
    4 
    5 Omni is an end-user GENI client that communicates with GENI Aggregate
    6 Managers and presents their resources with a uniform specification,
    7 known as the omnispec.  The Omni client can also communicate with
    8 control frameworks in order to create slices, delete slices, and
    9 enumerate available GENI Aggregate Managers.  Note that Omni also
    10 supports using control framework native RSpecs.
    11 
    12 To configure Omni, please copy src/omni_config to your ~/.gcf
    13 directory and fill in the parameters for at least one control
    14 framework.  The "omni" section should be filled in with the
    15 certificate and key that you use in your control framework.  Note that
    16 keys for the GCF framework are by default stored in ~/.gcf. Embedded
    17 comments describe the meaning of each field.
    18 
    19 The currently supported control frameworks are SFA (PlanetLab),
    20 ProtoGENI and GCF. The currently supported aggregates are SFA,
    21 ProtoGENI, OpenFlow and GCF.
    22 
    23 Omni performs the following functions:
    24  * Talks to each control framework in its native API
    25  * Contacts Aggregate Managers via the GENI API
    26  * Uses either native RSpecs or a common RSpec format called an
    27    omnispec.
    28 
    29 
    30 
    31 == Omni workflow ==
    32 
    33  1. Pick a Clearinghouse you want to use. That is the control framework you
    34     will use.
    35  2. Be sure the appropriate section of omni config for your framework
    36     (sfa/gcf/pg) has appropriate settings for contacting that
    37     Clearinghouse, and user credentials that are valid for that
    38     Clearinghouse.
    39  3. Run omni listresources > avail-resources.omnispec
    40   a. When you do this, Omni will contact your designated
    41      Clearinghouse, using your framework-specific user credentials.
    42   b. The clearinghouse will list the Aggregates it knows about. EG for
    43      GCF, the am_* entries in gcf_config. For SFA, it will return the
    44      contents of /etc/sfa/geni_aggregates.xml.
    45   c. Omni will then contact each of the Aggregates that the
    46      Clearinghouse told it about, and use the GENI AM API to ask each
    47      for its resources. Again, it will use your user credentials. So
    48      each Aggregate Manager must trust the signer of your user
    49      credentials, in order for you to talk to it. This is why you add
    50      the CH certificate to /etc/sfa/trusted_roots or to the -r
    51      argument of your GCF gcf-am.py.  d. Omni will then convert the
    52      proprietary RSPecs into a single 'omnispec'.
    53  4. Save this to a file. You can then edit this file to reserve
    54     resources, by changing 'allocate: false' to 'allocate: true'
    55     wherever the resource is not already allocated ('allocated: true').
    56  5. Create a Slice. Slices are created at your Clearinghouse. Slices
    57     are named based on the Clearinghouse authority that signs for
    58     them. Using the shorthand (just the name of your slice within PG,
    59     for example) allows Omni to ensure your Slice is named
    60     correctly. So run: omni.py createslice MyGreatTestSlice
    61  6. Allocate your Resources. Given a slice, and your edited omnispec
    62     file, you are ready to allocate resources by creating slivers at
    63     each of the Aggregate Managers.  Omni will contact your
    64     Clearinghouse again, to get the credentials for your slice. It
    65     will parse your omnispec file, converting it back into framework
    66     specific RSpec format as necessary.  It will then contact each
    67     Aggregate Manager in your omnispec where you are reserving
    68     resources, calling the GENI AM API CreateSliver call on each. It
    69     will supply your Slice Credentials (from the Clearinghouse) plus
    70     your own user certificate, and the RSpec. At this point, you have
    71     resources and can do your experiment.
    72  7. Renew or Delete.  After a while you may want to Renew your Sliver
    73     that is expiring, or Delete it. Omni will contact the
    74     Clearinghouse, get a list of all Aggregates, and invoke
    75     RenewSliver or DeleteSliver on each, for your slice name.
    76 
    77 
    78 == Example Configurations ==
    79 
    80 === PlanetLab (PLC) ===
    81 Here is a configuration example for plc for user jkarlin at site plc.bbn:
    82 
    83 {{{
    84 [plc]
    85 type=sfa
    86 authority=plc.bbn
    87 user=plc.bbn.jkarlin
    88 cert=~/.gcf/plc.bbn.jkarlin.gid
    89 key=~/.gcf/jkarlin.pkey
    90 registry=http://www.planet-lab.org:12345
    91 slicemgr=http://www.planet-lab.org:12347
    92 }}}
    93 
    94 The only lines you should have to change are authority, user, cert, and key.
    95 Be sure to use your own site name, such as plc.princeton and username.  The
    96 key is your private key associated with your planetlab account.  The cert must be
    97 acquired using the SFI tool provided by PlanetLab.
    98 
    99 To install SFI, follow the directions at http://svn.planet-lab.org/wiki/SFAInstallationGuide#InstallingtheCommandLineTools.
    100 
    101 After installing, configure SFI client by following the directions at http://svn.planet-lab.org/wiki/SFAConfigurationGuide#ConfiguringSFITools.
    102 
    103 Finally, run 'sfi.py get-gid' at the command line to download your certificate file.  It will download to ~/.sfi/plc.<yoursite>.<yourname>.gid  Now point your certificate file in omni_config to this file.
    104 
    105 You should now be configured to run omni on PLC.
    106 
    107 Try running getversion (the errors for VINI and PLJ are because they haven't upgraded to the GENI API yet but PLC and PLE work):
    108 
    109 {{{
    110 jkarlin@ubu:~/dev/gcf/src$ ./omni.py getversion
    111 INFO:omni:Using control framework plc
    112 INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345
    113 INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347
    114 urn:publicid:IDN+ple+authority+sa (http://planet-lab.eu:12346) {'geni_api': 1, 'sfa': 1}
    115 ERROR:omni:Failed to get version information for urn:publicid:IDN+plc:vini+authority+sa at (http://vini-veritas.net:12346).
    116 ERROR:omni:<Fault 109: ': GetVersion: Not implemented at interface get_version: aggregate'>
    117 urn:publicid:IDN+plc+authority+sa (http://planet-lab.org:12346) {'geni_api': 1, 'sfa': 1}
    118 jkarlin@ubu:~/dev/gcf/src$
    119 
    120 }}}
    121 
    122 
    123 
    124 === ProtoGENI ===
    125 
    126 == Running Omni ==
    127 
    128 === The following options are supported: ===
    129 
    130 -c FILE -- location of your config file (default ~/.gcf/omni_config)
    131 
    132 -f FRAMEWORK -- control framework to use (e.g. my_sfa), overriding
    133 default in config file.  The framework is a section named in the config file.
    134 
    135 --debug -- Enable debug output
    136 
    137 === The following commands are supported: ===
    138 
    139 ==== createslice ====
    140  * format:  omni.py createslice <slice-name>
    141  * example: omni.py createslice myslice
    142 
    143   Creates the slice in your chosen control framework.
    144 
    145   Default GCF certs require a slice named geni.net:gpo:gcf+slice+<name>
    146   based on the base_name section of gcf_config.  The shorthand notation
    147   is available for SFA and PG.  Shorthand works if your control framework is GCF
    148   only if you have configured the 'authority' line in the gcf section of
    149   omni_config.
    150 
    151 
    152 ==== deleteslice ====
    153  * format:  omni.py deleteslice <slice-name>
    154  * example: omni.py deleteslice myslice
    155 
    156   Deletes the slice in your chosen control framework
    157 
    158 
    159 ==== getversion ====
    160  * format:  omni.py getversion [-a AM-URL]
    161  * examples:
    162   * omni.py getversion
    163   * omni.py getversion -a http://localhost:12348
    164 
    165 
    166 ==== listresources ====
    167  * format:  omni.py listresources [-a AM-URL [-n]] [slice-name]
    168  * examples:
    169   * omni.py listresources
    170   * omni.py listresources myslice
    171   * omni.py listresources -a http://localhost:12348 myslice
    172   * omni.py listresources -a http://localhost:12348 -n myslice
    173                    
    174   This command will list the rspecs of all geni aggregates available
    175   through your chosen framework, and present them in omnispec form.
    176   Save the result to a file and edit the allocate value to true
    177   to set up a reservation RSpec, suitable for use in a call to
    178   createsliver.
    179 
    180   If a slice name is supplied, then resources for that slice only
    181   will be displayed.
    182 
    183   If an Aggregate Manager URL is supplied, only resources
    184   from that AM will be listed.
    185 
    186   If the "-n" flag s used the native RSpec is returned instead of an
    187   omnispec. The "-n" flag requires the "-a" flag also be used to
    188   specify an aggregate manager.
    189 
    190 
    191 ==== createsliver ====
    192  * format:  omni.py createsliver [-a AM-URL [-n]] <slice-name> <spec file>
    193  * examples:
    194   * omni.py createsliver myslice resources.ospec
    195   * omni.py createsliver -a http://localhost:12348 -n myslice resources.rspec
    196 
    197  * argument: the spec file should have been created by a call to
    198             listresources (e.g. omni.py listresources > resources.ospec)
    199             Then, edit the file and set "allocate": true, for each
    200                         resource that you want to allocate.
    201 
    202   This command will allocate the requested resources (those marked
    203   with allocate: true in the rspec).  It will send an rspec to each
    204   aggregate manager that a resource is requested for.
    205   This command can also operate in native mode "-n" by sending a
    206   native rspec to a single aggregate specified by the "-a" command.
    207 
    208 
    209 ==== deletesliver ====
    210  * format:  omni.py deletesliver [-a AM-URL] <slice-name>
    211  * examples:
    212   * omni.py deletesliver myslice
    213   * omni.py deletesliver -a http://localhost:12348 myslice
    214 
    215   This command will free any resources associated with your slice. 
    216 
    217 
    218 ==== renewsliver ====
    219  * format:  omni.py renewsliver [-a AM-URL] <slice-name> "<time>"
    220  * examples:
    221   * omni.py renewsliver myslice "12/12/10 4:15pm"
    222   * omni.py renewsliver myslice "12/12/10 16:15"
    223   * omni.py renewsliver -a http://localhost:12348 myslice "12/12/10 16:15"
    224 
    225   This command will renew your resources at each aggregate up to the
    226   specified time.  This time must be less than or equal to the time
    227   available to the slice.  Times are in UTC.
    228 
    229 
    230 ==== sliverstatus ====
    231  * format: omni.py sliverstatus [-a AM-URL] <slice-name>
    232  * examples:
    233   * omni.py sliverstatus myslice
    234   * omni.py sliverstatus -a http://localhost:12348 myslice
    235 
    236   This command will get information from each aggregate about the
    237   status of the specified slice
    238 
    239 
    240 ==== shutdown ====
    241  * format:  omni.py shutdown [-a AM-URL] <slice-name>
    242  * examples:
    243   * omni.py shutdown myslice
    244   * omni.py shutdown -a http://localhost:12348 myslice
    245 
    246   This command will stop the resources from running, but not delete
    247   their state.  This command should not be needed by most users.
    248 
    249 == Omnispecs ==
    250 
    251 Omni converts native advertisement rspecs into omnispecs on listresources calls.  The user can then edit the resulting omnispec file to select the resources that they would like to reserve.  Next, this edited omnispec is passed into omni during the createsliver call, is turned back into a native rspec, and is sent to the appropriate aggregate.
    252 
    253 This section presents example native rspecs and their respective omnispecs.
    254 
    255 TODO: Add links to protogeni/openflow/planetlab native rspec websites.
    256 
    257 === OpenFlow ===
    258 
    259 OpenFlow native RSpecs are different formats for advertisements and reservations. The Omni omnispec is an intermediate format - Omni will convert between the two formats for you, allowing you to edit the omnispec directly.
    260 
    261 The native advertisement RSpecs describe a network in terms of switches and links. To reserve flowspace you you will need to define a slice (needs a name and controller URL), a project (named), a user for logging in to FlowVisor, and then one or more flowspace entries (flowvisor rules). The flowspace needs the switches it applies to, and then the ports, dl, vlnan, nw, etc specs for what it should match. Everything in your reservation RSpec will go to the slice / controller you have specified.
    262 
    263 Note that in these RSpecs, all integers can be specified as hex or decimal. * or missing means wildcard.
    264 
    265 A native OpenFlow advertisement RSpec looks like this:
    266 {{{
    267 <rspec>
    268     <network name="Stanford" location="Stanford, CA, USA">
    269         <switches>
    270             <switch urn="urn:publicid:IDN+openflow:stanford+switch:0" />
    271             <switch urn="urn:publicid:IDN+openflow:stanford+switch:1" />
    272             <switch urn="urn:publicid:IDN+openflow:stanford+switch:2" />
    273         </switches>
    274         <links>
    275             <link
    276             src_urn="urn:publicid:IDN+openflow:stanford+switch:0+port:0"
    277             dst_urn="urn:publicid:IDN+openflow:stanford+switch:1+port:0"
    278             />
    279             <link
    280             src_urn="urn:publicid:IDN+openflow:stanford+switch:1+port:0"
    281             dst_urn="urn:publicid:IDN+openflow:stanford+switch:0+port:0"
    282             />
    283             <link
    284             src_urn="urn:publicid:IDN+openflow:stanford+switch:0+port:1"
    285             dst_urn="urn:publicid:IDN+openflow:stanford+switch:2+port:0"
    286             />
    287             <link
    288             src_urn="urn:publicid:IDN+openflow:stanford+switch:2+port:0"
    289             dst_urn="urn:publicid:IDN+openflow:stanford+switch:0+port:1"
    290             />
    291             <link
    292             src_urn="urn:publicid:IDN+openflow:stanford+switch:1+port:1"
    293             dst_urn="urn:publicid:IDN+openflow:stanford+switch:2+port:1"
    294             />
    295             <link
    296             src_urn="urn:publicid:IDN+openflow:stanford+switch:2+port:1"
    297             dst_urn="urn:publicid:IDN+openflow:stanford+switch:1+port:1"
    298             />
    299             </links>
    300             </network>
    301             <network name="Princeton" location="USA">
    302             <switches>
    303             <switch urn="urn:publicid:IDN+openflow:stanford+switch:3" />
    304             <switch urn="urn:publicid:IDN+openflow:stanford+switch:4" />
    305             </switches>
    306             <links>
    307             <link
    308             src_urn="urn:publicid:IDN+openflow:stanford+switch:3+port:0"
    309             dst_urn="urn:publicid:IDN+openflow:stanford+switch:4+port:0"
    310             />
    311             <link
    312             src_urn="urn:publicid:IDN+openflow:stanford+switch:4+port:0"
    313             dst_urn="urn:publicid:IDN+openflow:stanford+switch:3+port:0"
    314             />
    315         </links>
    316     </network>
    317 </rspec>
    318 }}}
    319 
    320 This advertisement RSpec gets converted by Omni (when you call listresources) into an omnispec. The omnispec can be edited directly, and Omni will convert that into the appropriate reservation RSpec. The OpenFlow omnispec looks like this:
    321 
    322 {{{
    323 {
    324     "urn": "urn:publicid:IDN+openflow:stanford+authority+am",
    325     "type": "rspec_of",
    326     "resources": {
    327         "urn:publicid:IDN+openflow:stanford+switch:2": {
    328             "name": "Stanford:openflow:stanford",
    329             "misc": {},
    330             "allocate": false,
    331             "allocated": false,
    332             "type": "switch",
    333             "options": {
    334                 "dl_type": "from=*, to=*",
    335                 "port:1": "switch:1 port:1",
    336                 "port:0": "switch:0 port:1",
    337                 "nw_dst": "from=*, to=*",
    338                 "dl_src": "from=*, to=*",
    339                 "nw_proto": "from=*, to=*",
    340                 "tp_dst": "from=*, to=*",
    341                 "tp_src": "from=*, to=*",
    342                 "dl_dst": "from=*, to=*",
    343                 "nw_src": "from=*, to=*",
    344                 "vlan_id": "from=*, to=*"
    345             },
    346             "description": "OpenFlow Switch"
    347         },
    348         "urn:publicid:IDN+openflow:stanford+switch:3": {
    349             "name": "Princeton:openflow:stanford",
    350             "misc": {},
    351             "allocate": false,
    352             "allocated": false,
    353             "type": "switch",
    354             "options": {
    355                 "dl_type": "from=*, to=*",
    356                 "port:0": "switch:4 port:0",
    357                 "nw_dst": "from=*, to=*",
    358                 "dl_src": "from=*, to=*",
    359                 "nw_proto": "from=*, to=*",
    360                 "tp_dst": "from=*, to=*",
    361                 "tp_src": "from=*, to=*",
    362                 "dl_dst": "from=*, to=*",
    363                 "nw_src": "from=*, to=*",
    364                 "vlan_id": "from=*, to=*"
    365             },
    366             "description": "OpenFlow Switch"
    367         },
    368         "urn:publicid:IDN+openflow:stanford+switch:0": {
    369             "name": "Stanford:openflow:stanford",
    370             "misc": {},
    371             "allocate": false,
    372             "allocated": false,
    373             "type": "switch",
    374             "options": {
    375                 "dl_type": "from=*, to=*",
    376                 "port:1": "switch:2 port:0",
    377                 "port:0": "switch:1 port:0",
    378                 "nw_dst": "from=*, to=*",
    379                 "dl_src": "from=*, to=*",
    380                 "nw_proto": "from=*, to=*",
    381                 "tp_dst": "from=*, to=*",
    382                 "tp_src": "from=*, to=*",
    383                 "dl_dst": "from=*, to=*",
    384                 "nw_src": "from=*, to=*",
    385                 "vlan_id": "from=*, to=*"
    386             },
    387             "description": "OpenFlow Switch"
    388         },
    389         "urn:publicid:IDN+openflow:stanford+switch:1": {
    390             "name": "Stanford:openflow:stanford",
    391             "misc": {},
    392             "allocate": false,
    393             "allocated": false,
    394             "type": "switch",
    395             "options": {
    396                 "dl_type": "from=*, to=*",
    397                 "port:1": "switch:2 port:1",
    398                 "port:0": "switch:0 port:0",
    399                 "nw_dst": "from=*, to=*",
    400                 "dl_src": "from=*, to=*",
    401                 "nw_proto": "from=*, to=*",
    402                 "tp_dst": "from=*, to=*",
    403                 "tp_src": "from=*, to=*",
    404                 "dl_dst": "from=*, to=*",
    405                 "nw_src": "from=*, to=*",
    406                 "vlan_id": "from=*, to=*"
    407             },
    408             "description": "OpenFlow Switch"
    409         },
    410         "urn:publicid:IDN+openflow:stanford+user+sliceinfo": {
    411             "name": "sliceinfo",
    412             "misc": {},
    413             "allocate": false,
    414             "allocated": false,
    415             "type": "user",
    416             "options": {
    417                 "project_description": "Internet performance research to ...",
    418                 "controller_url": "tcp:localhost:6633",
    419                 "slice_name": "Crazy Load Balancing Experiment",
    420                 "firstname": "John",
    421                 "lastname": "Doe",
    422                 "project_name": "Stanford Networking Group",
    423                 "fv_password": "slice_pass",
    424                 "slice_description": "Does crazy load balancing and plate spinning",
    425                 "email": "jdoe@geni.net"
    426             },
    427             "description": "Slice information for FlowVisor Access"
    428         },
    429         "urn:publicid:IDN+openflow:stanford+switch:4": {
    430             "name": "Princeton:openflow:stanford",
    431             "misc": {},
    432             "allocate": false,
    433             "allocated": false,
    434             "type": "switch",
    435             "options": {
    436                 "dl_type": "from=*, to=*",
    437                 "port:0": "switch:3 port:0",
    438                 "nw_dst": "from=*, to=*",
    439                 "dl_src": "from=*, to=*",
    440                 "nw_proto": "from=*, to=*",
    441                 "tp_dst": "from=*, to=*",
    442                 "tp_src": "from=*, to=*",
    443                 "dl_dst": "from=*, to=*",
    444                 "nw_src": "from=*, to=*",
    445                 "vlan_id": "from=*, to=*"
    446             },
    447             "description": "OpenFlow Switch"
    448         }
    449     }
    450 }
    451 }}}
    452 
    453 To allocate, simply set "allocate" to true for each switch that you want to allocate, and fill in the flowspace options for the flowspace on that switch (nw/dl/tp/vlan specifying the matching rules you want for your flowspace entry).  Next, fill in 'sliceinfo' section with your name, password, controller URL, etc and pass the omnispec in for the omni createsliver call.
    454 
    455 
    456 The native OpenFlow reservation RSpec looks like this:
    457 {{{
    458     <resv_rspec>
    459         <user
    460             firstname="John"
    461             lastname="Doe"
    462             email="john.doe@geni.net"
    463             password="slice_pass"
    464         />
    465         <project
    466             name="Stanford Networking Group"
    467             description="Internet performance research to ..."
    468         />
    469         <slice
    470             name="Crazy Load Balancer"
    471             description="Does this and that..."
    472             controller_url="tcp:controller.stanford.edu:6633"
    473         />
    474         <flowspace>
    475             <switches>
    476                 <switch urn="urn:publicid:IDN+openflow:stanford+switch:0">
    477                 <switch urn="urn:publicid:IDN+openflow:stanford+switch:2">
    478             </switches>
    479             <port from="1" to="4" />
    480             <dl_src from="22:33:44:55:66:77" to="22:33:44:55:66:77" />
    481             <dl_dst from="*" to="*" />
    482             <dl_type from="0x800" to="0x800" />
    483             <vlan_id from="15" to="20" />
    484             <nw_src from="192.168.3.0" to="192.168.3.255" />
    485             <nw_dst from="192.168.3.0" to="192.168.3.255" />
    486             <nw_proto from="17" to="17" />
    487             <tp_src from="100" to="100" />
    488             <tp_dst from="100" to="*" />
    489         </flowspace>
    490         <flowspace>
    491             <switches>
    492                 <switch urn="urn:publicid:IDN+openflow:stanford+switch:1">
    493             </switches>
    494             <tp_src from="100" to="100" />
    495             <tp_dst from="100" to="*" />
    496         </flowspace>
    497     </resv_rspec>
    498 }}}
    499 
    500 
    501 
    502 == Extending Omni ==
    503 
    504 Extending Omni to support additional types of Aggregate Managers with
    505 different RSpec formats requires adding a new omnispec/rspec
    506 conversion file.
    507 
    508 Extending Omni to support additional frameworks with their own
    509 clearinghouse APIs requires adding a new Framework extension class.
     1The information on this page has been moved to http://trac.gpolab.bbn.com/gcf/wiki/Omni.