Changes between Version 26 and Version 27 of GAPI_AM_API_DRAFT


Ignore:
Timestamp:
12/12/11 11:07:06 (13 years ago)
Author:
Aaron Helsinger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GAPI_AM_API_DRAFT

    v26 v27  
    55This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, many of the revisions documented here have been agreed to in discussions on the GENI developer mailing list, and during at least one GEC, but not formally adopted. Well behaved aggregates will typically already implement such revisions. Other revisions are in early discussions and subject to change or abandonment.
    66
    7 The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers.
    8 
    9 The current officially adopted version of the API is '''1''' and is documented on the main API page [wiki:GAPI_AM_API here].
    10 
    11 This page documents changes for AM API version '''2''' and proposed changes for AM API version '''3''' or later. These changes are grouped into sets. API Version 2 will be change sets A and B below, as agreed to at GEC12. Other change sets are still under discussion and will be targeted at a future release.
    12 
    13  A. To be in version 2, agreed upon and generally implemented: RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    14   * This set was modified slightly at the coding sprint session of GEC12 - that change is not yet implemented
     7The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is the set of resources allocated by one Aggregate to one Slice.
     8
     9The current officially adopted version of the API is '''2''' and is documented on [wiki:GAPI_AM_API the main API page].
     10
     11API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented [wiki:GAPI_AM_API_V2_DELTAS on a separate page]. They include:
     12 A. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    1513  * Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    1614  * Omni version 1.3 (released June 2011) added client software support for these changes.
    17  B. To be in version 2: Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.
    18  C. New and changing proposal: Add a new method: !UpdateSliver
    19  D. Undefined: Support for clients manipulating individual slivers or groups of slivers at an aggregate
     15 B. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.
     16
     17This page documents proposed changes for AM API version '''3'''. These changes are grouped into sets. API Version 3 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.
     18 C. Undefined proposal: Support for !UpdateSliver.
     19 D. Undefined: Support for clients manipulating individual slivers or groups of slivers at an aggregate.
    2020 E. Undefined: Tickets, otherwise known as negotiated reservations.
    2121
    2222== Summary of Proposed Changes ==
    23 === Change Set A: RSpecs are XML documents following GENI schemas ===
    24 '''Note''': At the GEC12 coding sprint, this change set was modified slightly from that which is widely implemented. This change
    25  - removed the {{{default_ad_rspec}}} value from getVersion
    26  - made the {{{rspec_version}}} argument to listResources required and renamed it {{{geni_rspec_version}}}
    27  - renamed the other return values from getVersion to use the {{{geni_}}} naming prefix
    28 
    29 Other parts of this change set have been discussed and well behaved aggregates already implement them. The community has agreed to include this change in version 2 of this API.
    30 
    31  * Specify that GENI RSpecs comply with GENI standard XML schemas as posted at http://www.geni.net/resources/rspec. GENI RSpec v3 is what was the ProtoGENI v2 schemas as documented [http://www.protogeni.net/trac/protogeni/wiki/RSpec here].
    32  * Include additional options in !GetVersion and !ListResources to allow aggregates to support the GENI RSpecs in addition to their own native format.
    33 
    34 === Change Set B: Flexible arguments and returns ===
    35 This change set is new and not implemented, but was adopted at GEC12 for inclusion in AM API version 2.
    36 
    37  * All methods take an ''options'' argument, which is a non null XML struct. No required options are added with this change - the struct may be empty.
    38  * Method returns are modified to return at least 3 name/value pairs, with arbitrary additional such pairs.
    39   * {{{code}}} indicates success or error return. This itself is an XMLRPC struct, whose content is described below.
    40   * {{{value}}} is the return value as specified in AM API v1 or as modified by Change Set A (RSpec, etc), and
    41   * {{{output}}} is a human readable indication of the nature of the return or error.
    42   * Aggregates are free to use other additional name/value pairs in the return struct.
    43 
    4423=== Change Set C: !UpdateSliver ===
    4524
     
    6645== Proposed Change Details ==
    6746
    68 === Change Set A ===
    69 
    70 '''Note''': At the GEC12 coding sprint, this change set was modified slightly from that which is widely implemented. This change
    71  - removed the {{{default_ad_rspec}}} value from getVersion
    72  - made the {{{rspec_version}}} argument to listResources required and renamed it {{{geni_rspec_version}}}
    73  - renamed the other return values from getVersion to use the {{{geni_}}} naming prefix
    74 
    75 ''Note: The change to GENI standard RSpec schemas originally referenced ProtoGENI v2 RSpec schemas, hosted at www.protogeni.net. These schemas have now been re-branded as GENI v3 RSpec schemas and are hosted at http://www.geni.net/resources/rspec/3.'''
    76 
    77 ''Note: Options in the geni_ namespace remain reserved, but not all GENI standard options need to be named with the geni_ prefix.''
    78 
    79 ''Note: Changes described here have been discussed, have been agreed upon, will be part of API version 2, and are implemented at well behaved aggregates and clients.''
    80 
    81 ==== Part 1: Standardized XML-based GENI RSpecs ====
    82 At GEC10 [wiki:GEC10RSpec the GENI community agreed] that GENI RSpecs would be in what was the [http://www.protogeni.net/trac/protogeni/wiki/RSpec ProtoGENI v2 format] and is now known as GENI v3. Aggregates are free to use an alternate format internally, but must accept and produce compliant RSpecs on demand. Note that individual aggregates may use RSpec extensions to describe custom resources or properties of resources. For RSpec extension information, see the ProtoGENI [http://www.protogeni.net/trac/protogeni/wiki/RSpecExtensions2 wiki].
    83 
    84 For more information:
    85  * [http://www.geni.net/resources/rspec/3 Official GENI v3 RSpec schemas]
    86  * [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard (was ProtoGENI v2) format]
    87  * [http://www.protogeni.net/trac/protogeni/wiki/RSpecExtensions2 RSpec extension information]
    88  * [http://www.protogeni.net/trac/protogeni/wiki/RSpecSchema2 RSpec schemas listing]
    89 
    90 Specific changes include:
    91  * !ListResources: Return value of !ListResources remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed.
    92 
    93  The return value is an RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] RSpec in text format if {{{geni_compressed}}} is unspecified or set to {{{false}}}. The return value will be a ZLib compressed and then base 64 encoded string representation of the RSpec if {{{geni_compressed}}} is specified and set to {{{true}}}.
    94 
    95 This RSpec will be a [http://www.geni.net/resources/rspec/3/ad.xsd advertisement RSpec] when invoked with no {{{geni_slice_urn}}} option, representing the resources available at this aggregate. When the client supplies the {{{geni_slice_urn}}} option, then the aggregate will return a [http://www.geni.net/resources/rspec/3/manifest.xsd manifest RSpec], representing all resources allocated to that slice by this aggregate.
    96 
    97  * !CreateSliver: rspec argument remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed.
    98 
    99   `rspec`::
    100     An RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] request RSpec [http://www.geni.net/resources/rspec/3/request.xsd schema] containing the resources that the caller is requesting for allocation to the slice specified in {{{slice_urn}}}. These are expected to be consistent with the resources returned by a previous invocation of [wiki:GAPI_AM_API#ListResources ListResources].
    101 
    102  * !CreateSliver: return value remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed.
    103 
    104  The return value is an RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] manifest RSpec [http://www.geni.net/resources/rspec/3/manifest.xsd schema] indicating the resources that were allocated to the slice. The result RSpec may contain additional information about the allocated resources.
    105 
    106 ==== Part 2: New RSpec Version Options ====
    107 In order to allow aggregates to support both a native RSpec format (e.g. the standard !PlanetLab / SFA RSpecs) as well as the GENI-standard format (GENI v3, was ProtoGENI v2), we need new options to allow a client to request RSpecs in a particular format.
    108 
    109 Specifically, aggregates will advertise (in !GetVersion) the list of Advertisement and Request RSpec types they are willing to accept. And they will produce manifest RSpecs in a format matching the format of the !CreateSliver input request RSpec (e.g. return a GENI v3 manifest RSpec when given a GENI v3 request RSpec, and a native !PlanetLab manifest RSpec when given a !PlanetLab request RSpec).
    110 
    111 ===== Contract details =====
    112 
    113 Aggregates advertise the {{{type}}} and {{{version}}} of RSpec formats that they support. If available, they specify the {{{schema}}}, {{{namespace}}} and {{{extensions}}} combination which is the authoritative definition of that format. Clients of the API should understand that combination in order to know how to understand the resources available at that aggregate.
    114 
    115 If an aggregate advertises a particular {{{type}}}/{{{version}}} (optionally defined with a combination of {{{schema}}}, {{{namespace}}} and {{{extensions}}}) in the {{{geni_ad_rspec_versions}}} attribute of !GetVersion, then it promises to send a correct Advertisement RSpec in response to a !ListResources call which supplies an {{{rspec_version}}} option containing that {{{type}}}/{{{version}}}. ({{{rspec_version}}} is a {{{struct}}} with 2 members, {{{type}}} and {{{version}}}. {{{type}}} and {{{version}}} are ''case-insensitive'' strings, matching those in {{{geni_ad_rspec_versions}}}).
    116 
    117 If an Aggregate advertises a particular {{{type}}}/{{{version}}} (optionally defined with a combination of {{{schema}}}, {{{namespace}}} and {{{extensions}}}) in the {{{geni_request_rspec_versions}}} attribute of !GetVersion then it promises to correctly honor a !CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (i.e. a GENI format request is answered with a GENI format manifest). The aggregate also promises to send a correctly formatted Manifest RSpec in response to a !ListResources call which supplies a valid {{{geni_slice_urn}}} option and an {{{geni_rspec_version}}} option containing that supported {{{type}}}/{{{version}}}.
    118 
    119 ===== New !GetVersion required attributes =====
    120 
    121 Return the version of the GENI Aggregate API and RSpecs supported by this aggregate.
    122 
    123 {{{
    124 struct GetVersion()
    125 }}}
    126 
    127 The result is an [http://www.xmlrpc.com/spec XMLRPC] struct with at least the following members:
    128 
    129 {{{
    130 {
    131   int geni_api;
    132   array geni_request_rspec_versions of {
    133     string type;
    134     string version;
    135     string schema;
    136     string namespace;
    137     array extensions of string;
    138   };
    139   array geni_ad_rspec_versions of {
    140     string type;
    141     string version;
    142     string schema;
    143     string namespace;
    144     array extensions of string;
    145   };
    146 }
    147 }}}
    148 
    149  `geni_api`::
    150     An integer indicating the revision of the Aggregate Manager API that an aggregate supports. This document (DRAFT revisions) describes API version 2 (two).
    151 
    152  `geni_request_rspec_versions`::
    153    An array of data structures indicating the RSpec types accepted by this AM in a request.
    154 
    155  `geni_ad_rspec_versions`::
    156    An array of data structures indicating what types of RSpec advertisements may be produced by this AM in !ListResources.
    157 
    158 Elements used within {{{geni_request_rspec_versions}}} and {{{geni_ad_rspec_versions}}}:
    159  `type`, `version`::
    160    Two ''case-insensitive'' strings which together comprise the type of RSpec. The RSpec `type` should be one of "geni", "protogeni", "sfa", "orca", "openflow", or "orbit" and `version` should be a type-specific version identifier as specified by the appropriate control framework. The "geni" type is reserved for GENI standard format RSpecs, following the schemas hosted at www.geni.net.
    161 
    162  `schema`::
    163    An optional URL pointing to a schema which can be used to verify the given type of RSpec.
    164 
    165  `namespace`::
    166    An optional XML namespace which the RSpec of the given type belongs to.
    167 
    168  `extensions`::
    169    An array of aggregate-specific strings denoting which extensions are supported. In the case of ProtoGENI, these are XML namespaces which denote the extension as a whole.
    170 
    171 Implementations can add additional members to the struct as desired. Implementations should choose an appropriate prefix to avoid conflicts, but avoid {{{geni_}}} which should be reserved for common options.
    172 
    173 This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#GetVersion GetVersion] operation. The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] specification does not include this operation other than through this method.
    174 
    175 ===== New !ListResources Option =====
    176 !ListResources will take an additional ''required'' option, {{{rspec_version}}}, allowing a user to request an Advertisement or Manifest Rspec in a particular format. This struct must contain a {{{type}}} and {{{version}}} matching one of this Aggregate's advertised {{{geni_ad_rspec_versions}}}.
    177 
    178 Specifics:
    179 
    180 Return information about available resources or resources allocated to a slice.
    181 
    182 {{{
    183 string ListResources(string credentials[], struct options)
    184 }}}
    185 
    186 ....
    187 
    188  `options`::
    189     An [http://www.xmlrpc.com/spec XMLRPC] struct containing members indicating the set of resources the caller is interested in or the format of the result. In addition to the members specified below, callers can pass additional members that specific aggregate manager implementations might honor. Implementations should choose an appropriate prefix to avoid conflicts, but avoid {{{geni_}}} which should be reserved for common options.
    190 
    191 The following members are available for use in the options parameter. All aggregate managers are required to implement these options.
    192 
    193 {{{
    194 {
    195   boolean geni_available;
    196   boolean geni_compressed;
    197   string geni_slice_urn;
    198   struct rspec_version {
    199     string type;
    200     string version;
    201   };
    202 }
    203 }}}
    204 
    205 ....
    206 
    207  `rspec_version`::
    208     An [http://www.xmlrpc.com/spec XMLRPC] struct indicating the type and version of Advertisement or Manifest RSpec to return. The struct contains 2 members, {{{type}}} and {{{version}}}. {{{type}}} and {{{version}}} are ''case-insensitive'' strings, matching those in {{{geni_ad_rspec_versions}}} as returned by !GetVersion at this aggregate. This option is ''required'', and aggregates are expected to return an geni_code 1 ('Bad Arguments') if it is missing.
    209 
    210     The returned Rspec will be an Advertisement RSpec when no {{{geni_slice_urn}}} option is supplied. When a valid {{{geni_slice_urn}}} option is supplied, the returned RSpec will be a Manifest RSpec of the type corresponding to {{{rspec_version}}}, but in manifest format.
    211 
    212     If the aggregate cannot support the requested {{{type}}}/{{{version}}} (that pair is not listed in {{{geni_ad_rspec_versions}}}), then the aggregate returns an error 13 (UNSUPPORTED).
    213 
    214 ===== New !CreateSliver behavior =====
    215 If an Aggregate advertises a {{{type}}}/{{{version}}} pair in its {{{geni_request_rspec_versions}}} as returned by !GetVersion, then it promises to correctly honor a !CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (ie a GENI v3 standard request is answered with a GENI v3 manifest).
    216 
    217 === Change Set B ===
    218 ''Note: This set of 2 distinct changes was discussed and adopted at GEC12 for inclusion in AM API version 2, but is not yet implemented.''
    219 
    220 ==== Part 1: Additional options argument ====
    221 Adding support for additional functionality in the AM API is currently difficult. Many possible areas of innovation are variations on existing functions. This change adds flexibility in arguments to support such innovation.
    222 
    223 Under this proposal, each method will take an argument which is a set of name-value pairs (an XMLRPC struct, aka a property list). This argument is required for all methods except !GetVersion (where it is optional), but may be empty (non NULL but containing no name/value pairs). Aggregates may support new entries in this 'options' argument to allow new non-standardized functionality.
    224 
    225 Specifically, this change adds a final {{{struct options}}} argument to each of !GetVersion, !CreateSliver, !SliverStatus, !RenewSliver, !DeleteSliver,  and Shutdown. !ListResources already has that argument. The argument is optional for !GetVersion, but required for all other functions.
    226 
    227 Aggregates are compliant with this API change by accepting this argument; only for !ListResources are they required to handle any specific options. Similarly, clients are required to supply this argument to talk to compatible aggregates, but are only required to supply any particular options for !ListResources.
    228 
    229 In !GetVersion, this argument remains optional. Clients that only talk AM API v1 will get an error invoking most functions, when they leave off the {{{options}}} argument. Experimenters can then call !GetVersion (without the {{{options}}} argument). AM API v2 compliant Aggregates shall include the {{{geni_api}}} argument as a top-level entry in the return struct, specifying {{{2}}} to indicate to clients that this AM speaks version 2 of the AM API.  This allows experimenters to understand that they need to upgrade their client, or might instruct a clever client tool to automatically switch to version 2 syntax.
    230 
    231 Aggregates should not ''require'' any new options to any method - they should always have a reasonable default for any such option. Clients must always be able to work with any aggregate by simply supplying the options required by this API.
    232 
    233 Aggregates are encouraged to document any new options which they accept in any method, to bootstrap coordination with clients, and provide documentation for human experimenters. This API does not specify how aggregates provide that documentation. One way to provide partial documentation, is to implement [http://xmlrpc-c.sourceforge.net/introspection.html XML-RPC introspection]. Through the use of method help, aggregates can provide human readable text describing options. Alternatively or additionally, aggregates may document options as part of their return from !GetVersion. This allows clients to either avoid communicating with an aggregate for which the client does not understand how to provide those options, or to tailor the client's request to provide those extra options.
    234 
    235 ==== Part 2: Richer return values ====
    236 In AM API v1, method failures come back sometimes as XMLRPC Faults, sometimes as ''False'', and is occasionally inconsistent across aggregates. Failures typically do not indicate how the Experimenter should modify their request for it to succeed, or if this is a server error. This proposed change expands and formalizes the return structures, to support semantically richer returns, allowing Experimenters better insight into both successes and failures, and how to respond.
    237 
    238 Allowing aggregates to return more information, on both errors and success, will allow for a richer client-server communication. It would also allow aggregates to give clients hints on how to use successful returns, or otherwise innovate within the bounds of the AM API.
    239 
    240 This change will modify all methods to return an XMLRPC struct (aka property list) on any application layer success, failure, and even on an error or for most exceptions. Note that a malformed XMLRPC request should still raise an XMLRPC Fault, and other Faults dictated by the XMLRPC specification should still be raised. This struct will contain the return value from the previous revision of the AM API as an entry, as modified by other changes adopted for API version 2. This struct will have 3 defined entries, and aggregates are free to include other entries to give more information to clients.
    241 
    242 The three required entries in the return structure are {{{code}}}, {{{value}}}, and {{{output}}}.
    243 
    244  * {{{code}}}: An XMLRPC struct containing 3 sub elements. For details on this (adopted) proposal, see: https://openflow.stanford.edu/display/FOAM/AM+return+code+proposal
    245 
    246 In summary however:
    247 
    248 {{{
    249 code {
    250   geni_code: XMLRPC integer, registered in an XML document off the GENI AM API page
    251   am_type: Optional: XMLRPC string, case insensitive, types in a registry (XML document) linked off the GENI AM API page
    252   am_code: Optional: XMLRPC integer
    253 }
    254 }}}
    255 
    256 Integers above are allowed to be negative.
    257 
    258 {{{am_type}}} and {{{am_code}}} are optional: AMs need not supply them and clients should not have to use them to understand the nature of the response.
    259 They serve to further specify the nature of the return as indicated by {{{geni_code}}}
    260 
    261 Success is indicated by {{{geni_code}}} value of 0
    262 
    263 Error codes and AM types are in an XML document off the GENI AM API page
    264 
    265  * {{{value}}}: On success, this is required. Optional on failure or error. Object representing the successful return value. This will be the object previously returned by the function (for example the manifest RSpec for !CreateSliver, or the struct for !SliverStatus). The value is not defined on error, though aggregates are free to use it.
    266   * For !GetVersion, the {{{value}}} is an XMLRPC struct
    267   * For !ListResources, the {{{value}}} is an RSpec
    268   * For !CreateSliver, the {{{value}}} is an RSpec
    269   * For !RenewSliver, the {{{value}}} is a boolean
    270   * For !DeleteSliver, the {{{value}}} is a boolean
    271   * For !SliverStatus, the {{{value}}} is an XMLRPC struct
    272   * For Shutdown, the {{{value}}} is a boolean
    273  * {{{output}}}: On failure or error, this is required. Optional on success. This is a String with a human readable message explaining the result. Specifically, this might include an error string, a stacktrace, or other useful messages to help the Experimenter resolve or report the failure or error. It is not defined on success, though aggregates are free to use it.
    274 
    275 Aggregates are encouraged to use {{{code}}} values and {{{output}}} messages that help experimenters and tools distinguish between bad input, other experimenter error, temporary server errors, or server bugs.
    276 
    277 GENI standard error codes will be documented on the GENI AM API Wiki page.
    278 
    279 Aggregates are similarly encouraged to provide hints on how to fix bad requests using the {{{value}}} entry to experimenters on error or failures. For example, a failed !RenewSliver call that failed because you are not allowed to renew your sliver that far in the future, might return a new date string in the {{{value}}} field that would be allowed. Similarly, a failed !CreateSliver call might return a modified request RSpec in the {{{value}}} field.
    280 
    281 Aggregates should avoid raising an error (XMLRPC Fault) for application layer errors or any other cases where the XMLRPC specification does not require a Fault, but rather should attempt to return this struct, providing any error messages and stack traces in the {{{output}}} field or other additional fields. Where the XMLRPC standard prescribes specific Faults, those will remain XMLRPC Faults.
    282 
    283 Aggregates are free to add additional return values to support aggregate or resource specific functionality, or to innovate within the bounds of the AM API. Aggregates are encouraged to document any such new return values which they return, to bootstrap coordination with clients, and provide documentation for human experimenters. One way to provide partial documentation, is to implement [http://xmlrpc-c.sourceforge.net/introspection.html XML-RPC introspection]. Through the use of method help, aggregates can provide human readable text describing return values. Alternatively or additionally, aggregates may document return values as part of their return from !GetVersion. We have not specified the format for advertising those extra return values in !GetVersion.
    284 
    285 For comparison, Orca functions return property lists internally. The ProtoGENI CMV2 API returns a struct with exactly these 3 values. ProtoGENI however uses a different range of return codes, and largely does not define the {{{value}}} slot on errors.
    286 
    287 For example, !SliverStatus could return on success:
    288 {{{
    289 {
    290   code: {
    291        geni_code: 0
    292         },
    293   value:
    294 {
    295   geni_urn: <sliver URN>
    296   geni_status: ready
    297   geni_resources: [ { geni_urn: <resource URN>
    298                       geni_status: ready
    299                       geni_error: ''},
    300                     { geni_urn: <resource URN>
    301                       geni_status: ready
    302                       geni_error: ''}
    303                   ]
    304 }
    305 
    306   output: <none>
    307 }
    308 }}}
    309 
    310 On failure, !DeleteSliver might return:
    311 {{{
    312 {
    313   code: {
    314         geni_code: 12
    315         }
    316   value: False
    317   output: 'No such slice here'
    318 }
    319 }}}
    320 (That code and output are merely examples.)
    321 
    322 An exception:
    323 At the top level, !GetVersion adds a required entry: 'geni_api'=2. This allows v1 clients to determine that they are indeed talking to a
    324 GENI AM, but since the version is 2, that is why other function calls will fail with an XMLRPC Fault.
    325 
    326 ==== Supporting multiple API versions ====
    327 Aggregates are free to support multiple versions of the API. They do so by providing different URLs for each version of the API that they support.
    328 Aggregates should have a 'default' URL (the one typically advertised). That version runs whichever version of the API the server chooses (could be the latest, could be something else.)
    329 Each server implementing the API must include all methods, including getVersion.
    330 
    331 This change modifies getVersion to include a new required member:
    332 {{{
    333 geni_api_versions: an XMLRPC struct containing 1+ entries of:
    334   Name: Integer - supported GENI AM API version
    335   Value: String - URL to the XMLRPC server implementing that version of the GENI AM API
    336 }}}
    337 
    338 For example
    339 {{{
    340 geni_api_versions: {
    341   1: <URL>,
    342   2: <Local URL>,
    343    ...
    344 }
    345 }}}
    346 
    347 Where the entries indicate versions of the API supported, and URLs are absolute URLs where that version of the API is supported. (As per Change Set B above, this new member is a sub-structure to the {{{value}}} member.)
    348 
    349 ==== Changes Summary: New Method Signatures ====
    350 When taken together, the new method signatures would be:
    351 
    352 {{{
    353 struct GetVersion(options=None)
    354 Success Return:
    355 {
    356   geni_api = 2
    357   code = {
    358        geni_code = 0
    359          }
    360   value
    361       {
    362         geni_api=2,
    363         geni_api_versions {
    364              2: <local URL>,
    365              <other versions and URLs as supported>
    366              }
    367         geni_request_rspec_versions [{
    368           type=GENI
    369           version=3
    370           schema=...
    371           namespace=....
    372           extensions []},
    373           <other version structs>]
    374         geni_ad_rspec_versions [{type, version, schema, namespace, extensions},]
    375       }
    376   output = <None>
    377 }
    378 
    379 struct ListResources(string credentials[], struct options)
    380 Success Return:
    381 {
    382    code= {geni_code=0}
    383    value= <GENI v3 Ad or Manifest RSpec string>
    384    output = <None>
    385 }
    386 
    387 struct CreateSliver(string slice_urn,
    388                     string credentials[],
    389                     <GENIv3 request RSpec schema compliant XML string> rspec,
    390                     struct users[],
    391                     struct options)
    392 Success Return:
    393 {
    394    code={geni_code=0}
    395    value= <GENI v3 Manifest RSpec string>
    396    output = <None>
    397 }
    398 
    399 struct DeleteSliver(string slice_urn, string credentials[], struct options)
    400 Success Return:
    401 {
    402    code={geni_code=0}
    403    value= boolean
    404    output = <None>
    405 }
    406 
    407 struct SliverStatus(string slice_urn, string credentials[], struct options)
    408 Success Return:
    409 {
    410    code={geni_code=0}
    411    value= struct (as defined in v1)
    412    output = <None>
    413 }
    414 
    415 struct RenewSliver(string slice_urn,
    416                     string credentials[],
    417                     string expiration_time, struct options)
    418 Success Return:
    419 {
    420    code={geni_code=0}
    421    value= boolean
    422    output = <None>
    423 }
    424 
    425 struct Shutdown(string slice_urn, string credentials[], struct options)
    426 Success Return:
    427 {
    428    code={geni_code=0}
    429    value= boolean
    430    output = <None>
    431 }
    432 }}}
    433 
    434 === Change Set C: !UpdateSliver ===
    435 ''Note: This set of changes is currently under active discussion and has gotten no official or unofficial agreement.''
    436 
    437 A common complaint among experimenters about the current AM API v1 is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.
     47=== Change Set C: !UpdateSliver ===
     48''Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.''
     49
     50A common complaint among experimenters about the current AM API V1 is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.
    43851
    43952The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] calls for an !UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".
    44053
    441 In the PlanetLab implementation of the SFA, !UpdateSliver is in fact a synonym for !CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.
     54In the !PlanetLab implementation of the SFA, !UpdateSliver is in fact a synonym for !CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.
    44255
    44356The ProtoGENI CMV2 API has [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#UpdateSliver UpdateSliver], which is described as the way to "Request a change of resources for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at !PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources. Then the experimenter manually calls !RestartSliver as needed on nodes that need to be rebooted, when the experimenter is ready to do so. This takes an extra step, but gives the experimenter full control over which nodes are restarted and when.
    44457
    445 Orca supports a modify function to modify properties of existing reserved resources. Similar functionality is in discussion to add or remove resources within a slice.
    446 
    447 Here are some key properties and differences of this method at those first two aggregate code-bases, with a note on the alternatives. The community must discuss the options:
    448  - In both ProtoGENI and !PlanetLab, the method takes the full description of what the experimenter wants, and the server computes the difference. Note that 2 experimenters with permissions to modify the slice (say, 2 students of a professor) could issue conflicting update calls on the same aggregate. An alternative would be that experimenters must compute the difference themselves, and would just specify the incremental changes that they want in their reservation.
    449  - Atomic: either the full request succeeds, or it fails. The alternative would be that if the experimenter wanted 5 more nodes and only 3 were available, then the server could give you those 3.
    450  - ProtoGENI and !PlanetLab differ on whether the changes are immediate or not. ProtoGENI uses tickets, allowing the experimenter to change their mind or coordinate their requests (See below). They also separate out reservation of the resources with restarting nodes. This allows the experimenter to control which nodes get rebooted and when. !PlanetLab in comparison handles all restarting of nodes for the experimenters, giving them a single operation to get their resources. This approach also more closely matches the behavior of !CreateSliver in the AM API v1.
     58Some key properties and differences of this method at those two aggregate code-bases, with a note on the alternatives. The community must discuss the options:
     59 - Takes the full description of what the experimenter wants, and the server computes the difference. Note that 2 experimenters with permissions to modify the slice (say, 2 students of a professor) could issue conflicting update calls on the same aggregate. And alternative would be that experimenters must compute the difference themselves, and would just specify the incremental changes they want in their reservation.
     60 - Atomic: either the full request succeeds, or it fails. The alternative would be that if the experimenter wanted 5 more nodes and only 3 were available, that the server could give you those 3.
     61 - The two implementations above differ on whether the changes are immediate or not. ProtoGENI uses tickets, allowing the experimenter to change their mind or coordinate their requests (See below). They also separate out reservation of the resources with restarting nodes. This allows the experimenter to control which nodes get rebooted and when. !PlanetLab in comparison handles all restarting of nodes for the experimenters, giving them a single operation to get their resources. This approach also more closely matches the behavior of !CreateSliver in the AM API V1.
    45162
    45263''The community must discuss the alternatives above.''
     64
     65This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
    45366
    45467The proposed new method:
     
    45669struct UpdateSliver(string slice_urn,
    45770                    string credentials[],
    458                     <GENIv3 request RSpec schema compliant XML string> rspec,
     71                    <GENIV3 request RSpec schema compliant XML string> rspec,
    45972                    struct users[],
    46073                    struct options)
     
    46275{
    46376   code=0
    464    value= <GENI v3 Manifest RSpec string>
    465    output = <None>
     77   value= <GENI V3 Manifest RSpec string>
     78   option = <None>
    46679}
    46780}}}
    46881
    469 Proposed method semantics:
     82Proposed semantics:
    47083 - RSpec argument is a complete request RSpec with the desired new end state for the slice at this aggregate. ''Open questions: Does the request schema support specifying sliver_ids or other attributes that were in the manifest RSpec, and which the experimenter might want or need to preserve in this RSpec?''
    47184 - Atomic: the request either succeeds entirely or fails and the slice is not changed at the aggregate at all, with the aggregate returning a message indicating that it failed and why. Aggregates are encouraged to return enough information to allow the experimenter to modify the request such that it can succeed.
    47285 - Resources are allocated and started or restarted immediately, on behalf of the experimenter (no tickets, and no separate call to restart the sliver)
    473  - The API provides no guarantees about whether existing resources in the slice at the aggregate will be left undisturbed or will be restarted. Aggregates should document any guarantees or the lack thereof for their resources. For example, an aggregate may move virtual machines from one physical rack to another in order to accommodate an experimenter's modified requirements for internal network topology or disk space.
    474 
    475 === Change Set D: Slivers and Sliver Groups ===
    476 ''Note: This set of changes is currently under active discussion and has gotten no official or unofficial agreement.''
     86
     87
     88=== Change Set D: Slivers and !SliverGroups ===
     89''Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.''
    47790
    47891''The current proposal in this change set is to do nothing. See the proposal for !UpdateSliver instead.''
     92
     93This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
    47994
    48095The current AM API calls take a Slice URN, and operate on all resources under that ID at the given aggregate - all the resources for that slice at the aggregate are allocated, renewed, and deleted together. There is no provision for releasing some of the resources allocated to the slice at that aggregate, or for adding new resources to the reservation for that slice at a given aggregate.
     
    490105Specifically, this change would provide ways to:
    491106 - Add resources to a slice at a given aggregate, where resources are already reserved - without having to delete and recreate the reservation at the aggregate. But for this capability, see the !UpdateSliver proposal above.
    492  - Delete specific resources (slivers) from a reservation at an aggregate, without losing other resources. For this capability, see the !UpdateSliver proposal above. Alternatively, !DeleteSliver could be modified to take ''either'' a {{{slice_urn}}} (as it does currently), or a {{{sliver_urn}}}, such as is returned in a manifest RSpec.
     107 - Delete specific resources (slivers) from a reservation at an aggregate, without losing other resources. For this capability, see the !UpdateSliver proposal above. Alternatively, !DeleteSliver could be modified to take 'either' a {{{slice_urn}} (as it does currently), or a {{{sliver_urn}}}, such as is returned in a manifest RSpec.
    493108 - Renew the reservation on specific resources (slivers) within a reservation at an aggregate, while allowing other resource reservations to expire. However, it is unclear whether this particular function would be useful.
    494109 - Support fine grained manipulation of resources in other future AM API calls (such as !StartSliver, !StopSliver, !RestartSliver). However, those other functions are not yet supported in the AM API.
    495110
    496 
    497 === Change Set E: Tickets ===
    498 ''Note: This set of changes is not defined, currently under active discussion and has gotten no official or unofficial agreement.''
     111NOTE: It is important to define the way of referring to the sliver (sliver name or sliver_urn) and the restrictions on such a name with regards to supported characters and length.
     112
     113
     114=== Change Set E: Tickets ===
     115''Note: This set of changes is not defined, currently under active discussion and has gotten no unofficial or official agreement.''
     116
     117This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
    499118
    500119The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.
    501120
    502 Tickets are used in the [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2 ProtoGENI CMV2 interface], and are discussed on their wiki [http://www.protogeni.net/trac/protogeni/wiki/Tickets here]. Tickets (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see [https://geni-orca.renci.org/trac/attachment/wiki/WikiStart/ORCA%20Book.pdf 'the Orca Book']. However, each of these uses of the notion of tickets differs.
     121Tickets are used in the [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2 ProtoGENI CMV2 interface], and are discussed on  [http://www.protogeni.net/trac/protogeni/wiki/Tickets the PG wiki]. Tickets (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see [https://geni-orca.renci.org/trac/attachment/wiki/WikiStart/ORCA%20Book.pdf 'the Orca Book']. However, each of these uses of the notion of tickets differs.
    503122
    504123Tickets would enable a number of useful and possibly critical features:
     
    515134 - semantics for what promises a ticket provides, and what it means to redeem and return tickets
    516135 - many other details.
     136[[PageOutline]]
     137
     138= GENI Aggregate Manager API Draft Revisions =
     139
     140This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, many of the revisions documented here have been agreed to in discussions on the GENI developer mailing list, and during at least one GEC, but not formally adopted. Well behaved aggregates will typically already implement such revisions. Other revisions are in early discussions and subject to change or abandonment.
     141
     142The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is the set of resources allocated by one Aggregate to one Slice.
     143
     144The current officially adopted version of the API is '''2''' and is documented on [wiki:GAPI_AM_API the main API page].
     145
     146API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented [wiki:GAPI_AM_API_V2_DELTAS on a separate page]. They include:
     147 A. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
     148  * Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
     149  * Omni version 1.3 (released June 2011) added client software support for these changes.
     150 B. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.
     151
     152This page documents proposed changes for AM API version '''3'''. These changes are grouped into sets. API Version 3 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.
     153 C. Undefined proposal: Support for !UpdateSliver.
     154 D. Undefined: Support for clients manipulating individual slivers or groups of slivers at an aggregate.
     155 E. Undefined: Tickets, otherwise known as negotiated reservations.
     156
     157== Summary of Proposed Changes ==
     158=== Change Set C: !UpdateSliver ===
     159
     160This change set is new, not implemented, and currently under discussion.
     161
     162 * Add a method !UpdateSliver, which adds, removes or modifies resources allocated the slice at the given aggregate, to match the new request RSpec supplied as an argument
     163 * See the discussion below.
     164
     165=== Change Set D: Slivers and Sliver groups ===
     166
     167This change is TBD. The current proposal is to do nothing. See the discussion below.
     168
     169=== Change Set E: Tickets ===
     170
     171This change is TBD. There is no concrete proposal yet on how to introduce tickets. See the discussion below.
     172
     173== Proposing Additional Changes ==
     174GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.
     175
     176Technical discussions are generally held on the [mailto:dev@geni.net Developers mailing list]
     177
     178Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net
     179
     180== Proposed Change Details ==
     181
     182=== Change Set C: !UpdateSliver ===
     183''Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.''
     184
     185A common complaint among experimenters about the current AM API V1 is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.
     186
     187The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] calls for an !UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".
     188
     189In the !PlanetLab implementation of the SFA, !UpdateSliver is in fact a synonym for !CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.
     190
     191The ProtoGENI CMV2 API has [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#UpdateSliver UpdateSliver], which is described as the way to "Request a change of resources for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at !PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources. Then the experimenter manually calls !RestartSliver as needed on nodes that need to be rebooted, when the experimenter is ready to do so. This takes an extra step, but gives the experimenter full control over which nodes are restarted and when.
     192
     193Some key properties and differences of this method at those two aggregate code-bases, with a note on the alternatives. The community must discuss the options:
     194 - Takes the full description of what the experimenter wants, and the server computes the difference. Note that 2 experimenters with permissions to modify the slice (say, 2 students of a professor) could issue conflicting update calls on the same aggregate. And alternative would be that experimenters must compute the difference themselves, and would just specify the incremental changes they want in their reservation.
     195 - Atomic: either the full request succeeds, or it fails. The alternative would be that if the experimenter wanted 5 more nodes and only 3 were available, that the server could give you those 3.
     196 - The two implementations above differ on whether the changes are immediate or not. ProtoGENI uses tickets, allowing the experimenter to change their mind or coordinate their requests (See below). They also separate out reservation of the resources with restarting nodes. This allows the experimenter to control which nodes get rebooted and when. !PlanetLab in comparison handles all restarting of nodes for the experimenters, giving them a single operation to get their resources. This approach also more closely matches the behavior of !CreateSliver in the AM API V1.
     197
     198''The community must discuss the alternatives above.''
     199
     200This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
     201
     202The proposed new method:
     203{{{
     204struct UpdateSliver(string slice_urn,
     205                    string credentials[],
     206                    <GENIV3 request RSpec schema compliant XML string> rspec,
     207                    struct users[],
     208                    struct options)
     209Success Return:
     210{
     211   code=0
     212   value= <GENI V3 Manifest RSpec string>
     213   option = <None>
     214}
     215}}}
     216
     217Proposed semantics:
     218 - RSpec argument is a complete request RSpec with the desired new end state for the slice at this aggregate. ''Open questions: Does the request schema support specifying sliver_ids or other attributes that were in the manifest RSpec, and which the experimenter might want or need to preserve in this RSpec?''
     219 - Atomic: the request either succeeds entirely or fails and the slice is not changed at the aggregate at all, with the aggregate returning a message indicating that it failed and why. Aggregates are encouraged to return enough information to allow the experimenter to modify the request such that it can succeed.
     220 - Resources are allocated and started or restarted immediately, on behalf of the experimenter (no tickets, and no separate call to restart the sliver)
     221
     222
     223=== Change Set D: Slivers and !SliverGroups ===
     224''Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.''
     225
     226''The current proposal in this change set is to do nothing. See the proposal for !UpdateSliver instead.''
     227
     228This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
     229
     230The current AM API calls take a Slice URN, and operate on all resources under that ID at the given aggregate - all the resources for that slice at the aggregate are allocated, renewed, and deleted together. There is no provision for releasing some of the resources allocated to the slice at that aggregate, or for adding new resources to the reservation for that slice at a given aggregate.
     231
     232This ties closely to the precise definition of a Slice vs a Sliver. The current AM API methods imply that a sliver represents all resources at an aggregate for a given slice. However, this does not match the definition that previous GENI documents have used, nor the functionality that experimenters desire.
     233
     234''A sliver is the smallest set of resources at an aggregate that can be independently reserved and allocated. A given slice may contain multiple slivers at a single aggregate. A sliver may contain multiple components.''
     235
     236As such, the current AM API methods in fact operate on a group of slivers.
     237
     238This change set would provide a means for experimenters to operate on individual slivers within their slice at a given aggregate, or perhaps to define and operate on arbitrary groups of slivers at a given aggregate.
     239
     240Specifically, this change would provide ways to:
     241 - Add resources to a slice at a given aggregate, where resources are already reserved - without having to delete and recreate the reservation at the aggregate. But for this capability, see the !UpdateSliver proposal above.
     242 - Delete specific resources (slivers) from a reservation at an aggregate, without losing other resources. For this capability, see the !UpdateSliver proposal above. Alternatively, !DeleteSliver could be modified to take 'either' a {{{slice_urn}} (as it does currently), or a {{{sliver_urn}}}, such as is returned in a manifest RSpec.
     243 - Renew the reservation on specific resources (slivers) within a reservation at an aggregate, while allowing other resource reservations to expire. However, it is unclear whether this particular function would be useful.
     244 - Support fine grained manipulation of resources in other future AM API calls (such as !StartSliver, !StopSliver, !RestartSliver). However, those other functions are not yet supported in the AM API.
     245
     246NOTE: It is important to define the way of referring to the sliver (sliver name or sliver_urn) and the restrictions on such a name with regards to supported characters and length.
     247
     248
     249=== Change Set E: Tickets ===
     250''Note: This set of changes is not defined, currently under active discussion and has gotten no unofficial or official agreement.''
     251
     252This proposal was discussed at the [http://groups.geni.net/geni/wiki/GEC12GeniAmAPI GEC12 AM API session].
     253
     254The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.
     255
     256Tickets are used in the [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2 ProtoGENI CMV2 interface], and are discussed on  [http://www.protogeni.net/trac/protogeni/wiki/Tickets the PG wiki]. Tickets (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see [https://geni-orca.renci.org/trac/attachment/wiki/WikiStart/ORCA%20Book.pdf 'the Orca Book']. However, each of these uses of the notion of tickets differs.
     257
     258Tickets would enable a number of useful and possibly critical features:
     259 - Brokers: 3rd parties consolidating, scheduling and allocating resources on behalf of a number of other aggregates
     260 - Lending resources to other experimenters
     261 - Coordinated or negotiated reservations: reserving resources from aggregate B only if aggregate A can give you a complementary resource. For example, a VLAN tag. This is related to stitching, both network stitching and the more general form.
     262 - Two phase commit reservations (similar to the above)
     263 - Scheduled reservations in the future
     264 - Giving experimenters explicit control over when resources are started, stopped, and restarted (see the discussion on !UpdateSliver above)
     265
     266This change would introduce the concept of tickets, and new methods to use tickets. It requires:
     267 - a ticket schema
     268 - methods to get, redeem, split, and maybe return tickets
     269 - semantics for what promises a ticket provides, and what it means to redeem and return tickets
     270 - many other details.