[[PageOutline]] = GENI Aggregate Manager API Draft Revisions = This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, many of the revisions documented here have been agreed to in discussions on the GENI developer mailing list, and during at least one GEC, but not formally adopted. Well behaved aggregates will typically already implement such revisions. Other revisions are in early discussions and subject to change or abandonment. The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is the set of resources allocated by one Aggregate to one Slice. The current officially adopted version of the API is '''1''' and is documented on the main API page [wiki:GAPI_AM_API here]. This page documents proposed changes for AM API version '''2'''. These changes are grouped into sets. API Version 2 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release. A. Agreed upon and generally implemented: RSpec related changes, specifying that RSpecs are XML following GENI standard schemas. * Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes. * Omni version 1.3 (released June 2011) added client software support for these changes. B. New and changing proposal: Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list. C. Undefined proposal: Support for !UpdateSliver. D. Undefined: Support for clients adding slivers at an aggregate that already has 1 sliver for that slice, or deleting or renewing individual slivers. E. Undefined: Tickets, otherwise known as negotiated reservations. == Summary of Proposed Changes == === Change Set A: RSpecs are XML documents following GENI schemas === This change set has been discussed and well behaved aggregates already implement this change. * Specify that GENI RSpecs comply with GENI standard XML schemas as posted at http://www.geni.net/resources/rspec. GENI RSpec V3 is what was the ProtoGENI V2 schemas as documented [http://www.protogeni.net/trac/protogeni/wiki/RSpec here]. * Include additional options in !GetVersion and !ListResources to allow aggregates to support the GENI RSpecs in addition to their own native format. === Change Set B: Flexible arguments and returns === This change set is new, not implemented, and currently under discussion. * All methods take an ''options'' argument, which is a non null XML struct. No required options are added with this change - the struct may be empty. * Method returns are modified to return at least 3 name/value pairs, with arbitrary additional such pairs. * {{{code}}} is an integer, with non-0 indicating error. * {{{value}}} is the return value as specified in AM API V1 (RSpec, etc), and * {{{output}}} is a human readable indication of the nature of the return or error. * Aggregates are free to use other additional name/value pairs in the return struct. === Change Set C: !UpdateSliver === This change is TBD. === Change Set D: Slivers and Slivergroups === This change is TBD. === Change Set E: Tickets === This change is TBD. == Proposing Additional Changes == GENI community members are encouraged to propose changes to the GENI Aggregate Manager API. Technical discussions are generally held on the [mailto:dev@geni.net Developers mailing list] Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net == Proposed Change Details == === Change Set A === ''Note: The change to GENI standard RSpec schemas originally referenced ProtoGENI V2 RSpec schemas, hosted at www.protogeni.net. These schemas have now been re-branded as GENI V3 RSpec schemas and are hosted at http://www.geni.net/resources/rspec/3.''' ''Note: Options in the geni_ namespace remain reserved, but not all GENI standard options need to be named with the geni_ prefix.'' ''Note: Changes described here have been discussed, are generally agreed upon, and are implemented at well behaved aggregates and clients.'' ==== Part 1: Standardized XML-based GENI RSpecs ==== At GEC10 [wiki:GEC10RSpec the GENI community agreed] that GENI RSpecs would be in what was the [http://www.protogeni.net/trac/protogeni/wiki/RSpec ProtoGENI V2 format] and is now known as GENI V3. Aggregates are free to use an alternate format internally, but must accept and produce compliant RSpecs on demand. Note that individual aggregates may use RSpec extensions to describe custom resources or properties of resources. For RSpec extension information, see the ProtoGENI [http://www.protogeni.net/trac/protogeni/wiki/RSpecExtensions2 wiki]. For more information: * [http://www.geni.net/resources/rspec/3 Official GENI V3 RSpec schemas] * [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard (was ProtoGENI V2) format] * [http://www.protogeni.net/trac/protogeni/wiki/RSpecExtensions2 RSpec extension information] * [http://www.protogeni.net/trac/protogeni/wiki/RSpecSchema2 RSpec schemas listing] Specific changes include: * !ListResources: Return value of !ListResources remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed. The return value is an RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] RSpec in text format if {{{geni_compressed}}} is unspecified or set to {{{false}}}. The return value will be a ZLib compressed and then base 64 encoded string representation of the RSpec if {{{geni_compressed}}} is specified and set to {{{true}}}. This RSpec will be a [http://www.geni.net/resources/rspec/3/ad.xsd advertisement RSpec] when invoked with no {{{geni_slice_urn}}} option, representing the resources available at this aggregate. When the client supplies the {{{geni_slice_urn}}} option, then the aggregate will return a [http://www.geni.net/resources/rspec/3/manifest.xsd manifest RSpec], representing all resources allocated to that slice by this aggregate. * !CreateSliver: rspec argument remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed. `rspec`:: An RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] request RSpec [http://www.geni.net/resources/rspec/3/request.xsd schema] containing the resources that the caller is requesting for allocation to the slice specified in {{{slice_urn}}}. These are expected to be consistent with the resources returned by a previous invocation of [wiki:GAPI_AM_API#ListResources ListResources]. * !CreateSliver: return value remains an [http://www.xmlrpc.com/spec XMLRPC] string, but its format and meaning are now proscribed. The return value is an RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] manifest RSpec [http://www.geni.net/resources/rspec/3/manifest.xsd schema] indicating the resources that were allocated to the slice. The result RSpec may contain additional information about the allocated resources. ==== Part 2: New RSpec Version Options ==== In order to allow aggregates to support both a native RSpec format (e.g. the standard !PlanetLab / SFA RSpecs) as well as the GENI-standard format (GENI V3, was ProtoGENI V2), we need new options to allow a client to request RSpecs in a particular format. Specifically, aggregates will advertise (in !GetVersion) the list of Advertisement and Request RSpec types they are willing to accept, as well as the default Advertisement RSpec type. And they will produce manifest RSpecs in a format matching the format of the !CreateSliver input request RSpec (e.g. return a GENI V3 manifest RSpec when given a GENI V3 request RSpec, and a native !PlanetLab manifest RSpec when given a !PlanetLab request RSpec). ===== Contract details ===== Aggregates advertise the {{{type}}} and {{{version}}} of RSpec formats that they support. If available, they specify the {{{schema}}}, {{{namespace}}} and {{{extensions}}} combination which is the authoritative definition of that format. Clients of the API should understand that combination in order to know how to understand the resources available at that aggregate. If an aggregate advertises a particular {{{type}}}/{{{version}}} (optionally defined with a combination of {{{schema}}}, {{{namespace}}} and {{{extensions}}}) in the {{{ad_rspec_versions}}} attribute of !GetVersion, then it promises to send a correct Advertisement RSpec in response to a !ListResources call which supplies an {{{rspec_version}}} option containing that {{{type}}}/{{{version}}}. ({{{rspec_version}}} is a {{{struct}}} with 2 members, {{{type}}} and {{{version}}}. {{{type}}} and {{{version}}} are ''case-insensitive'' strings, matching those in {{{ad_rspec_versions}}}). If an Aggregate advertises a particular {{{type}}}/{{{version}}} (optionally defined with a combination of {{{schema}}}, {{{namespace}}} and {{{extensions}}}) in the {{{request_rspec_versions}}} attribute of !GetVersion then it promises to correctly honor a !CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (i.e. a GENI format request is answered with a GENI format manifest). The aggregate also promises to send a correctly formatted Manifest RSpec in response to a !ListResources call which supplies a valid {{{geni_slice_urn}}} option and an {{{rspec_version}}} option containing that supported {{{type}}}/{{{version}}}. The !GetVersion attribute {{{default_ad_rspec}}} will be one of the values in the !GetVersion {{{ad_rspec_versions}}} array. ===== New !GetVersion required attributes ===== Return the version of the GENI Aggregate API and RSpecs supported by this aggregate. {{{ struct GetVersion() }}} The result is an [http://www.xmlrpc.com/spec XMLRPC] struct with at least the following members: {{{ { int geni_api; array request_rspec_versions of { string type; string version; string schema; string namespace; array extensions of string; }; array ad_rspec_versions of { string type; string version; string schema; string namespace; array extensions of string; }; default_ad_rspec of { string type; string version; }; } }}} `geni_api`:: An integer indicating the revision of the Aggregate Manager API that an aggregate supports. This document (DRAFT revisions) describes API version 2 (two). `request_rspec_versions`:: An array of data structures indicating the RSpec types accepted by this AM in a request. `ad_rspec_versions`:: An array of data structures indicating what types of RSpec advertisements may be produced by this AM in !ListResources. `default_ad_rspec`:: A data structure indicating the default type of advertisement RSpec produced by this AM in !ListResources. Matches one of the values from {{{ad_rspec_versions}}} Elements used within {{{request_rspec_versions}}}, {{{ad_rspec_versions}}}, and {{{default_ad_rspec}}}: `type`, `version`:: Two ''case-insensitive'' strings which together comprise the type of RSpec. The RSpec `type` should be one of "geni", "protogeni", "sfa", "orca", "openflow", or "orbit" and `version` should be a type-specific version identifier as specified by the appropriate control framework. The "geni" type is reserved for GENI standard format RSpecs, following the schemas hosted at www.geni.net. `schema`:: An optional URL pointing to a schema which can be used to verify the given type of RSpec. `namespace`:: An optional XML namespace which the RSpec of the given type belongs to. `extensions`:: An array of aggregate-specific strings denoting which extensions are supported. In the case of ProtoGENI, these are XML namespaces which denote the extension as a whole. Implementations can add additional members to the struct as desired. Implementations should choose an appropriate prefix to avoid conflicts, but avoid {{{geni_}}} which should be reserved for common options. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#GetVersion GetVersion] operation. The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] specification does not include this operation other than through this method. ===== New !ListResources Option ===== !ListResources will take an additional option, {{{rspec_version}}}, allowing a user to request an Advertisement or Manifest Rspec in a particular format. This struct must contain a {{{type}}} and {{{version}}} matching one of this Aggregate's advertised {{{ad_rspec_versions}}}. Specifics: Return information about available resources or resources allocated to a slice. {{{ string ListResources(string credentials[], struct options) }}} .... `options`:: An [http://www.xmlrpc.com/spec XMLRPC] struct containing members indicating the set of resources the caller is interested in or the format of the result. In addition to the members specified below, callers can pass additional members that specific aggregate manager implementations might honor. Implementations should choose an appropriate prefix to avoid conflicts, but avoid {{{geni_}}} which should be reserved for common options. The following members are available for use in the options parameter. All aggregate managers are required to implement these options. {{{ { boolean geni_available; boolean geni_compressed; string geni_slice_urn; struct rspec_version { string type; string version; }; } }}} .... `rspec_version`:: An [http://www.xmlrpc.com/spec XMLRPC] struct indicating the type and version of Advertisement or Manifest RSpec to return. The struct contains 2 members, {{{type}}} and {{{version}}}. {{{type}}} and {{{version}}} are ''case-insensitive'' strings, matching those in {{{ad_rspec_versions}}} as returned by !GetVersion at this aggregate. If this option is not included in the request, then the returned RSpec will be of the type specified in {{{default_ad_rspec}}}. That Rspec will be an Advertisement RSpec when no {{{geni_slice_urn}}} option is supplied. When a valid {{{geni_slice_urn}}} option is supplied, the returned RSpec will be a Manifest RSpec of the type corresponding to {{{default_ad_rspec}}}, but in manifest format. If the aggregate cannot support the requested {{{type}}}/{{{version}}} (that pair is not listed in {{{ad_rspec_versions}}}), then the aggregate returns an Exception. ===== New !CreateSliver behavior ===== If an Aggregate advertises a {{{type}}}/{{{version}}} pair in its {{{request_rspec_versions}}} as returned by !GetVersion, then it promises to correctly honor a !CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (ie a GENI V3 standard request is answered with a GENI V3 manifest). === Change Set B === ''Note: This set of 2 distinct changes is currently under discussion and has gotten no unofficial or official agreement.'' ''Note: There are 2 parts to this change set. They could be adopted independently, but are described as going together. Additionally, they are described as cumulative to Change Set A above, but are independent.'' ==== Part 1: Additional options argument ==== Adding support for additional functionality in the AM API is currently difficult. Many possible areas of innovation are variations on existing functions. This change adds flexibility in arguments to support such innovation. Under this proposal, each method will take an argument which is a set of name-value pairs (an XMLRPC struct, aka a property list). This argument is required for all methods except !GetVersion (where it is optional), but may be empty (non NULL but containing no name/value pairs). Aggregates may support new entries in this 'options' argument to allow new non-standardized functionality. Specifically, this change adds a final {{{struct options}}} argument to each of !GetVersion, !CreateSliver, !SliverStatus, !RenewSliver, !DeleteSliver, and Shutdown. !ListResources already has that argument. The argument is optional for !GetVersion, but required for all other functions. Aggregates are compliant with this API change by accepting this argument; only for !ListResources are they required to handle any specific options. Similarly, clients are required to supply this argument to talk to compatible aggregates, but are only required to supply any particular options for !ListResources. In !GetVersion, this argument remains optional. Clients that only talk AM API V1 will get an error invoking most functions, when they leave off the {{{options}}} argument. Experimenters can then call !GetVersion (without the {{{options}}} argument). AM API V2 compliant Aggregates shall include the {{{geni_api}}} argument as a top-level entry in the return struct, specifying {{{2}}} to indicate to clients that this AM speaks version 2 of the AM API. This allows experimenters to understand that they need to upgrade their client, or might instruct a clever client tool to automatically switch to version 2 syntax. Aggregates are encouraged to document any new options which they either accept or require in any method. One way to provide partial documentation, is to implement [http://xmlrpc-c.sourceforge.net/introspection.html XML-RPC introspection]. Through the use of method help, aggregates can provide human readable text describing options. Alternatively or additionally, aggregates may document options as part of their return from !GetVersion. This allows clients to either avoid communicating with an aggregate for which the client does not understand how to provide those options, or to tailor the client's request to provide those extra options. '''We have not yet specified the format for advertising those extra options in !GetVersion''' A sample possible option advertisement in !GetVersion: {{{ methodOptions: { GetVersion: { myNewOption1: { type=String, description="A useful but not critical option. This is a human parsable description.", required=False }, myOtherNewOption:..... }, ListResources.... } }}} ==== Part 2: Richer return values ==== In AM API V1, method failures come back sometimes as XMLRPC Faults, sometimes as ''False'', and is occasionally inconsistent across aggregates. Failures typically do not indicate how the Experimenter should modify their request for it to succeed, or if this is a server error. This proposed change expands and formalizes the return structures, to support semantically richer returns, allowing Experimenters better insight into both successes and failures, and how to respond. Allowing aggregates to return more information, on both errors and success, will allow for a richer client-server communication. It would also allow aggregates to give clients hints on how to use successful returns, or otherwise innovate within the bounds of the AM API. This change will modify all methods to return an XMLRPC struct (aka property list) on any application layer success, failure, and even on an error or for most exceptions. Note that a malformed XMLRPC request should still raise an XMLRPC Fault, and other Faults dictated by the XMLRPC specification should still be raised. This struct will contain the return value from the previous revision of the AM API as an entry. This struct will have 3 defined entries, and aggregates are free to include other entries to give more information to clients. The three required entries in the return structure are {{{code}}}, {{{value}}}, and {{{option}}}: * {{{code}}}: An error code: integer, non-zero on error. * 0 = Success * <0 = XMLRPC required error codes * 1-1000 = GENI negotiated return codes (none so far) * 1001-2000 = ProtoGENI specific return codes * 2001-3000 = !PlanetLab specific return codes * 3001-4000 = Orca specific return codes * 4001-5000 = !OpenFlow specific return codes * others as needed * {{{value}}}: On success, this is required. Optional on failure or error. Object representing the successful return value. This will be the object previously returned by the function (for example the manifest RSpec for !CreateSliver, or the struct for !SliverStatus). The value is not defined on error, though aggregates are free to use it. * For !GetVersion, the {{{value}}} is an XMLRPC struct * For !ListResources, the {{{value}}} is an RSpec * For !CreateSliver, the {{{value}}} is an RSpec * For !RenewSliver, the {{{value}}} is a boolean * For !DeleteSliver, the {{{value}}} is a boolean * For !SliverStatus, the {{{value}}} is an XMLRPC struct * For Shutdown, the {{{value}}} is a boolean * {{{option}}}: On failure or error, this is required. Optional on success. This is a String with a human readable message explaining the result. Specifically, this might include an error string, a stacktrace, or other useful messages to help the Experimenter resolve or report the failure or error. It is not defined on success, though aggregates are free to use it. Aggregates are encouraged to use {{{code}}} values and {{{output}}} messages that help experimenters and tools distinguish between bad input, other experimenter error, temporary server errors, or server bugs. Aggregates are similarly encouraged to provide hints on how to fix bad requests using the {{{value}}} entry to experimenters on error or failures. For example, a failed !RenewSliver call that failed because you are not allowed to renew your sliver that far in the future, might return a new date string in the {{{value}}} field that would be allowed. Similarly, a failed !CreateSliver call might return a modified request RSpec in the {{{value}}} field. Aggregates should avoid raising an error (XMLRPC Fault) for application layer errors or any other cases where the XMLRPC specification does not require a Fault, but rather should attempt to return this struct, providing any error messages and stack traces in the {{{output}}} field or other additional fields. For comparison, Orca functions return property lists internally. The ProtoGENI CMV2 API returns a struct with exactly these 3 values. ProtoGENI however uses a different range of return codes, and largely does not define the {{{value}}} slot on errors. For example, !SliverStatus could return on success: {{{ { code: 0 value: { geni_urn: geni_status: ready geni_resources: [ { geni_urn: geni_status: ready geni_error: ''}, { geni_urn: geni_status: ready geni_error: ''} ] } option: } }}} On failure, !DeleteSliver might return: {{{ { code: 1 value: False option: 'No such slice here' } }}} (That code and option are merely examples.) An exception: At the top level, !GetVersion adds a required entry: 'geni_api'=2. This allows V1 clients to determine that they are indeed talking to a GENI AM, but since the version is 2, that is why other function calls will fail. ==== Changes Summary: New Method Signatures ==== When taken together, the new method signatures would be: {{{ struct GetVersion(options=None) Success Return: { geni_api = 2 code = 0 value { geni_api=2, methodOptions: { } } option = } struct ListResources(string credentials[], struct options) Success Return: { code=0 value= option = } struct CreateSliver(string slice_urn, string credentials[], rspec, struct users[], struct options) Success Return: { code=0 value= option = } struct DeleteSliver(string slice_urn, string credentials[], struct options) Success Return: { code=0 value= boolean option = } struct SliverStatus(string slice_urn, string credentials[], struct options) Success Return: { code=0 value= struct (as defined in V1) option = } struct RenewSliver(string slice_urn, string credentials[], string expiration_time, struct options) Success Return: { code=0 value= boolean option = } struct Shutdown(string slice_urn, string credentials[], struct options) Success Return: { code=0 value= boolean option = } }}}