wiki:DRAFT_GAPI_AM_API_V2_DELTAS

Version 4 (modified by Aaron Helsinger, 12 years ago) (diff)

--

GENI AM API Changes from version 1 to version 2

This page documents changes to the GENI Aggregate Manager API from version 1 to version 2. It consists of the text describing the changes to the API which was used to define and adopt the changes to the API for version 2.

Summary of Changes

Change Set A: RSpecs are XML documents following GENI schemas

Note: At the GEC12 coding sprint, this change set was modified slightly from that which is widely implemented. This change

  • removed the default_ad_rspec value from GetVersion
  • made the rspec_version argument to ListResources required and renamed it geni_rspec_version
  • renamed the other return values from GetVersion to use the geni_ naming prefix

Other parts of this change set have been discussed and most current aggregates already implement them. The community has agreed to include this change in version 2 of this API.

  • Specify that GENI RSpecs comply with GENI standard XML schemas as posted at http://www.geni.net/resources/rspec. GENI RSpec v3 is what was the ProtoGENI v2 schemas. RSpec and the ProtoGENI schemas are documented on the ProtoGENI wiki.
  • Include additional options in GetVersion and ListResources to allow aggregates to support the GENI RSpecs in addition to their own native format.

Change Set B: Flexible arguments and returns

  • All methods take an options argument, which is a non null XML struct. No required options are added with this change - the struct may be empty.
  • Method returns are modified to return at least 3 name/value pairs, with arbitrary additional such pairs.
    • code indicates success or error return. This itself is an XML-RPC struct, whose content is described below.
    • value is the return value as specified in AM API v1 or as modified by Change Set A (RSpec, etc), and
    • output is a human readable indication of the nature of the return or error.
    • Aggregates are free to use other additional name/value pairs in the return struct.

Adopted Change Details

Change Set A

Note: At the GEC12 coding sprint, this change set was modified slightly from that which is widely implemented. This change

  • removed the default_ad_rspec value from GetVersion
  • made the rspec_version argument to ListResources required and renamed it geni_rspec_version
  • renamed the other return values from GetVersion to use the geni_ naming prefix

Note: The change to GENI standard RSpec schemas originally referenced ProtoGENI v2 RSpec schemas, hosted at www.protogeni.net. These schemas have now been re-branded as GENI v3 RSpec schemas and are hosted at http://www.geni.net/resources/rspec/3.

Note: Options in the geni_ namespace remain reserved, but not all GENI standard options need to be named with the geni_ prefix.

Part 1: Standardized XML-based GENI RSpecs

At GEC10 the GENI community agreed that GENI RSpecs would be in what was the ProtoGENI v2 format and is now known as GENI v3. Aggregates are free to use an alternate format internally, but must accept and produce compliant RSpecs on demand. Note that individual aggregates may use RSpec extensions to describe custom resources or properties of resources. For RSpec extension information, see the ProtoGENI wiki.

For more information:

Specific changes include:

  • ListResources: Return value of ListResources remains an XML-RPC string, but its format and meaning are now proscribed.

The return value is an RSpec matching the GENI standard RSpec in text format if geni_compressed is unspecified or set to false. The return value will be a ZLib compressed and then base 64 encoded string representation of the RSpec if geni_compressed is specified and set to true.

This RSpec will be a advertisement RSpec when invoked with no geni_slice_urn option, representing the resources available at this aggregate. When the client supplies the geni_slice_urn option, then the aggregate will return a manifest RSpec, representing all resources allocated to that slice by this aggregate.

  • CreateSliver: rspec argument remains an XML-RPC string, but its format and meaning are now proscribed.
rspec
An RSpec matching the GENI standard request RSpec schema containing the resources that the caller is requesting for allocation to the slice specified in slice_urn. These are expected to be consistent with the resources returned by a previous invocation of ListResources.
  • CreateSliver: return value remains an XML-RPC string, but its format and meaning are now proscribed.

The return value is an RSpec matching the GENI standard manifest RSpec schema indicating the resources that were allocated to the slice. The result RSpec may contain additional information about the allocated resources.

Part 2: New RSpec Version Options

In order to allow aggregates to support both a native RSpec format (e.g. the standard PlanetLab / SFA RSpecs) as well as the GENI-standard format (GENI v3, was ProtoGENI v2), we need new options to allow a client to request RSpecs in a particular format.

Specifically, aggregates will advertise (in GetVersion) the list of Advertisement and Request RSpec types they are willing to accept. And they will produce manifest RSpecs in a format matching the format of the CreateSliver input request RSpec (e.g. return a GENI v3 manifest RSpec when given a GENI v3 request RSpec, and a native PlanetLab manifest RSpec when given a PlanetLab request RSpec).

Contract details

Aggregates advertise the type and version of RSpec formats that they support. If available, they specify the schema, namespace and extensions combination which is the authoritative definition of that format. Clients of the API should understand that combination in order to know how to understand the resources available at that aggregate.

If an aggregate advertises a particular type/version (optionally defined with a combination of schema, namespace and extensions) in the geni_ad_rspec_versions attribute of GetVersion, then it promises to send a correct Advertisement RSpec in response to a ListResources call which supplies an geni_rspec_version option containing that type/version. (geni_rspec_version is a struct with 2 members, type and version. type and version are case-insensitive strings, matching those in geni_ad_rspec_versions).

If an Aggregate advertises a particular type/version (optionally defined with a combination of schema, namespace and extensions) in the geni_request_rspec_versions attribute of GetVersion then it promises to correctly honor a CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (i.e. a GENI format request is answered with a GENI format manifest). The aggregate also promises to send a correctly formatted Manifest RSpec in response to a ListResources call which supplies a valid geni_slice_urn option and an geni_rspec_version option containing that supported type/version.

New GetVersion required attributes

Return the version of the GENI Aggregate Manager API and RSpecs supported by this aggregate.

struct GetVersion()

The result is an XML-RPC struct with at least the following members:

{
  int geni_api;
  array geni_request_rspec_versions of {
    string type;
    string version;
    string schema;
    string namespace;
    array extensions of string;
  };
  array geni_ad_rspec_versions of {
    string type;
    string version;
    string schema;
    string namespace;
    array extensions of string;
  };
}
geni_api
An integer indicating the revision of the Aggregate Manager API that an aggregate supports. This document describes API version 2 (two).
geni_request_rspec_versions
An array of data structures indicating the RSpec types accepted by this AM in a request.
geni_ad_rspec_versions
An array of data structures indicating what types of RSpec advertisements may be produced by this AM in ListResources.

Elements used within geni_request_rspec_versions and geni_ad_rspec_versions:

type, version
Two case-insensitive strings which together comprise the type of RSpec. The RSpec type should be one of "geni", "protogeni", "sfa", "orca", "openflow", or "orbit" and version should be a type-specific version identifier as specified by the appropriate control framework. The "geni" type is reserved for GENI standard format RSpecs, following the schemas hosted at www.geni.net.
schema
An optional URL pointing to a schema which can be used to verify the given type of RSpec.
namespace
An optional XML namespace which the RSpec of the given type belongs to.
extensions
An array of aggregate-specific strings denoting which extensions are supported. In the case of ProtoGENI, these are XML namespaces which denote the extension as a whole.

Implementations can add additional members to the struct as desired. Implementations should choose an appropriate prefix to avoid conflicts, but avoid geni_ which should be reserved for common options.

This operation is similar to ProtoGENI's GetVersion operation. The SFA specification does not include this operation other than through this method.

New ListResources Option

ListResources will take an additional required option, geni_rspec_version, allowing a user to request an Advertisement or Manifest Rspec in a particular format. This struct must contain a type and version matching one of this Aggregate's advertised geni_ad_rspec_versions.

Specifics:

Return information about available resources or resources allocated to a slice.

string ListResources(string credentials[], struct options)

....

options
An XML-RPC struct containing members indicating the set of resources the caller is interested in or the format of the result. In addition to the members specified below, callers can pass additional members that specific aggregate manager implementations might honor. Implementations should choose an appropriate prefix to avoid conflicts, but avoid geni_ which should be reserved for common options.

The following members are available for use in the options parameter. All aggregate managers are required to implement these options.

{
  boolean geni_available;
  boolean geni_compressed;
  string geni_slice_urn;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

....

geni_rspec_version
An XML-RPC struct indicating the type and version of Advertisement or Manifest RSpec to return. The struct contains 2 members, type and version. type and version are case-insensitive strings, matching those in geni_ad_rspec_versions as returned by GetVersion at this aggregate. This option is required, and aggregates are expected to return a geni_code 1 ('Bad Arguments') if it is missing.

The returned Rspec will be an Advertisement RSpec when no geni_slice_urn option is supplied. When a valid geni_slice_urn option is supplied, the returned RSpec will be a Manifest RSpec of the type corresponding to geni_rspec_version, but in manifest format.

If the aggregate cannot support the requested type/version (that pair is not listed in geni_ad_rspec_versions), then the aggregate returns an error 13 (UNSUPPORTED).

New CreateSliver behavior

If an Aggregate advertises a type/version pair in its geni_request_rspec_versions as returned by GetVersion, then it promises to correctly honor a CreateSliver call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (ie a GENI v3 standard request is answered with a GENI v3 manifest).

Change Set B

Part 1: Additional options argument

Adding support for additional functionality in the AM API is currently difficult. Many possible areas of innovation are variations on existing functions. This change adds flexibility in arguments to support such innovation.

Under this proposal, each method will take an argument which is a set of name-value pairs (an XML-RPC struct, aka a property list). This argument is required for all methods except GetVersion (where it is optional), but may be empty (non NULL but containing no name/value pairs). Aggregates may support new entries in this 'options' argument to allow new non-standardized functionality.

Specifically, this change adds a final struct options argument to each of GetVersion, CreateSliver, SliverStatus, RenewSliver, DeleteSliver, and Shutdown. ListResources already has that argument. The argument is optional for GetVersion, but required for all other functions.

Aggregates are compliant with this API change by accepting this argument; only for ListResources are they required to handle any specific options. Similarly, clients are required to supply this argument to talk to compatible aggregates, but are only required to supply any particular options for ListResources.

In GetVersion, this argument remains optional. Clients that only talk AM API v1 will get an error invoking most functions, when they leave off the options argument. Experimenters can then call GetVersion (without the options argument). AM API v2 compliant Aggregates shall include the geni_api argument as a top-level entry in the return struct, specifying 2 to indicate to clients that this AM speaks version 2 of the AM API. This allows experimenters to understand that they need to upgrade their client, or might instruct a clever client tool to automatically switch to version 2 syntax.

Aggregates should not require any new options to any method - they should always have a reasonable default for any such option. Clients must always be able to work with any aggregate by simply supplying the options required by this API.

Aggregates are encouraged to document any new options which they accept in any method, to bootstrap coordination with clients, and provide documentation for human experimenters. This API does not specify how aggregates provide that documentation. One way to provide partial documentation, is to implement XML-RPC introspection. Through the use of method help, aggregates can provide human readable text describing options. Alternatively or additionally, aggregates may document options as part of their return from GetVersion. This allows clients to either avoid communicating with an aggregate for which the client does not understand how to provide those options, or to tailor the client's request to provide those extra options.

Part 2: Richer return values

In AM API v1, method failures come back sometimes as XML-RPC Faults, sometimes as False, and is occasionally inconsistent across aggregates. Failures typically do not indicate how the Experimenter should modify their request for it to succeed, or if this is a server error. This proposed change expands and formalizes the return structures, to support semantically richer returns, allowing Experimenters better insight into both successes and failures, and how to respond.

Allowing aggregates to return more information, on both errors and success, will allow for a richer client-server communication. It would also allow aggregates to give clients hints on how to use successful returns, or otherwise innovate within the bounds of the AM API.

This change will modify all methods to return an XML-RPC struct (aka property list) on any application layer success, failure, and even on an error or for most exceptions. Note that a malformed XML-RPC request should still raise an XML-RPC Fault, and other Faults dictated by the XML-RPC specification should still be raised. This struct will contain the return value from the previous revision of the AM API as an entry, as modified by other changes adopted for API version 2. This struct will have 3 defined entries, and aggregates are free to include other entries to give more information to clients.

The three required entries in the return structure are code, value, and output.

In summary however:

code {
  geni_code: XML-RPC integer, registered in an XML document off the GENI AM API page
  am_type: Optional: XML-RPC string, case insensitive, types in a registry (XML document) linked off the GENI AM API page
  am_code: Optional: XML-RPC integer
}

Integers above are allowed to be negative.

am_type and am_code are optional: AMs need not supply them and clients should not have to use them to understand the nature of the response. They serve to further specify the nature of the return as indicated by geni_code

Success is indicated by geni_code value of 0

Error codes and AM types are in an XML document off the GENI AM API page

  • value: On success, this is required. Optional on failure or error. Object representing the successful return value. This will be the object previously returned by the function (for example the manifest RSpec for CreateSliver, or the struct for SliverStatus). The value is not defined on error, though aggregates are free to use it.
    • For GetVersion, the value is an XML-RPC struct
    • For ListResources, the value is an RSpec
    • For CreateSliver, the value is an RSpec
    • For RenewSliver, the value is a boolean
    • For DeleteSliver, the value is a boolean
    • For SliverStatus, the value is an XML-RPC struct
    • For Shutdown, the value is a boolean
  • output: On failure or error, this is required. Optional on success. This is a String with a human readable message explaining the result. Specifically, this might include an error string, a stacktrace, or other useful messages to help the Experimenter resolve or report the failure or error. It is not defined on success, though aggregates are free to use it.

Aggregates are encouraged to use code values and output messages that help experimenters and tools distinguish between bad input, other experimenter error, temporary server errors, or server bugs.

GENI standard error codes will be documented on the GENI AM API Wiki page.

Aggregates are similarly encouraged to provide hints on how to fix bad requests using the value entry to experimenters on error or failures. For example, a failed RenewSliver call that failed because you are not allowed to renew your sliver that far in the future, might return a new date string in the value field that would be allowed. Similarly, a failed CreateSliver call might return a modified request RSpec in the value field.

Aggregates should avoid raising an error (XML-RPC Fault) for application layer errors or any other cases where the XML-RPC specification does not require a Fault, but rather should attempt to return this struct, providing any error messages and stack traces in the output field or other additional fields. Where the XML-RPC standard prescribes specific Faults, those will remain XML-RPC Faults.

Aggregates are free to add additional return values to support aggregate or resource specific functionality, or to innovate within the bounds of the AM API. Aggregates are encouraged to document any such new return values which they return, to bootstrap coordination with clients, and provide documentation for human experimenters. One way to provide partial documentation, is to implement XML-RPC introspection. Through the use of method help, aggregates can provide human readable text describing return values. Alternatively or additionally, aggregates may document return values as part of their return from GetVersion. We have not specified the format for advertising those extra return values in GetVersion.

For comparison, Orca functions return property lists internally. The ProtoGENI CMV2 API returns a struct with exactly these 3 values. ProtoGENI however uses a different range of return codes, and largely does not define the value slot on errors.

For example, SliverStatus could return on success:

{
  code: {
            geni_code: 0
        },
  value: 
        {
            geni_urn: <sliver URN>
            geni_status: ready
            geni_resources: [ { geni_urn: <resource URN>
                                geni_status: ready
                                geni_error: ''},
                              { geni_urn: <resource URN>
                                geni_status: ready
                                geni_error: ''}
                            ]
        },
  output: <none>
}

On failure, DeleteSliver might return:

{
  code: {
        geni_code: 12
        }
  value: False
  output: 'No such slice here'
}

(That code and output are merely examples.)

An exception: At the top level, GetVersion adds a required entry: 'geni_api'=2. This allows v1 clients to determine that they are indeed talking to a GENI AM, but since the version is 2, that is why other function calls will fail with an XML-RPC Fault.

Supporting multiple API versions

Aggregates are free to support multiple versions of the API. They do so by providing different URLs for each version of the API that they support. Aggregates should have a 'default' URL (the one typically advertised). That version runs whichever version of the API the server chooses (could be the latest, could be something else.) Each server implementing the API must include all methods, including GetVersion.

This change modifies GetVersion to include a new required member:

geni_api_versions: an XML-RPC struct containing 1+ entries of:
  Name: Integer - supported GENI AM API version
  Value: String - URL to the XML-RPC server implementing that version of the GENI AM API

For example

geni_api_versions: {
  1: <URL>,
  2: <Local URL>,
   ...
}

Where the entries indicate versions of the API supported, and URLs are absolute URLs where that version of the API is supported. (As per Change Set B above, this new member is a sub-structure to the value member.)

Changes Summary: New Method Signatures

When taken together, the new method signatures would be:

struct GetVersion(options=None)
Success Return:
{ 
  geni_api = 2
  code = {
       geni_code = 0
         }
  value
      {
        geni_api=2,
        geni_api_versions {
             2: <local URL>,
             <other versions and URLs as supported>
             }
        geni_request_rspec_versions [{
          type=GENI
          version=3
          schema=...
          namespace=....
          extensions []},
          <other version structs>]
        geni_ad_rspec_versions [{type, version, schema, namespace, extensions},]
      }
  output = <None>
}
struct ListResources(string credentials[], struct options)
Success Return:
{
   code= {geni_code=0}
   value= <GENI v3 Ad or Manifest RSpec string>
   output = <None>
}
struct CreateSliver(string slice_urn,
                    string credentials[],
                    <GENIv3 request RSpec schema compliant XML string> rspec,
                    struct users[],
  		    struct options)
Success Return:
{
   code={geni_code=0}
   value= <GENI v3 Manifest RSpec string>
   output = <None>
}
struct DeleteSliver(string slice_urn, string credentials[], struct options)
Success Return:
{
   code={geni_code=0}
   value= boolean
   output = <None>
}
struct SliverStatus(string slice_urn, string credentials[], struct options)
Success Return:
{
   code={geni_code=0}
   value= struct (as defined in v1)
   output = <None>
}
struct RenewSliver(string slice_urn,
                    string credentials[],
                    string expiration_time, struct options)
Success Return:
{
   code={geni_code=0}
   value= boolean
   output = <None>
}
struct Shutdown(string slice_urn, string credentials[], struct options)
Success Return:
{
   code={geni_code=0}
   value= boolean
   output = <None>
}