wiki:GAPI_AM_API_V3

Version 22 (modified by Aaron Helsinger, 12 years ago) (diff)

--

GENI Aggregate Manager API Version 3

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers.

This page documents the GENI Aggregate Manager API Version 3.

See Also:

For a summary of the Aggregate Manager API common concepts, arguments, and returns, see this sub page.

Versions

  • Latest: 3 - Documented here. Changes since version 2 include:
    • All new method names
    • Addressable slivers: the ability to add or delete resources from an existing slice reservation at a single aggregate
    • New sliver allocation states & methods.
      • Including a cheap 'allocated' state, that tools can use to negotiate resource reservations, and coordinate resource reservations among multiple aggregates (necessary for VLAN stitching)
    • Operational states, and a way for aggregates to advertise resource specific operational states and methods in the Ad RSpec.
    • Generalized credential argument, allowing aggregates to accept and tools to supply credentials of alternate types
    • Return sliver expiration in most method returns
    • Specify SSH node login information in a standard RSpec extension
    • Standardize certificate contents, specifically including a UUID and email address with particular semantics
    • Standardize slice and user name character and length restrictions
    • Split ListResources in two: ListResources for Ad RSpecs, and Describe for a slice' manifest RSpec
  • Previous: 2 - Documented on the v2 page. Changes since v1 include:
    • Includes Change Sets A and B from the v2 Deltas page
    • Specify that GENI RSpecs are XML documents following standard schemas published on geni.net and documented on the ProtoGENI wiki
    • Include additional options in GetVersion and ListResources to allow aggregates to specify the RSpec formats they support
    • Add an 'options' argument, an XML-RPC struct, to all methods with no required values. This allows Aggregates to innovate or support resource-specific functionality.
    • Modify all methods to return an XML-RPC struct consisting of at least (1) a return 'code', (2) the 'value' specified by v1 of the API, and (3) a human-readable 'output' on errors. Aggregates are free to innovate by adding additional return values.
  • Original: Version 1
  • Draft changes for version 4+
    • Covers changes proposed for future versions of this API. Documented changes include:
    • Add an UpdateSliver method to add/remove/modify resources from a slice at an aggregate
    • Others to be proposed

API Overview

The GENI Aggregate Manager API is the control plane interface by which experimenter tools discover, reserve and control resources at resource providers. It does not include resource specific interactions, application level interactions, or monitoring and management functions.

API Protocols and Data Structures

GENI specifies that the AM API is provided via XML-RPC over an SSL connection. Aggregate Managers shall require client side GENI certificates to authenticate users, accepting only certificates that comply with the adopted GENI certificates standards. The GENI AM API therefore assumes that users have already been authenticated, and that the aggregate manager has available the client certificate to identify the user.

Clients are authorized to take actions at aggregates using GENI credentials. To that end, all methods that require authorization take an argument credentials. In particular, operations on a single GENI slice require a credential (set) that authorizes the client whose certificate was used to authenticate to operate on the slice named by a urn argument to the method or on the slice containing the slivers named by a urns argument.

The primary data structure used within this API is a resource specification, known as an RSpec. These XML documents follow a specific set of schemas. They are used by aggregates to list and describe local resources (advertisement RSpecs), by experimenters to describe desired resources (request RSpecs), and then by aggregates to describe reserved resources (manifest RSpecs). For more information on RSpecs, see the RSpecs section on the detail page.

Using the GENI AM API

Clients (experimenters via their tools) use the AM API to discover resources (ListResources), request resources (Allocate), provision reserved resources (Provision), start resources (PerformOperationalAction), check the status of resources as they are started (Status), extend their reservation (Renew), and then return the resources when done (Delete). Client tools may use GetVersion to ensure aggregates speak a compatible version of the AM API and known formats for RSpecs. Administrators may call Shutdown to stop the resources of a slice at this aggregate, perhaps if that slice is misbehaving.

ListResources returns to the client an advertisement RSpec - a detailed listing of the resources available at that aggregate. From this information, the experimenter may determine which resources to reserve for their use. The RSpec should also have enough information to help the experimenter set the initial configuration for their resources.

Once the experimenter has selected the resources they want and how to configure them, they produce a request RSpec, detailing the resources they want and how they should be configured. They separately contact their slice authority to obtain a slice credential (or set of credentials), granting them rights to reserve resources for that slice. The experimenter then uses their tool and calls Allocate on this API, passing in both the slice credential and the request RSpec. The aggregate then attempts to satisfy the experimenter's resource request. If the aggregate can satisfy the request, the aggregate reserves the resources for the experimenter. The resources have not been provisioned yet, giving the experimenter a chance to verify the reservation, or check for corresponding resource availability in another aggregate. If it is acceptable, the experimenter (via their tool) calls Provision to set up the resources. The aggregate then starts the process of instantiating the resources and configuring them as requested in the request RSpec. Once that process has started, the Provision call returns with a manifest RSpec, listing the resources as reserved and initially configured for the experimenter.

The experimenter tool can then poll the aggregate manager to watch as the resources are configured and become ready for use, by calling Status, looking for an operational state other than geni_pending_allocation. The actual operational state that the sliver will change to depends on the sliver and aggregate type. Operational states are sliver type and aggregate specific, and defined in the aggregate's advertisement RSpec. In many cases, the aggregate indicates that the sliver is ready with a geni_operational_state value of geni_notready. Once the resources are ready for use, the experimenter tool will typically call PerformOperationalAction(geni_start) to start the resources (e.g. boot a machine). The experimenter (or their tool) will also call Renew to request that their reservation lasts as long as they require the resources for. When the experimenter is done using the resources, they call Delete to end their reservation. The aggregate then stops and clears the resources, freeing them for use by other clients.

Typical client work flow:

  1. <Experimenter gets a GENI certificate and slice credential>
  2. GetVersion(): learn RSpec formats supported at this aggregate
  3. ListResources(<user credential>, options): get Ad RSpec describing available resources
  4. <Experimenter constructs a request RSpec>
  5. Allocate(<slice URN>, <slice credential>, <request RSpec>, {}):
    • Aggregate reserves resources
    • Return is a manifest RSpec describing the reserved resources
    • Optionally Delete some slivers, if you made a mistake, or don't like what the aggregate picked for you.
  6. Provision(<slice URN or sliver URNs>, <slice credential>, <request RSpec>, <users struct>, {}):
    • Aggregate instantiates resources
    • Return is a manifest RSpec describing the reserved resources, plus any instantiation-specific configuration information
  7. Status(<slice URN or sliver URNs>, <slice credential>, {}) to check that resources are provisioned (e.g. look for operational state geni_notready.
  8. PerformOperationalAction(<slice URN>, <slice credential>, "geni_start", {}):
    • Aggregate starts resources
  9. Status(<slice URN or sliver URNs>, <slice credential>, {}) to check that resources have started
  10. Renew(<slice URN or sliver URNs>, <slice credential>, <newtime>, {}) to extend reservation
  11. <Experimenter uses resources>
  12. Delete(<slice URN or sliver URNs>, <slice credential>, {}) when done

Changes from AM API v2

This version of the AM API includes substantial changes since version 2 of the AM API. For experimenters, a few things are worth noting:

  • The old CreateSliver operation has now been broken into 3 steps:
    • Allocate to reserve the resources
    • Provision to instantiate the resources, which may take time to complete
    • PerformOperationalAction(geni_start) to start (e.g. boot) the resources, which also may take time to complete
  • Use the new intermediate geni_allocated state after Allocate to coordinate reservations across aggregates, e.g. to ensure another aggregate can give you nodes to be the other end of a requested link.
  • Multiple methods have been renamed, typically by removing the Sliver term from method names.
  • Sliver expiration is available in the return from several other methods, like Provision
  • You no longer use ListResources to see the contents of your slice - use Describe instead. ListResources is only for the AM's Ad RSpec.
  • Experimenters can select when to start or stop resources, e.g. when to boot a VM. Consult the operational state machine in the AM's Ad RSpec, and use PerformOperationalAction.
  • SSH login names and keys should be available in manifest RSpecs in a standard format.
  • Slice name restrictions have been codified and standardized.
    • Slice names are <=19 characters, only alphanumeric plus hyphen (no hyphen in first character): '^[a-zA-Z0-9][-a-zA-Z0-9]\{0,18\}$'

Tool developers should also be aware:

  • The credentials argument to methods is now a struct, including a type and version for each credential. AMs should advertise which credential types they accept. SAs should advertise which type they provide.
  • Aggregates may have their own operational states and actions. The advertisement RSpec should define these, probably by sliver_type.

GetVersion

Query static configuration information about this aggregate manager implementation, such as API and RSpec versions supported.

struct GetVersion([optional: struct options])

The options argument may be omitted entirely by clients.

The result is an XML-RPC struct with at least the following members:

{
  int geni_api;
  struct code = {
       int geni_code;
       [optional: string am_type;]
       [optional: int am_code;]
         }
  struct value
      {
        int geni_api;
        struct geni_api_versions {
             URL <this API version #>; # value is a URL, name is a number
             [optional: other supported API versions and the URLs where they run]
        }
        array geni_request_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_ad_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_credential_types of {
             string geni_type <case insensitive>;
             string geni_version <containing an integer>;
       };
       [optional: boolean geni_single_allocation; <optional only if using default of 0>]
       [optional: string geni_allocate; <case insensitive. optional only if using default of geni_single>]
      }
  string output;
}

Return Summary

As with all AM API methods, the return is an XML-RPC struct. For GetVersion, it includes:

  • The standard AM API code, value, and output entries. For details, see the Common Concepts page
  • geni_api: integer version of this API (3). Repeated here for backwards compatibility.

GetVersion value required return members:

For details on GetVersion arguments, returns, and semantics, and a sample minimal return, see the Details page.

ListResources

Return a listing and description of available resources at this aggregate. The resource listing and description provides sufficient information for clients to select among available resources. These listings are known as RSpecs.

struct ListResources(struct credentials[], struct options)

The following members are available for use in the options parameter. All aggregate managers are required to honor these options.:

{
  boolean geni_available <optional>;
  boolean geni_compressed <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, except geni_rspec_version which is required.

Return: On success, the value field of the return struct will contain: A geni.rspec advertisment RSpec. For details on RSpecs, see the Common Concepts page.

Note that this is only part of what ListResources did in earlier versions of this API. For the what-is-in-my-slice functionality, see Describe().

This operation is similar to ProtoGENI's DiscoverResources operation and to the SFA's GetResources operation (sec. 6.2.4).

For details on the options (which are required), see the Details page.

Describe

Retrieve a manifest RSpec describing the resources contained by the entities named, e.g. a single slice or a set of the slivers in a slice. This listing and description should be sufficient to allow experimenters to use the resources.

Note this functionality was previously part of ListResources when supplied a slice URN.

struct Describe(string urns[], struct credentials[], struct options[])

The following members are available for use in the options parameter. All aggregate managers are required to honor these options.:

{
  boolean geni_compressed <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, exception geni_rspec_version which is required.

For details on the options, see the Details page.

This method is part of what ListResources used to do, and is similar to ProtoGENI Resolve.

Return: On success, the value field of the return struct will contain at struct:

{
   geni_rspec: <geni.rspec, Manifest >
   geni_urn: <string slice urn of the containing slice>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - allocated or ?? >,
                  geni_operational_status: <string sliver operational state>,
                  geni_error: <optional string, may be omitted entirely, explaining any failure for a sliver>
               },
               ...
         ]
}

Note that the manifest RSpec for allocated slivers may contain less detail than for provisioned slivers. Aggregates are expected to combine the manifests of all requested slivers into a single manifest RSpec. Note that a manifest returned here for only some of the slivers in a slice at this aggregate, may contain references to resources not described in this manifest (they are in other slivers). As a result, such manifests may not be directly usable as a subsequent request.

If a slice urn is supplied and there are no slivers in the given slice at this aggregate, then geni_rspec shall be a valid manifest RSpec, containing 0 node or link elements - that is, specifying no resources. geni_slivers may be an empty array, or may be an array of previous slivers that have since been deleted or expired.

Manifests are not necessarily static. In general, the manifest of a given sliver should be static once it has reached the operational state geni_ready (e.g., fully booted). However, this API does not require that to be true.

See the Common Concepts page about the credentials, rspec, and urns arguments, and also for details on allocation states, operational states, and datetime values. See GeniApiIdentifiers for details on URN structure.

Allocate

Allocate resources as described in a request RSpec argument to a slice with the named URN. On success, one or more slivers are allocated, containing resources satisfying the request, and assigned to the given slice. This method returns a listing and description of the resources reserved for the slice by this operation, in the form of a manifest RSpec. Allocated slivers are held for an aggregate-determined period. Clients must Renew or Provision slivers before the expiration time (given in the return struct), or the aggregate will automatically Delete them. Aggregates should implement Allocate() as quick, cheap, and not impacting provisioned resources, such that it can be readily undone. Allocate is an all or nothing request: if the aggregate cannot completely satisfy the request RSpec, it should fail the request entirely.

This method is the first part of what was previously known as CreateSliver.

struct Allocate(string slice_urn,
                    struct credentials[],
                    geni.rspec rspec,
                    struct options)

This is the first part of what CreateSliver used to do. The second part is now done by Provision, and the final part is done by PerformOperationalAction.

This operation is similar to ProtoGENI's GetTicket operation.

Return: On success, the value field of the return struct will contain a struct:

{
 geni_rspec: <geni.rspec manifest>,
 geni_slivers: [
        {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from Status>,
                  geni_allocation_status: <string sliver state - allocated or ?? >
        },
        ...
    ]
}

The manifest is a manifest of only newly allocated slivers.

Arguments:

  • slice_urn: The URN of the slice to which the resources specified in rspec will be allocated. For details on GENI AM API URN identifiers, see the GENI wiki page.
  • rspec: An RSpec matching the GENI standard request RSpec schema containing the resources that the caller is requesting for allocation to the slice specified in slice_urn. See the Common Concepts page.
  • credentials: The standard authorization argument. See the Common Concepts page.

Options include:

{
  [optional geni_end_time: <dateTime.rfc3339: requested expiration of all new slivers, may be ignored by aggregates>]
}

See the Common Concepts page for details on geni_end_time, RSpecs, expiration dateTime, and allocation states.

Renew

Request that the named slivers be renewed, with their expiration extended. If possible, the aggregate should extend the slivers to the requested expiration time, or to a sooner time if policy limits apply. This method applies to slivers that are geni_allocated or to slivers that are geni_provisioned, though different policies may apply to slivers in the different states, resulting in much shorter max expiration times for geni_allocated slivers.

struct Renew(string urns[],
                    struct credentials[],
                    dateTime.rfc3339 expiration_time, 
                    struct options)

This operation used to be called RenewSliver. Use Renew(<slice_urn>) to get the equivalent functionality.

Options include geni_best_effort, specifying whether the client prefers all included slivers to be renewed or none, or wants a partial success if possible. See the Common Concepts page under geni_best_effort and Operations on Individual Slivers.

When Renew is called with geni_best_effort false, the entire method will fail (return non-zero code) if any requested sliver cannot be renewed to the requested time, and all slivers will keep their original expiration time. When Renew is called with geni_best_effort true, some sliver may fail to be renewed. In this case, the allocation state and expiration times do not change. geni_error may optionally be returned by the aggregate to explain this failure.

Arguments: expiration_time: The date-time string in RFC 3339 format in UTC when the reservation(s) should be extended until. See the Common Concepts page.

See the Common Concepts page for details on the urns and credentials arguments.

Return: On success, the value field of the return struct will contain a list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_operational_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
   geni_error: <optional string, may be omitted entirely, explaining any renewal failure for this sliver>
  },
  ...
]

Attempting to Renew an unknown sliver will result in an error (non zero geni_code). Attempting to Renew a previously deleted or expired sliver may result in an error - the aggregate may have forgotten about it entirely. Attempting to Renew a slice with no current slivers at this aggregate may return an empty list of slivers, or may return a list of previous slivers that have since been deleted.

It is legal to attempt to renew a sliver to a sooner expiration time than the sliver was previously due to expire. Not all aggregates will support this however.

See the Common Concepts page for details on allocation and operational states, operations on individual slivers, geni_best_effort, sliver expiration times, urns, and credentials.

Provision

Request that the named geni_allocated slivers be made geni_provisioned, instantiating or otherwise realizing the resources, such that they have a valid geni_operational_status and may possibly be made geni_ready for experimenter use. This operation is synchronous, but may start a longer process, such as creating and imaging a virtual machine.

struct Provision(string urns[], struct credentials[],  
                                   struct options)

This operation is part of what CreateSliver used to do. The first part of what CreateSliver did is now in Allocate(). Note that resources are not necessarily ready for experimenter use after the work that this function initiates finally completes. Consult the geni_operational_status, and the advertised operational state machine. Consider calling PerformOperationalAction, e.g. with the command name geni_start.

The operation is similar to ProtoGENI's RedeemTicket method.

Options include:

  • geni_best_effort: Do all slivers fail if any fail?
  • geni_end_time: Requested sliver expiration time
  • struct geni_users[]: Resource login information. See the Common Concepts page.

Return: On success, the value field of the return struct will contain a struct:

  geni_rspec: <geni.rspec, RSpec manifest>,
  geni_slivers: 
  [
    {
     geni_sliver_urn: <string>,
     geni_allocation_status: <string>,
     geni_operational_status: <string>,
     geni_expires <dateTime.rfc3339 when the sliver expires from its current state>,
     geni_error: <optional string, may be omitted entirely, explaining any failure to Provision this sliver>
    },
    ...
  ],

The returned manifest covers only newly provisioned slivers. Use Describe to get a manifest of all provisioned slivers. When geni_best_effort is true, all requested slivers are returned, but some slivers may have failed: geni_allocation_status will remain geni_allocated. Check geni_error for details.

Note that previously allocated slivers may have expired (been deleted) by the time you call Provision.

See the Common Concepts page for details on allocation and operational states, operations on individual slivers, geni_best_effort, sliver expiration times, urns, and credentials, and geni_users.

Status

Get the status of a sliver or slivers belonging to a single slice at the given aggregate. Status may include other dynamic reservation or instantiation information as required by the resource type and aggregate. This method is used to provide updates on the state of the resources after the completion of Provision, which began to asynchronously provision the resources. This should be relatively dynamic data, not descriptive data as returned in the manifest RSpec.

struct Status(string urns[], struct credentials[], struct options)

This operation used to be called SliverStatus in earlier versions of the AM API. geni_slivers has replaced geni_resources and geni_sliver_urn replaces geni_urn. geni_status is replaced with 2 fields, geni_allocation_status and geni_operational_status.

This method has no required options.

Return: On success, the value field of the return struct will contain a struct:

{
  geni_urn: <slice URN>
  geni_slivers: [ 
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                     },
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                      }
                  ]
}

Note that aggregates may return other information, such as details on sliver contents, etc.

Attempting to get status for an unknown sliver will result in an error. Attempting to get status for a previously deleted sliver may result in an error - the aggregate may have forgotten about it entirely. Attempting to get status for a slice with no current slivers at this aggregate may return an empty list for geni_slivers, or may return a list of previous slivers that have since been deleted.

See the Common Concepts page for details on allocation and operational states, sliver expiration times, urns, and credentials.

PerformOperationalAction

Perform the named operational action on the named slivers, possibly changing the geni_operational_status of the named slivers. E.G. 'start' a VM. For valid operations and expected states, consult the state diagram advertised in the aggregate's advertisement RSpec.

struct PerformOperationalAction (string urns[], struct credentials[], string action, 
                                                    struct options={})

This operation is similar to ProtoGENI functions like StartSliver, StopSliver, and RestartSliver in the PG CMv2 API.

Aggregate Managers SHOULD return an error code of 13 (UNSUPPORTED) if they do not support a given action for a given resource. An AM SHOULD constrain actions based on the current operational state of the resource. This is a fast synchronous operation, and MAY start long-running sliver transitions whose status can be queried using Status. This method should only be called, and is only valid, when the sliver is fully allocated (allocation state is not geni_pending_allocation).

While the action argument may be aggregate and sliver type specific (none are required for all aggregates and sliver types), this API does define three common actions that AMs should support if possible: geni_start, geni_stop, and geni_restart. Calling PerformOperationalAction with the actiongeni_start corresponds to the final part of what CreateSliver did in AM API v2. For more information, see the Common Concepts page section on operational actions.

Options include: geni_best_effort. Default is false (action applies to all slivers equally or none).

Return: On success, the value field of the return struct will contain a list of structs:

[ {
        geni_sliver_urn : <string>,
        geni_allocation_status: <string, eg provisioned>,
        geni_operational_status : <string>,
        geni_expires: <dateTime.rfc3339 of individual sliver expiration>,
        [optional: 'geni_resource_status' : string],
        [optional: 'geni_error': string explanation of operation failure for this sliver]
        }, 
        ... 
]
;

See the Common Concepts page for details on allocation and operational states, operational methods, operations on individual slivers, geni_best_effort, sliver expiration times, urns, and credentials.

Delete

Delete the named slivers, making them geni_unallocated. Resources are stopped if necessary, and both de-provisioned and de-allocated. No further AM API operations may be performed on slivers that have been deleted.

struct Delete(string urns[], struct credentials[], struct options)

This operation used to be called DeleteSliver in earlier versions of this API. To get the functionality of DeleteSliver, call Delete with the slice URN.

This operation is similar to ProtoGENI's DeleteSliver operation and to the SFA's DeleteSlice operation (sec. 6.2.3).

Options include: geni_best_effort

Return: On success, the value field of the return struct will contain a list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
   [optional: 'geni_error': string indicating any AM failure deleting the sliver.]
  },
  ...
]

Note that aggregates will automatically delete slivers whose expiration time is reached.

Attempting to delete an unknown sliver will result in an error. Attempting to delete a previously deleted sliver may result in an error - the aggregate may have forgotten about it entirely. Attempting to delete a slice with no current slivers at this aggregate may return an empty list of slivers, or may return a list of previous slivers that have since been deleted.

See the Common Concepts page for details on allocation and operational states, operations on individual slivers, geni_best_effort, sliver expiration times, urns, and credentials.

Shutdown

Perform an emergency shutdown on the slivers in the given slice at this aggregate. Resources should be taken offline, such that experimenter access (on both the control and data plane) is cut off. No further actions on the slivers in the given slice should be possible at this aggregate, until an un-specified operator action restores the slice's slivers. This operation is intended for operator use. The slivers are shut down but remains available for further forensics.

struct Shutdown(string slice_urn, struct credentials[], struct options)

This operation is similar to ProtoGENI's Shutdown operation. The SFA specification does not include this operation.

No options are required.

Return: On success, the value field of the return struct will contain: XML-RPC boolean, indicating whether the resources associated with this reservation were successfully shut down to a state suitable for forensics. Return should be true, or else an error should be returned.

If the given slice has no resources locally, or was previously Shutdown, return shall be true, indicating the slice is shut down.

See the Common Concepts page for details on the credentials argument.