[[PageOutline]] = GENI Aggregate Manager API Draft Revisions Method Signatures Summary = This page documents the DRAFT GENI Aggregate Manager API, if all proposed revisions are adopted. This includes proposals that have already been adopted, and others still under discussion. The current officially adopted version of the API is '''2''' and is documented on [wiki:GAPI_AM_API the main API page]. The current adopted or proposed changes for the AM API, whose impacts are reflected here, are documented on [wiki:AaronHelsinger/GAPI_AM_API_DRAFT the AM API Draft wiki page]. For a summary of the Aggregate Manager API common concepts, arguments, and returns, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts this sub page]. == API Overview == The GENI Aggregate Manager API is the control plane interface by which experimenters discover, reserve and control resources at resource providers. It does not include resource specific interactions, application level interactions, or monitoring and management functions. === API Protocols and Data Structures === GENI specifies that the AM API is provided via [http://www.xmlrpc.com/spec XML-RPC] over an SSL connection. Aggregate Managers shall require client side [wiki:GeniApiCertificates GENI certificates] to authenticate users, accepting only certificates that comply with the adopted [wiki:GeniApiCertificates GENI certificates] standards. The GENI AM API therefore assumes that users have already been authenticated, and that the aggregate manager has available the client certificate to identify the user. Clients are authorized to take actions at aggregates using [wiki:GeniApiCredentials GENI credentials]. To that end, all methods that require authorization take an argument {{{credentials}}}. In particular, operations on a single GENI slice will require a credential (set) that authorizes the client whose certificate was used to authenticate to operate on the slice named by a {{{urn}}} argument to the method or on the slice containing the slivers named by a {{{urns}}} argument. The primary data structure used within this API is a resource specification, known as an [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#RSpecdatatype RSpec]. These XML documents follow a specific set of schemas. They are used by aggregates to list and describe local resources (advertisement RSpecs), by experimenters to describe desired resources (request RSpecs), and then by aggregates to describe reserved resources (manifest RSpecs). For more information on RSpecs, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#RSpecdatatype the RSpecs section on the detail page]. === Using the GENI AM API === Clients (experimenters) use the AM API to discover resources (`ListResources`), request resources (`Allocate`), provision reserved resources (`Provision`), start resources (`PerformOperationalAction`), check the status of resources as they are started (`Status`), extend their reservation (`Renew`), and then return the resources when done (`Delete`). Client tools may use `GetVersion` to ensure aggregates speak a compatible version of the AM API and known formats for RSpecs. Administrators may call `Shutdown` to stop the resources of a slice at this aggregate, perhaps if that slice is misbehaving. `ListResources` returns to the client an advertisement RSpec - a detailed listing of the resources available at that aggregate. From this information, the experimenter may determine which resources to reserve for their use. The RSpec should also have enough information to help the experimenter set the initial configuration for their resources. Once the experimenter has selected the resources they want and how to configure them, they produce a request RSpec, detailing the resources they want and how they should be configured. They separately contact their slice authority to obtain a slice credential (or set of credentials), granting them rights to reserve resources for that slice. The experimenter then calls `Allocate` on this API, passing in both the slice credential and the request RSpec. The aggregate then attempts to satisfy the experimenter's resource request. If the aggregate can satisfy the request, the aggregate reserves the resources for the experimenter. The resources have not been provisioned yet, giving the experimenter a chance to verify the reservation, or check for corresponding resource availability in another aggregate. If it is acceptable, the experimenter calls `Provision` to set up the resources. The aggregate then starts the process of instantiating the resources and configuring them as requested in the request RSpec. Once that process has started, the `Provision` call returns with a manifest RSpec, listing the resources as reserved and initially configured for the experimenter. The experimenter can then poll the aggregate manager to watch as the resources are configured and become ready for use, by calling `Status`, looking for an operational state other than `geni_pending_allocation. A given aggregate and sliver type may use a different set of states once provisioning is complete, and further operational actions are possible - see the AM's Ad RSpec. In many cases, this indication comes with a `geni_operational_state` value of `geni_notready`. Once the resources are ready for use, the experimenter will typically call `!PerformOperationalAction(geni_start)` to start the resources (e.g. boot a machine). The experimenter will also call `Renew` to request that their reservation lasts as long as they require the resources for. When the experimenter is done using the resources, they call `Delete` to end their reservation. The aggregate then stops and clears the resources, freeing them for use by other clients. Typical client work flow: 0. 1. {{{GetVersion()}}}: learn RSpec formats supported at this aggregate 2. {{{ListResources(, options)}}}: get Ad RSpec describing available resources 3. 4. {{{Allocate(, , , {})}}}: * Aggregate reserves resources * Return is a manifest RSpec describing the reserved resources * Optionally {{{Delete}}} some slivers, if you made a mistake, or don't like what the aggregate picked for you. 5. {{{Provision(, , , , {})}}}: * Aggregate instantiates resources * Return is a manifest RSpec describing the reserved resources, plus any instantiation-specific configuration information 6. {{{Status(, , {})}}} to check that resources are provisioned (e.g. look for operational state `geni_notready`. 7. {{{PerformOperationalAction(, , geni_start, {})}}}: * Aggregate starts resources 8. {{{Status(, , {})}}} to check that resources have started 9. {{{Renew(, , newtime, {})}}} to extend reservation 10. 11. {{{Delete(, , {})}}} when done === Changes from AM API v2 === This version of the AM API includes multiple changes since version 2 of the AM API. For experimenters, a few things are worth noting: - The old `CreateSliver` operation has now been broken into 3 steps: - `Allocate` to reserve the resources - `Provision` to instantiate the resources, which may take time to complete - `PerformOperationalAction(geni_start)` to start (e.g. boot) the resources, which also may take time to complete - Use the new intermediate `geni_allocated` state after `Allocate` to coordinate reservations across aggregates, e.g. to ensure another aggregate can give you nodes to be the other end of a requested link. - Multiple methods have been renamed, typically by removing the `Sliver` term from method names. - Sliver expiration is available in the return from multiple other methods, like `Provision` - You no longer use `ListResources` to see the contents of your slice - use `Describe` instead. `ListResources` is only for the AM's Ad RSpec. - Experimenters can select when to start or stop resources, e.g. when to boot a VM. Consult the operational state machine in the AM's Ad RSpec, and use `PerformOperationalAction`. - SSH login names and keys should be available in manifest RSpecs in a standard format. - Slice name restrictions have been codified and standardized. - Slice names are <=19 characters, only alphanumeric plus hyphen (no hyphen in first character): `'^[a-zA-Z0-9][-a-zA-Z0-9]+$'` Tool developers should also be aware: - The `credentials` argument to methods is now a struct, including a type and version for each credential. AMs should advertise which credential types they accept. SAs should advertise which type they provide. - Aggregates may have their own operational states and actions. The Ad RSpec should define these, probably by `sliver_type`. ----- == !GetVersion == Query static configuration information about this aggregate manager implementation, such as API and RSpec versions supported. For details on the arguments, return structure, and semantics, see [wiki:GAPI_AM_API_V2_DETAILS#GetVersionDetails the details page]. The !GetVersion operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#GetVersion GetVersion] operation. The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] specification does not include this operation. {{{ struct GetVersion([optional: struct options]) }}} The `options` argument may be omitted entirely by clients. The result is an [http://www.xmlrpc.com/spec XML-RPC] struct with at least the following members: {{{ { int geni_api; struct code = { int geni_code; [optional: string am_type;] [optional: int am_code;] } struct value { int geni_api; struct geni_api_versions { URL ; # value is a URL, name is a number [optional: other supported API versions and the URLs where they run] } array geni_request_rspec_versions of { string type; string version; string schema; string namespace; array extensions of string; }; array geni_ad_rspec_versions of { string type; string version; string schema; string namespace; array extensions of string; }; array geni_credential_types of { string geni_type ; string geni_version ; }; } string output; } }}} === Return Summary === As with all AM API methods, the return is an XML-RPC struct. For !GetVersion, it includes: * The standard AM API {{{code}}}, {{{value}}}, and {{{output}}} entries. For details, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#ReturnStruct the Common Concepts page] * {{{geni_api}}}: integer version of this API ('''3'''). Repeated here for backwards compatibility. !GetVersion {{{value}}} required return members: * {{{geni_api}}} = '''3''' (Integer current version of this API). * List of versions of the API supported by this aggregate. For details on doing this, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#SupportingMultipleAPIVersions the Common Concepts page]. * List of request RSpec formats supported by this aggregate. For details on RSpecs, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#RSpecs the Common Concepts page]. * List of advertisement RSpec formats supported by this aggregate. For details on RSpecs, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts#RSpecs the Common Concepts page]. * List of supported credential types and versions. For details, see [wiki:AaronHelsinger/GAPI_AM_API_DRAFT/MethodSignatures/CommonConcepts the Common Concepts page]. For details on !GetVersion arguments, returns, and semantics, and a sample minimal return, see [wiki:GAPI_AM_API_V2_DETAILS#GetVersionDetails the Details page]. == !ListResources == Return a listing and description of available resources at this aggregate. The resource listing and description provides sufficient information for clients to select among available resources. These listings are known as RSpecs. {{{ struct ListResources(struct credentials[], struct options) }}} The following members are available for use in the options parameter. All aggregate managers are required to honor these options.: {{{ { boolean geni_available ; boolean geni_compressed ; struct geni_rspec_version { string type; string version; }; } }}} Note that all options may be omitted by the client, except `geni_rspec_version` which is required. Return: A `geni.rspec` advertisment RSpec. Note that this is only part of what !ListResources did in earlier versions of this API. For the what-is-in-my-slice functionality, see `Describe()`. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#DiscoverResources DiscoverResources] operation and to the [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA]'s !GetResources operation (sec. 6.2.4). == Describe == AKA !ListResources when supplied a slice URN. Retrieve a manifest RSpec describing the resources contained by the entities named, e.g. a single slice or a set of the slivers in a slice. This listing and description should be sufficient to allow experimenters to use the resources. {{{ struct Describe(string urns[], struct credentials[], struct options[]) }}} The following members are available for use in the options parameter. All aggregate managers are required to honor these options.: {{{ { boolean geni_compressed ; struct geni_rspec_version { string type; string version; }; } }}} Note that all options may be omitted by the client, exception `geni_rspec_version` which is required. This method is part of what !ListResources used to do, and is similar to ProtoGENI [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#Resolve Resolve]. Return struct: {{{ { geni_rspec: geni_urn: geni_slivers: [ { geni_sliver_urn: geni_expires: , geni_allocation_status: , geni_operational_status: }, ... ] } }}} Note that the manifest RSpec for allocated slivers may contain less detail than for provisioned slivers. Aggregates are expected to combine the manifests of all requested slivers into a single manifest RSpec. Note that a manifest returned here for only some of the slivers in a slice at this aggregate, may contain references to resources not described in this manifest (they are in other slivers). As a result, such manifests may not be directly usable as a subsequent request. == Allocate == AKA !CreateSlivers() Allocate resources as described in a request RSpec argument to a slice with the named URN. On success, one or more slivers are allocated, containing resources satisfying the request, and assigned to the given slice. This method returns a listing and description of the resources reserved for the slice by this operation, in the form of a manifest RSpec. Allocated slivers are held for an aggregate-determined period. Clients must Renew or Provision slivers before the expiration time (given in the return struct), or the aggregate will automatically Delete them. Aggregates should implement Allocate() as quick, cheap, and not impacting Provisioned resources, such that it can be readily undone. {{{ struct Allocate(string slice_urn, struct credentials[], geni.rspec rspec, struct options) }}} This is the first part of what !CreateSliver used to do. The next part is now done by Provision. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#GetTicket GetTicket] operation. * {{{slice_urn}}}: The URN of the slice to which the resources specified in {{{rspec}}} will be allocated. For details on GENI AM API URN identifiers, see the [wiki:GeniApiIdentifiers GENI wiki page]. * {{{rspec}}}: An RSpec matching the [http://www.protogeni.net/trac/protogeni/wiki/RSpec GENI standard] request RSpec [http://www.geni.net/resources/rspec/3/request.xsd schema] containing the resources that the caller is requesting for allocation to the slice specified in {{{slice_urn}}}. Options include: {{{ { [optional geni_end_time: ] } }}} Return struct: {{{ { geni_rspec: , geni_slivers: [ { geni_sliver_urn: geni_expires: , geni_allocation_status: }, ... ] } }}} The manifest is a manifest of only newly allocated slivers. == Renew == AKA !RenewSliver() or !RenewAllocated Request that the named slivers be renewed, with their expiration extended. If possible, the aggregate should extend the slivers to the requested expiration time, or to a sooner time if policy limits apply. This method applies to slivers that are `geni_allocated` or to slivers that are `geni_provisioned`, though different policies may apply to slivers in the different states, resulting in much shorter max expiration times for `geni_allocated` slivers. {{{ struct Renew(string urns[], struct credentials[], dateTime.rfc3339 expiration_time, struct options) }}} This operation used to be called !RenewSliver. Use Renew() to get the equivalent functionality. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#RenewSlice RenewSlice] operation. Options include {{{geni_best_effort}}}, specifying whether the client prefers all included slivers to be renewed or none, or wants a partial success if possible. {{{expiration_time}}}: The date-time string in [http://tools.ietf.org/html/rfc3339 RFC 3339] format in UTC when the reservation(s) should be extended until. Return value is a list of structs: {{{ [ { geni_sliver_urn: , geni_allocation_status: , geni_operational_status: , geni_expires: , }, ... ] }}} == !UpdateAllocations == AKA !UpdateAllocated() This method modifies a set of allocated slivers in place. The existing reservations are not lost if the newly requested allocations cannot be satisfied by the aggregate. Details are suppressed pending agreement on this method. {{{ #!comment {{{ struct UpdateAllocations(string urns[], struct credentials[], geni.rspec rspec, struct options) }}} `credentials` is the standard argument defined above. Options include: - {{{geni_end_time}}}: - {{{geni_best_effort}}} Return struct: {{{ { geni_rspec: , geni_slivers: [ { geni_sliver_urn: geni_expires: , geni_allocation_status: }, ... ] } }}} The manifest is a manifest of only referenced allocated slivers. FIXME: Is this the same method as for updating provisioned slivers? }}} == Provision == AKA !ProvisionSlivers() Request that the named `geni_allocated` slivers be made `geni_provisioned`, instantiating or otherwise realizing the resources, such that they have a valid `geni_operational_status` and may possibly be made `geni_ready` for experimenter use. This operation is synchronous, but may start a longer process, such as creating and imaging a virtual machine. {{{ struct Provision(string urns[], struct credentials[], struct options) }}} This operation is part of what !CreateSliver used to do. The first part of what !CreateSliver did is now in `Allocate()`. Note that resources are not necessarily ready for experimenter use after the work that this function initiates finally completes. Consult the `geni_operational_status`, and the advertised operational state machine. Consider calling `PerformOperationalAction`, e.g. with the command name `geni_start`. The operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#RedeemTicket RedeemTicket] method. Options include: - {{{geni_best_effort}}} - {{{geni_end_time}}} - `struct geni_users[]` Return struct: {{{ geni_rspec: , geni_slivers: [ { geni_sliver_urn: , geni_allocation_status: , geni_operational_status: , geni_expires , }, ... ], }}} The returned manifest covers only newly provisioned slivers. Use `Describe` to get a manifest of all provisioned slivers. == !UpdateSlivers == AKA Update() This method requests a modification to 1 or more provisioned slivers. The slivers end up in a new `geni_allocated+geni_provisioned` state. Details are suppressed pending agreement on this method. {{{ #!comment {{{ struct UpdateSlivers(string urns[], struct credentials[], geni.rspec rspec, struct options) }}} `credentials` is the standard argument defined above. Options include: - {{{geni_end_time}}} - {{{geni_best_effort}}} - `struct geni_users[]` Return struct: {{{ { rspec: , geni_slivers: [ { geni_sliver_urn: geni_expires: , geni_allocation_status: }, ... ] } }}} The RSpec argument may be a new delta format. The manifest is a manifest of only referenced provisioned slivers. FIXME: Is this the same method as for updating allocated slivers? FIXME: Return both a manifest and a delta RSpec? }}} == Status == AKA !SliverStatus Get the status of a sliver or slivers belonging to a single slice at the given aggregate. Status may include other dynamic reservation or instantiation information as required by the resource type and aggregate. This method is used to provide updates on the state of the resources after the completion of Provision, which began to asynchronously provision the resources. This should be relatively dynamic data, not descriptive data as returned in the manifest RSpec. {{{ struct Status(string urns[], struct credentials[], struct options) }}} This operation used to be called !SliverStatus in earlier versions of the AM API. `geni_slivers` has replaced `geni_resources` and `geni_sliver_urn` replaces `geni_urn`. `geni_status` is replaced with 2 fields, `geni_allocation_status` and `geni_operational_status`. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#SliverStatus,WaitForStatus SliverStatus] operation. The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] specification does not include this operation. Return `value` is a struct: {{{ { geni_urn: geni_slivers: [ { geni_sliver_urn: geni_allocation_status: geni_operational_status: geni_expires: geni_error: , }, { geni_sliver_urn: geni_allocation_status: geni_operational_status: geni_expires: geni_error: , } ] } }}} == !PerformOperationalAction == Perform the named operational action on the named slivers, possibly changing the `geni_operational_status` of the named slivers. E.G. 'start' a VM. For valid operations and expected states, consult the state diagram advertised in the Aggregate's Ad RSpec. {{{ struct PerformOperationalAction (string urns[], struct credentials[], string action, struct options={}) }}} This operation is similar to ProtoGENI functions like `StartSliver`, `StopSliver`, and `RestartSliver` in the [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2 PG CMv2 API]. Aggregate Managers SHOULD return an error code of `13` (`UNSUPPORTED`) if they do not support a given action for a given resource. An AM SHOULD constrain actions based on the current operational state of the resource. This is a fast synchronous operation, and MAY start long-running sliver transitions whose status can be queried using `Status`. This method should only be called, and is only valid, when the sliver is fully allocated (allocation state is not `geni_pending_allocation`). Options include: {{{geni_best_effort}}}. Default is false (action applies to all slivers equally or none. Return value is a list of structs: {{{ [ { geni_sliver_urn : , geni_allocation_status: geni_operational_status : , geni_expires: [optional: 'geni_resource_status' : string] }, ... ] ; }}} == Delete == AKA !DeleteSliver Delete the named slivers, making them `geni_unallocated`. Resources are stopped in necessary, and both de-provisioned and de-allocated. No further AM API operations may be performed on slivers that have been deleted. {{{ struct Delete(string urns, struct credentials[], struct options) }}} This operation used to be called !DeleteSliver in earlier versions of this API. To get the functionality of !DeleteSliver, call Delete with the slice URN. This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#DeleteSliver DeleteSliver] operation and to the [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA]'s !DeleteSlice operation (sec. 6.2.3). Options include: {{{geni_best_effort}}} Return list of structs: {{{ [ { geni_sliver_urn: , geni_allocation_status: , geni_expires: , }, ... ] }}} == Shutdown == Perform an emergency shutdown on the slice at this aggregate. Resources should be taken offline, such that experimenter access (on both the control and data plane) is cut off. No further actions on the slice should be possible at this aggregate, until an un-specified operator action restores the slice. This operation is intended for operator use. The sliver is shut down but remains available for further forensics. {{{ struct Shutdown(string slice_urn, struct credentials[], struct options) }}} This operation is similar to ProtoGENI's [http://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPIV2#Shutdown Shutdown] operation. The [http://svn.planet-lab.org/attachment/wiki/WikiStart/sfa.pdf SFA] specification does not include this operation. Return: XML-RPC boolean, indicating whether the resources associated with this reservation were successfully shut down to a state suitable for forensics.