wiki:GAPI_AM_API_V3

Version 4 (modified by Aaron Helsinger, 8 years ago) (diff)

--

GENI Aggregate Manager API Version 3

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers.

This page documents the GENI Aggregate Manager API Version 3.

See Also:

For a summary of the Aggregate Manager API common concepts, arguments, and returns, see this sub page.

Versions

  • Latest: 3 - Documented here. Changes since version 2 include:
    • All new method names
    • Change Set D: Addressable slivers: the ability to add or delete resources from an existing slice reservation at a single aggregate
    • Change Set F3: New sliver allocation states & methods.
      • Including a cheap 'allocated' state, that tools can use to negotiate resource reservations, and coordinate resource reservations among multiple aggregates (necessary for VLAN stitching)
    • Change Sets F4&5: Operational states, and a way for aggregates to advertise resource specific operational states and methods in the Ad RSpec.
    • Change Set G: Generalized credential argument, allowing aggregates to accept and tools to supply credentials of alternate types
    • Change Set I: Return sliver expiration in most method returns
    • Change Set I: Specify SSH node login information in a standard RSpec extension
    • Change Set K: Standardize certificate contents, specifically including a UUID and email address with particular semantics
    • Change Set K: Standardize slice and user name character and length restrictions
    • Change Set M: Split ListResources in two: ListResources for Ad RSpecs, and Describe for a slice' manifest RSpec
  • Previous: 2 - Documented on the v2 page. Changes since v1 include:
    • Includes Change Sets A and B from the v2 Deltas page
    • Specify that GENI RSpecs are XML documents following standard schemas published on geni.net and documented on the ProtoGENI wiki
    • Include additional options in GetVersion and ListResources to allow aggregates to specify the RSpec formats they support
    • Add an 'options' argument, an XML-RPC struct, to all methods with no required values. This allows Aggregates to innovate or support resource-specific functionality.
    • Modify all methods to return an XML-RPC struct consisting of at least (1) a return 'code', (2) the 'value' specified by v1 of the API, and (3) a human-readable 'output' on errors. Aggregates are free to innovate by adding additional return values.
  • Original: Version 1
  • Draft changes for version 4+
    • Covers changes proposed for future versions of this API. Documented changes include:
    • Change Set C: Add an UpdateSliver method to add/remove/modify resources from a slice at an aggregate
    • Change Set E: Add the use of Tickets to API methods
    • Others to be proposed

API Overview

The GENI Aggregate Manager API is the control plane interface by which experimenters discover, reserve and control resources at resource providers. It does not include resource specific interactions, application level interactions, or monitoring and management functions.

API Protocols and Data Structures

GENI specifies that the AM API is provided via XML-RPC over an SSL connection. Aggregate Managers shall require client side GENI certificates to authenticate users, accepting only certificates that comply with the adopted GENI certificates standards. The GENI AM API therefore assumes that users have already been authenticated, and that the aggregate manager has available the client certificate to identify the user.

Clients are authorized to take actions at aggregates using GENI credentials. To that end, all methods that require authorization take an argument credentials. In particular, operations on a single GENI slice will require a credential (set) that authorizes the client whose certificate was used to authenticate to operate on the slice named by a urn argument to the method or on the slice containing the slivers named by a urns argument.

The primary data structure used within this API is a resource specification, known as an RSpec. These XML documents follow a specific set of schemas. They are used by aggregates to list and describe local resources (advertisement RSpecs), by experimenters to describe desired resources (request RSpecs), and then by aggregates to describe reserved resources (manifest RSpecs). For more information on RSpecs, see the RSpecs section on the detail page.

Using the GENI AM API

Clients (experimenters) use the AM API to discover resources (ListResources), request resources (Allocate), provision reserved resources (Provision), start resources (PerformOperationalAction), check the status of resources as they are started (Status), extend their reservation (Renew), and then return the resources when done (Delete). Client tools may use GetVersion to ensure aggregates speak a compatible version of the AM API and known formats for RSpecs. Administrators may call Shutdown to stop the resources of a slice at this aggregate, perhaps if that slice is misbehaving.

ListResources returns to the client an advertisement RSpec - a detailed listing of the resources available at that aggregate. From this information, the experimenter may determine which resources to reserve for their use. The RSpec should also have enough information to help the experimenter set the initial configuration for their resources.

Once the experimenter has selected the resources they want and how to configure them, they produce a request RSpec, detailing the resources they want and how they should be configured. They separately contact their slice authority to obtain a slice credential (or set of credentials), granting them rights to reserve resources for that slice. The experimenter then calls Allocate on this API, passing in both the slice credential and the request RSpec. The aggregate then attempts to satisfy the experimenter's resource request. If the aggregate can satisfy the request, the aggregate reserves the resources for the experimenter. The resources have not been provisioned yet, giving the experimenter a chance to verify the reservation, or check for corresponding resource availability in another aggregate. If it is acceptable, the experimenter calls Provision to set up the resources. The aggregate then starts the process of instantiating the resources and configuring them as requested in the request RSpec. Once that process has started, the Provision call returns with a manifest RSpec, listing the resources as reserved and initially configured for the experimenter.

The experimenter can then poll the aggregate manager to watch as the resources are configured and become ready for use, by calling Status, looking for an operational state other than geni_pending_allocation. A given aggregate and sliver type may use a different set of states once provisioning is complete, and further operational actions are possible - see the AM's Ad RSpec. In many cases, this indication comes with a geni_operational_state value of geni_notready. Once the resources are ready for use, the experimenter will typically call PerformOperationalAction(geni_start) to start the resources (e.g. boot a machine). The experimenter will also call Renew to request that their reservation lasts as long as they require the resources for. When the experimenter is done using the resources, they call Delete` to end their reservation. The aggregate then stops and clears the resources, freeing them for use by other clients.

Typical client work flow:

  1. <Experimenter gets a GENI certificate and slice credential>
  2. GetVersion(): learn RSpec formats supported at this aggregate
  3. ListResources(<user credential>, options): get Ad RSpec describing available resources
  4. <Experimenter constructs a request RSpec>
  5. Allocate(<slice URN>, <slice credential>, <request RSpec>, {}):
    • Aggregate reserves resources
    • Return is a manifest RSpec describing the reserved resources
    • Optionally Delete some slivers, if you made a mistake, or don't like what the aggregate picked for you.
  6. Provision(<slice URN or sliver URNs>, <slice credential>, <request RSpec>, <users struct>, {}):
    • Aggregate instantiates resources
    • Return is a manifest RSpec describing the reserved resources, plus any instantiation-specific configuration information
  7. Status(<slice URN or sliver URNs>, <slice credential>, {}) to check that resources are provisioned (e.g. look for operational state geni_notready.
  8. PerformOperationalAction(<slice URN>, <slice credential>, geni_start, {}):
    • Aggregate starts resources
  9. Status(<slice URN or sliver URNs>, <slice credential>, {}) to check that resources have started
  10. Renew(<slice URN or sliver URNs>, <slice credential>, newtime, {}) to extend reservation
  11. <Experimenter uses resources>
  12. Delete(<slice URN or sliver URNs>, <slice credential>, {}) when done

Changes from AM API v2

This version of the AM API includes multiple changes since version 2 of the AM API. For experimenters, a few things are worth noting:

  • The old CreateSliver operation has now been broken into 3 steps:
    • Allocate to reserve the resources
    • Provision to instantiate the resources, which may take time to complete
    • PerformOperationalAction(geni_start) to start (e.g. boot) the resources, which also may take time to complete
  • Use the new intermediate geni_allocated state after Allocate to coordinate reservations across aggregates, e.g. to ensure another aggregate can give you nodes to be the other end of a requested link.
  • Multiple methods have been renamed, typically by removing the Sliver term from method names.
  • Sliver expiration is available in the return from multiple other methods, like Provision
  • You no longer use ListResources to see the contents of your slice - use Describe instead. ListResources is only for the AM's Ad RSpec.
  • Experimenters can select when to start or stop resources, e.g. when to boot a VM. Consult the operational state machine in the AM's Ad RSpec, and use PerformOperationalAction.
  • SSH login names and keys should be available in manifest RSpecs in a standard format.
  • Slice name restrictions have been codified and standardized.
    • Slice names are <=19 characters, only alphanumeric plus hyphen (no hyphen in first character): '^[a-zA-Z0-9][-a-zA-Z0-9]+$'

Tool developers should also be aware:

  • The credentials argument to methods is now a struct, including a type and version for each credential. AMs should advertise which credential types they accept. SAs should advertise which type they provide.
  • Aggregates may have their own operational states and actions. The Ad RSpec should define these, probably by sliver_type.

GetVersion

Query static configuration information about this aggregate manager implementation, such as API and RSpec versions supported.

For details on the arguments, return structure, and semantics, see the details page.

The GetVersion operation is similar to ProtoGENI's GetVersion operation. The SFA specification does not include this operation.

struct GetVersion([optional: struct options])

The options argument may be omitted entirely by clients.

The result is an XML-RPC struct with at least the following members:

{
  int geni_api;
  struct code = {
       int geni_code;
       [optional: string am_type;]
       [optional: int am_code;]
         }
  struct value
      {
        int geni_api;
        struct geni_api_versions {
             URL <this API version #>; # value is a URL, name is a number
             [optional: other supported API versions and the URLs where they run]
        }
        array geni_request_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_ad_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_credential_types of {
             string geni_type <case insensitive>;
             string geni_version <containing an integer>;
       };
       boolean geni_single_allocation; <optional if using default of 0>
       string geni_allocate; <case insensitive. optional if using default of geni_single>
      }
  string output;
}

Return Summary

As with all AM API methods, the return is an XML-RPC struct. For GetVersion, it includes:

  • The standard AM API code, value, and output entries. For details, see the Common Concepts page
  • geni_api: integer version of this API (3). Repeated here for backwards compatibility.

GetVersion value required return members:

For details on GetVersion arguments, returns, and semantics, and a sample minimal return, see the Details page.

ListResources

Return a listing and description of available resources at this aggregate. The resource listing and description provides sufficient information for clients to select among available resources. These listings are known as RSpecs.

struct ListResources(struct credentials[], struct options)

The following members are available for use in the options parameter. All aggregate managers are required to honor these options.:

{
  boolean geni_available <optional>;
  boolean geni_compressed <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, except geni_rspec_version which is required.

Return: A geni.rspec advertisment RSpec.

Note that this is only part of what ListResources did in earlier versions of this API. For the what-is-in-my-slice functionality, see Describe().

This operation is similar to ProtoGENI's DiscoverResources operation and to the SFA's GetResources operation (sec. 6.2.4).

Describe

AKA ListResources when supplied a slice URN.

Retrieve a manifest RSpec describing the resources contained by the entities named, e.g. a single slice or a set of the slivers in a slice. This listing and description should be sufficient to allow experimenters to use the resources.

struct Describe(string urns[], struct credentials[], struct options[])

The following members are available for use in the options parameter. All aggregate managers are required to honor these options.:

{
  boolean geni_compressed <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, exception geni_rspec_version which is required.

This method is part of what ListResources used to do, and is similar to ProtoGENI Resolve.

Return struct:

{
   geni_rspec: <geni.rspec, Manifest >
   geni_urn: <string slice urn of the containing slice>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - allocated or ?? >,
                  geni_operational_status: <string sliver operational state>
               },
               ...
         ]
}

Note that the manifest RSpec for allocated slivers may contain less detail than for provisioned slivers. Aggregates are expected to combine the manifests of all requested slivers into a single manifest RSpec. Note that a manifest returned here for only some of the slivers in a slice at this aggregate, may contain references to resources not described in this manifest (they are in other slivers). As a result, such manifests may not be directly usable as a subsequent request.

Allocate

AKA CreateSlivers()

Allocate resources as described in a request RSpec argument to a slice with the named URN. On success, one or more slivers are allocated, containing resources satisfying the request, and assigned to the given slice. This method returns a listing and description of the resources reserved for the slice by this operation, in the form of a manifest RSpec. Allocated slivers are held for an aggregate-determined period. Clients must Renew or Provision slivers before the expiration time (given in the return struct), or the aggregate will automatically Delete them. Aggregates should implement Allocate() as quick, cheap, and not impacting Provisioned resources, such that it can be readily undone.

struct Allocate(string slice_urn,
                    struct credentials[],
                    geni.rspec rspec,
                    struct options)

This is the first part of what CreateSliver used to do. The next part is now done by Provision.

This operation is similar to ProtoGENI's GetTicket operation.

  • slice_urn: The URN of the slice to which the resources specified in rspec will be allocated. For details on GENI AM API URN identifiers, see the GENI wiki page.
  • rspec: An RSpec matching the GENI standard request RSpec schema containing the resources that the caller is requesting for allocation to the slice specified in slice_urn.

Options include:

{
  [optional geni_end_time: <dateTime.rfc3339: requested expiration of all new slivers, may be ignored by aggregates>]
}

Return struct:

{
 geni_rspec: <geni.rspec manifest>,
 geni_slivers: [
        {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from Status>,
                  geni_allocation_status: <string sliver state - allocated or ?? >
        },
        ...
    ]
}

The manifest is a manifest of only newly allocated slivers.

Renew

AKA RenewSliver() or RenewAllocated Request that the named slivers be renewed, with their expiration extended. If possible, the aggregate should extend the slivers to the requested expiration time, or to a sooner time if policy limits apply. This method applies to slivers that are geni_allocated or to slivers that are geni_provisioned, though different policies may apply to slivers in the different states, resulting in much shorter max expiration times for geni_allocated slivers.

struct Renew(string urns[],
                    struct credentials[],
                    dateTime.rfc3339 expiration_time, 
                    struct options)

This operation used to be called RenewSliver. Use Renew(<slice_urn>) to get the equivalent functionality.

This operation is similar to ProtoGENI's RenewSlice operation.

Options include geni_best_effort, specifying whether the client prefers all included slivers to be renewed or none, or wants a partial success if possible.

expiration_time: The date-time string in RFC 3339 format in UTC when the reservation(s) should be extended until.

Return value is a list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_operational_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
  },
  ...
]

Provision

AKA ProvisionSlivers() Request that the named geni_allocated slivers be made geni_provisioned, instantiating or otherwise realizing the resources, such that they have a valid geni_operational_status and may possibly be made geni_ready for experimenter use. This operation is synchronous, but may start a longer process, such as creating and imaging a virtual machine.

struct Provision(string urns[], struct credentials[],  
                                   struct options)

This operation is part of what CreateSliver used to do. The first part of what CreateSliver did is now in Allocate(). Note that resources are not necessarily ready for experimenter use after the work that this function initiates finally completes. Consult the geni_operational_status, and the advertised operational state machine. Consider calling PerformOperationalAction, e.g. with the command name geni_start.

The operation is similar to ProtoGENI's RedeemTicket method.

Options include:

  • geni_best_effort
  • geni_end_time
  • struct geni_users[]

Return struct:

  geni_rspec: <geni.rspec, RSpec manifest>,
  geni_slivers: 
  [
    {
     geni_sliver_urn: <string>,
     geni_allocation_status: <string>,
     geni_operational_status: <string>,
     geni_expires <dateTime.rfc3339 when the sliver expires from its current state>,
    },
    ...
  ],

The returned manifest covers only newly provisioned slivers. Use Describe to get a manifest of all provisioned slivers.

Status

AKA SliverStatus

Get the status of a sliver or slivers belonging to a single slice at the given aggregate. Status may include other dynamic reservation or instantiation information as required by the resource type and aggregate. This method is used to provide updates on the state of the resources after the completion of Provision, which began to asynchronously provision the resources. This should be relatively dynamic data, not descriptive data as returned in the manifest RSpec.

struct Status(string urns[], struct credentials[], struct options)

This operation used to be called SliverStatus in earlier versions of the AM API. geni_slivers has replaced geni_resources and geni_sliver_urn replaces geni_urn. geni_status is replaced with 2 fields, geni_allocation_status and geni_operational_status.

This operation is similar to ProtoGENI's SliverStatus operation. The SFA specification does not include this operation.

Return value is a struct:

{
  geni_urn: <slice URN>
  geni_slivers: [ 
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                     },
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                      }
                  ]
}

PerformOperationalAction

Perform the named operational action on the named slivers, possibly changing the geni_operational_status of the named slivers. E.G. 'start' a VM. For valid operations and expected states, consult the state diagram advertised in the Aggregate's Ad RSpec.

struct PerformOperationalAction (string urns[], struct credentials[], string action, 
                                                    struct options={})

This operation is similar to ProtoGENI functions like StartSliver, StopSliver, and RestartSliver in the PG CMv2 API.

Aggregate Managers SHOULD return an error code of 13 (UNSUPPORTED) if they do not support a given action for a given resource. An AM SHOULD constrain actions based on the current operational state of the resource. This is a fast synchronous operation, and MAY start long-running sliver transitions whose status can be queried using Status. This method should only be called, and is only valid, when the sliver is fully allocated (allocation state is not geni_pending_allocation).

Options include: geni_best_effort. Default is false (action applies to all slivers equally or none.

Return value is a list of structs:

[ {
        geni_sliver_urn : <string>,
        geni_allocation_status: <string, eg provisioned>
        geni_operational_status : <string>,
        geni_expires: <dateTime.rfc3339 of individual sliver expiration>
        [optional: 'geni_resource_status' : string]
        }, 
        ... 
]
;

Delete

AKA DeleteSliver Delete the named slivers, making them geni_unallocated. Resources are stopped in necessary, and both de-provisioned and de-allocated. No further AM API operations may be performed on slivers that have been deleted.

struct Delete(string urns, struct credentials[], struct options)

This operation used to be called DeleteSliver in earlier versions of this API. To get the functionality of DeleteSliver, call Delete with the slice URN.

This operation is similar to ProtoGENI's DeleteSliver operation and to the SFA's DeleteSlice operation (sec. 6.2.3).

Options include: geni_best_effort

Return list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
  },
  ...
]

Shutdown

Perform an emergency shutdown on the slice at this aggregate. Resources should be taken offline, such that experimenter access (on both the control and data plane) is cut off. No further actions on the slice should be possible at this aggregate, until an un-specified operator action restores the slice. This operation is intended for operator use. The sliver is shut down but remains available for further forensics.

struct Shutdown(string slice_urn, struct credentials[], struct options)

This operation is similar to ProtoGENI's Shutdown operation. The SFA specification does not include this operation.

Return: XML-RPC boolean, indicating whether the resources associated with this reservation were successfully shut down to a state suitable for forensics.