wiki:GAPI_AM_API_DRAFT

Version 80 (modified by Aaron Helsinger, 12 years ago) (diff)

--

GENI Aggregate Manager API Draft Revisions

This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, some of the revisions documented here have been discussed on the GENI developer mailing list, and during at least one GEC. Other revisions are in early discussions and subject to change or abandonment.

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is a set of resources allocated by one Aggregate to one Slice. See below for a proposed complete definition.

The current officially adopted version of the API is 3 and is documented on the main API page.

API Version 3 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They are detailed below.

API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They include:

  1. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    • Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    • Omni version 1.3 (released June 2011) added client software support for these changes.
  2. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.

This page documents proposed changes for AM API version 4. These changes are grouped into sets. API Version 4 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.

Proposing Additional Changes

GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.

Technical discussions are generally held on the Developers mailing list

Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net

Change Sets Adopted in GENI Aggregate Manager API version 3

Version 3 has been adopted.

Details of the change sets included in AM API version 3 are at the v3 Deltas page.

Proposals adopted after GEC13:

  • Change Set D: Slivers: Change methods to clarify that there may be multiple slivers per slice at an AM, and to allow operating on individual slivers
  • Change Set F3: Sliver Allocation States and methods
  • Change Set F4: Method to perform Sliver Operational actions
  • Change Set F5: Sliver Operational States
  • Change Set M: New method signatures, incorporating all previous adopted change sets

Proposals adopted at GEC13:

  • ADOPTED: Change Set G: Generalize the credentials argument, allowing ABAC support
  • ADOPTED: Change Set I1: SliversStatus return structure includes sliver expiration
  • ADOPTED with changes: Change Set I2: SliversStatus return includes SSH logins/key for nodes that support SSH access
  • ADOPTED: Change Set I3: CreateSlivers return becomes a struct, adds sliver expiration
  • ADOPTED: Change Set K: Standardize certificate contents, etc.
    • Include a real serial number, holder email, holder uuid, and optionally authority URL in certificates
    • Define slice ID as the UUID plus URN in slice certificates
    • Define slice name, sliver name, and user name restrictions, and similar for URNs
    • Publish schemas for credentials and certificates

Proposed Changes for GENI Aggregate Manager API version 4

Change Set C: Update()

Add an ability for experimenters to modify their allocated resources at an aggregate without deleting (and possibly losing) existing resource allocations.

This change was briefly discussed at GEC13, discussed at the GEC14 coding sprint, and is a topic for ongoing discussion.

Motivation

A common complaint among experimenters about the current AM API is that there is no way to atomically add/remove and modify resources at an aggregate. AM API v3 allows aggregates to support adding or deleting some resources from the slice at the aggregate, but not all aggregates can support that without the ability to also modify other resources. For example, you cannot add a link without adding an interface to one or more nodes, and there is no way to modify the node without first deleting it. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resuorces for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources.

This topic was discussed at the GEC12 AM API session and on the GENI dev mailing list (in October and November).

This topic was also discussed on the geni-dev mailing list here: http://lists.geni.net/pipermail/dev/2012-March/000588.html and here: http://lists.geni.net/pipermail/dev/2012-March/000643.html and elsewhere in March 2012.

A competing related proposal from Gary Wong is here: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

Jon Duerig made a proposal prior to GEC14, which was discussed at the coding sprint. This proposal is a revised version of that proposal.

Proposal Summary

This proposal adds an Update() method, a Cancel() method, and a new geni_allocation_state: geni_updating. The proposal includes a new state guarantee RSpec extension, a new geni_cancelled option to Describe(), and calls for some minor changes to GENI v3 RSpecs.

Update()

Begins a transaction to modify resources which are currently in the geni_allocated or geni_provisioned state. Update() itself changes only the internal allocation of slivers, not their operational state. It is therefore fast and synchronous.

struct Update(string urns[], struct credentials[], struct geni.rspec rspec, struct options)

This call accepts the geni_best_effort option; when supplied, the aggregate should attempt to partially satisfy the request.

This call accepts the geni_end_time option; when supplied, the aggregate may attempt to renew the slivers being created or modified to the requested time, according to aggregate-local policy.

Since Update() changes the allocation state only, it has an immediate effect on slivers currently in the geni_allocated or geni_updating states. However, updating slivers in the geni_provisioned state is a two-step process involving first changing the allocation via Update() and then committing those changes with Provision().

The input RSpec is a complete request specification for all slivers included in the urns list, as you would like the slivers to exist when the call is fully committed. Any difference in the update RSpec from the current slivers shall be interpreted as a modification request, including removing interfaces or links, or deleting entire slivers.

The RSpec argument to Update() should be a request RSpec, but RSpecs passing the manifest schema are allowed. Resources in the RSpec should include the sliver_id tag when the experimenter is requesting a modification to the given resource. Resources without a sliver_id are to be interpreted as new resource requests.

If a sliver is defined in the RSpec and contained in the urns list, (either by URN or by indirect inclusion because the slice URN is in the urns list), then the user is requesting that the sliver be modified. If the sliver was in the geni_allocated state, the allocation of that sliver is changed immediately and the sliver remains in a geni_allocated state. If the sliver was in the geni_provisioned state, the operational state of that sliver is preserved and it is placed in the geni_updating state where it can be Provision()ed to implement the modification or Cancel()led.

If the RSpec does not define one or more slivers which are specified in the urns list (that is, does not include resources containing the given sliver_id tag), then the user is requesting that the given sliver(s) should be deleted. If the sliver was in the geni_allocated state, the allocated sliver is deleted and the sliver ceases to exist (becomes geni_unallocated). If the sliver was in the geni_provisioned state, then the operational state of that sliver is preserved and it is placed in the geni_updating allocation state where it can be Provision()ed to delete the sliver or Cancel()led to preserve it.

Any slivers which are not defined in the RSpec and do not appear in the urns list will remain unchanged. They remain in their current allocation state.

Any resources requested in the RSpec that are not specified in the urns list will be instantiated and be in the geni_allocated state, if the aggregate has the resources available.

After a successful Update()

After a successful call to Update(), all slivers in the urns list which began in the geni_provisioned state will be in the geni_updating state (and their configuration will revert in the case of a Cancel() call described below). This includes those slivers which are marked for deletion as described above. These slivers will not actually be deleted until the Provision() call.

Slivers which were geni_provisioned and are now geni_updating will retain their prior existing expiration time. If the expiration time is reached before the sliver is Provision()ed, then the resources are freed and the sliver is geni_unallocated. The sliver does not return to the geni_provisioned state on sliver expiration.

All slivers which began in the geni_allocated state will remain in the geni_allocated state, or will have been deleted and there will be no way (except for attempting another Update()) to revert their configuration. Slivers that are updated and remain in the geni_allocated state will have a new expiration time, as determined by the aggregate, but typically set just as for new slivers returned by Allocate(). As with other geni_allocated slivers, if the sliver expires before it is Provision()ed, then the resources are freed and the sliver is geni_unallocated.

Any slivers which began in the geni_updating state will remain in the geni_updating state, with their prior existing expiration time. If the expiration time is reached before the sliver is Provision()ed, then the resources are freed and the sliver is geni_unallocated. The sliver does not return to the geni_provisioned state on sliver expiration.

Any new resources requested in the RSpec not associated with a current sliver will be in the geni_allocated state and will be added to slivers, just as after a call to Allocate().

Return

On success, the value field of the return struct will contain a struct:

{
 geni_rspec: <geni.rspec manifest of newly allocated slivers>,
 geni_slivers: [
        {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from Status>,
                  geni_allocation_status: <string sliver state - e.g. geni_allocated, geni_updating>,
                  geni_error: <optional string, may be omitted entirely, explaining any failure for a sliver>
        },
        ...
    ]
}

The list of returned slivers includes even those that will be deleted.

State Guarantees

The manifest RSpec may include an extension describing what guarantees the AM will make about state preservation. Omitting the extension means the AM provides no guarantees. It is proposed that the extension have the following RNC schema:

default namespace = "http://www.geni.net/resources/rspec/ext/preserve/1"

# This is meant to extend a node or link
Preserved = element preserve {
  attribute guarantee { "geni_none" | "geni_persistent_state" |
                        "geni_dynamic_state" | "geni_no_disruption" }
}

start = Preserved

Guarantee levels are defined to mean:

  • geni_none is returned when the AM provides no state preservation guarantees for a sliver.
  • geni_persistent_state is returned when the AM may wipe out dynamic state (i.e. reboot a node) but will preserve persistent state.
  • geni_dynamic_state is returned when the AM retains the dynamic state but there may be other perturbations such as packet loss or service restarts.
  • geni_no_disruption is returned when the AM will not make any changes that can perturb the sliver.

Cancel()

Cancels an Update() or an Allocate().

struct Cancel(string urns[], string credentials[], struct options)

This call accepts the geni_best_effort option; as with Delete(), then this option is supplied the aggregate should attempt to de-allocate individual slivers.

When applied to slivers in the geni_allocated state, Cancel() acts just like Delete(). When applied to slivers in the geni_updating state, those slivers are returned to the geni_provisioned state with no change to the operational state or attributes of the existing slivers.

Return: On success, the value field of the return struct will contain a struct:

{
   geni_rspec: <geni.rspec, a Manifest RSpec>
   geni_urn: <string slice urn of the containing slice>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - e.g. geni_allocated or geni_provisioned >,
                  geni_operational_status: <string sliver operational state>,
                  geni_error: <optional string, may be omitted entirely, explaining any failure for a sliver>
               },
               ...
         ]
}

Changes to Provision()

When applied to slivers in the geni_updating state, this call will either remove those slivers (if they were marked for deletion as described in the Update() call) or move them to the geni_provisioned state, provisioning them according to the manifest RSpec returned from that call.

Changes to Describe()

The addition of an Update() operation creates an ambiguity on the slivers being described. Should the Aggregate Describe() the slivers as they will be instantiated on a successful Provision() (including all slivers that are geni_allocated, geni_updating, and geni_provisioned)? Or should the AM describe the slivers which would remain after a Cancel() (including only the slivers that are geni_allocated or geni_provisioned, plus the updated slivers as they existed before being updated)?

In order to account for this ambiguity, Describe() accepts a new option, geni_cancelled which defaults to false. When geni_cancelled is true, Describe() returns the state of all currently geni_provisioned slivers, showing the user what their slice will look like if they cancel all geni_updating and geni_allocated slivers. When geni_cancelled is false, the manifest and status will reflect the changed slivers in the geni_updating and geni_allocated states.

Changes to Delete()

A call to Delete() can be used to delete slivers in any state.

Change to geni_single_allocation

When an AM reports geni_single_allocation as true in GetVersion(), the following additional restrictions apply to calls:

  • When calling Update(), the urns list must either contain a slice URN or it must contain the URN of every sliver at this Aggregate Manager in the given slice.
  • When calling Provision(), the urns list must contain the sliver URNs of all slivers in either the geni_allocated or geni_updating states in the given slice.
  • When calling Cancel(), the urns list must contain the sliver URNs of all slivers in either the geni_allocated or geni_updating states in the given slice.

Mixed State Calls

A Renew() call which contains sliver URNs in multiple allocation states will attempt to renew them all to the same requested expiration time.

A Delete() call which contains sliver URNs in multiple allocation states will attempt to delete all slivers and put them in the geni_unallocated state.

Provision() and Cancel() will ignore any sliver URNs which are not in the geni_allocated or geni_updating states.

Update() can be called on slivers in any state. It will have no effect on slivers in the geni_unallocated state, and otherwise operates as described above.

RSpec Change

To easily accommodate submitting Update() requests, RSpecs will be changed:

  • sliver_id will be allowed in request RSpecs
  • Manifests will be modified to be re-usable as request RSpecs. In particular, the manifest schema will become a simple extension of the request schema, but this change is not required to support Update().

Change Set N: Add information to GetVersion

This change set adds some additional information to the return from GetVersion. If adopted, aggregates may implement this as part of their AM API v3 implementation, but will be required to do so as part of AM API v4.

Change Set N1: Add geni_am_code_version

Aggregates use varying ways to identify the software version their AM uses. PlanetLab and ProtoGENI use code_tag, Orca uses orca_version, and FOAM uses foam_version. Automatically identifying software versions is important for monitoring and GENI operations. Aggregates are therefore strongly encouraged to advertise their current software revision using this standard field, though for security reasons some aggregates may choose not to do so.

This change set proposes adding geni_am_code_version to the return from GetVersion.

  • The field is optional. Aggregate developers are expected to include this option, but site operators may select not to expose it.
  • The value is a string.
  • Legal characters are: alphanumeric, space, - (hyphen), ., : (colon), #, _ (underscore), +, (, )
  • Regular expression: '^[a-zA-Z0-9-\.:#_\+\(\)]+$'

Change Set N2: Add geni_am_type

Aggregates are allowed to include AM specific return values, or even implement custom methods as part of the same interface.

Currently, there is no consistent way to identify what type of aggregate a given instance is, so as to know what additional attributes or options may be used. This proposal would add a way for aggregates to identify what kind of aggregate this is, and therefore what aggregate specific options or returns are applicable. Aggregates of aggregates may identify as multiple types. One of those types would indicate that this is such an aggregate of aggregates, and other listed types would indicate that clients may interact with the aggregate as though it is any of the listed types.

This change set proposes adding geni_am_type to the return from GetVersion.

  • The field is required.
  • The value is a list of strings, of length at least one. It should generally be a list of length 1. Aggregates of aggregates may list multiple types.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

Change Set N3: Use consistent types and prefixes

Make am_type, geni_am_type, and AM specific prefixes for new options consistent.

The AM API specifies an am_type field as part of the code in the basic return structure. This field has a set of prescribed values.

An earlier proposal adds a geni_am_type return to GetVersion.

The AM API encourages AM specific options and returns. These options and returns should have unique names, but the API does not specify how to ensure this.

This proposal aims to standardize these fields.

am_type and geni_am_type:

  • The value is a string.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

Aggregate specific options and return attributes should be named with a prefix denoting the aggregate type. It should be the same as the am_type and geni_am_type for this aggregate. Therefore, character restrictions are:

  • The value is a string.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

For example: ProtoGENI includes an AM-specific status in the v2 API return from SliverStatus called pg_status. By this proposal, that attribute should be renamed protogeni_status.

Change Set O: Refine character restrictions

If adopted, aggregates may implement this as part of their AM API v3 implementation, but will be required to do so as part of AM API v4.

Change Set O1: Allow other characters in sliver names

Currently we heavily constrain legal characters in the 'name' portion of sliver URNs. See GeniApiIdentifiers. This is awkward. In particular characters like periods, and underscores are not allowed, but are very useful as separators.

This change set proposes modifying the name portion of the URN rules as follows. Note that these changes loosen existing restrictions, and so are backwards compatible (existing sliver names remain legal).

Sliver:

  • May use only alphanumeric characters plus hyphen, underscore, or period: '^[a-zA-Z0-9-_\.]+$'

Change Set O2: Explicitly define legal characters in some strings

Various fields in the AM API specification are defined only as strings, without explicit limits on legal characters. This makes it awkward to pass these values to and from scripts. In particular, many fields should only be alphanumeric, plus some limited number of separator characters.

This change set proposes constraining several options/return values that are currently defined only as 'string'.

  1. From getversion and listresources and describe, the RSpec type:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From getversion and listresources and describe, the RSpec version:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From getversion, the RSpec schema, and namespace: Any character valid in a URL (see e.g. http://www.w3.org/Addressing/URL/url-spec.txt)
  1. From getversion and the credentials argument, the credential geni_type:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From the return of multiple methods, geni_operational_status: alphanumeric plus underscore; alphanumeric only in first character: '^[a-zA-Z0-9][a-zA-Z0-9_]*$'

Change Set O3: Allow unicode values

Many values defined by this API appear to be limited to ASCII characters - both arguments and returns. We would like to support internationalization. Some values in this API (e.g. those defined as URLs) implicitly support unicode. Currently, a well behaved aggregate should be able to handle unicode values in at least some arguments. Similarly, a well behaved client should be able to handle unicode returns.

This change would clarify the support for internationalization in this API, and allow aggregates to specify unicode support. This change would specify:

Aggregates are encouraged to support non-ASCII letters as input. Clients are encouraged to support non-ASCII returns from aggregates. However, this change set does not require that either aggregates or clients support non-ASCII strings, and aggregates should not require clients to accept non-ASCII results where the API does not explicitly require it.

This change would allow an aggregate to explicitly specify that arguments may be unicode, and returns may be unicode if the content demands it. By default, aggregates are expected to use the ASCII only regular expressions that the rest of this API specification uses, and to limit returns to ASCII characters where possible. This change would add an optional return from GetVersion that allows an aggregate to specify that unicode arguments are acceptable (ASCII strings must still be accepted), and that all returns may be unicode. When the aggregate specifies this option, it shall still attempt to support ASCII inputs to allow clients to operate only in ASCII where possible.

Specifically, this change adds geni_unicode_compliant to the return from GetVersion.

  • This field is optional.
  • Default is false. All strings specified in this API are to be treated as ASCII unless otherwise specified, (e.g. the format explicitly references a standard that supports unicode). This includes both arguments and returns.
  • When true, all strings specified in this API may be unicode. In particular, regular expressions that limit to ASCII characters shall be interpreted to include the unicode equivalents. The aggregate will still accept ASCII input. Returns will remain ASCII where the API defines ASCII constants or content allows it.

Older Proposals

Older proposals, withdrawn, superseded, or postponed:

  • Original Change Set C: On hold (waiting for a revised proposal). Add the ability to UpdateSlivers to immediately modify your reservation
  • Change Set E: Tickets: On hold (waiting for a revised proposal). Add methods using tickets to do negotiated reservations
  • Change Set F1: Superseded by F3 and F4: Define sliver states, and the state changes that various methods cause
  • Change Set F2: Superseded by F3 and F4: Add a new general ActOnSlivers method allowing AMs to support AM and resource-type specific operations
  • Change Set H: Superseded by F3: Clarify: A second call to CreateSlivers without an intervening DeleteSlivers is an error.
  • Change Set I4: Superseded by F3 and F4: CreateSlivers optionally does not start resources.
  • Withdrawn: Change Set J: Support proxy aggregates with 1 new option and 1 new GetVersion entry
  • Postponed: Change Set L: Standardize slice credential privileges

Unspecified items

  • UpdateSlivers: Update a reservation or sliver in place, without risking losing resources
  • Publish credential schema
  • Define error codes returned by new methods, conditions

Changes Not Included

RSpec changes resulting in GENI v4 RSpecs

  • Support unbound manifests
  • Make manifest an extension of Request, so you can readily edit & resubmit a manifest
  • Make configuration information in request and manifest optional, so it can be supplied/returned separately
  • Fully implement the compute ontology from Ilia
  • Ilia's other requests (Openflow related information)
  • Document process for updates per my dev list email
  • Be consistent: ref vs idref vs href
  • Include AM name/URL in RSpecs? Experimenter who allocated it URN?
  • Incorporate stitching extension as part of the 'base' RSpec

Misc

  • Allow Shutdown on a single sliver or a list of slivers
  • Add geni_am_info block to GetVersion return (name, id, url, location, description, is_proxy, proxy: {(a geni_am_info block)}, proxy_for[] (list of geni_am_info blocks))
  • Allow the update methods to take a generic rspec argument, allowing AMs to accept full or diff RSpecs
  • Tickets
    • Remove requestor certificate?
    • Support brokers: Make ticket methods return multiple tickets. Define tickets as optionally diffs (additive). Make RedeemTicket and UpdateTicket take a list of tickets. Tickets are delegatable via signing, but the delegated ticket must be for a strict subset of the resources in the original.
    • Drop ticket struct, not a signed document. Just reference by ID
  • Support message passing
    • Add sendMsg (and getMsgs?) methods that take a signed document, allowing freely passing messages instead of XMLRPC?

Stitching

AMs must support the stitching extension where Layer 2 connections are available:

Update stitching schema per changes here: https://geni.maxgigapop.net/twiki/bin/view/GENI/NetworkStitchingGeniApiAndRspec

Attachments (1)

Download all attachments as: .zip