wiki:GAPI_AM_API_DRAFT/Archive

Version 2 (modified by Aaron Helsinger, 10 years ago) (diff)

--

GENI Aggregate Manager API Archived Proposed Revisions

This page documents archived older DRAFT revisions to the GENI Aggregate Manager API, previously proposed for the API, but then withdrawn, superseded, or postponed.

The current officially adopted version of the API is 3 and is documented on the main API page.

Current proposals for the next version of the AM API are on the [GAPI_AM_API_DRAFT AM API Draft wiki page].

Proposals on this page:

  • Change Set C: On hold (waiting for a revised proposal). Add the ability to UpdateSlivers to immediately modify your reservation
  • Change Set E: Tickets: On hold (waiting for a revised proposal). Add methods using tickets to do negotiated reservations
  • Change Set F1: Superseded by F3 and F4: Define sliver states, and the state changes that various methods cause
  • Change Set F2: Superseded by F3 and F4: Add a new general ActOnSlivers method allowing AMs to support AM and resource-type specific operations
  • Change Set H: Superseded by F3: Clarify: A second call to CreateSlivers without an intervening DeleteSlivers is an error.
  • Change Set I4: Superseded by F3 and F4: CreateSlivers optionally does not start resources.
  • Withdrawn: Change Set J: Support proxy aggregates with 1 new option and 1 new GetVersion entry
  • Postponed: Change Set L: Standardize slice credential privileges
  • Change Set O3: Allow unicode values
  • No Reservation Allocate

Change Set C: UpdateSlivers

Add an ability for experimenters to modify their allocated resources at an aggregate without deleting (and possibly losing) existing resource allocations.

This change was briefly discussed at GEC13, and is a topic for ongoing discussion.

Motivation

A common complaint among experimenters about the current AM API is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resuorces for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources.

This topic was discussed at the GEC12 AM API session and on the GENI dev mailing list (in October and November).

This topic was also discussed on the geni-dev mailing list here: http://lists.geni.net/pipermail/dev/2012-March/000588.html and here: http://lists.geni.net/pipermail/dev/2012-March/000643.html and elsewhere in March 2012.

A competing related proposal from Gary Wong is here: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

A further revised proposal is expected from Jon Duerig.

UpdateSlivers

This change would add a new method UpdateSlivers, which takes a full request RSpec of the desired final state of the slice at this aggregate. This proposal calls for adding this functionality in the context of tickets: the UpdateSlivers method returns a ticket. See Change Set E: Tickets for details.

Some points about this change:

  • The method takes a full request RSpec - not a diff.
    • Note that we want the manifest to be readily modifiable to be a request (include component_ids and sliver_ids), but it is not yet.
    • AMs may, as always, return UNSUPPORTED - EG if they are incapable of determining what changes to apply (computing a diff).
  • The request is either fully satisfied, or fails (returns an error code).
  • AMs must document the level of service they provide: will any state be lost on existing resources?
  • Options includes geni_end_time, a RFC3339 requested end time for the reservation. See below for details.
    • If omitted, the AM may reserve the resources for the default sliver duration.
    • AMs should follow the logic of RenewSlivers to determine if the requested duration of the sliver is acceptable.
    • The request should Fail (return an error code) if the resources cannot be reserved until the requested time.

Aggregates and experimenters should use an RSpec extension for specifying a guarantee on state loss per-resource: EG:

  <node ... sliver_id="urn:publicid:IDN+jonlab.tbres.emulab.net+sliver+250">
    <sliver_type name="raw-pc"/>
...
    <preserve:preserve guarantee="persistent-state" />
  </node>

This uses the RNC schema:

default namespace = "http://www.protogeni.net/resources/rspec/ext/preserve/1"

# This is meant to extend a node or link
Preserved = element preserve {
  attribute guarantee { "none" | "persistent-state" |
                        "dynamic-state" | "no-disruption" }
}

start = Preserved

In the above schema, the states represent increasing levels of state preservation guarantee.

struct UpdateSlivers(string slice_urn, string credentials[], string rspec, 
                                                 struct options)

Returns a struct:

{
  ticket: <string, the ticket>
  geni_status: <string, the sliver state - ticketed>,
 <others that are AM specific>
}

See Change Set E: Tickets for details on the ticket returned and ticket semantics.

Change Set E: Tickets

AM APIv3 adds support for negotiated reservations or two-phase commit, by adding methods that allow an experimenter to reserve resources for a slice without committing to using them, or forcing the AM to incur the cost of instantiating them.

This change is actively under discussion. See for example http://lists.geni.net/pipermail/dev/2012-March/000616.html and http://lists.geni.net/pipermail/dev/2012-March/000648.html

For an alternative proposal, see: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

The later proposal for allocation states (below) supersedes this. See Change Set F3.

Motivation

This possible change was discussed at the GEC12 AM API session.

The SFA defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.

Tickets are used in the ProtoGENI CMV2 interface, and are discussed on the PG wiki. Tickets with a slightly different semantics (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see the Orca Book. However, each of these uses of the notion of tickets differs.

Tickets would potentially enable a number of useful and possibly critical features:

  • Coordinated or negotiated reservations: reserving resources from aggregate B only if aggregate A can give you a complementary resource. For example, a VLAN tag. This is related to stitching, both network stitching and the more general form.
  • Two phase commit reservations (similar to the above).
  • Scheduled reservations in the future.
  • Brokers: 3rd parties consolidating, scheduling and allocating resources on behalf of a number of other aggregates
  • Lending resources to other experimenters
  • Giving experimenters explicit control over when resources are started, stopped, and restarted (see the discussion on UpdateSliver).

Tickets semantics

This proposal would add tickets to the existing AM API, allowing experimenters to reserve (hold) resources temporarily and cheaply. Tickets represent a promise to the named slice to allocate the specified resources, if the ticket is 'redeemed' while the ticket is still valid. Tickets describe a complete specification of the resources that will be allocated to the slice at this aggregate, if the ticket is redeemed.

Some key properties of the tickets proposed here:

  • Tickets are IOUs from an AM to a slice (not to an experimenter - no delegation is necessary or possible).
  • Experimenters do not need to use tickets to reserve resources: existing methods without tickets still work.
  • A ticket is a promise by the AM to give the specified resources to the slice if an authorized slice member requests them.
    • The aggregate is saying they will not give away these resources to other slices, but only to this slice.
    • AMs must document how firm their promises are. See the attribute geni_ticket_service_class.
      • Some aggregates may only offer soft promises, as in PlanetLab.
  • Tickets are signed by the AM: non repudiatable.
  • Tickets are bound to a slice: they contain the slice certificate.
  • Tickets may be passed from 1 researcher on a slice to another freely - no explicit delegation is required.
    • Indeed, any experimenter with appropriate slice credentials can retrieve the ticket from the aggregate.
    • Tickets may not be delegated to another slice or other entity; these tickets do not support brokers.
  • Tickets promise a particular set of resources: they include an RSpec. Note that this may be an unbound RSpec.
    • Note that we do not currently have unbound manifest RSpecs. For now we specify only that this is an RSpec.
  • Tickets are good for a limited time.
    • They must be redeemed by a specified time, redeem_before, after which the aggregate is free to assign the resources elsewhere.
      • Aggregates determine redeem_before, which is some epsilon in the near future.
      • Aggregates may accept a new option geni_reserve_until which is a request for a particular redeem_before, but are not required to support this (they may ignore the option).
    • Tickets specify when the resources will be available from (starts, typically essentially now), and when they will be available until (typically now plus the aggregate-local default sliver expiration time).
      • The resources may be available even longer, but that would require a separate RenewSlivers call.
  • Tickets specify the full final state of the slice after applying this ticket.
    • Tickets are not incremental changes, and are not additive.
    • The implication is that there may be only 1 ticket outstanding for a slice per aggregate (except for scheduled reservations, see below).
    • This also implies that these tickets are not suitable for use by brokers.
  • Aggregates must attempt to honor their promises. As a result, aggregates must remember all outstanding tickets until they are redeemed or expire.
  • All ticket related timestamps must be in the format of RFC3339 (http://www.ietf.org/rfc/rfc3339.txt)
    • Full date and time with explicit timezone: offset from UTC or in UTC.
    • eg: 1985-04-12T23:20:50.52Z or 1996-12-19T16:39:57-08:00

Change Set F: Support AM and resource-type specific methods.

Define the control API (the AM API) as about moving slivers through various states at an AM.

The proposal here elicited concerns (the method ActOnSlivers is an ioctl, and the states mix allocation and operational states).

A later alternative proposal, see http://lists.geni.net/pipermail/dev/2012-March/000721.html

At the GEC13 coding sprint, a variant on the above was approved. It is documented as Change Set F3.

A variant on the operational states proposal is defined as Change Set F4 and documented here: https://openflow.stanford.edu/display/FOAM/GENI+-+PerformOperationalAction

Motivation

AM API methods logically change the state of the slivers at this AM. But the API is not clear what experimenters should expect, and does not provide easy ways for experimenters to control when and how states change. There is in particular no way to move slivers through states and change them in ways otherwise undefined by the API.

Change Set F1: Define Sliver States

This change is superseded by Change Sets F3 and F4.

Currently the AM API defines several possible states as valid returns in SliversStatus: configuring, ready, unknown, and failed. This change changes and expands that list of valid states, and explicitly defines the expected states after each AM API method call. Additionally, this change provides a mechanism for aggregates to supply their own states.

The GENI AM API can be thought of as manipulating slivers. As such, each method potentially changes the state of 1 or more slivers. With the changes proposed here, several of the methods return a new geni_status field, whose value is one of the standard GENI sliver status values. Aggregates must use one of the standard GENI values for that return.

Change Set F2: ActOnSlivers

This change is superseded by Change Sets F3 and F4.

This change introduces a new method, providing a generic way to act on slivers in an AM or resource type specific way. This method shall be used to 'start' or 'stop' or 'restart' resources that have been allocated but not started by CreateSlivers or RedeemTicket. It may also be used to change the state of slivers (or their contained resources) in an aggregate or resource specific way. Some aggregates may use this method to change configuration details of allocated resources. This might include changing acceptable login keys.

Changes to existing methods

Modify a few existing methods to make certain operations easier or more experimenter friendly.

Change Sets H&I: Misc other method changes

Change Set H

A second call to CreateSlivers without an intervening DeleteSlivers is an error.

  • This change just clarifies expected behavior that was left under-specified in AM API v1.
  • CreateSlivers takes a full Ad RSpec, it is not a way to 'add' resources.
  • Silently replacing the existing slivers with new slivers (similar to a call to UpdateSlivers) is not acceptable.

Change Set I4

CreateSlivers optionally does not start resources.

Currently, CreateSlivers auto starts resources, moving them from allocated through changing to ready.

  • Add a new option geni_donotstart:
    • If supplied and true (boolean: 0 or 1 in XML-RPC), final state is allocated, and experimenter must explicitly start resources using ActOnSlivers.
    • If omitted or false (0) (default), AM automatically starts resources as before, and state will be changing and then ready.

Withdrawn: Change Set J

This change set has been withdrawn.

Change Set L: Change SFA credentials' privileges

This proposal has not been fully discussed and is likely to be postponed.

Our goal is to simplify and standardize privilege strings used in SFA credentials. Currently there are wildcards, bind, embed, and others. They are confusing. We also want extensibility to use these credentials elsewhere in future.

Credentials should support these kinds of operations:

  • Learn about the slice
  • Add/Modify/Delete resources in the slice
  • Read slice details like I&M?
  • Use the slice
  • Operator shutdown

Proposal - Replace all existing privileges with only the following possible strings (case insensitive):

  • CanWrite
    • If present in a valid slice credential, aggregates may permit CreateSlivers, RenewSlivers, DeleteSlivers, Shutdown, plus new methods ActOnSlivers, UpdateSlivers, GetTicket, RedeemTicket, UpdateTicket, ReleaseTicket
    • Thus it replaces bind, embed, control, instantiate, sa, pi, or * in various places
  • CanRead
    • If present in a valid slice credential, aggregates may permit ListResources with a slice_urn, SliversStatus
    • Thus it replaces info or * in various places
  • CanReadDetails
  • CanUse

Note that those last 2 may never get used, but are there in case I&M or opt-in make those useful.

Note also that operators who wish to shut down a slice would need a slice credential with the CanWrite privilege.

Privilege and credential semantics are defined as follows:

  • Aggregates may only grant access using current SFA credentials to a method if at least one such valid credential:
    • grants the required privilege or privileges (if any)
    • to the caller of the API method
      • (identified by their SSL client certificate and the owner_gid in the credential)
    • over the slice (if any) on which they are operating
      • (target_gid in the credential).
  • Other privileges may be present in the same or other credentials, and other non-SFA credentials may be used to authorize actions (per Change Set G).
  • Local aggregate policy may grant or deny access to a particular method regardless of the presence of a valid credential granting the required privilege. This depends in part on federation policy governing aggregates.
  • Some operations (e.g. GetVersion) may either simply require a valid credential with no particular privilege, or have no credentials argument at all.

Note also that some current AMs do not require any particular privileges to do ListResources, even with a slice_urn. This change suggests that aggregates require a valid slice credential with CanRead privileges to authorize this operation using current slice credentials.


Deferred: Change Set O3: Allow unicode values

Many values defined by this API appear to be limited to ASCII characters - both arguments and returns. We would like to support internationalization. Some values in this API (e.g. those defined as URLs) implicitly support unicode. Currently, a well behaved aggregate should be able to handle unicode values in at least some arguments. Similarly, a well behaved client should be able to handle unicode returns.

This change set was discussed and deferred at GEC19. We do not yet have sufficient motivation, and there are questions about this proposal.

This change would clarify the support for internationalization in this API, and allow aggregates to specify unicode support. This change should not require any clients or aggregates to make changes. This change would specify:

Aggregates are encouraged to support non-ASCII letters as input. Clients are encouraged to support non-ASCII returns from aggregates. However, this change set does not require that either aggregates or clients support non-ASCII strings, and aggregates should not require clients to accept non-ASCII results where the API does not explicitly require it.

This change would allow an aggregate to explicitly specify that arguments may be unicode, and returns may be unicode if the content demands it. By default, aggregates are expected to use the ASCII only regular expressions that the rest of this API specification uses, and to limit returns to ASCII characters where possible. This change would add an optional return from GetVersion that allows an aggregate to specify that unicode arguments are acceptable (ASCII strings must still be accepted), and that all returns may be unicode. When the aggregate specifies this option, it shall still attempt to support ASCII inputs to allow clients to operate only in ASCII where possible.

Specifically, this change adds geni_unicode_compliant to the return from GetVersion.

  • This field is optional.
  • Default is false. All strings specified in this API are to be treated as ASCII unless otherwise specified, (e.g. the format explicitly references a standard that supports unicode). This includes both arguments and returns.
  • When true, all strings specified in this API may be unicode. In particular, regular expressions that limit to ASCII characters shall be interpreted to include the unicode equivalents. The aggregate will still accept ASCII input. Returns will remain ASCII where the API defines ASCII constants or content allows it.

Declined: No Reservation Allocate

Tools should be able to do complex topology embedding to figure out where a particular request can be instantiated. Aggregates know best what they can do. This proposal provides a cheap way to ask "could you satisfy this request, and if so what would I get?"

This proposal is declined: Allocate itself is pretty cheap.

  • Add a new option to Allocate named geni_no_reservation. If supplied with a non-empty value (not null or an empty string), the request is not asking for the resources to actually be reserved. Instead, the aggregate should calculate what it would reserve, and return the usual allocate return structure, but without actually making the reservation. As such, geni_expires should be set to the current time, and the status should remain geni_unallocated, and the geni_sliver_urn element should be empty (no sliver is created or assigned). Since there is no sliver URN, no calls to other methods should return any information on this sliver.