wiki:AaronHelsinger/GAPI_AM_API_DRAFT

Version 40 (modified by Aaron Helsinger, 7 years ago) (diff)

--

  1. GENI Aggregate Manager API Draft Revisions
    1. Proposing Additional Changes
  2. Proposed changes for GENI Aggregate Manager API version 3
  3. Summary
    1. Proposed Changes
    2. Unspecified items
  4. Change Set C: UpdateSlivers
    1. Motivation
    2. UpdateSlivers
  5. Change Set D: Sliver-specific operations
    1. Motivation
    2. Define sliver
    3. Addressable Slivers
  6. Change Set E: Tickets
    1. Motivation
    2. Tickets semantics
  7. Change Set F: Support AM and resource-type specific methods.
    1. Motivation
    2. Change Set F1: Define Sliver States
    3. Change Set F2: ActOnSlivers
    4. Change Set F3: Sliver Allocation States and methods
    5. Change Set F4: Sliver Operations Method
    6. Change Set F5: Sliver Operational States
  8. Adopted: Change Set G: Credentials are general authorization tokens.
    1. Motivation
    2. Make credentials more general
    3. Advertising supported credentials
    4. Specifying Type of Supplied Credentials
  9. Changes to existing methods
    1. Change Sets H&I: Misc other method changes
    2. Withdrawn: Change Set J
  10. Adopted: Change Set K: Standardize certificates and credentials
    1. Motivation
    2. Changes
  11. Change Set L: Change SFA credentials' privileges
  12. Change Set M: New Method Signatures
    1. M1: users struct an option
    2. M2: Split ListResources
  13. Changes Not Included
    1. RSpec changes resulting in GENI v4 RSpecs
    2. Misc
    3. Stitching
  14. Change summary - method signatures
    1. Common Arguments and Returns
      1. structs and optional arguments
      2. datetime data type
      3. RSpec data type
      4. credentials
      5. geni_end_time
      6. geni_best_effort
      7. geni_users
      8. options
      9. urns[]
    2. GetVersion
    3. ListResources
    4. Allocate
    5. Renew
    6. UpdateAllocations
    7. Provision
    8. UpdateSlivers
    9. Status
    10. PerformOperationalAction
    11. !Delete
    12. Shutdown

GENI Aggregate Manager API Draft Revisions

This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, some of the revisions documented here have been discussed on the GENI developer mailing list, and during at least one GEC. Other revisions are in early discussions and subject to change or abandonment.

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is a set of resources allocated by one Aggregate to one Slice. See below for a proposed complete definition.

The current officially adopted version of the API is 2 and is documented on the main API page.

API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They include:

  1. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    • Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    • Omni version 1.3 (released June 2011) added client software support for these changes.
  2. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.

This page documents proposed changes for AM API version 3. These changes are grouped into sets. API Version 3 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.

Proposing Additional Changes

GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.

Technical discussions are generally held on the Developers mailing list

Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net

Proposed changes for GENI Aggregate Manager API version 3

This page documents a long list of proposed changes for AM API version 3. These changes provide ways for aggregates to expand GENI functionality, without requiring further API modifications.

There are many changes here. As such, aggregates may implement these to varying degrees.

  • Clients are reminded that these methods are requests - based on the AM type and resource types, these requests may fail or not make sense. Clients should watch for UNSUPPORTED returns.
  • AMs are encouraged to implement as much of this API as reasonable to provide a common front for clients. When a function is not possible, return UNSUPPORTED, document publicly what functions do work, and suggest alternative ways to get the result the client desired.

Summary

Proposed Changes

Multiple change sets have been proposed.

Below, you will see that some change sets have been withdrawn (J). Others have been superseded, meaning that other proposals implement similar concepts in an alternative way (Change Sets F1 and F2). Some sets may be postponed, to be considered for a later version of this API (L). In addition, two change sets are waiting on revised proposals (C and E).

Given that, readers of this page should focus on Change Sets F, G, I, K, and M.

At the top level, the proposed changes for AM API v3 include:

  • Change Set C: On hold (waiting for a revised proposal). Add the ability to UpdateSlivers to immediately modify your reservation
  • Change Set D: Slivers: Partially superseded. Change methods to clarify that there may be multiple slivers per slice at an AM, and to allow operating on individual slivers
  • Change Set E: Tickets: On hold (waiting for a revised proposal). Add methods using tickets to do negotiated reservations
  • Change Set F1: Superseded by F3 and F4: Define sliver states, and the state changes that various methods cause
  • Change Set F2: Superseded by F3 and F4: Add a new general ActOnSlivers method allowing AMs to support AM and resource-type specific operations
  • Change Set F3: Sliver Allocation States and methods
  • Change Set F4: Method to perform Sliver Operational actions
  • Change Set F5: Sliver Operational States
  • ADOPTED: Change Set G: Generalize the credentials argument, allowing ABAC support
  • Change Set H: Clarify: A second call to CreateSlivers without an intervening DeleteSlivers is an error.
  • ADOPTED: Change Set I1: SliversStatus return structure includes sliver expiration
  • ADOPTED with changes: Change Set I2: SliversStatus return includes SSH logins/key for nodes that support SSH access
  • ADOPTED: Change Set I3: CreateSlivers return becomes a struct, adds sliver expiration
  • Change Set I4: CreateSlivers optionally does not start resources.
  • Withdrawn: Change Set J: Support proxy aggregates with 1 new option and 1 new GetVersion entry
  • ADOPTED: Change Set K: Standardize certificate contents, etc
    • Include a real serial number, holder email, holder uuid, and optionally authority URL in certificates
    • Define slice ID as the UUID plus URN in slice certificates
    • Define slice name, sliver name, and user name restrictions, and similar for URNs
    • Publish schemas for credentials and certificates
  • Postponed: Change Set L: Standardize slice credential privileges
  • Change Set M: New method signatures, incorporating all previous adopted change sets

For a full listing of the proposed API methods after all these changes, see below.

Unspecified items

  • Semantics of adding or removing slivers from the full set of slivers in a slice at an aggregate
  • An acceptable Update proposal
  • RSpec for advertising operational states for a sliver type
  • Publish credential schema
  • Define error codes returned by new methods, conditions

Change Set C: UpdateSlivers

Add an ability for experimenters to modify their allocated resources at an aggregate without deleting (and possibly losing) existing resource allocations.

This change was briefly discussed at GEC13, and is a topic for ongoing discussion.

Motivation

A common complaint among experimenters about the current AM API is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resuorces for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources.

This topic was discussed at the GEC12 AM API session and on the GENI dev mailing list (in October and November).

This topic was also discussed on the geni-dev mailing list here: http://lists.geni.net/pipermail/dev/2012-March/000588.html and here: http://lists.geni.net/pipermail/dev/2012-March/000643.html and elsewhere in March 2012.

A competing related proposal from Gary Wong is here: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

A further revised proposal is expected from Jon Duerig.

UpdateSlivers

This change would add a new method UpdateSlivers, which takes a full request RSpec of the desired final state of the slice at this aggregate. This proposal calls for adding this functionality in the context of tickets: the UpdateSlivers method returns a ticket. See Change Set E: Tickets for details.

Some points about this change:

  • The method takes a full request RSpec - not a diff.
    • Note that we want the manifest to be readily modifiable to be a request (include component_ids and sliver_ids), but it is not yet.
    • AMs may, as always, return UNSUPPORTED - EG if they are incapable of determining what changes to apply (computing a diff).
  • The request is either fully satisfied, or fails (returns an error code).
  • AMs must document the level of service they provide: will any state be lost on existing resources?
  • Options includes geni_end_time, a RFC3339 requested end time for the reservation. See below for details.
    • If omitted, the AM may reserve the resources for the default sliver duration.
    • AMs should follow the logic of RenewSlivers to determine if the requested duration of the sliver is acceptable.
    • The request should Fail (return an error code) if the resources cannot be reserved until the requested time.

Aggregates and experimenters should use an RSpec extension for specifying a guarantee on state loss per-resource: EG:

  <node ... sliver_id="urn:publicid:IDN+jonlab.tbres.emulab.net+sliver+250">
    <sliver_type name="raw-pc"/>
...
    <preserve:preserve guarantee="persistent-state" />
  </node>

This uses the RNC schema:

default namespace = "http://www.protogeni.net/resources/rspec/ext/preserve/1"

# This is meant to extend a node or link
Preserved = element preserve {
  attribute guarantee { "none" | "persistent-state" |
                        "dynamic-state" | "no-disruption" }
}

start = Preserved

In the above schema, the states represent increasing levels of state preservation guarantee.

struct UpdateSlivers(string slice_urn, string credentials[], string rspec, 
                                                 struct options)

Returns a struct:

{
  ticket: <string, the ticket>
  geni_status: <string, the sliver state - ticketed>,
 <others that are AM specific>
}

See Change Set E: Tickets for details on the ticket returned and ticket semantics.

Change Set D: Sliver-specific operations

A slice may have multiple slivers at a single AM. Experimenters can operate on slivers independently, if the AM supports it. AMs define slivers as groups of resources, and give them locally unique sliver_urns for identifying that group of resources.

This change was briefly discussed at GEC13, and remains open for discussion. Concerns include whether aggregates can support individually operating on slivers, and whether we should make this change without that functionality.

See http://lists.geni.net/pipermail/dev/2012-March/000593.html as well.

Motivation

This change set was discussed at the GEC12 AM API session.

The current AM API calls take a Slice URN, and operate on all resources under that label at the given aggregate - all the resources for that slice at the aggregate are allocated, renewed, and deleted together. There is no provision for releasing some of the resources allocated to the slice at that aggregate, or for adding new resources to the reservation for that slice at a given aggregate.

This ties closely to the precise definition of a Slice vs a Sliver. The current AM API methods imply that a sliver represents all resources at an aggregate for a given slice. However, this does not match the definition that previous GENI documents have used, nor the functionality that experimenters desire.

Previous GENI documents have used this definition: A sliver is the smallest set of resources at an aggregate that can be independently reserved and allocated. A given slice may contain multiple slivers at a single aggregate. A sliver may contain multiple components.

Given this definition, the current AM API methods in fact operate on a group of slivers.

This change set would provide a means for experimenters to operate on individual slivers within their slice at a given aggregate.

Define sliver

A Sliver is an aggregate defined grouping of resources within a slice at this aggregate, whose URN identifies the sliver, and can be used as an argument to methods such as DeleteSlivers or RenewSlivers, and whose status can be independently reported in the return from SliversStatus. The AM defines 1 or more of these groupings to satisfy a given resource request for a slice. All reserved resources are directly contained by exactly 1 such sliver container, which is in precisely 1 slice.

Slivers are identified by an aggregate selected URN. See other change proposals for details on standardizing such URNs.

Addressable Slivers

Considering the clarified sliver definition, several API names are misleading. This change proposal modifies those method names to clarify that they may work with multiple slivers. Additionally, some methods can logically operate on individual slivers: this change modifies those methods' arguments to allow specifying a particular sliver.

This proposal is missing a supplementary proposal of the semantics involved in operating on individual slivers, part of the slice at an AM. In particular, defining whether CreateSlivers can be called multiple times, and the semantics of doing DeleteSlivers or RenewSlivers on only some of the slivers in a slice at an aggregate.

The specific method signature changes of this proposal are superseded by proposal F3 and the proposed summary methods at the bottom of this page.

  1. Rename some existing methods to clarify that they act on 1+ slivers:
    • CreateSliver -> CreateSlivers
    • RenewSliver -> RenewSlivers
    • DeleteSliver -> DeleteSlivers
    • SliverStatus -> SliversStatus
  1. Some methods that take slice_urn now take a urn that may be a slice or sliver:
    • RenewSlivers, DeleteSlivers, SliversStatus
    • AMs are responsible for distinguishing whether the request operates on a slice or a sliver (see Change Set K which defines how slice and sliver URNs differ).
    • AMs are free to refuse to Renew, Delete, or provide status on an individual sliver, if the local AM or that resource type does not support it.
      • AMs should return an error message. Clients may often use UpdateSlivers instead to similar effect.

Open Question: Should the methods take a list of urns, 1 of which may be a slice_urn?

Change Set E: Tickets

AM APIv3 adds support for negotiated reservations or two-phase commit, by adding methods that allow an experimenter to reserve resources for a slice without committing to using them, or forcing the AM to incur the cost of instantiating them.

This change is actively under discussion. See for example http://lists.geni.net/pipermail/dev/2012-March/000616.html and http://lists.geni.net/pipermail/dev/2012-March/000648.html

For an alternative proposal, see: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

The later proposal for allocation states (below) supersedes this. See Change Set F3.

Motivation

This possible change was discussed at the GEC12 AM API session.

The SFA defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.

Tickets are used in the ProtoGENI CMV2 interface, and are discussed on the PG wiki. Tickets with a slightly different semantics (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see the Orca Book. However, each of these uses of the notion of tickets differs.

Tickets would potentially enable a number of useful and possibly critical features:

  • Coordinated or negotiated reservations: reserving resources from aggregate B only if aggregate A can give you a complementary resource. For example, a VLAN tag. This is related to stitching, both network stitching and the more general form.
  • Two phase commit reservations (similar to the above).
  • Scheduled reservations in the future.
  • Brokers: 3rd parties consolidating, scheduling and allocating resources on behalf of a number of other aggregates
  • Lending resources to other experimenters
  • Giving experimenters explicit control over when resources are started, stopped, and restarted (see the discussion on UpdateSliver).

Tickets semantics

This proposal would add tickets to the existing AM API, allowing experimenters to reserve (hold) resources temporarily and cheaply. Tickets represent a promise to the named slice to allocate the specified resources, if the ticket is 'redeemed' while the ticket is still valid. Tickets describe a complete specification of the resources that will be allocated to the slice at this aggregate, if the ticket is redeemed.

Some key properties of the tickets proposed here:

  • Tickets are IOUs from an AM to a slice (not to an experimenter - no delegation is necessary or possible).
  • Experimenters do not need to use tickets to reserve resources: existing methods without tickets still work.
  • A ticket is a promise by the AM to give the specified resources to the slice if an authorized slice member requests them.
    • The aggregate is saying they will not give away these resources to other slices, but only to this slice.
    • AMs must document how firm their promises are. See the attribute geni_ticket_service_class.
      • Some aggregates may only offer soft promises, as in PlanetLab.
  • Tickets are signed by the AM: non repudiatable.
  • Tickets are bound to a slice: they contain the slice certificate.
  • Tickets may be passed from 1 researcher on a slice to another freely - no explicit delegation is required.
    • Indeed, any experimenter with appropriate slice credentials can retrieve the ticket from the aggregate.
    • Tickets may not be delegated to another slice or other entity; these tickets do not support brokers.
  • Tickets promise a particular set of resources: they include an RSpec. Note that this may be an unbound RSpec.
    • Note that we do not currently have unbound manifest RSpecs. For now we specify only that this is an RSpec.
  • Tickets are good for a limited time.
    • They must be redeemed by a specified time, redeem_before, after which the aggregate is free to assign the resources elsewhere.
      • Aggregates determine redeem_before, which is some epsilon in the near future.
      • Aggregates may accept a new option geni_reserve_until which is a request for a particular redeem_before, but are not required to support this (they may ignore the option).
    • Tickets specify when the resources will be available from (starts, typically essentially now), and when they will be available until (typically now plus the aggregate-local default sliver expiration time).
      • The resources may be available even longer, but that would require a separate RenewSlivers call.
  • Tickets specify the full final state of the slice after applying this ticket.
    • Tickets are not incremental changes, and are not additive.
    • The implication is that there may be only 1 ticket outstanding for a slice per aggregate (except for scheduled reservations, see below).
    • This also implies that these tickets are not suitable for use by brokers.
  • Aggregates must attempt to honor their promises. As a result, aggregates must remember all outstanding tickets until they are redeemed or expire.
  • All ticket related timestamps must be in the format of RFC3339 (http://www.ietf.org/rfc/rfc3339.txt)
    • Full date and time with explicit timezone: offset from UTC or in UTC.
    • eg: 1985-04-12T23:20:50.52Z or 1996-12-19T16:39:57-08:00

Change Set F: Support AM and resource-type specific methods.

Define the control API (the AM API) as about moving slivers through various states at an AM.

The proposal here elicited concerns (the method ActOnSlivers is an ioctl, and the states mix allocation and operational states).

A later alternative proposal, see http://lists.geni.net/pipermail/dev/2012-March/000721.html

At the GEC13 coding sprint, a variant on the above was approved. It is documented here as Change Set F3.

A variant on the operational states proposal is defines as Change Set F4 and documented here: https://openflow.stanford.edu/display/FOAM/GENI+-+PerformOperationalAction

Motivation

AM API methods logically change the state of the slivers at this AM. But the API is not clear what experimenters should expect, and does not provide easy ways for experimenters to control when and how states change. There is in particular no way to move slivers through states and change them in ways otherwise undefined by the API.

Change Set F1: Define Sliver States

This change is superseded by Change Sets F3 and F4.

Currently the AM API defines several possible states as valid returns in SliversStatus: configuring, ready, unknown, and failed. This change changes and expands that list of valid states, and explicitly defines the expected states after each AM API method call. Additionally, this change provides a mechanism for aggregates to supply their own states.

The GENI AM API can be thought of as manipulating slivers. As such, each method potentially changes the state of 1 or more slivers. With the changes proposed here, several of the methods return a new geni_status field, whose value is one of the standard GENI sliver status values. Aggregates must use one of the standard GENI values for that return.

Change Set F2: ActOnSlivers

This change is superseded by Change Sets F3 and F4.

This change introduces a new method, providing a generic way to act on slivers in an AM or resource type specific way. This method shall be used to 'start' or 'stop' or 'restart' resources that have been allocated but not started by CreateSlivers or RedeemTicket. It may also be used to change the state of slivers (or their contained resources) in an aggregate or resource specific way. Some aggregates may use this method to change configuration details of allocated resources. This might include changing acceptable login keys.

Change Set F3: Sliver Allocation States and methods

This change was discussed and adopted at the GEC13 Coding Sprint.

For meeting minutes, see: the GEC13 Coding Sprint agenda page.

  • We agreed to use two kinds of states: allocation states, and operational states. We put off discussion of operational states (ie is the node booted), noting however that this is critical. See Change Set F4.
  • We debated whether the API should specify a limited number of states, or allow for aggregate or resource specific states. We agreed that for allocation states, the API should define a limited set of states, while operational states might be more permissive.
  • We discussed the pros and cons of including a single all-in-one method to change allocation states, or a single method per desired transition. There is at least 1 case where there are 2 paths between the same 2 allocation states with very different meaning. As a result, we agreed to use a separate method per allocation state change.

We agreed on 3 allocation states for slivers and an enumeration of methods for transitioning between those states.

Allocation states:

  1. geni_unallocated (alternatively called 'null'). The sliver does not exist. This is the small black circle in typical state diagrams.
  2. geni_allocated (alternatively called 'offered' or 'promised'). The sliver exists, defines particular resources, and is in a sliver. The aggregate has not (if possible) done any time consuming or expensive work to instantiate the resources, provision them, or make it difficult to revert the slice to the state prior to allocating this sliver. This state is what the aggregate is offering the experimenter.
  3. geni_provisioned. The aggregate has started instantiating resources, and otherwise making changes to resources and the slice to make the resources available to the experimenter. At this point, operational states are valid to specify further when the resources are available for experimenter use.

The key change is the addition of state 2, representing resources that have been allocated to a slice without provisioning the resources. This represents a cheap and un-doable resource allocation, such as we previously discussed in the context of tickets. This compares reasonably well to the 'transaction' proposal written up by Gary Wong (http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals). When a sliver is created and moved into state 2 (geni_allocated), the aggregate produces a manifest RSpec identifying which resources are included in the sliver. This is something like the current CreateSliver, except that it does not provision nor start the resources. These resources are exclusively available to the containing sliver, but are not ready for use. In particular, allocating a sliver should be a cheap and quick operation, which the aggregate can readily un-do without impacting the state of slivers which are fully provisioned. For some aggregates, transitioning to this state may be a no-op.

States 2 and 3 (geni_allocated and geni_provisioned) have aggregate and possibly resource specific timeouts. By convention the geni_allocated state timeout is typically short, like the redeem_before in ProtoGENI tickets, or the commit_by in Gary's transactions proposal. The geni_provisioned state timeout is the existing sliver expiration. If the client does not transition the sliver from geni_allocated to geni_provisioned before the end of the geni_allocated state timeout, the sliver reverts to geni_unallocated. If the experimenter needs more time, the experimenter should be allowed to request a renewal of either timeout. Note that typically the sliver expiration time (timeout for state 3, geni_provisioned) will be notably longer than the timeout for state 2, geni_allocated.

State 3, geni_provisioned, is the state of the sliver allocation after the aggregate begins to instantiate the sliver. Note that fully provisioning a sliver may take noticeable time. This state also includes a timeout - the sliver expiration time (which is not necessarily related to the time it takes to provision a resource). RenewSliver extends this timeout. For some aggregates and resource types, moving to this state from state 2 (geni_allocated) may be a no-op.

If the transition from one state to another fails, the sliver shall remain in its original state.

These are the only allocation states supported by this API. Since the state transitions are finite, but include potentially multiple transitions between the same two states, this API uses separate methods to perform each state transition, rather than a single method for requesting a new state for the sliver.

  1. Allocate moves 1+ slivers from geni_unallocated (state 1) to geni_allocated (state 2). This method can be described as creating an instance of the state machine for each sliver. If the aggregate cannot fully satisfy the request, the whole request fails. This is a change from the version 2 CreateSliver, which also provisioned the resources, and 'started' them. That is Allocate does 1 of the 3 things that CreateSliver did previously.
  2. Delete moves 1+ slivers from either state 2 or 3 (geni_allocated or geni_provisioned), back to state 1 (geni_unallocated). This is similar to the AM API version 2 DeleteSliver.
  3. Renew, when given allocated slivers, requests an extended timeout for slivers in state 2 (geni_allocated).
  4. Renew can also be used to request an extended timeout for slivers in state 3 - the geni_provisioned state. That is, this method's semantics can be the same as RenewSliver from AM API v2.
  5. Provision moves 1+ slivers from state 2 (geni_allocated) to state 3 (geni_provisioned). This is some of what version 2 CreateSliver did. Note however that this does not 'start' the resources, or otherwise change their operational state. This method only fully instantiates the resources in the slice. This may be a no-op for some aggregates or resources.

These states apply to each sliver individually. Logically, the state transition methods then take a single sliver URN. For convenience, these methods accept a list of sliver URNs, or a slice URN as a simple alias for all slivers in this slice at this aggregate.

Since each method may operate on multiple slivers, each of these methods returns a list of structs as the value:

value = [
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_expires: <time when the sliver expires from its current state>,
   <others AM or method specific>
   <Provision returns geni_operational_status>
  },
  ...
]

Allocate returns a single manifest RSpec, plus the above list of structs.

Aggregates must be consistent across all these methods whether they are all or nothing, or support partial success.

These methods all take a new option (aggregates must support it, clients do not need to supply it):

   geni_best_effort: <XML-RPC boolean 1/0, default 0>

If false, the client is requesting that the aggregate either fully satisfy the request, moving all listed slivers to the desired state, or fully fail the request, leaving all slivers in their original state. If the aggregate cannot guarantee all or nothing success or failure given the included slivers and resource types, the aggregate shall fail the request, returning an appropriate error code. If this option is true, then some slivers may transition to the new state, and some note. Aggregates must examine the return closely to know the state of their slivers.

Note: Allocate remains (like v2 CreateSliver) all or nothing (either the aggregate can allocate all desired resources as requested, or the call fails).

Note: These calls are synchronous - when they return, the slivers shall be in their final state. In particular, the transition from state 2 to 3 (geni_allocated to geni_provisioned) should be quick. The resource that is now in the 'provisioned' state may take a long time to actually be ready for operational use (e.g. imaging and booting the node) -- this remains true as in version 2 after CreateSliver. Note that the geni_allocated state is by definition cheap, such that transitioning to this state should also be quick.

SliverStatus, where it currently includes geni_status for each geni_resource, shall now return geni_allocation_status with one of the above defined values, and geni_operational_status. The values of geni_operational_status are still under discussion.

Currently, SliverStatus returns a single geni_status for the entire slice at this aggregate. With this change, the top-level allocation status for the slice is not defined, and that field is not required.

Open Questions:

  • What about an UpdateAllocations method, similar to UpdateTickets or UpdateTransactions from other similar proposals, for modifying allocated resources in place, without losing allocated resources?

Change Set F4: Sliver Operations Method

This proposal was discussed on the geni-dev mailing list: http://lists.geni.net/pipermail/dev/2012-March/000743.html

The canonical source for documentation on this proposal is here: https://openflow.stanford.edu/display/FOAM/GENI+-+PerformOperationalAction

Open questions include:

  • Do we need a state discovery mechanism? A method? An RSpec ad extension?
  • Should the method default to all or nothing? Or to partial success?
  • What are the defined operational states, and GENI standard actions for transitions?
  • Does the API specify how operational state of slivers rolls up to the state of the slice at the aggregate?
  • Make option names and returns consistent between this change set and change set F3
struct PerformOperationalAction (string urn[], struct credentials[], string action, 
                                         struct options={})
{
  struct code = {
       int geni_code;
       [optional: string am_type;]
       [optional: int am_code;]
         }
  struct value = [ {
        'geni_urn' : string,
        'geni_operational_status' : string,
        [optional: 'geni_resource_status' : string]
        }, ... ];
  string output;
}

Performs the given action on the given sliver_urn(s) (or slice_urn as a proxy for "all slivers"). Actions are constrained to the set of default GENI actions, as well as resource-specific actions which reasonably perform operational resource tasks as defined by the aggregate manager for the given resource type. This method is not intended to allow for reconfiguration of options found in the request rspec. Aggregate Managers SHOULD return an error code of 13 (UNSUPPORTED) if they do not support a given action for a given resource. Actions are performed on all slivers, or none - if an action cannot be performed on a sliver given, the entire operation MUST fail. Passing the option geni_best_effort allows for partial success.

An AM SHOULD constrain actions based on the current operational state of the resource, such that attempting to perform the action geni_stop on a resource that is geni_busy will fail, but SHOULD also be idempotent for all actions which result in a steady state.

geni_operational_status MUST be the current operational status of the sliver after this action (as would be returned by SliverStatus). An optional geni_resource_status field MAY be returned for each sliver which contains a resource-specific status that may be more nuanced than the options for geni_operational_status.

Calling this method with a slice_urn functions as if all the child sliver_urn's had been passed in - specifically the action is performed on all slivers and all sliver_urn's and their statuses are returned. No status is returned for the slice as a whole.

This is a fast synchronous operation, and MAY start long-running sliver transitions whose status can be queried using SliverStatus.

Change Set F5: Sliver Operational States

Currently, geni_status in SliverStatus can have values configuring, ready, failed, unknown.

This proposal modifies that list, and renames those to use the standard 'geni_' prefix.

These states would be reported by various AM API methods, specifically SliverStatus. And would be used in reasoning about valid operations in PerformOperationalAction

The AM API defines a few operational states with particular semantics. AMs are not required to support them for a given set of resources, but if they use them, they must follow the given semantics. AMs are however STRONGLY encouraged to support them, to provide maximum utility.

Similarly, the API defines a few operational actions: these need not be supported. AMs are encouraged to support these if possible, but only if they can be supported following the defined semantics.

AMs may have their own operational states/state-machine internally. AMs are required to advertise such states and actions that experimenters may see or use, by using Ad RSpec extensions. Operational states which the experimenter never sees, need not be advertised. Operational states and actions are generally by resource type. The standard RSpec extension attaches such definitions to the sliver_type element of RSpecs.

TODO: Jon Duerig will propose this extension, with examples covering PG/Emulab sliver_types.

Tools must use the operational states and actions advertisement to determine what operational actions to offer to experimenters, and what actions to perform for the experimenter. Tools may choose to offer actions which the tool does not understand, relying on the experimenter to understand the meaning of the new action.

States should be defined in terms of (a) whether the resource is accessible to the experimenter on the data or control planes, (b) whether an experimenter action is required to change from this state, and if so, (c) what action or actions are useful. If the resource will change states without explicit experimenter action, what is the expected next state on success.

Note that states represent the AM's view of the operational condition of the resource. This state may be wrong due to faulty information at the AM.

Any operational action may fail. When this happens, the API method should return an error code. The sliver may remain in the original state. In some cases, the sliver may transition to the geni_failed state.

There is no busy state. Instead, AMs are encouraged to define separate such transition states for each separate transition path, allowing experimenters to distinguish the start and end states for this transition.

shutdown is not an operational state for a sliver. The Shutdown() API method applies to an entire slice.

GENI defined operational states:

  • geni_notready: A final state. The resource is not usable / accessible by the experimenter, and requires explicit experimenter action before it is usable/accessible by the experimenter. For some resources, geni_start will move the resource out of this state and towards geni_ready.
  • geni_configuring: A wait state. The resource is in process of changing to geni_ready, and on success will do so without additional experimenter action. For example, the resource may be powering on.
  • geni_ready: A final state. The resource is usable/accessible by the experimenter, and ready for slice operations.
  • geni_ready_busy: A wait state. The resource is performing some operational action, but remains accessible/usable by the experimenter. Upon completion of the action, the resource will return to geni_ready.
  • geni_failed: A final state. Some operational action failed, rendering the resource unusable. An administrator action, undefined by this API, may be required to return the resource to operations.

GENI defined operational actions:

  • geni_start: This action results in the sliver becoming geni_ready eventually. The operation may fail (move to geni_failed), or move through some number of transition states. See EG booting a VM.
  • geni_restart: This action results in the sliver becoming geni_ready eventually. The operation may fail (move to geni_failed), or move through some number of transition states. During this operation, the resource may largely remain accessible/usable. Dynamic state associated with this resource may be lost by performing this operation. See EG re-booting a VM.
  • geni_stop: This action results in the sliver becoming geni_notready eventually. The operation may fail (move to geni_failed), or move through some number of transition states. See EG powering down a VM.

Actions are performed using the above proposed PerformOperationalAction.

Adopted: Change Set G: Credentials are general authorization tokens.

This change was adopted at GEC13.

Motivation

Most AM API methods take a list of credentials to authorize operations. Currently the API requires credentials in a particular format, and would disallow others, such as ABAC. The API should allow for other innovative authorization tokens.

Make credentials more general

This change modifies the credentials argument to all methods. Each credential is now defined only as a signed document. A given list of credentials may contain credentials in multiple formats. The list may be empty. A given authorization policy at an AM may require 0, 1, or many credentials. Aggregates are required to allow credentials which are not used by local authorization policy or engines, using only credentials locally relevant.

  • An AM must pick credentials out of the list that it understands and be robust to receiving credentials it does not understand.
    • Current slice and user credentials will be recognizable for following the schema defined in Change Set K.
  • AMs are required to continue to accept current-format credentials.
    • In particular, a single standard slice credential remains sufficient for most authorization policies.
  • Other credential formats acceptable by some aggregates might include ABAC x509 Attribute certificates, eg.
  • AMs may get other authorization material from other sources: EG a future Credential Store service.

Advertising supported credentials

We agreed that aggregates must advertise what types of credentials they accept so clients know how to gain authorization for API methods.

Aggregates are required to return a new entry in GetVersion:

geni_credential_types = <a list of structs>: [
  {
   geni_type: <string, case insensitive>,
   geni_version: <string containing an integer>,
   <others fields optionally. EG A URL for more info, or a schema>
  }
]

We agreed that "sfa" slice credentials as defined pre AM API version 3 will have type=geni_sfa and version=2. "sfa" slice credentials as of AM API version 3 will be type=geni_sfa, version=3. ABAC credentials as of AM API version 3 will be type=geni_abac, version=1.

For example, an aggregate that accepts ABAC credentials, SFA slice credentials that were issued prior to AM API v3, and SFA slice credentials from AM API version 3, would include this in GetVersion:

geni_credential_types = [
  {
   geni_type = "geni_sfa",
   geni_version = "2"
  },
  {
   geni_type="geni_sfa",
   geni_version = "3"
  },
  {
   geni_type="geni_abac",
   geni_version="1"
  }
]

Note that there might be multiple geni_type entries to support multiple versions of SFA credentials. Note there might also be multiple kinds of ABAC credentials: identity certificates, attribute certificates, references to those certificates, bundles of those certificates. We briefly considered adding a sub_type field or a flag for 'credentials' that are really references to credentials.

Specifying Type of Supplied Credentials

For AMs to understand the type of each supplied credential, per-AM heuristics are not sufficient - they might differ at different AMs.

The API will require that credentials be explicitly typed. This change makes methods take in the credentials argument a struct:

credentials = [
   {
    geni_type: <string>, 
    geni_version: <string>,
    geni_value: <string>,
    <others>
   }
]

Note that the value may be a credential, a URL, an XLink compliant string, etc. Clients are required to identify the type of each credential they supply. Instead of requiring clients to apply similar heuristics, authorities are required to identify credentials they supply with the same type and version fields. Specifically, ProtoGENI representatives suggested that their slice authorities would issue credentials as these structures in the near future.

Changes to existing methods

Modify a few existing methods to make certain operations easier or more experimenter friendly.

Change Sets H&I: Misc other method changes

  • Change Set H: A second call to CreateSlivers without an intervening DeleteSlivers is an error.
    • This change just clarifies expected behavior that was left under-specified in AM API v1.
    • CreateSlivers takes a full Ad RSpec, it is not a way to 'add' resources.
    • Silently replacing the existing slivers with new slivers (similar to a call to UpdateSlivers) is not acceptable.
  • Adopted: Change Set I1: Add geni_expires to return from SliversStatus for whole slice and then each sliver
    • This change was adopted at GEC13
    • This change standardizes behavior necessary for experimenters to determine their sliver expiration times.
    • Format is RFC3339 (http://www.ietf.org/rfc/rfc3339.txt)
      • Full date and time with explicit timezone: offset from UTC or in UTC)
      • eg: 1985-04-12T23:20:50.52Z or 1996-12-19T16:39:57-08:00
  • Adopted: Change Set I2: Add SSH logins/keys to each node that supports SSH login in the manifest RSpec

This change standardizes behavior so experimenters can readily find how to log in to reserved resources. Aggregates that allocate resources that an experimenter can 'log in to', should use this struct to return that information. Other aggregates will not use this at all.

This change was adopted at GEC13.

Aggregates shall use a new RSpec extension to include all login information in manifest RSpecs. This extension is version controlled in the GENI RSpec git repository.

The extension adds information to the <services> tag, which already has the <login> tag.

The <login> tag tells you the kind of authentication (ssh), the port, and the username. The new extension adds an entry per login username

  • URN of the user
  • 1+ public SSH keys that can be used under that login

Note that 1 of the <user:services_user login>s in the extension duplicates the default username already in the base <login> tag. The extension allows specifying the keys usable with that login username.

EG:

.......
  <services>
    <login authentication="ssh-keys" hostname="pc27.emulab.net" port="22" username="flooby"/>
    <ssh-user:services_user login="flooby" user-urn="http://urn:publicid:IDN+jonlab.tbres.emulab.net+user+flooby">
      <ssh-user:public-key>asdfasdfasdf;lkasdf=foo@bar</ssh-user:public-key>
      <ssh-user:public-key>asdfasdfasdf;lkjasdf;lasdf=foobar@barfoo</ssh-user:public-key>
    </ssh-user:services_user>
    <ssh-user:services_user login="io" user-urn="http://urn:publicid:IDN+jonlab.tbres.emulab.net+user+io">
      <ssh-user:public-key>asdfasdfasdf;lkasdf=foo@bar</ssh-user:public-key>
      <ssh-user:public-key>asdfasdfasdf;lkjasdf;lasdf=foobar@barfoo</ssh-user:public-key>
    </ssh-user:services_user>
  </services>

And the RNC schema:

# An extension for describing user SSH login credentials in the manifest

default namespace = "http://www.protogeni.net/resources/rspec/ext/ssh_user/1"

# This is meant to extend the services element
Services = element services_user {
  attribute login { string } &
  attribute user_urn { string }? &
  element public-key { string }*
}

# Both of the above are start elements.
start = Services
  • A note on distinguishing ListResources from SliversStatus:
    • ListResources in the context of a slice URN is for listing the reserved resources. It provides mostly static information. (But if the manifest contains things which can change, then the manifest must change when those things (like say IP addresses) change.)
    • SliversStatus is for everything else: anything which the AM can change for you using API calls, or which changes over time. So it has up/down state, expiration time, and now login keys. It provides that for your whole slice at this aggregate and all contained slivers.
  • Adopted: Change Set I3: Return sliver expiration from CreateSlivers

This change was adopted at GEC13.

Experimenters currently do not know the expiration of their slivers without explicitly asking. This change makes the CreateSlivers return value become a struct:

{
 rspec: <string manifest>,
 geni_expires: =<RFC3339 sliver expiration string, as in geni_expires from SliversStatus>,
 geni_allocation_status: <string sliver state - allocated or changing or ready>,
 <others that are AM specific>
}
  • Change Set I4: CreateSlivers optionally does not start resources.

Currently, CreateSlivers auto starts resources, moving them from allocated through changing to ready.

  • Add a new option geni_donotstart:
    • If supplied and true (boolean: 0 or 1 in XML-RPC), final state is allocated, and experimenter must explicitly start resources using ActOnSlivers.
    • If omitted or false (0) (default), AM automatically starts resources as before, and state will be changing and then ready.

Withdrawn: Change Set J

This change set has been withdrawn.

Adopted: Change Set K: Standardize certificates and credentials

This proposal was adopted at GEC13

Motivation

The current AM API specifies that certificates and credentials follow a particular format, using URNs that are based on experimenter supplied names. However that specification is not sufficiently specific, and there are currently differences in implementation among existing certificate and credential producers. This has led to errors, experimenter confusion, and messy code.

Changes

This proposal requires that certificates include a UUID and email address for the subject. It codifies restrictions on usernames, sliver names, and slice names. The proposal specifies that slices have a UUID to be used to help identify slices in a consistent way over time (slice names may be re-used).

Some overall points:

  • Aggregates are expected to fail requests that use certificates or URNs or names that violate this API.
  • Schemas for certificates & credentials will be published on geni.net.

Certificates:

  • GENI uses x509v3 identity certificates to identity users, slices, aggregates, and authorities, and these restrictions apply to all such certificates.
  • See http://groups.geni.net/geni/wiki/GeniApiCertificates.
  • Aggregates are required to properly validate all certificates to authenticate access to AM API calls, and fail calls that supply invalid certificates.

Certificate contents:

  • Version shall be properly marked: 3
  • serialNum is required to be unique within the certificate authority: each newly issued certificate must have a unique serial number.
  • The Distinguished Name should include a human readable identifier, for both subject and issuer. Details are not specified
  • Only authority certificates (but all authorities that issue certificates) shall be marked CA:TRUE in the x509 v3 basic constraints; Slices and users shall be marked FALSE.
  • The Subject Alternative Name field must include 3 pieces of information
    • Entries are comma separated (', '), and may be in any order.
    • The URN identifier, following GENI URN standards as described here: http://groups.geni.net/geni/wiki/GeniApiIdentifiers
      • The URN is identifiable by looking for the entry beginning "URI:urn:publicid:IDN", for example: URI:urn:publicid:IDN+emulab.net+user+stoller.
    • A UUID, providing a unique ID for the entity.
      • The UUID must be used with the URN to fully identify the slice or user. UUID alone should not be accepted. This ensures that the authority certifying the slice or user is always identified when referring to the slice or user.
      • In the hexadecimal digit string format given in RFC 4122
      • The UUID is identified with this prefix: "URI:urn:uuid" (as specified by RFC4122), for example: URI:urn:uuid:33178d77-a930-40b1-9469-3aae08755743.
      • The COPY tag is not supported.
    • The email address is an RFC2822 compliant and working address for contacting the subject of the certificate (experimenter, authority administrator, or slice owner).
      • The email entry is identified by the prefix "email:", for example: email:stoller@emulab.net
      • The COPY tag is not supported.
      • Note that the slice and user email addresses are addresses for contacting the responsible party - the slice owner or creator and the user. These may be aliases.
  • Recommendation: Authorities are encouraged but not required to include a URL where more information about the subject is available (eg slice authority registry URL). That URL may be included in a certificate extension, in the DN, or in the subjectAltName.

Slices:

  • Slice ID for use over time and space is the UUID plus the URN in the slice certificate.
    • Currently URNs identify slices, but they are not unique over time. This change adds UUIDs to slice identifiers. URNs remain the identifier for slices in AM API calls, and uniquely identify slices for a moment in time. UUID plus URN together uniquely identify slices over time, and can be used for forensics, or for use by authorization modules, such as ABAC. UUIDs alone should not be used to identify slices, but only in conjunction with the URN, which scopes the UUID to the authority which generated the UUID. The UUID essentially may only be used to distinguish between slices with identical URNs.
    • Monitoring and instrumentation interfaces will likely require both the URN and the UUID for recording slice measurements that can be used past the life of the slice, and aggregates must be prepared to provide both.
  • Slice URN alone is a label - unique at a point in time but not over time.
    • Format: urn:publicid:IDN+<SA name>+slice+<slice name>
  • Slice names are <=19 characters, only alphanumeric plus hyphen (no hyphen in first character): '^[a-zA-Z0-9][-a-zA-Z0-9]+$'
  • Aggregates are required to accept any compliant slice name and URN.
    • Note that this currently causes problems at PlanetLab/SFA aggregates, where node login names are based on slice names and are limited to 31 characters.

Slivers:

  • Have a URN (returned in manifest RSpec), determined by the aggregate.
  • This URN should be unique over time within an AM for record-keeping / operations purposes.
    • Format: urn:publicid:IDN+<AM name>+sliver+<sliver name>
  • Sliver names
    • Must be unique over time within that AM, and are selected by the AM.
    • May use only alphanumeric characters plus hyphen.

Usernames:

  • Usernames (user identifiers to the system) are set at the authority.
  • Usernames are case-insensitive internally, though they may be case-sensitive in display.
    • EG JohnSmth as a display name is johnsmth internally, and there cannot also be a user JOHNSMTH.
  • Usernames should begin with a letter and be alphanumeric or underscores - no hyphen or '.': ('^[a-zA-Z][\w]+$').
  • Usernames are limited to 8 characters.
  • User URNs (which contain the authority name and the username) are required to be temporally and globally unique.

Change Set L: Change SFA credentials' privileges

This proposal has not been fully discussed and is likely to be postponed.

Our goal is to simplify and standardize privilege strings used in SFA credentials. Currently there are wildcards, bind, embed, and others. They are confusing. We also want extensibility to use these credentials elsewhere in future.

Credentials should support these kinds of operations:

  • Learn about the slice
  • Add/Modify/Delete resources in the slice
  • Read slice details like I&M?
  • Use the slice
  • Operator shutdown

Proposal - Replace all existing privileges with only the following possible strings (case insensitive):

  • CanWrite
    • If present in a valid slice credential, aggregates may permit CreateSlivers, RenewSlivers, DeleteSlivers, Shutdown, plus new methods ActOnSlivers, UpdateSlivers, GetTicket, RedeemTicket, UpdateTicket, ReleaseTicket
    • Thus it replaces bind, embed, control, instantiate, sa, pi, or * in various places
  • CanRead
    • If present in a valid slice credential, aggregates may permit ListResources with a slice_urn, SliversStatus
    • Thus it replaces info or * in various places
  • CanReadDetails
  • CanUse

Note that those last 2 may never get used, but are there in case I&M or opt-in make those useful.

Note also that operators who wish to shut down a slice would need a slice credential with the CanWrite privilege.

Privilege and credential semantics are defined as follows:

  • Aggregates may only grant access using current SFA credentials to a method if at least one such valid credential:
    • grants the required privilege or privileges (if any)
    • to the caller of the API method
      • (identified by their SSL client certificate and the owner_gid in the credential)
    • over the slice (if any) on which they are operating
      • (target_gid in the credential).
  • Other privileges may be present in the same or other credentials, and other non-SFA credentials may be used to authorize actions (per Change Set G).
  • Local aggregate policy may grant or deny access to a particular method regardless of the presence of a valid credential granting the required privilege. This depends in part on federation policy governing aggregates.
  • Some operations (e.g. GetVersion) may either simply require a valid credential with no particular privilege, or have no credentials argument at all.

Note also that some current AMs do not require any particular privileges to do ListResources, even with a slice_urn. This change suggests that aggregates require a valid slice credential with CanRead privileges to authorize this operation using current slice credentials.

Change Set M: New Method Signatures

If all the other adopted change set proposals are adopted, there will be new method signatures.

In some cases, the proposals are not clear in terms of the details of the resulting method signatures. This proposal consolidates those separate proposals, to propose a new set of method signatures.

There are a few other small changes that this change set covers.

The details of the proposed final method signatures are listed below.

M1: users struct an option

Previously, the CreateSliver method took a users[] struct to specify information for logging in to resources. But this struct is not universally applicable. This change moves that struct to be an option within the options struct, named geni_users[].

M2: Split ListResources

Currently, ListResources has two forms: get an advertisement general to the aggregate, and get a manifest specific to a slice. This proposal splits those two modes into two separate methods, ListResources, and Describe.

ListResources would no longer take a slice_urn option, and no longer ever return a manifest RSpec. Describe would be used to achieve the same functionality.

struct Describe(string urns[], struct credentials[], struct options[])

Where options include:

{
  boolean geni_compressed <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, exception geni_rspec_version which is required. The aggregate must honor all supplied options.

credentials is the standard argument defined elsewhere.

Note that the manifest RSpec for allocated slivers may contain less detail than for provisioned slivers.

Return struct:

{
   geni_rspec: <geni.rspec, Manifest - may be empty though)>
   geni_urn: <string slice urn, as requested>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - allocated or ?? >,
                  geni_operational_status: <string sliver operational state>
               },
               ...
         ]
}

Changes Not Included

RSpec changes resulting in GENI v4 RSpecs

  • Support unbound manifests
  • Make manifest an extension of Request, so you can readily edit & resubmit a manifest
  • Make configuration information in request and manifest optional, so it can be supplied/returned separately
  • Fully implement the compute ontology from Ilia
  • Ilia's other requests (Openflow related information)
  • Document process for updates per my dev list email
  • Be consistent: ref vs idref vs href
  • Include AM name/URL in RSpecs? Experimenter who allocated it URN?
  • Incorporate stitching extension as part of the 'base' RSpec

Misc

  • Allow Shutdown on a single sliver or a list of slivers
  • Split ListResources with slice_urn from ListResources without. ListResources with slice_urn we call Resolve(slice_urn).
  • Add geni_am_info block to GetVersion return (name, id, url, location, description, is_proxy, proxy: {(a geni_am_info block)}, proxy_for[] (list of geni_am_info blocks))
  • Allow the update methods to take a generic rspec argument, allowing AMs to accept full or diff RSpecs
  • Tickets
    • Remove requestor certificate?
    • Support brokers: Make ticket methods return multiple tickets. Define tickets as optionally diffs (additive). Make RedeemTicket and UpdateTicket take a list of tickets. Tickets are delegatable via signing, but the delegated ticket must be for a strict subset of the resources in the original.
    • Drop ticket struct, not a signed document. Just reference by ID
  • Support message passing
    • Add sendMsg (and getMsgs?) methods that take a signed document, allowing freely passing messages instead of XMLRPC?

Stitching

AMs must support the stitching extension where Layer 2 connections are available:

Update stitching schema per changes here: https://geni.maxgigapop.net/twiki/bin/view/GENI/NetworkStitchingGeniApiAndRspec


Change summary - method signatures

If the generally agreed-upon change sets listed here are adopted, the final method signatures will be as follows:

Common Arguments and Returns

structs and optional arguments

Unless otherwise specified, all arguments and returns of type struct may include aggregate or resource-specific entries. As arguments, such options must be optional for the client to supply, with the aggregate providing a reasonable default.

datetime data type

All datetime arguments and returns in this API shall conform to RFC 3339. This represents a subset of the valid date/time strings permissible by the standard XML-RPC date/time data type, dateTime.iso8601.

In the specification of this API, this is described as dateTime.rfc3339.

RSpec data type

Throughout this API, multiple arguments and returns are labeled as an RSpec. These fields shall be understood as XML documents following one of the schemas advertised in the return from GetVersion. All such RSpecs must pass an XML schema validator, must list all used schemas and namespaces within the document, using schemas that are publicly available. The GetVersion return advertises schemas for advertisment and request RSpecs; the schemas for manifest RSpecs are assumed to be available at the same base URL, but using a corresponding manifest schema.

A fully GENI AM API compliant aggregate will always support the GENI standard schemas for RSpecs, available at http://www.geni.net/resources/rspec. As of 4/2012, the current GENI RSpec version is 3. Aggregates are free to use an alternate format internally, but must accept and produce compliant RSpecs on demand.

The Aggregate Manager API requires this contract: Aggregates advertise the type and version of RSpec formats that they support. If available, they specify the schema, namespace and extensions combination which is the authoritative definition of that format. Clients of the API should understand that combination in order to know how to understand the resources available at that aggregate.

If an aggregate advertises a particular type/version (optionally defined with a combination of schema, namespace and extensions) in the geni_ad_rspec_versions attribute of GetVersion, then it promises to send a correct Advertisement RSpec in response to a ListResources call which supplies a geni_rspec_version option containing that type/version. (geni_rspec_version is a struct with 2 members, type and version. type and version are case-insensitive strings, matching those in geni_ad_rspec_versions).

If an Aggregate advertises a particular type/version (optionally defined with a combination of schema, namespace and extensions) in the geni_request_rspec_versions attribute of GetVersion then it promises to correctly honor an Allocate (was CreateSliver in API v2) call containing a request RSpec in the given format, and then to return a Manifest RSpec in the corresponding format (i.e. a GENI format request is answered with a GENI format manifest). The aggregate also promises to send a correctly formatted Manifest RSpec in response to a ListResources call which supplies a valid geni_slice_urn option and an geni_rspec_version option containing that supported type/version.

In this API, such RSpec fields are labeled as type geni.rspec.

credentials

Many methods take an array of credentials. This is an array of credential type, version, and string value.

credentials = [
   {
    geni_type: <string>, 
    geni_version: <string>,
    geni_value: <string>,
    <others>
   }
]

At least one credential must be a valid slice credential for the slice specified in slice_urn if that is an argument, or for the slice that contains the named slivers, if sliver urns are an argument, or a valid administrative credential with sufficient privileges. When sliver_urns are supplied, all such slivers must belong to the same slice, over which the given credential provides access. Credentials must be valid (signed by a valid GENI certificate authority either directly or by chain, not expired, and grant privileges to the client identified by the SSL client certificate). Each method requires specific privileges, which must be granted by the provided credentials. Note that the semantics of this argument is not clear: most implementations require a single credential to provide all needed privileges. Alternative interpretations might, for example, accumulate privileges from each valid credential to determine overall caller permissions. For details on GENI AM API format credentials, see the GENI wiki.

geni_end_time

The geni_end_time argument requests an expiration of the specified slivers. It is in dateTime.rfc3339 format (defined above). When an explicit argument, it is required, and aggregates must honor the request to the extent local policy permits. When an option in the options struct, clients may omit the option, and AMs may choose not to or be unable to honor this option, but may still succeed the overall request.

geni_best_effort

geni_best_effort: <XML-RPC boolean 1 or 0, default false (0)>

Clients may omit this option, but aggregates must honor the option if possible. This option modifies the way that the operation applies to all named slivers. By default (geni_best_effort=false), the operation must apply equally to all slivers, either succeeding or failing for all. When true, the aggregate may succeed the operation for some slivers, while failing the operation for other slivers.

geni_users

struct geni_users[] is an option for some methods.

Clients may omit this option. Aggregates should honor this option for any resource that accepts the provided login keys, and ignore it for other resources. This option is an array of user structs, which contain information about the users that might login to the sliver that the AM needs to know about. For example, this option is the mechanism by which users supply their SSH public keys, permitting SSH login to allocated nodes. In such cases, the corresponding manifest RSpec will contain the ssh-users element on each such node, showing the login username and applicable public keys. When this option is supplied, each struct must include the key 'keys', which is an array of strings and can be empty. The struct must also include the key 'urn', which is the user’s URN string. For example:

[
  {
    urn: urn:publicid:IDN+geni.net:gcf+user+alice
    keys: [<ssh key>, <ssh key>]
  },
  {
    urn: urn:publicid:IDN+geni.net:gcf+user+bob
    keys: [<ssh key>]
  }
]

options

An XML-RPC struct. For GetVersion only, this argument is optional. In all other methods, it is required. Only ListResources has required entries in the options struct. Aggregates are compliant with this API change by accepting this argument. Aggregates may accept entries to this struct. Aggregates should not require any new options to any method - they should always have a reasonable default for any such option. Aggregates should document new options arguments. The prefix geni_ is reserved for members that are part of this API specification. Implementations should choose an appropriate prefix to avoid conflicts.

urns[]

Several methods take some URNs to identify what to operate on. These methods are defined as accepting a list of arbitrary strings we call URNs. This API defines two kinds of URNs that may be supplied here, slice URNs and sliver URNs (see the GENI identifiers page, and Change Set K). Some aggregates may understand other URNs, but these are not defined or required here. Aggregates that accept only URNs defined by this API will return an error when given URNs not in one of those forms. This API requires that aggregates accept either a single slice URN, or 1+ sliver URNs that all belong to the same slice. Aggregates are not required to accept both a slice URN and sliver URNs, 2+ slice URNs, or a set of sliver URNs that crosses multiple slices. Some aggregates may choose to accept other such combinations of URNs. Aggregates that accept only arguments defined by this API will return an error when given more than 1 slice URN, a combination of both slice and sliver URNs, or a set of sliver URNs that belong to more than 1 slice.

GetVersion

Query static configuration information about this aggregate manager implementation, such as API and RSpec versions supported.

struct GetVersion([optional: struct options])

The options argument may be omitted entirely by clients.

Return struct:

      {
        int geni_api;
        struct geni_api_versions {
             URL <this API version #>; # value is a URL, name is a number
             [optional: other supported API versions and the URLs where they run]
        }
        array geni_request_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_ad_rspec_versions of {
             string type;
             string version;
             string schema;
             string namespace;
             array extensions of string;
        };
        array geni_credential_types of {
             string geni_type <case insensitive>;
             string geni_version <containing an integer>;
       };
    }

ListResources

Retrieve RSpecs listing resources at this aggregate. Without arguments, retrieve an Advertisement RSpec of available resources. With arguments naming a particular slice and/or slivers, retrieve a Manifest RSpec of resources allocated to the given slivers.

struct ListResources(struct credentials[], struct options)

Where options include:

{
  boolean geni_available <optional>;
  boolean geni_compressed <optional>;
  string geni_slice_urn <optional>;
  string geni_sliver_urns[] <optional>;
  struct geni_rspec_version {
    string type;
    string version;
  };
}

Note that all options may be omitted by the client, exception geni_rspec_version which is required. The aggregate must honor all supplied options.

credentials is the standard argument defined above.

Where supplying geni_sliver_urns, whether or not the geni_slice_urn is also supplied, means return a manifest RSpec showing the (allocated or provisioned) slivers listed - not necessarily the complete slice at this aggregate.

Note that the manifest RSpec for allocated slivers may contain less detail than for provisioned slivers.

Return struct:

{
   geni_rspec: <geni.rspec, ad or Manifest - may be empty though)>
   geni_urn: <string slice urn, as requested>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - allocated or ?? >,
                  geni_operational_status: <string sliver operational state>
               },
               ...
         ]
}

FIXME: Include the slivers struct? FIXME: Break this into 3 methods: List(), List(slice), List(slivers)

Allocate

AKA CreateSlivers() Request resources described by the given request RSpec to be allocated to the given slice. On success, one or more slivers are allocated, containing resources satisfying the request, and assigned to the given slice. The manifest RSpec describing those resources is returned.

struct Allocate(string slice_urn,
                    struct credentials[],
                    geni.rspec rspec,
                    struct options)

credentials is the standard argument defined above.

Options include:

{
  [optional geni_end_time: <dateTime.rfc3339>]
}

Return struct:

{
 geni_rspec: <geni.rspec manifest>,
 geni_slivers: [
        {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - allocated or ?? >
        },
        ...
    ]
}

The manifest is a manifest of only newly allocated slivers.

Renew

AKA RenewSliver() or RenewAllocated Request that the named slivers be renewed, with their expiration extended. If possible, the aggregate should extend the slivers to the requested expiration time, or to a sooner time if policy limits apply. This method applies to slivers that are geni_allocated or to slivers that are geni_provisioned, though different policies may apply to slivers in the different states, resulting in much shorter max expiration times for geni_allocated slivers.

struct Renew(string urns[],
                    struct credentials[],
                    dateTime.rfc3339 expiration_time, 
                    struct options)

credentials is the standard argument defined above.

Options include geni_best_effort

Return list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_operational_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
  },
  ...
]

UpdateAllocations

AKA UpdateAllocated() This method modifies a set of allocated slivers in place. The existing reservations are not lost if the newly requested allocations cannot be satisfied by the aggregate.

Details are suppressed pending agreement on this method.

Provision

AKA ProvisionSlivers() Request that the named geni_allocated slivers be made geni_provisioned, instantiating or otherwise realizing the resources, such that they have a valid geni_operational_status and may possibly be made geni_ready for experimenter use.

struct Provision(string urns[], struct credentials[],  
                                   struct options)

credentials is the standard argument defined above.

Options include:

  • geni_best_effort
  • geni_end_time
  • struct geni_users[]

Return struct:

  geni_rspec: <geni.rspec, RSpec manifest>,
  geni_slivers: 
  [
    {
     geni_sliver_urn: <string>,
     geni_allocation_status: <string>,
     geni_operational_status: <string>,
     geni_expires <dateTime.rfc3339 when the sliver expires from its current state>,
    },
    ...
  ],

FIXME: Is the returned manifest only for newly provisioned slivers? Or all slivers at this AM for this slice?

UpdateSlivers

AKA Update() This method requests a modification to 1 or more provisioned slivers. The slivers end up in a new geni_allocated+geni_provisioned state.

Details are suppressed pending agreement on this method.

Status

AKA SliverStatus Retrieve status information about the named slivers. This should be relatively dynamic data, not descriptive data as returned in the manifest RSpec.

struct Status(string urns[], struct credentials[], struct options)

credentials is the standard argument defined above.

Return:

{
  geni_urn: <slice URN>
  geni_slivers: [ 
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                     },
                    { geni_sliver_urn: <sliver URN>
                      geni_allocation_status: <string, eg provisioned>
                      geni_operational_status: <string, eg ready>
                      geni_expires: <dateTime.rfc3339 of individual sliver expiration>
                      geni_error: <string, eg ''>,
                      }
                  ]
}

PerformOperationalAction

Perform the named operational action on the named slivers, possibly changing the geni_operational_status of the named slivers.

struct PerformOperationalAction (string urns[], struct credentials[], string action, 
                                                    struct options={})

credentials is the standard argument defined above.

Options include: geni_best_effort

Return list of structs:

[ {
        geni_sliver_urn : <string>,
        geni_allocation_status: <string, eg provisioned>
        geni_operational_status : <string>,
        geni_expires: <dateTime.rfc3339 of individual sliver expiration>
        }, 
        ... 
]
;

!Delete

AKA DeleteSliver Delete the named slivers, making them geni_unallocated. Resources are de-provisioned. No further AM API operations may be performed on slivers that have been deleted.

struct Delete(string urns, struct credentials[], struct options)

credentials is the standard argument defined above.

Options include: geni_best_effort

Return list of structs:

[
  {
   geni_sliver_urn: <string>,
   geni_allocation_status: <string>,
   geni_expires: <dateTime.rfc3339 when the sliver expires from its current state>,
  },
  ...
]

Shutdown

Perform an emergency shutdown on the slice at this aggregate. Resources should be taken offline, such that experimenter access (on both the control and data plane) is cut off. No further actions on the slice should be possible at this aggregate, until an un-specified operator action restores the slice.

struct Shutdown(string slice_urn, struct credentials[], struct options)

credentials is the standard argument defined above.

Return: XML-RPC boolean

Attachments (1)

Download all attachments as: .zip