wiki:GAPI_AM_API_DRAFT

Version 111 (modified by Aaron Helsinger, 10 years ago) (diff)

--

  1. GENI Aggregate Manager API Draft Revisions
    1. Proposing Additional Changes
  2. Change Sets Adopted in GENI Aggregate Manager API version 3
  3. Proposed Changes for GENI Aggregate Manager API version 4
    1. Change Set C: Update()
      1. Motivation
      2. Proposal Summary
      3. Update()
      4. After a successful Update()
        1. Return
      5. State Guarantees
      6. Cancel()
      7. Changes to Provision()
      8. Changes to Describe()
      9. Changes to Delete()
      10. Changes to PerformOperationalAction()
      11. Changes to Status()
      12. Changes to Renew()
      13. Change to geni_single_allocation
      14. Mixed State Calls
      15. RSpec Change
    2. Change Set N: Add information to GetVersion
      1. Change Set N1: Add geni_am_code_version
      2. Change Set N2: Add geni_am_type
      3. Change Set N3: Use consistent types and prefixes
    3. Change Set O: Refine character restrictions
      1. Change Set O1: Allow other characters in sliver names
      2. Change Set O2: Explicitly define legal characters in some strings
      3. Change Set O3: Allow unicode values
    4. Change Set P: Support proxy clients that 'Speak For' an experimenter
      1. Tool Certificates
    5. Change Set Q: Support changing users and keys on existing compute slivers
      1. Specifying users and keys
      2. Semantics with Update
      3. Changing users/keys through an operational action
    6. Change Set R: Allow Renew to extend slivers as much as possible
      1. Comments
    7. Change Set S: More info from GetVersion
    8. Change Set T: Long Lived Slices
    9. Change Set U: Scheduling
      1. An Analogy
      2. Goals
      3. Proposal
      4. Questions
    10. Change Set V: Create and Manage Disk Images
      1. CreateImage
      2. DeleteImage
      3. ListImages
      4. Image URN type
      5. Import Disk Images
    11. Declined: No Reservation Allocate
    12. Change Set W: Rename fields to drop geni prefix
    13. Change Set X: Split API into services advertised in GetVersion
    14. Change Set Y: Support UTF8
    15. Change Set Z: Support Shutdown on a single sliver
    16. Change Set AA: Batch Allocate
    17. Change Set AB: Case Sensitivity
    18. Older Proposals
    19. Unspecified items
  4. Changes Not Included
    1. RSpec changes resulting in GENI v4 RSpecs
    2. Misc
    3. Stitching

GENI Aggregate Manager API Draft Revisions

This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, some of the revisions documented here have been discussed on the GENI developer mailing list, and during at least one GEC. Other revisions are in early discussions and subject to change or abandonment.

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is a set of resources allocated by one Aggregate to one Slice. See below for a proposed complete definition.

The current officially adopted version of the API is 3 and is documented on the main API page.

API Version 3 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They are detailed below.

API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They include:

  1. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    • Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    • Omni version 1.3 (released June 2011) added client software support for these changes.
  2. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.

This page documents proposed changes for AM API version 4. These changes are grouped into sets. API Version 4 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.

Proposing Additional Changes

GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.

Technical discussions are generally held on the Developers mailing list

Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net

Change Sets Adopted in GENI Aggregate Manager API version 3

Version 3 has been adopted.

Details of the change sets included in AM API version 3 are at the v3 Deltas page.

Proposals adopted after GEC13:

  • Change Set D: Slivers: Change methods to clarify that there may be multiple slivers per slice at an AM, and to allow operating on individual slivers
  • Change Set F3: Sliver Allocation States and methods
  • Change Set F4: Method to perform Sliver Operational actions
  • Change Set F5: Sliver Operational States
  • Change Set M: New method signatures, incorporating all previous adopted change sets

Proposals adopted at GEC13:

  • ADOPTED: Change Set G: Generalize the credentials argument, allowing ABAC support
  • ADOPTED: Change Set I1: SliversStatus return structure includes sliver expiration
  • ADOPTED with changes: Change Set I2: SliversStatus return includes SSH logins/key for nodes that support SSH access
  • ADOPTED: Change Set I3: CreateSlivers return becomes a struct, adds sliver expiration
  • ADOPTED: Change Set K: Standardize certificate contents, etc.
    • Include a real serial number, holder email, holder uuid, and optionally authority URL in certificates
    • Define slice ID as the UUID plus URN in slice certificates
    • Define slice name, sliver name, and user name restrictions, and similar for URNs
    • Publish schemas for credentials and certificates

Proposed Changes for GENI Aggregate Manager API version 4

From the below, Change Sets C, N, and O were all adopted at the GEC15 coding sprint. Change Set R was adopted at the GEC 18 coding sprint.

Change Set C: Update()

This change was adopted at the GEC15 coding sprint for inclusion in API v4. Aggregates are permitted to include this method in the API v3 implementations.

Add an ability for experimenters to modify their allocated resources at an aggregate without deleting (and possibly losing) existing resource allocations.

This change was briefly discussed at GEC13, discussed at the GEC14 coding sprint, and the GEC15 AM Topics and Coding Sprint.

Motivation

A common complaint among experimenters about the current AM API is that there is no way to atomically add/remove and modify resources at an aggregate. AM API v3 allows aggregates to support adding or deleting some resources from the slice at the aggregate, but not all aggregates can support that without the ability to also modify other resources. For example, you cannot add a link without adding an interface to one or more nodes, and there is no way to modify the node without first deleting it. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resuorces for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources.

This topic was discussed at the GEC12 AM API session and on the GENI dev mailing list (in October and November).

This topic was also discussed on the geni-dev mailing list here: http://lists.geni.net/pipermail/dev/2012-March/000588.html and here: http://lists.geni.net/pipermail/dev/2012-March/000643.html and elsewhere in March 2012.

A competing related proposal from Gary Wong is here: http://www.protogeni.net/trac/protogeni/wiki/AM_API_proposals

Jon Duerig made a proposal prior to GEC14, which was discussed at the coding sprint. This proposal is a revised version of that proposal.

Proposal Summary

This proposal adds an Update() method, a Cancel() method, and a new geni_allocation_state: geni_updating. The proposal includes a new state guarantee RSpec extension, a new geni_cancelled option to Describe(), and calls for some minor changes to GENI v3 RSpecs.

Update()

Begins a transaction to modify resources which are currently in the geni_allocated or geni_provisioned state. Update() itself changes only the internal allocation of slivers, not their operational state. It is therefore fast and synchronous.

struct Update(string urns[], struct credentials[], struct geni.rspec rspec, struct options)

This call accepts the geni_best_effort option; when supplied, the aggregate should attempt to partially satisfy the request.

This call accepts the geni_end_time option; when supplied, the aggregate may attempt to renew the slivers being created or modified to the requested time, according to aggregate-local policy.

This change adds a new geni_updating state, for slivers that were geni_provisioned but have been updated. Here is a state diagram:

Since Update() changes the allocation state only, it has an immediate effect on slivers currently in the geni_allocated or geni_updating states. However, updating slivers in the geni_provisioned state is a two-step process involving first changing the allocation via Update() and then committing those changes with Provision().

The input RSpec is a complete request specification for all slivers included in the urns list, as you would like the slivers to exist when the call is fully committed. Any difference in the update RSpec from the current slivers shall be interpreted as a modification request, including removing interfaces or links, or deleting entire slivers.

The RSpec argument to Update() should be a request RSpec, but RSpecs passing the manifest schema are allowed. Resources in the RSpec should include the sliver_id tag when the experimenter is requesting a modification to the given resource. Resources without a sliver_id are to be interpreted as new resource requests.

If a sliver is defined in the RSpec and contained in the urns list, (either by URN or by indirect inclusion because the slice URN is in the urns list), then the user is requesting that the sliver be modified. If the sliver was in the geni_allocated state, then the allocation of that sliver is changed immediately and the sliver remains in a geni_allocated state. If the sliver was in the geni_provisioned state, then the operational state of that sliver is preserved and it is placed in the geni_updating state where it can be Provision()ed to implement the modification or Cancel()led.

If the RSpec does not define one or more slivers which are specified in the urns list (that is, does not include resources containing the given sliver_id tag), then the user is requesting that the given sliver(s) should be deleted. If the sliver was in the geni_allocated state, the allocated sliver is deleted and the sliver ceases to exist (becomes geni_unallocated). If the sliver was in the geni_provisioned state, then the operational state of that sliver is preserved and it is placed in the geni_updating allocation state where it can be Provision()ed to delete the sliver or Cancel()led to preserve it.

Any slivers which are not defined in the RSpec and do not appear in the urns list will remain unchanged. They remain in their current allocation state.

Any resources requested in the RSpec that are not specified in the urns list will be instantiated and be in the geni_allocated state, if the aggregate has the resources available and otherwise can provide the resources following the aggregate's policy for Allocate.

Note that at least one urn must be provided in the urns argument (slice or sliver). If a sliver urn is supplied and that sliver is unknown or expired, then Update shall result in an error (e.g. SEARCHFAILED, EXPIRED or ERROR geni_code) (unless geni_best_effort is true, in which case the method may succeed, but return a geni_error for each sliver that failed). If Update is called with a slice urn which identifies a slice with no existing reservation at this aggregate, then Update behaves like Allocate. Not however that if the supplied RSpec identifies any specific slivers (via inclusion of a sliver_id), then those sections are treated as new resource requests.

After a successful Update()

After a successful call to Update(), all slivers in the urns list which began in the geni_provisioned state will be in the geni_updating state (and their configuration will revert in the case of a Cancel() call described below). This includes those slivers which are marked for deletion as described above. These slivers will not actually be deleted until the Provision() call.

Slivers which were geni_provisioned and are now geni_updating will retain their prior existing expiration time. If the expiration time is reached before the sliver is Provision()ed, then the resources are freed and the sliver is geni_unallocated. The sliver does not return to the geni_provisioned state on sliver expiration.

All slivers which began in the geni_allocated state will remain in the geni_allocated state, or will have been deleted and there will be no way (except for attempting another Update()) to revert their configuration. Slivers that are updated and remain in the geni_allocated state will have a new expiration time, as determined by the aggregate, but typically set just as for new slivers returned by Allocate(). As with other geni_allocated slivers, if the sliver expires before it is Provision()ed, then the resources are freed and the sliver is geni_unallocated.

Any slivers which began in the geni_updating state will remain in the geni_updating state, with their prior existing expiration time. If the expiration time is reached before the sliver is Provision()ed, then the resources are freed and the sliver is geni_unallocated. The sliver does not return to the geni_provisioned state on sliver expiration.

Any new resources requested in the RSpec not associated with a current sliver will be in the geni_allocated state and will be added to slivers, just as after a call to Allocate().

While slivers are geni_updating, operational actions are not permitted (PerformOperationalAction() should return an error code - 2, 14, or 16).

Return

On success, the value field of the return struct will contain a struct:

{
 geni_rspec: <geni.rspec manifest of newly allocated slivers>,
 geni_slivers: [
        {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from Status>,
                  geni_allocation_status: <string sliver state - e.g. geni_allocated, geni_updating>,
                  geni_next_allocation_status: <string state of the sliver after the next Provision() call - geni_unallocated or geni_provisioned>,
                  geni_error: <optional string, may be omitted entirely, explaining any failure for a sliver>
        },
        ...
    ]
}

The list of returned slivers includes even those that will be deleted. For those slivers, geni_next_allocation_status will be geni_unallocated.

State Guarantees

The manifest RSpec may include an extension describing what guarantees the AM will make about state preservation. Omitting the extension means the AM provides no guarantees. It is proposed that the extension have the following RNC schema:

default namespace = "http://www.geni.net/resources/rspec/ext/preserve/1"

# This is meant to extend a node or link
Preserved = element preserve {
  attribute guarantee { "geni_none" | "geni_persistent_state" |
                        "geni_dynamic_state" | "geni_no_disruption" }
}

start = Preserved

Guarantee levels are defined to mean:

  • geni_none is returned when the AM provides no state preservation guarantees for a sliver.
  • geni_persistent_state is returned when the AM may wipe out dynamic state (i.e. reboot a node) but will preserve persistent state.
  • geni_dynamic_state is returned when the AM retains the dynamic state but there may be other perturbations such as packet loss or service restarts.
  • geni_no_disruption is returned when the AM will not make any changes that can perturb the sliver.

Cancel()

Cancels an Update() or an Allocate().

struct Cancel(string urns[], string credentials[], struct options)

This call accepts the geni_best_effort option; as with Delete(), when this option is supplied the aggregate should attempt to de-allocate individual slivers.

When applied to slivers in the geni_allocated state, Cancel() acts just like Delete(). When applied to slivers in the geni_updating state, those slivers are returned to the geni_provisioned state with no change to the operational state or attributes of the existing slivers.

Return: On success, the value field of the return struct will contain a struct:

{
   geni_rspec: <geni.rspec, a Manifest RSpec>
   geni_urn: <string slice urn of the containing slice>
   geni_slivers: [
               {
                  geni_sliver_urn: <string sliver urn>
                  geni_expires: <dateTime.rfc3339 allocation expiration string, as in geni_expires from SliversStatus>,
                  geni_allocation_status: <string sliver state - e.g. geni_allocated or geni_provisioned >,
                  geni_operational_status: <string sliver operational state>,
                  geni_error: <optional string, may be omitted entirely, explaining any failure for a sliver>
               },
               ...
         ]
}

Changes to Provision()

When applied to slivers in the geni_updating state, this call will either remove those slivers (if they were marked for deletion as described in the Update() call) or move them to the geni_provisioned state, provisioning them according to the manifest RSpec returned from that call.

Changes to Describe()

The addition of an Update() operation creates an ambiguity on the slivers being described. Should the Aggregate Describe() the slivers as they will be instantiated on a successful Provision() (including all slivers that are geni_allocated, geni_updating, and geni_provisioned)? Or should the AM describe the slivers which would remain after a Cancel() (including only the slivers that are geni_allocated or geni_provisioned, plus the updated slivers as they existed before being updated)?

In order to account for this ambiguity, Describe() accepts a new option, geni_cancelled which defaults to false. When geni_cancelled is true, Describe() returns the state of all currently geni_provisioned slivers, showing the user what their slice will look like if they cancel all geni_updating and geni_allocated slivers. When geni_cancelled is false, the manifest and status will reflect the changed slivers in the geni_updating and geni_allocated states.

Additionally, the per-sliver return structure for Describe() adds a new attribute, just as for Update():

                  geni_next_allocation_status: <string state of the sliver after the next Provision() call - geni_unallocated or geni_provisioned. May be an empty string.>

This attribute is an empty string (not None or null) when the current allocation state is not geni_allocated or geni_updating.

Changes to Delete()

A call to Delete() can be used to delete slivers in any state.

Changes to PerformOperationalAction()

While slivers are geni_updating, operational actions are not permitted (PerformOperationalAction() should return an error code - 2, 14, or 16).

Changes to Status()

The per-sliver return structure for Status() adds a new attribute, just as for Update():

                  geni_next_allocation_status: <string state of the sliver after the next Provision() call - geni_unallocated or geni_provisioned. May be an empty string.>

This attribute is an empty string (not None or null) when the current allocation state is not geni_allocated or geni_updating.

Changes to Renew()

The per-sliver return structure for Renew() adds a new attribute, just as for Update():

                  geni_next_allocation_status: <string state of the sliver after the next Provision() call - geni_unallocated or geni_provisioned. May be an empty string.>

This attribute is an empty string (not None or null) when the current allocation state is not geni_allocated or geni_updating.

Change to geni_single_allocation

When an AM reports geni_single_allocation as true in GetVersion(), the following additional restrictions apply to calls:

  • When calling Update(), the urns list must either contain a slice URN or it must contain the URN of every sliver at this Aggregate Manager in the given slice.
  • When calling Provision(), the urns list must contain the sliver URNs of all slivers in either the geni_allocated or geni_updating states in the given slice.
  • When calling Cancel(), the urns list must contain the sliver URNs of all slivers in either the geni_allocated or geni_updating states in the given slice.

Mixed State Calls

A Renew() call which contains sliver URNs in multiple allocation states will attempt to renew them all to the same requested expiration time.

A Delete() call which contains sliver URNs in multiple allocation states will attempt to delete all slivers and put them in the geni_unallocated state.

Provision() and Cancel() will ignore any sliver URNs which are not in the geni_allocated or geni_updating states.

Update() can be called on slivers in any state. It will have no effect on slivers in the geni_unallocated state, and otherwise operates as described above.

RSpec Change

To easily accommodate submitting Update() requests, RSpecs will be changed:

  • sliver_id will be allowed in request RSpecs
  • Manifests will be modified to be re-usable as request RSpecs. In particular, the manifest schema will become a simple extension of the request schema, but this change is not required to support Update().

Change Set N: Add information to GetVersion

This change set adds some additional information to the return from GetVersion.

This change set was adopted at the GEC15 coding sprint. Aggregates may implement this as part of their AM API v3 implementation, but will be required to do so as part of AM API v4.

Change Set N1: Add geni_am_code_version

Aggregates use varying ways to identify the software version their AM uses. PlanetLab and ProtoGENI use code_tag, Orca uses orca_version, and FOAM uses foam_version. Automatically identifying software versions is important for monitoring and GENI operations. Aggregates are therefore strongly encouraged to advertise their current software revision using this standard field, though for security reasons some aggregates may choose not to do so.

This change set proposes adding geni_am_code_version to the return from GetVersion.

  • The field is optional. Aggregate developers are expected to include this option, but site operators may select not to expose it.
  • The value is a string.
  • Legal characters are: alphanumeric, space, - (hyphen), ., : (colon), #, _ (underscore), +, (, )
  • Regular expression: '^[a-zA-Z0-9-\.:#_\+\(\)]+$'

Change Set N2: Add geni_am_type

Aggregates are allowed to include AM specific return values, or even implement custom methods as part of the same interface.

Currently, there is no consistent way to identify what type of aggregate a given instance is, so as to know what additional attributes or options may be used. This proposal would add a way for aggregates to identify what kind of aggregate this is, and therefore what aggregate specific options or returns are applicable. Aggregates of aggregates may identify as multiple types. One of those types would indicate that this is such an aggregate of aggregates, and other listed types would indicate that clients may interact with the aggregate as though it is any of the listed types.

This change set proposes adding geni_am_type to the return from GetVersion.

  • The field is required.
  • The value is a list of strings, of length at least one. It should generally be a list of length 1. Aggregates of aggregates may list multiple types.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

Change Set N3: Use consistent types and prefixes

Make am_type, geni_am_type, and AM specific prefixes for new options consistent.

The AM API specifies an am_type field as part of the code in the basic return structure. This field has a set of prescribed values.

An earlier proposal adds a geni_am_type return to GetVersion.

The AM API encourages AM specific options and returns. These options and returns should have unique names, but the API does not specify how to ensure this.

This proposal aims to standardize these fields.

am_type and geni_am_type:

  • The value is a string.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

Aggregate specific options and return attributes should be named with a prefix denoting the aggregate type. It should be the same as the am_type and geni_am_type for this aggregate. Therefore, character restrictions are:

  • The value is a string.
  • Legal characters: alphanumeric
  • Values should be one of the defined GENI AM types if applicable, as defined by the AM API geni-am-types.xml. (As of this proposal, one of orca, foam, protogeni, sfa. More GENI AM types may be added in the future.)

For example: ProtoGENI includes an AM-specific status in the v2 API return from SliverStatus called pg_status. By this proposal, that attribute should be renamed protogeni_status.

Change Set O: Refine character restrictions

This change set was adopted at the GEC15 coding sprint. Aggregates may implement this as part of their AM API v3 implementation, but will be required to do so as part of AM API v4.

Change Set O1: Allow other characters in sliver names

Currently we heavily constrain legal characters in the 'name' portion of sliver URNs. See GeniApiIdentifiers. This is awkward. In particular characters like periods, and underscores are not allowed, but are very useful as separators.

This change set proposes modifying the name portion of the URN rules as follows. Note that these changes loosen existing restrictions, and so are backwards compatible (existing sliver names remain legal). Note also that this change simple allows aggregates to use these other characters in their own identifiers, if they so choose.

Sliver:

  • May use only alphanumeric characters plus hyphen, underscore, or period: '^[a-zA-Z0-9-_\.]+$'

Change Set O2: Explicitly define legal characters in some strings

Various fields in the AM API specification are defined only as strings, without explicit limits on legal characters. This makes it awkward to pass these values to and from scripts. In particular, many fields should only be alphanumeric, plus some limited number of separator characters.

This change set proposes constraining several options/return values that are currently defined only as 'string'.

  1. From getversion and listresources and describe, the RSpec type:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From getversion and listresources and describe, the RSpec version:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From getversion, the RSpec schema, and namespace: Any character valid in a URL (see e.g. http://www.w3.org/Addressing/URL/url-spec.txt)
  1. From getversion and the credentials argument, the credential geni_type:

May use only alphanumeric characters plus hyphen, underscore, period, or colon; alphanumeric only in first character. '^[a-zA-Z0-9][a-zA-Z0-9-_\.:]*$'

  1. From the return of multiple methods, geni_operational_status: alphanumeric plus underscore; alphanumeric only in first character: '^[a-zA-Z0-9][a-zA-Z0-9_]*$'

Change Set O3: Allow unicode values

Many values defined by this API appear to be limited to ASCII characters - both arguments and returns. We would like to support internationalization. Some values in this API (e.g. those defined as URLs) implicitly support unicode. Currently, a well behaved aggregate should be able to handle unicode values in at least some arguments. Similarly, a well behaved client should be able to handle unicode returns.

This change would clarify the support for internationalization in this API, and allow aggregates to specify unicode support. This change should not require any clients or aggregates to make changes. This change would specify:

Aggregates are encouraged to support non-ASCII letters as input. Clients are encouraged to support non-ASCII returns from aggregates. However, this change set does not require that either aggregates or clients support non-ASCII strings, and aggregates should not require clients to accept non-ASCII results where the API does not explicitly require it.

This change would allow an aggregate to explicitly specify that arguments may be unicode, and returns may be unicode if the content demands it. By default, aggregates are expected to use the ASCII only regular expressions that the rest of this API specification uses, and to limit returns to ASCII characters where possible. This change would add an optional return from GetVersion that allows an aggregate to specify that unicode arguments are acceptable (ASCII strings must still be accepted), and that all returns may be unicode. When the aggregate specifies this option, it shall still attempt to support ASCII inputs to allow clients to operate only in ASCII where possible.

Specifically, this change adds geni_unicode_compliant to the return from GetVersion.

  • This field is optional.
  • Default is false. All strings specified in this API are to be treated as ASCII unless otherwise specified, (e.g. the format explicitly references a standard that supports unicode). This includes both arguments and returns.
  • When true, all strings specified in this API may be unicode. In particular, regular expressions that limit to ASCII characters shall be interpreted to include the unicode equivalents. The aggregate will still accept ASCII input. Returns will remain ASCII where the API defines ASCII constants or content allows it.

Change Set P: Support proxy clients that 'Speak For' an experimenter

GENI tools invoke AM API methods on behalf of experimenters. When that tool instance is well known and trusted by the experimenter, and runs local to a single experimenter, then it is reasonable for that tool to be given the experimenter's private key and public certificate, and 'speak as' the experimenter. But for a hosted tool that might act for multiple experimenters it is both more secure and more clear what is happening for such tools to have their own identity certificate and private key by which each tool instance authenticates to AM API calls, and use a new 'Speaks For' credential (or credential set) to authorize the tool instance to take a given action. The experimenter will issue such credentials to the tool instance (possibly scoped by time, slice, aggregate, or other dimension), and the aggregate can then properly assign the resources to the experimenter, while logging and reporting that the tool performed the operation. This functionality can also be used for an experimenter to authorize a 'proxy' aggregate or an aggregate of aggregates.

Two changes are required in the AM API to support this functionality. First, not all aggregates will support 'Speaks For' yet, so tools must be able to distinguish. Second, we require an explicit way for the aggregate to determine on whose behalf a tool is taking this action.

  1. Add a new optional return entry from GetVersion:

geni_handles_speaksfor: A boolean (0 or 1 in XML-RPC), default is false (0).

When present and true, the aggregate is capable of authorizing AM API actions using a GENI 'Speaks For' credential (or credential set), and accepting the option geni_speaking_for (see below). When the aggregate supports 'Speaks For', tools should authenticate using their own GENI issued certificate and key, supply a 'Speaks For' credential (or credential set) from the experimenter authorizing the tool instance to perform the given action, and supply the URN of the experimenter on whose behalf the tool is acting in the geni_speaking_for option. When the aggregate does not support 'Speaks For', the tool is expected to 'speak as' the experimenter, using the experimenter's own GENI issued certificate and key. Aggregates should however attempt to support 'Speaks For', as 'Speaks As' is undesirable deprecated behavior. If a client supplies 'Speaks For' arguments to an aggregate that does not advertise support for 'Speaks For', the aggregate will typically ignore these, and attempt to authorize the client directly (though that behavior is not required), likely resulting in an authorization failure.

  1. Add a new option to the options field of each AM API call:

geni_speaking_for. <string URN>

When supplied, this is the URN of the experimenter on whose behalf the client is acting. The aggregate should expect to find a 'Speaks For' credential or credential set in the credentials argument, by which the given experimenter authorizes the client (as authenticated by the SSL client certificate) to perform the given action on their behalf. If such a credential is not found, the aggregate should fail the request with an authorization error (e.g. error code 3: FORBIDDEN). When omitted, the aggregate need not look for a 'Speaks For' credential, but should look to authorize the client itself ('speaks as'). If the client is not itself directly authorized to perform the given action, the aggregate should fail the request with an authorization error (e.g. error code 3: FORBIDDEN).

Two other additions are required: we must define the 'Speaks For' credential and its semantics, and we must define the URN and certificates for tools.

The 'Speaks For' credential will be a signed XML document encoding of an ABAC credential (GENI type geni_abac version 1) as specified here. Several general points are worth noting:

  • The credential includes the certificate of the user and an identifier for the tool. For the credential to be accepted, each certificate must itself be trusted by the aggregate; current recommended GENI policy requires a particular format for that certificate, and requires that it be signed (directly or indirectly) by a trusted GENI root.
  • The credential includes an expiration
  • The credential may in future include scope limitations (including slice, aggregate, operation)
  • When the aggregate authorizes a 'Speaks For' operation, the aggregate must treat the operation as though performed by the experimenter, but also log that it was done via the given tool. That is, resources will be owned by the experimenter, and logs and monitoring reports will include both the experimenter URN and the tool URN.

The specific ABAC assertion that this 'Speaks For' credential encodes is: Experimenter.speaks_for_Experimenter <- Tool. That is, the experimenter signs an assertion saying that the Tool has the attribute in the Experimenter's namespace speaks_for_Experimenter, where the Experimenter and Tool are identified (as with all ABAC assertions) by the SHA1 hash of their public key. As an example, here is a selection from a 'Speaks For' credential:

    <expires>2014-02-12T20:10:32Z</expires>
    <abac>
      <rt0>
        <version>1.1</version>
        <head>
          <ABACprincipal>
            <keyid>097f010966eacbc0f8e2fc8c66c8abfdd55f6036</keyid>
          </ABACprincipal>
          <role>speaks_for_097f010966eacbc0f8e2fc8c66c8abfdd55f6036</role>
        </head>
        <tail>
          <ABACprincipal>
            <keyid>66dd9f3018e64c12130068f4a71d364fc9cbdfb6</keyid>
          </ABACprincipal>
        </tail>
      </rt0>
    </abac>

Tool Certificates

Tool instances may be issued GENI identity certificates, using the same format and rules as for users.

  • The URN will be the URN of the tool instance. With this change, we introduce a new 'type' for the URN field: tool. The name of a tool is subject to the same restrictions as the name for users:
    • Tool names are case-insensitive internally, though they may be case-sensitive in display.
      • EG JohnSmth as a display name is johnsmth internally, and there cannot also be a user JOHNSMTH.
    • Tool names should begin with a letter and be alphanumeric or underscore, hyphen, at sign or period: ('^[a-zA-Z][a-ZA-Z0-9\-_@\.]{0,63}$').
    • Tool names are limited to 64 characters.
    • Tool URNs (which contain the authority name and the tool instance name) are required to be temporally and globally unique.
    • Tool names should encode both the tool type and instance. For example portal-gpo or genidesktop-uky.
  • The tool email address should be a way to contact the administrators of the tool instance - the organization or individual who applied for the certificate and who stands behind its integrity.

Note that this functionality also supports proxy aggregates, or aggregates of aggregates. See a previous related change proposal: http://groups.geni.net/geni/wiki/GAPI_AM_API_DRAFT?version=47#ChangeSetJ:Proxyaggregatemanagersaresupported

Change Set Q: Support changing users and keys on existing compute slivers

Experimenters often want to change the users and/or SSH keys installed on existing running compute nodes, in a way that is persistent and consistent with any aggregate manager controlled processes. This is theoretically possible through the use of Update(), but the semantics are not clear there.

This change set proposes a way to do this using PerformOperationalAction() and also proposes the semantics of providing a new set of users/keys to the Provision() call after a call to Update().

This change set may be implemented as part of an aggregate's AM API v2 implementation, not just the AM API v3 implementation

Specifying users and keys

Users to be installed and SSH keys to be installed for those users are specified using the geni_users[] struct as specified in AM API v3.

When applied to an existing sliver (through a new operational action or through a call to Update() and Provision()):

  • This same user struct is to be used
  • New users may be added
  • Omitting a user means there will be no change to the keys installed for that user. It does not mean the user should be removed.
  • Existing users will have all SSH keys replaced by the new set. Note that a user may have an empty list of SSH keys specified, effectively preventing the user from accessing the node.

Semantics with Update

After a call to Update(), users must call Provision() to commit the change. Provision() takes the geni_users[] struct. When called after Update(), the geni_users[] struct will have the add/replace semantics described above for any existing nodes; new nodes will treat all the users/keys as additions as normal.

Changing users/keys through an operational action

Users and SSH keys can be changed using 2 new operational actions (through PerformOperationalAction()):

  1. geni_update_users: The credentials[] argument must include credentials over the slice as usual. The options struct must include the geni_users option as specified in AM API v3 and with the semantics described above. This action is only legal on slivers in the geni_ready operational state. This action immediately moves all such slivers to a new geni_updating_users operational state. Slivers stays in that state until the aggregate completes the needed changes, at which time the slivers change back to the geni_ready operational state. Slivers may be in the geni_updating_users state for several minutes; during this time no other operational actions can be taken on the slivers.

Besides the usual return error codes, the PerformOperationalAction method with this action may return REFUSED if the sliver is in the wrong operational state.

Note that as described here, aggregates supporting this action and operational state will advertise this fact in their advertisement RSpec.

Sample JSON value for the geni_users option to be supplied:

{
 "geni_users": [
  {
   "urn": "urn:publicid:IDN+ch.geni.net+user+jdoe",
   "keys": ["ssh-rsa jdoegoodkey"]
  },
  {
   "urn": "urn:publicid:IDN+ch.geni.net+user+jsmith",
   "keys": ["somekey", "someotherkey"]
  }
 ]
}
  1. geni_updating_users_cancel: This action requires no options. It cancels any pending geni_update_users action on the named slivers, returning those slivers to the geni_ready operational state. This action is only legal on slivers in the geni_updating_users operational state. This action may be used on slivers which fail to complete the geni_update_users action. After a successful geni_updating_users_cancel, the state of users and keys on the sliver(s) is not defined; some may have the same users/keys as they had prior to beginning the geni_update_users action, and others may have already updated to the new set of users and keys.

Besides the usual return error codes, the PerformOperationalAction method with this action may return REFUSED if the sliver is in the wrong operational state.

Change Set R: Allow Renew to extend slivers as much as possible

Adopted at the GEC 18 Coding Sprint. This proposal was discussed at the GEC 18 Coding Sprint. ProtoGENI/InstaGENI, ION/MAX, Orca/ExoGENI, and FOAM all agreed to implement and support this option.

In the current AM API, aggregates are free to restrict how long a reservation can be extended with local policy. AM API v2 says the aggregate should return false if the reservation cannot be extended as requested. V3 allows the geni_best_effort option, which would allow some slivers to be extended, while other requests failed. But it still does not permit an aggregate to extend a reservation for less than the full request. This makes things difficult for experimenters.

Currently, if an experiment requests a renewal past the aggregate's local policy maximum, the request fails - and there is no standard way to determine from the error message in v3 (you may get none in v2) what renewal time 'would' work. The problem is compounded if you try to renew slivers at multiple aggregates.

This proposed change set would accommodate an experimenter attempting to extend their reservation as long as possible

Proposal: Add a new option: geni_extend_alap (as long as possible), default is False (XML-RPC boolean) (preserving current behavior). Option is valid in AM API v2 in RenewSliver, or in v3 or higher in Renew. AMs are not required to support it. When True, the AM should extend the life of the slice/sliver (depending on the API version), to the minimum of the requested new expiration time, the expiration time in the slice credential (of course), and the AM local maximum based on local policy (which may be resource or slice dependent).

When the AM successfully extends the life of the slice/sliver using this new option:

  • return value is success (True or the API v3 struct)
  • AM API v3 Renew return struct will be as previously defined, including the new sliver expiration time
  • For AM API v2 aggregates, the output slot of the AM API triple should be the new sliver expiration time - either the time requested, or the 'as long as possible' time. This expiration time should be a standard GENI AM API datetime complying with RFC 3339.

When the AM cannot extend the life of the slice/sliver at all (because it does not honor the new option, or because no expiration is possible), then it fails the request (return False in APIv2, return an error code in APIv3 unless geni_best_effort is True, in which case it may return success and indicate the individual per-sliver errors in the return struct; v3 error code may be one of UNSUPPORTED for an AM that does not support this option, or BADARGS or ERROR for other failures).

Note that this would be a new option that AMs are not required to honor - making things backwards compatible.

Comments

  • Maybe we should try to convey AM local policy on max renewal times (which may be per resource type) in GetVersion or the Ad RSpec ('before' you try to reserve or renew the resources). That is a separate discussion.

Change Set S: More info from GetVersion

GetVersion should return an additional field indicate the URN(s) of the Aggregate Manager at this endpoint. This corresponds to the URN IDs of the component_manager or component_manager_id from the advertisement RSpec produced by this aggregate.

At many aggregates, this is a single URN. But at some aggregates that proxy for multiple component managers, this may be a list of URNs.

FIXME:

  • geni_urn? geni_component_manager_urn?
  • Presumably this must be a list?
  • Do aggregates ever have a single URN identifier?

Change Set T: Long Lived Slices

This topic was raised at the GEC18 Coding Sprint.

This proposal provides a way to support long running experiments that require slices and resource reservations with long lives. This proposal requires changes to Slice Authorities and aggregates.

Proposal:

Augment existing slice credentials to add a new possible privilege value longlived that means that the issuing Slice Authority (SA) believes that the given slice should be allowed to have an extra long reservation. The proposal does not specify how the SA decides to include this privilege. Requesting a credential for such a slice results in a slice credential that adds this privilege.

Aggregates must accept slice credentials that include the new privilege and understand that a privilege value of * does not include this privilege. When an aggregate receives a slice credential that includes this privilege, the aggregate should allow resources in this slice to be renewed beyond the usual maximum resource reservation lifetime at that aggregate. However, the maximum expiration time for the resources remains subject to local aggregate policy. Some aggregate may allow arbitrary expirations for some resources when given a long lived slice credential. Some aggregates may double the maximum duration of the reservation. And other aggregates for scarce resources may only allow a slight increase over the usual duration.

Question: Does the privilege say anything about how long the slice' reservations should be good for? I think not.

Alternative: Add a new credential, in addition to the existing slice credential, which looks just like a normal slice credential, but adds this additional privilege. This requires tools to retrieve and pass around multiple credentials.

This change requires aggregates to accept and handle these new credentials.

Change Set U: Scheduling

The ability to schedule resource reservations in the future has been discussed multiple times as desirable. This proposal would add support in the AM API to allow aggregates to provide this ability.

This proposal is incomplete and requires further discussion.

An Analogy

First, an analogy to reserving a table at a restaurant:

The current AM API does not support reservations - we only support walk-ins. In AM API v2, we support "Can I have a table?" (Createsliver), with the possible answers "No" or "I'll bring you your menus."

In AM API v3, we support "Can I have a table?" (Allocate) with the answers "No" or "I have a table free, would you like it?". Then the experimenter can say "No" (Delete) or "Yes, I'll take it" (Provision), or "Can you give me a minute to consult with my friends?" (Renew)

With this proposal, we want to support reservations:

Experimenter: Can I have a table on Tuesday? (Allocate)

Aggregate: I have a table by the door, would you like it?

  • And the restaurant will not give away that table until the experimenter answers, but only waiting a short time.

E: Give me a minute to consult? (Renew)

or

E: No thanks (Delete)

or

E: Yes, please reserve that table for me on Tuesday. (Provision?)

Then when Tuesday rolls around, the restaurant will only hold the table for a limited time.

E: I'm here, ready to sit down. (Provision? PerformOperationalAction of geni_start?)

or

E: Never mind, I don't want the table. (Delete)

Or the experimenter doesn't show up, and the reservation "expires" and the restaurant is free to give the table away.

Goals

  • Experimenters can specify both start and end time for when they want resources
  • Experimenters must commit to using the resources, so that aggregates are not holding resources that the experimenter will never use
  • Experimenters know when they have the resources
  • Experimenters know how long they may be able to hold the resources, and when the resources are likely taken by someone else
  • Support for scheduling is optional

Proposal

  • Allocate takes both a geni_start_time (new) and geni_end_time (already allowed).
    • The geni_start_time option to Allocate indicates when the experimenter wants the reservation to start. The semantics are such that the aggregate may give the client the resources early, or up to 10 minutes later than requested, but no more. If the aggregate cannot reserve the requested resources for the client within that time window, then the request must fail with the error code indicating the resources are UNAVAILABLE. Aggregates may include in the error some message indicating if a slightly different start time would succeed, if known.
    • Aggregates that do not support scheduled reservations will ignore this option, always treat the request as a request for resources now. Clients may examine the return from Allocate to and see that geni_start_time and geni_end_time is not specified in the return, indicating that this is not a scheduled reservation, but an immediate reservation.
    • Note that geni_start_time is not an option to Provision
  • Add a geni_max_expiration field wherever we currently return a sliver expiration. This is advisory, indicating the latest datetime that you should expect to be able to renew your reservation until. This may be based on local aggregate policy only, or based on known other scheduled future reservations for the same resource(s). Note though that this is not a guarantee: a request to extend a reservation up until this time may still be denied (other resource reservations that come in in the interim for example), and it is even possible that a request to renew past this time might succeed, though clients should not expect this.
  • Add a geni_start_time field to the return wherever we currently return a sliver expiration. This field is required only for reservations that begin in the future.
  • geni_expires is slightly redefined. This now indicates the end of the time window in which you must act on the sliver(s) / reservation, or else the slivers will expire and be given to others. This is consistent with how the field is used by aggregates that do not support scheduling today. Aggregates that do support scheduling however will use this and a new option.
  • Add a geni_end_time return element wherever geni_expires is returned, to be required only for aggregates that support scheduling, and only when both a geni_start_time and geni_end_time was supplied in the original reservation request. This element specifies the time at which the reservation has been requested to end. For example, geni_start_time might be tomorrow at 1pm, geni_end_time might be tomorrow at 3pm, and `geni_expires might be 10 minutes from now. This indicates that the client has 10 minutes in which to confirm the reservation for resources tomorrow from 1 to 3pm.
  • After Allocate, scheduled requests are geni_allocated as before. As before, such slivers have a short expiration time (given in geni_expires), during which time Delete and Renew are both valid.
    • Renew does not change the requested start and end times of the reservation, it only extends the time window in which the client must confirm the reservation. To change the requested start or end times of an allocated reservation, use Update, or call Delete and then Allocate.
    • Experimenters must call Provision as before (arguments are unchanged) to confirm the reservation during this time window (before geni_expires).
      • If the client does not call Provision during that time window, the slivers expire and the reservation is cancelled.
      • If the client does call Provision for a reservation in the future (has a geni_start_time), then the sliver(s) transition to the new geni_scheduled state. The aggregate does not begin instantiating the resources at this point. The aggregate only changes the state of the slivers and the geni_expires time at which the slivers will expire (see below).
  • Add a new geni_scheduled allocation state after geni_allocated and before geni_provisioned. Slivers in this state are being held for the client for some future time. The client has already confirmed that they want the resources as promised, but the requested start time has not arrived or the experimenter has not confirmed that they are ready to start using their reservation. This state is only used by aggregates that support scheduling and only for requests for resources in the future.
    • While in this state, the client may call Describe, Status, Delete, or Update. Renew is not legal on slivers in the geni_scheduled state: aggregates may raise an error, or simply return the existing expiration times and states.
      • Update moves the slivers to the geni_updating state; clients must confirm the new request with a new call to Provision before the time in geni_expires, or else the update request expires, the slivers return to the geni_scheduled state, and the reservation remains unchanged. Regardless of the specified geni_start_time, resources will not be instantiated until the slivers move out of the geni_updating state. Update may be used to request a change to both what resources are reserved, and the requested geni_start_time and geni_end_time.
    • When Provision is used on geni_allocated slivers with a geni_start_time in the future, it moves the slivers to the geni_scheduled state. The return must include geni_start_time, geni_end_time, and geni_expires. geni_end_time is the end of the requested reservation window. geni_expires is a time some small delta after the geni_start_time. This indicates the time at which the slivers will expire, if the client has not claimed the slivers with a second call to Provision.
  • When the geni_start_time arrives, the client must call Provision (arguments are unchanged), indicating the client is ready to use the resources. Only at this point does the aggregate begin to instantiate the resources. The slivers become geni_provisioned as before.
    • If the client does not call Provision before the sliver expiration time given in geni_expires, then the slivers expire and become geni_unallocated (as before).
    • Renew has no effect on slivers that are geni_scheduled.
  • Update on slivers that are already geni_provisioned will not change the geni_start_time. It may request a change to the geni_end_time, along with changes to what resources are reserved.

Questions

  • Is there any way to query the reservation schedule of a particular resource?

Change Set V: Create and Manage Disk Images

This proposal would allow creating, listing and deleting disk images based on the state of existing disks in your slice at an aggregate.

This proposal is based on the functionality exported by ProtoGENI based aggregates. FIXME: This proposal must be made more generic.

Some general notes:

  • Disk images are typically created with snapshots of the disk of an existing compute resource. At some aggregates this might disrupt the state of the compute resource.
  • Images exist local to an aggregate: They are created, listed and deleted at a particular aggregate
  • Images are identified by a URN. At some aggregates, they are also identifiable by a URL (for purposes of exporting an image to another aggregate)
  • Exporting an image to another aggregate involves copying the image from one aggregate to another.
    • Not all aggregates will support exporting images
    • An image imported from another aggregate may not function at the new aggregate (due to hardware differences, etc)
    • Once imported, the disk image now exists local to the new aggregate as well

FIXME

  • Add a separate SERVICE indicating if images are exportable to other aggregates (EXPORT_IMAGES?)
  • Define rules for legal disk image names
  • Define URL rules for exporting images
  • Define authorization rules for deleting and listing images
  • Are exportable images always public? Or does public only control what is listable?

CreateImage

Create a disk snapshot or image based on the state of the disk for an existing sliver.

Arguments:

  • slice URN
  • name for the image: An alphanumeric non empty string
  • sliver URN of the machine whose disk should be snapshotted
  • credentials giving the caller rights on this slice
  • options
    • The option global defines whether this image will be publicly visible or not. The value is an XML-RPC boolean indicating whether the image should be public, defaulting to False meaning private to your project or sub-SA. "Private" images are visible and usable by any experimenter in the same project (or sub slice authority) at this aggregate. "Public" images are usable by anyone with the URL or URN of the image.

Returns:

  • URN naming the new image. The image URN looks like urn:publicid:IDN+<aggregate authority>+image+<aggregate authority with punctuation replace by hyphens>:<image name supplied in the argument>, for example urn:publicid:IDN+foo.net+image+foo-net:booboo.
  • URL for referencing the image, for use at other aggregates, for example: https://www.foo.net/image_metadata.php?uuid=c0b47d37-f6cd-11e1-9f72-001143e453fe

Image names must be unique at this aggregate. If an image of the same name already exists and is "owned" (was created or imported by this user), then this image replaces the previous image of the same name. For other experimenters, this method shall return an error code BADARGS indicating that the image name is taken.

If local policy determines that this experimenter or project or slice has exceeded some quota on number of images or disk space, then this command may fail with an error message explaining the issue (code 2 or 7).

This function should return quickly, asynchronously snapshotting the disk. While the disk is being snapshotted, do not modify the state of your VM - results will be unpredictable.

Only slivers in the geni_ready state may be imaged. While the image is being created, the sliver will be in a new operational state, geni_imaging. While the sliver is in this state, no operational actions are permitted, and at some aggregates, the sliver may be inaccessible. Note that at some aggregates, creating an image may disrupt the running and state of the resource. When imaging is complete (successfully or due to error), the operational state transitions back to geni_ready.

The aggregate will contact the user who created the snapshot, typically at the email address found in the user's certificate, providing status and results.

DeleteImage

Delete the named disk image owned by / created by the named user.

Arguments:

  • image URN
  • credentials, including a valid slice credential for any slice in the same project / sub-slice authority as the slice from which the image was created (or imported)
  • options
    • creator_urn: image creator URN (a valid "user" URN). If not supplied, the current user is assumed

Return: XML-RPC boolean for success. On error, return codes include SEARCHFAILED, FORBIDDEN and ERROR.

Only users in the same project / sub-Slice Authority may delete an image (as authorized by the supplied slice credential).

Note that the experimenter who created the image locally or imported it from another aggregate is the image 'creator' for purposes of this method.

ListImages

List the disk images created by the given user at the aggregate you are calling.

Arguments:

  • creator URN
  • credentials, including optionally a valid slice credential for the project / sub-slice authority whose images you want to list
  • options

The creator URN must be from the same authority as the caller (i.e. the portion of the user URN before user must be identical).

Note that the experimenter who created the image locally or imported it from another aggregate is the image 'creator' for purposes of this method.

Return: a list of structs, each containing the url and urn of the images at this aggregate. Note that only public images will be provided in general, plus images in the same project / sub-slice authority as the slice whose valid slice credential was supplied will be listed.

 [ 
   {"urn" => <image_urn>,
    "url" => <image_url>
   },
     ....
 ]

Only users in the same project / sub-Slice Authority may list private images (as authorized by the supplied slice credential).

FIXME: Are those the right authorization rules?

Image URN type

This proposal introduces a new "type" of URN, image, to be used to name disk images.

FIXME: Restrictions on the characters legal in the name of an image or image name length?

  • Image names should begin with a letter and be alphanumeric or underscore, hyphen, at sign or period: ('[a-zA-Z][a-ZA-Z0-9\-_@\.]{0,63}$').
    • FIXME: PG uses ^[a-zA-Z0-9][-\w\.+]+$
  • Image names are limited to 64 characters.
  • The authority in an image URN is the aggregate hosting the image

Import Disk Images

A disk image that was created / is stored at one aggregate may be used at another aggregate, if the source aggregate supports it, and if the new aggregate accepts a URL for disk images in the request RSpec. If both are true, then an experimenter may specify this remote disk image by URL in the name slot of the disk_image element in a request RSpec. The aggregate exporting the image serves up the image at the URL. The aggregate importing the image:

  • May fail the import if imports are not supported, or local policy prohibits, or local disk space or quotes are exceeded
  • Stores the image locally
  • Gives it a local name / URN, locally unique
  • Lists the experimenter who imported the image as the image 'creator'
  • Associates the image with the project/sub slice authority of the slice for which the image was imported
  • FIXME: Makes the image public? Private? Does the experimenter get to say or change it?
  • Contacts the 'creator' with the new URN and URL for the local copy of the image (typically via email)

When an image is imported to another aggregate, the experimenter that imports the image is treated as the local image "creator", and the slice in which they import the image determines the local project / sub-slice authority for controlling local access to the image.

This proposal does not specify how aggregates manage local disk space or might limit the number of disk images a particular experimenter or project can have. Importing a disk image therefore might fail due to local AM policy. Additionally, disk images might 'expire' at certain aggregates. This proposal does not specify this one way or another.

Declined: No Reservation Allocate

Tools should be able to do complex topology embedding to figure out where a particular request can be instantiated. Aggregates know best what they can do. This proposal provides a cheap way to ask "could you satisfy this request, and if so what would I get?"

This proposal is declined: Allocate itself is pretty cheap.

  • Add a new option to Allocate named geni_no_reservation. If supplied with a non-empty value (not null or an empty string), the request is not asking for the resources to actually be reserved. Instead, the aggregate should calculate what it would reserve, and return the usual allocate return structure, but without actually making the reservation. As such, geni_expires should be set to the current time, and the status should remain geni_unallocated, and the geni_sliver_urn element should be empty (no sliver is created or assigned). Since there is no sliver URN, no calls to other methods should return any information on this sliver.

Change Set W: Rename fields to drop geni prefix

In the interest of making this API better apply to international federations, change all options and return elements that are currently named geni_foo to just plain foo. Non standard options and return elements should use a local aggregate type specific prefix.

That is, where the spec currently says:

The prefix geni_ is reserved for members that are part of this API specification. Implementations should choose an appropriate prefix to avoid conflicts.

Instead, it should say:

Implementations may add additional options or return elements that are not standard to this API.
To avoid conflicts with options in this specification or other implementations, implementations
should name these non-standard members with their aggregate type specific prefix.

See Change Set N3.

Change Set X: Split API into services advertised in GetVersion

The Uniform Federation API defines multiple "services". Each service is a coherent piece of functionality that a given server / authority may implement. For example, some slice authorities support projects and some do not. This proposal would split up the AM API into services, and add a SERVICES element in the return from GetVersion that is a list of the string names of services supported by this aggregate.

FIXME

Proposed services:

  • SCHEDULING: Indicates that the aggregate supports scheduled reservations per the previous proposal. Aggregates that support this will accept a geni_start_time option to Allocate, others will ignore such an option.
  • IMAGES: This aggregate allows creating and deleting and listing disk images, per the previous proposal.
  • COMPUTE: This aggregate supports reserving nodes and will list nodes in its Ad RSpec.
  • OPENFLOW: This aggregate supports reserving or configuration OpenFlow links
  • STITCHING: This aggregate supports the stitching extension and creating stitched links between aggregates.
  • STORAGE: This aggregate supports reserving storage resources.
  • UPDATE: This aggregate supports the Update method

Change Set Y: Support UTF8

To more fully support international users and sites, support UTF8 characters in usernames, slice names, sliver names, authorities, tool names, and RSpecs.

Regular expressions defining those fields must change appropriately. Slice authorities, tools and aggregates must all change.

See Change Set O3.

FIXME: Is that proposal sufficient as is?

Change Set Z: Support Shutdown on a single sliver

This proposal would allow calling Shutdown with a specific sliver or slivers specified, and would only shut down the named slivers, not the entire slice at this aggregate.

Questions:

  • What if there are interdependencies among slivers such that other slivers must also be shut down?
  • Can an aggregate specify that only the whole slice may be shut down?
  • Is this like the options for how Allocate works?

FIXME

Change Set AA: Batch Allocate

Scheduling (see above) is complicated. Often, what an experimenter really wants is to get a particular set of resources whenever they are available. This change set proposes a way for a client to submit an asynchronous request for resources.

In this proposal, an experimenter submits a resource request (via Allocate), but does not immediately receive a manifest. Instead, the aggregate contacts the experimenter (typically via email) when and if the requested resources are available. At that time, the resources are geni_allocated and the experimenter can Provision and use the resources as usual.

In detail:

  • Add a new option to Allocate: geni_when_ready. If supplied, the experimenter is requesting the resources described in the request RSpec whenever they become available. The value of geni_when_ready may be None (empty), meaning that the experimenter wants these resources however long it takes to get them. If the value is a datetime, then the experimenter wants these resources only if they become available by the specified date.
  • If an aggregate does not support this option, it will ignore it, and either allocate the resources immediately, or fail the request (with an UNAVAILABLE error).
  • The return from Allocate when this option is accepted is different than usual:
    • The geni_rspec will be the request rspec as submitted, not a manifest RSpec
    • The geni_slivers list will contain a single sliver. This sliver identifies this new request (distinguishing it from other resource reservations in the same slice at the same aggregate, if any).
      • geni_expires is in general not defined, but will be the time specified by geni_when_ready if any
      • geni_allocation_status will not be geni_allocated, but a new allocation state: geni_batch_requested
  • Add a new allocation state, geni_batch_requested: The single sliver that results from a call to Allocate with the geni_when_ready option will be in the geni_batch_requested state until:
    • The aggregate can allocate the resource (and the sliver(s) become geni_allocated), or
    • The experimenter calls Delete on the sliver, and the sliver becomes geni_unallocated
  • Update on a batch requested sliver is legal, modifying the request that has been batched. geni_when_ready is not accepted in Update.
  • Delete on a batch requested sliver cancels the request, making the sliver geni_unallocated
  • If a time was specified in geni_when_ready and the resources have not become available, then:
    • The aggregate shall contact the requester (typically using the email address in the client certificate), specifying that the request could not be satisfied (including the slice urn and sliver urn)
    • The sliver becomes geni_unallocated
  • Otherwise, when the resources become available:
    • The resources are reserved and change to geni_allocated as usual. The expiration time should be larger than for usual allocated resources, to allow time for the experimenter to receive the notification and respond. Typically at least 1 business day.
    • The aggregate shall contact the requester (typically using the email address in the client certificate), specifying that the request has been satisfied, including the usual return structure from Allocate plus the slice URN and aggregate URN
  • Once a batched request has been made geni_allocated, it behaves like any other set of allocated slivers. You can call Update, Renew, Delete, or Provision on it as usual.

Change Set AB: Case Sensitivity

This API specification calls out that certain fields are case insensitive (RSpec and credential and AM type fields, for example, plus usernames). But for most fields, this specification is silent. This has led to some confusion. FOAM and ORCA/EG are generally case sensitive. ProtoGENI/IG and SFA/Planetlab are generally case insensitive. The GENI Clearinghouse is generally case sensitive. This causes problems. In particular, if a Slice Authority can generate 2 slices whose names differ only by case, and some aggregates will treat those as duplicates, that is a problem.

Some notes on relevant specifications:

  • DNS names are case insensitive (RFC4343)
    • This suggests that the authority field of a URN which is generally a hostname should be case insensitive.
    • It also means that aggregates that generate hostnames based on slice URNs may be inclined to treat those fields as case insensitive
  • URNs are generally case sensitive, except for the NSS and NID portions (in GENI, the 'publicid:IDN' portion), per RFC 2141
  • In URLs, the domain name is case insensitive per RFC 4343, but URLs in general are case sensitive, http://www.w3.org/TR/WD-html40-970708/htmlweb.html

Options

  • Define slice names and authority names as case insensitive, but requiring (suggesting?) that services are case preserving
    • Note however that in the context of a URN that is case sensitive, this is a little funny
  • Define these strings, or all URNs, as strictly lowercase (so case sensitivity is not a problem)
  • Note that while URNs are case sensitive, some services that operate on URNs are case insensitive, so URN generating services and clients must avoid conflicts based on case - either by being case insensitive, or by generating URNs that follow a fixed case pattern of their choosing.
  • Get a new GENI namespace (not publicid) and define our own subset of the URN spec that codifies one of these alternatives (strictly lowercase, case insensitive)

FIXME FIXME

Older Proposals

Older proposals, withdrawn, superseded, or postponed:

  • Original Change Set C: On hold (waiting for a revised proposal). Add the ability to UpdateSlivers to immediately modify your reservation
  • Change Set E: Tickets: On hold (waiting for a revised proposal). Add methods using tickets to do negotiated reservations
  • Change Set F1: Superseded by F3 and F4: Define sliver states, and the state changes that various methods cause
  • Change Set F2: Superseded by F3 and F4: Add a new general ActOnSlivers method allowing AMs to support AM and resource-type specific operations
  • Change Set H: Superseded by F3: Clarify: A second call to CreateSlivers without an intervening DeleteSlivers is an error.
  • Change Set I4: Superseded by F3 and F4: CreateSlivers optionally does not start resources.
  • Withdrawn: Change Set J: Support proxy aggregates with 1 new option and 1 new GetVersion entry
  • Postponed: Change Set L: Standardize slice credential privileges

Unspecified items

  • UpdateSlivers: Update a reservation or sliver in place, without risking losing resources
  • Publish credential schema
  • Define error codes returned by new methods, conditions

Changes Not Included

RSpec changes resulting in GENI v4 RSpecs

  • Support unbound manifests
  • Make manifest an extension of Request, so you can readily edit & resubmit a manifest
  • Make configuration information in request and manifest optional, so it can be supplied/returned separately
  • Fully implement the compute ontology from Ilia
  • Ilia's other requests (Openflow related information)
  • Document process for updates per my dev list email
  • Be consistent: ref vs idref vs href
  • Include AM name/URL in RSpecs? Experimenter who allocated it URN?
  • Incorporate stitching extension as part of the 'base' RSpec

Misc

  • Allow Shutdown on a single sliver or a list of slivers
  • Add geni_am_info block to GetVersion return (name, id, url, location, description, is_proxy, proxy: {(a geni_am_info block)}, proxy_for[] (list of geni_am_info blocks))
  • Allow the update methods to take a generic rspec argument, allowing AMs to accept full or diff RSpecs
  • Tickets
    • Remove requestor certificate?
    • Support brokers: Make ticket methods return multiple tickets. Define tickets as optionally diffs (additive). Make RedeemTicket and UpdateTicket take a list of tickets. Tickets are delegatable via signing, but the delegated ticket must be for a strict subset of the resources in the original.
    • Drop ticket struct, not a signed document. Just reference by ID
  • Support message passing
    • Add sendMsg (and getMsgs?) methods that take a signed document, allowing freely passing messages instead of XMLRPC?

Stitching

AMs must support the stitching extension where Layer 2 connections are available:

Attachments (1)

Download all attachments as: .zip