Changes between Initial Version and Version 1 of DRAFT_GAPI_AM_API_V3

02/13/12 11:52:01 (7 years ago)




    v1 v1  
     3= Proposed changes for GENI Aggregate Manager API version 3 =
     5If this is all approved, there will be a lot in APIv3.
     7Clients are reminded that these methods are requests - based on the AM type and resource types, these requests may fail or not make sense. Clients should watch for {{{UNSUPPORTED}}} returns.
     9AMs are encouraged to implement as much of this API as reasonable to provide a common front for clients. When a function is not possible, return UNSUPPORTED, document publicly what functions do work, and suggest alternative ways to get the result the client desired.
     11We will aim for an agreement on what the new methods/constructs will be at GEC13. We plan on driving out details (syntax, semantics, error codes) after the GEC. We will not worry before or at GEC13 about specific error codes, details of certificate or credential formats, or such things.
     13= Summary =
     14== Proposed Changes ==
     15At the top level, the proposed changes for AM API v3 include:
     16 - Add methods using tickets to do negotiated reservations
     17 - Add !UpdateSliversNow to immediately modify your reservation
     18 - Change methods to clarify that there may be multiple slivers per slice at an AM, and to allow operating on individual slivers
     19 - Define sliver states, and the state changes that various methods cause
     20 - Add a new general !ActOnSlivers method allowing AMs to support AM and resource-type specific operations
     21 - Generalize the credentials argument, allowing ABAC support
     22 - Clarify: a 2nd call to !CreateSliver means replace the previous reservation for the slice
     23 - !SliverStatus return structure includes sliver expiration
     24 - !SliverStatus return includes SSH logins/key for nodes that support SSH access
     25 - Support proxy aggregates with 2 new options and aggregate SSL client certificate authorization rules
     26 - Standardize certificate contents to include a real serial number, holder email, holder uuid, and authority URL
     27 - Define slice ID as the UUID in slice certificates
     28 - Define slice name, sliver name, and username restrictions, and similar for URNs
     29 - Standardize credential privileges
     30 - Publish schemas for credentials and certificates
     31 - Various RSpec schema changes, resulting in GENI v4 RSpecs
     33= Tickets =
     34APIv3 adds support for negotiated reservations or two-phase commit, by ADDing methods that allow an experimenter to reserve resources for a slice without committing to using them, or forcing the AM to incur the cost of instantiating them.
     37== Methods ==
     38 1. !GetTicket (slice_urn, credentials[], requestRSpec, options {geni_start_time, geni_end_time, others})
     39 - Return: ticket; state: ticketed
     40 - Re start/end: if left out then start is 'now or really soon' and end is the default sliver duration.
     41 2. !RedeemTicket(slice_urn, credentials[], ticket, users (as in !CreateSlivers), options)
     42 - Return: manifest RSpec & end time; state: ??
     43 - FIXME: State after !RedeemTicket: started? Or do you have to use !ActOnSlivers to do that?
     44 3. !ReleaseTicket(slice_urn, credentials, ticket, options)
     45 - Return: True or error
     46 4. !UpdateTicket (atomic release/get) (slice_urn, credentials[], requestRSpec, ticket, options {geni_start_time, geni_end_time, others})
     47 - Re start/end: if left out then start is 'now or really soon' and end is the default sliver duration.
     48 - Return: Ticket; state: ticketed
     49 5. !UpdateSliversWithTicket (slice_urn, credentials[], requestRSpec (or edited manifest), options {geni_start_time, geni_end_time})
     50 - Re start/end: if left out then start is 'now or really soon' and end is the default sliver duration.
     51 - Return: ticket; state: ?ticketed?
     53= !UpdateSliversNow and sliver-specific operations =
     54A slice may have multiple slivers at a single AM. Experimenters can operate on slivers independently, if the AM supports it. AMs define slivers as groups of resources, and give them locally unique sliver_urns for identifying that group of resources.
     57== !UpdateSliversNow ==
     58 - Add new method !UpdateSliversNow
     59  - Takes full request RSpec of desired final state
     60   - FIXME: Want manifest to be readily modifiable to be a request (include component_ids and sliver_ids)
     61   - FIXME: Could call this just 'RSpec' and let the AMs decide if this is diff or full? This leaves the window open for Ilia to implement the diff version if he wants. But it is less clean I think. So I think no.
     62  - AMs may, as always, return {{{UNSUPPORTED}}}
     63  - As with !CreateSlivers, AM should start/restart resources immediately, as necessary.
     64  - Request is either fully satisfied, or fails.
     65  - See !UpdateSliversWithTicket above
     66  - AMs must document the level of service they provide: will any state be lost on existing resources?
     67   - FIXME: ideally this would be a per node or resource-type specification
     68   - FIXME: Could allow experimenter to specify what level of disruption they can tolerate? EG I can live with losing in memory state, but not with losing disk state.
     71struct UpdateSliversNow(string slice_urn,
     72                    string credentials[],
     73                    <GENIV3 request RSpec schema compliant XML string> rspec,
     74                    struct users[],
     75                    struct options)
     76return value is GENI v3 Manifest RSpec string
     79This method moves the overall sliver to configuring and then ready if it succeeds.
     82== Addressable Slivers ==
     83 - Modify some existing methods to clarify that they act on 1+ slivers
     84  - !CreateSliver -> !CreateSlivers
     85  - !RenewSliver -> !RenewSlivers
     86  - !DeleteSliver -> !DeleteSlivers
     87  - !SliverStatus -> !SliversStatus
     88 - Some methods that take {{{slice_urn}}} now take {{{urn}}} that may be slice or sliver
     89  - !RenewSlivers, !DeleteSlivers, !SliversStatus
     90  - AMs are free to refuse to Renew, Delete, or provide status on an individual sliver, if the local AM or that resource type does not support it.
     91   - AMs should return an error message that indicates 'use !UpdateSliversNow!'
     92   - E.G. renewing part of a reservation but not all has the same issues as !UpdateSliversNow with only a partial RSpec: dependencies or implications on other parts of the slice at this AM may be unclear.
     93   - FIXME: If we dislike this, then we only rename the methods and don't allow the sliver_urn argument here
     96= Support AM and resource-type specific methods. =
     97In other words, define the control API (the AM API) as about moving slices through various states at an AM.
     99== !ActOnSliver ==
     100Generic action for acting on slivers in an AM or resource type specific way
     101 - !ActOnSlivers(command, credentials[], URN, requested_state, options)
     102 - return is a struct that includes: URN, newState, and any others
     103 - URN may be slice or sliver
     104 - state is the state of the sliver or slice. Values may be one of the geni status values, or AM-specific
     106== Define Sliver States ==
     107Define explicitly states of slivers or the whole slice at an AM:
     108 1) add 'ticketed', 'allocated', 'shutdown', 'closed' as standard {{{geni_status}}} values in !SliversStatus
     109(ticketed being the state after !GetTicket, allocated after !RedeemTicket but before start, Shutdown for after shutdown, and closed for after Delete)
     111 2) State that AMs are encouraged to add their own states (ie AMs are encouraged to have an am_specific status field with other values)
     112   - AMs must document these states. This API does not define how or where.
     113(Question: Can they use those values where we call for a single state? Do we add a parallel am_specific_state? If so, we need some geni states that make sense being modified.)
     115 3) Specify an additional optional member of the return struct for CreateSlivers, DeleteSlivers, Shutdown: {{{geni_status}}}, which contains one of those values
     116   - plus for a bunch of the new methods
     118 4) Clarify in the API that calling !CreateSlivers moves the sliver through ticketed, allocated, configuring, and then ready (or failed or unknown on error)
     119 5) Similarly clarify that !DeleteSlivers changes from allocated/configuring/ready/unknown/failed to closed (or failed or unknown)
     120 6) Shutdown moves from allocated/configuring/ready/unknown/failed to shutdown
     122There are specific rules for rolling up states or operating.
     123 - If not all resources in the sliver/slice can be moved to the desired next state, then the call fails. (All or nothing)
     124 - Status that is reported rolls up similarly: The slice is ready only when all slivers are ready. The slice is failed when any slivers are failed. Whem moving from state 1 to 2, the slice is in 1 until all slivers are in 2 (ticketed->allocated, allocated -> configuring, configuring->ready)
     125 - unknown and closed are different: some slivers can be unknown - slice follows the rest. Similarly closed. Unless all slivers are unknown/closed.
     128= Credentials are general authorization tokens. =
     129This change modifies the credentials argument to all methods. Each credential is now defined only as a signed document. A given list of credentials may contain credentials in multiple formats. The list may be empty. A given authorization policy at an AM may require 0, 1, or many credentials.
     130 - AM may get other authorization material from other sources: EG a future Credential Store service.
     131 - An AM must pick credentials out of the list that it understands and be robust to receiving credentials it does not understand.
     132 - AMs are required to continue to accept current-format credentials.
     133  - In particular, a single standard slice credential remains sufficient for most authorization policies.
     134 - Other credential formats might include ABAC x509 Attribute certificates, eg.
     136= Changes to existing methods =
     137== Proxy aggregate managers are supported ==
     138 - Define a new option {{{geni_am_id}}} for all methods. Proxy AMs may use this ID to look up the URL of the real AM, and pass the call along.
     139 - Define a new option {{{geni_experimenter_urn}}}. Proxy AMs will retrieve the experimenter URN from the subjectAltName of the client SSL certificate, and then supply this value in calls to the 'real AM' where the call is being redirected.
     140 - Require that AMs support separately authorizing accepted SSL client certficiates, from authorizing accepted certificates in credentials. In particular, proxiable AMs must support restricting client connections to their configured proxy AM, while succesfully validating all certificates in authorized slice credentials.
     142== Others ==
     143 - !ListResources returns a struct, 1 member is RSpec, other is list of tickets (but see below)
     144 To support retrieving open tickets.
     145 - !CreateSlivers takes full RSpec, it is not 'add'. 2nd such call means update.
     146 Just clarifying from AM API v1
     147 - Add ssh logins/keys to each node that supports SSH login in !SliversStatus (but see below)
     149'users' => [{'urn'   => $user1_urn.
     150             'login' => $login,
     151             'keys'  => [...] },
     152            {'urn'   => $user2_urn.
     153             'login' => $login,
     154             'keys'  => [...] }
     155           ]
     159= Standardize certificates and credentials =
     160 - Standardize contents
     161  - Certificates: real serial number, users have email, slices have email, slices have uuid, users too?,
     162   - URL of the issuing authority?
     163 - Specify a slice ID (require UUID in slice certs?)
     164 - Standardize slice and user name and URN restrictions
     165  - Usernames are case-insensitive internally, though they may be case-sensitive in display
     166   - . allowed? Limited to 8 characters?
     167  - Slice names are <=19 characters, only alphanumeric plus hyphen not in first character ({{{'^[a-zA-Z0-9][-a-zA-Z0-9]+$'}}})
     168   - Require PL do something?
     169 - Define sliver_urn as following same rules as slice_urns, ditto sliver names
     170  - urn:publicid:IDN:<am name>+sliver+<sliver name>
     171 - Revocation/expiration policy & procedures
     172 - Publish schemas for certificates & credentials on
     175= Change SFA credentials' privileges =
     176Our goal is to simplify and standardize privilege strings. Currently there are wildcards, bind, embed, and others.
     177We also want extensibility.
     178Kinds of operations:
     179 - Learn about the slice
     180 - Add/Modify/Delete resources in the slice
     181 - Read slice details like I&M?
     182 - Use the slice
     185 - !CanWrite
     186 - !CanRead
     187 - !CanReadDetails
     188 - !CanUse
     191= RSpec changes resulting in GENI v4 RSpecs =
     192 - Support unbound manifests
     193 - Make manifest an extension of Request, so you can readily edit & resubmit a manifest
     194 - Make configuration information in request and manifest optional, so it can be supplied/returned separately
     195 - Fully implement the compute ontology from Ilia
     196 - Ilia's other requests
     197 - Document process for updates per my dev list email