wiki:GAPI_AM_API_DRAFT

Version 27 (modified by Aaron Helsinger, 12 years ago) (diff)

--

GENI Aggregate Manager API Draft Revisions

This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, many of the revisions documented here have been agreed to in discussions on the GENI developer mailing list, and during at least one GEC, but not formally adopted. Well behaved aggregates will typically already implement such revisions. Other revisions are in early discussions and subject to change or abandonment.

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is the set of resources allocated by one Aggregate to one Slice.

The current officially adopted version of the API is 2 and is documented on the main API page.

API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They include:

  1. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    • Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    • Omni version 1.3 (released June 2011) added client software support for these changes.
  2. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.

This page documents proposed changes for AM API version 3. These changes are grouped into sets. API Version 3 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.

  1. Undefined proposal: Support for UpdateSliver.
  2. Undefined: Support for clients manipulating individual slivers or groups of slivers at an aggregate.
  3. Undefined: Tickets, otherwise known as negotiated reservations.

Summary of Proposed Changes

Change Set C: UpdateSliver

This change set is new, not implemented, and currently under discussion.

  • Add a method UpdateSliver, which adds, removes or modifies resources allocated the slice at the given aggregate, to match the new request RSpec supplied as an argument
  • See the discussion below.

Change Set D: Slivers and Sliver groups

This change is TBD. The current proposal is to do nothing. See the discussion below.

Change Set E: Tickets

This change is TBD. There is no concrete proposal yet on how to introduce tickets. See the discussion below.

Proposing Additional Changes

GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.

Technical discussions are generally held on the Developers mailing list

Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net

Proposed Change Details

Change Set C: UpdateSliver

Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.

A common complaint among experimenters about the current AM API V1 is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resources for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources. Then the experimenter manually calls RestartSliver as needed on nodes that need to be rebooted, when the experimenter is ready to do so. This takes an extra step, but gives the experimenter full control over which nodes are restarted and when.

Some key properties and differences of this method at those two aggregate code-bases, with a note on the alternatives. The community must discuss the options:

  • Takes the full description of what the experimenter wants, and the server computes the difference. Note that 2 experimenters with permissions to modify the slice (say, 2 students of a professor) could issue conflicting update calls on the same aggregate. And alternative would be that experimenters must compute the difference themselves, and would just specify the incremental changes they want in their reservation.
  • Atomic: either the full request succeeds, or it fails. The alternative would be that if the experimenter wanted 5 more nodes and only 3 were available, that the server could give you those 3.
  • The two implementations above differ on whether the changes are immediate or not. ProtoGENI uses tickets, allowing the experimenter to change their mind or coordinate their requests (See below). They also separate out reservation of the resources with restarting nodes. This allows the experimenter to control which nodes get rebooted and when. PlanetLab in comparison handles all restarting of nodes for the experimenters, giving them a single operation to get their resources. This approach also more closely matches the behavior of CreateSliver in the AM API V1.

The community must discuss the alternatives above.

This proposal was discussed at the GEC12 AM API session.

The proposed new method:

struct UpdateSliver(string slice_urn,
                    string credentials[],
                    <GENIV3 request RSpec schema compliant XML string> rspec,
                    struct users[],
  		    struct options)
Success Return:
{
   code=0
   value= <GENI V3 Manifest RSpec string>
   option = <None>
}

Proposed semantics:

  • RSpec argument is a complete request RSpec with the desired new end state for the slice at this aggregate. Open questions: Does the request schema support specifying sliver_ids or other attributes that were in the manifest RSpec, and which the experimenter might want or need to preserve in this RSpec?
  • Atomic: the request either succeeds entirely or fails and the slice is not changed at the aggregate at all, with the aggregate returning a message indicating that it failed and why. Aggregates are encouraged to return enough information to allow the experimenter to modify the request such that it can succeed.
  • Resources are allocated and started or restarted immediately, on behalf of the experimenter (no tickets, and no separate call to restart the sliver)

Change Set D: Slivers and SliverGroups

Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.

The current proposal in this change set is to do nothing. See the proposal for UpdateSliver instead.

This proposal was discussed at the GEC12 AM API session.

The current AM API calls take a Slice URN, and operate on all resources under that ID at the given aggregate - all the resources for that slice at the aggregate are allocated, renewed, and deleted together. There is no provision for releasing some of the resources allocated to the slice at that aggregate, or for adding new resources to the reservation for that slice at a given aggregate.

This ties closely to the precise definition of a Slice vs a Sliver. The current AM API methods imply that a sliver represents all resources at an aggregate for a given slice. However, this does not match the definition that previous GENI documents have used, nor the functionality that experimenters desire.

A sliver is the smallest set of resources at an aggregate that can be independently reserved and allocated. A given slice may contain multiple slivers at a single aggregate. A sliver may contain multiple components.

As such, the current AM API methods in fact operate on a group of slivers.

This change set would provide a means for experimenters to operate on individual slivers within their slice at a given aggregate, or perhaps to define and operate on arbitrary groups of slivers at a given aggregate.

Specifically, this change would provide ways to:

  • Add resources to a slice at a given aggregate, where resources are already reserved - without having to delete and recreate the reservation at the aggregate. But for this capability, see the UpdateSliver proposal above.
  • Delete specific resources (slivers) from a reservation at an aggregate, without losing other resources. For this capability, see the UpdateSliver proposal above. Alternatively, DeleteSliver could be modified to take 'either' a slice_urn}} (as it does currently), or a {{{sliver_urn, such as is returned in a manifest RSpec.
  • Renew the reservation on specific resources (slivers) within a reservation at an aggregate, while allowing other resource reservations to expire. However, it is unclear whether this particular function would be useful.
  • Support fine grained manipulation of resources in other future AM API calls (such as StartSliver, StopSliver, RestartSliver). However, those other functions are not yet supported in the AM API.

NOTE: It is important to define the way of referring to the sliver (sliver name or sliver_urn) and the restrictions on such a name with regards to supported characters and length.

Change Set E: Tickets

Note: This set of changes is not defined, currently under active discussion and has gotten no unofficial or official agreement.

This proposal was discussed at the GEC12 AM API session.

The SFA defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.

Tickets are used in the ProtoGENI CMV2 interface, and are discussed on the PG wiki. Tickets (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see the Orca Book. However, each of these uses of the notion of tickets differs.

Tickets would enable a number of useful and possibly critical features:

  • Brokers: 3rd parties consolidating, scheduling and allocating resources on behalf of a number of other aggregates
  • Lending resources to other experimenters
  • Coordinated or negotiated reservations: reserving resources from aggregate B only if aggregate A can give you a complementary resource. For example, a VLAN tag. This is related to stitching, both network stitching and the more general form.
  • Two phase commit reservations (similar to the above)
  • Scheduled reservations in the future
  • Giving experimenters explicit control over when resources are started, stopped, and restarted (see the discussion on UpdateSliver above)

This change would introduce the concept of tickets, and new methods to use tickets. It requires:

  • a ticket schema
  • methods to get, redeem, split, and maybe return tickets
  • semantics for what promises a ticket provides, and what it means to redeem and return tickets
  • many other details.

GENI Aggregate Manager API Draft Revisions

This page documents DRAFT revisions to the GENI Aggregate Manager API, proposed for the next version of the API. As indicated below, many of the revisions documented here have been agreed to in discussions on the GENI developer mailing list, and during at least one GEC, but not formally adopted. Well behaved aggregates will typically already implement such revisions. Other revisions are in early discussions and subject to change or abandonment.

The GENI Aggregate Manager API allows aggregates to advertise resources and to allocate resources to Slices in the form of Slivers. A Sliver is the set of resources allocated by one Aggregate to one Slice.

The current officially adopted version of the API is 2 and is documented on the main API page.

API Version 2 was adopted based on changes previously listed on this page. Those changes have been removed from this page, and are now documented on a separate page. They include:

  1. RSpec related changes, specifying that RSpecs are XML following GENI standard schemas.
    • Since June 2011, the latest software from ProtoGENI and SFA (as of code tag 1.0-24) has complied with these changes.
    • Omni version 1.3 (released June 2011) added client software support for these changes.
  2. Supporting flexible arguments and returns. Specifically, adding a property list to all calls, and making all returns be a property list.

This page documents proposed changes for AM API version 3. These changes are grouped into sets. API Version 3 will be the collection of changes from the change sets below which we next agree on. Change sets still under discussion will then be targeted at a future release.

  1. Undefined proposal: Support for UpdateSliver.
  2. Undefined: Support for clients manipulating individual slivers or groups of slivers at an aggregate.
  3. Undefined: Tickets, otherwise known as negotiated reservations.

Summary of Proposed Changes

Change Set C: UpdateSliver

This change set is new, not implemented, and currently under discussion.

  • Add a method UpdateSliver, which adds, removes or modifies resources allocated the slice at the given aggregate, to match the new request RSpec supplied as an argument
  • See the discussion below.

Change Set D: Slivers and Sliver groups

This change is TBD. The current proposal is to do nothing. See the discussion below.

Change Set E: Tickets

This change is TBD. There is no concrete proposal yet on how to introduce tickets. See the discussion below.

Proposing Additional Changes

GENI community members are encouraged to propose changes to the GENI Aggregate Manager API.

Technical discussions are generally held on the Developers mailing list

Specific questions may be directed to the software team at the GPO (Tom Mitchell, Aaron Helsinger, and Sarah Edwards) at {tmitchel, ahelsing, sedwards} at geni.net

Proposed Change Details

Change Set C: UpdateSliver

Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.

A common complaint among experimenters about the current AM API V1 is that there is no way to add or remove resources from a slice at an aggregate, without completely deleting the slice at that aggregate and recreating it, possibly losing resources to another experimenter and certainly losing state. This proposal aims to address that, by introducing a method to update the slice at an aggregate.

The SFA calls for an UpdateSlice method, "to request that additional resources—as specified in the RSpec—be allocated to the slice".

In the PlanetLab implementation of the SFA, UpdateSliver is in fact a synonym for CreateSliver - the server will ensure that your allocated resources match your request RSpec, adding, removing and modifying resources as needed. It immediately allocates and boots nodes to match the request RSpec.

The ProtoGENI CMV2 API has UpdateSliver, which is described as the way to "Request a change of resources for an existing sliver. The new set of resources that are desired are specified in the rspec." At ProtoGENI as at PlanetLab, this method takes the full RSpec description of resources the experimenter wants, and the server computes the difference with what the experimenter already has. At ProtoGENI though, this method returns a ticket. The experimenter must then redeem the ticket to actually acquire the resources. Then the experimenter manually calls RestartSliver as needed on nodes that need to be rebooted, when the experimenter is ready to do so. This takes an extra step, but gives the experimenter full control over which nodes are restarted and when.

Some key properties and differences of this method at those two aggregate code-bases, with a note on the alternatives. The community must discuss the options:

  • Takes the full description of what the experimenter wants, and the server computes the difference. Note that 2 experimenters with permissions to modify the slice (say, 2 students of a professor) could issue conflicting update calls on the same aggregate. And alternative would be that experimenters must compute the difference themselves, and would just specify the incremental changes they want in their reservation.
  • Atomic: either the full request succeeds, or it fails. The alternative would be that if the experimenter wanted 5 more nodes and only 3 were available, that the server could give you those 3.
  • The two implementations above differ on whether the changes are immediate or not. ProtoGENI uses tickets, allowing the experimenter to change their mind or coordinate their requests (See below). They also separate out reservation of the resources with restarting nodes. This allows the experimenter to control which nodes get rebooted and when. PlanetLab in comparison handles all restarting of nodes for the experimenters, giving them a single operation to get their resources. This approach also more closely matches the behavior of CreateSliver in the AM API V1.

The community must discuss the alternatives above.

This proposal was discussed at the GEC12 AM API session.

The proposed new method:

struct UpdateSliver(string slice_urn,
                    string credentials[],
                    <GENIV3 request RSpec schema compliant XML string> rspec,
                    struct users[],
  		    struct options)
Success Return:
{
   code=0
   value= <GENI V3 Manifest RSpec string>
   option = <None>
}

Proposed semantics:

  • RSpec argument is a complete request RSpec with the desired new end state for the slice at this aggregate. Open questions: Does the request schema support specifying sliver_ids or other attributes that were in the manifest RSpec, and which the experimenter might want or need to preserve in this RSpec?
  • Atomic: the request either succeeds entirely or fails and the slice is not changed at the aggregate at all, with the aggregate returning a message indicating that it failed and why. Aggregates are encouraged to return enough information to allow the experimenter to modify the request such that it can succeed.
  • Resources are allocated and started or restarted immediately, on behalf of the experimenter (no tickets, and no separate call to restart the sliver)

Change Set D: Slivers and SliverGroups

Note: This set of changes is currently under active discussion and has gotten no unofficial or official agreement.

The current proposal in this change set is to do nothing. See the proposal for UpdateSliver instead.

This proposal was discussed at the GEC12 AM API session.

The current AM API calls take a Slice URN, and operate on all resources under that ID at the given aggregate - all the resources for that slice at the aggregate are allocated, renewed, and deleted together. There is no provision for releasing some of the resources allocated to the slice at that aggregate, or for adding new resources to the reservation for that slice at a given aggregate.

This ties closely to the precise definition of a Slice vs a Sliver. The current AM API methods imply that a sliver represents all resources at an aggregate for a given slice. However, this does not match the definition that previous GENI documents have used, nor the functionality that experimenters desire.

A sliver is the smallest set of resources at an aggregate that can be independently reserved and allocated. A given slice may contain multiple slivers at a single aggregate. A sliver may contain multiple components.

As such, the current AM API methods in fact operate on a group of slivers.

This change set would provide a means for experimenters to operate on individual slivers within their slice at a given aggregate, or perhaps to define and operate on arbitrary groups of slivers at a given aggregate.

Specifically, this change would provide ways to:

  • Add resources to a slice at a given aggregate, where resources are already reserved - without having to delete and recreate the reservation at the aggregate. But for this capability, see the UpdateSliver proposal above.
  • Delete specific resources (slivers) from a reservation at an aggregate, without losing other resources. For this capability, see the UpdateSliver proposal above. Alternatively, DeleteSliver could be modified to take 'either' a slice_urn}} (as it does currently), or a {{{sliver_urn, such as is returned in a manifest RSpec.
  • Renew the reservation on specific resources (slivers) within a reservation at an aggregate, while allowing other resource reservations to expire. However, it is unclear whether this particular function would be useful.
  • Support fine grained manipulation of resources in other future AM API calls (such as StartSliver, StopSliver, RestartSliver). However, those other functions are not yet supported in the AM API.

NOTE: It is important to define the way of referring to the sliver (sliver name or sliver_urn) and the restrictions on such a name with regards to supported characters and length.

Change Set E: Tickets

Note: This set of changes is not defined, currently under active discussion and has gotten no unofficial or official agreement.

This proposal was discussed at the GEC12 AM API session.

The SFA defines the concept of a ticket. SFA1.0 section 5.3 says "A component signs an RSpec to produce a ticket, indicating a promise by the component to bind resources to the ticket-holder at some point in time." Tickets are promises to allocate resources.

Tickets are used in the ProtoGENI CMV2 interface, and are discussed on the PG wiki. Tickets (and leases) are also used extensively in Orca. For details on the use of leases and tickets in Orca, see the Orca Book. However, each of these uses of the notion of tickets differs.

Tickets would enable a number of useful and possibly critical features:

  • Brokers: 3rd parties consolidating, scheduling and allocating resources on behalf of a number of other aggregates
  • Lending resources to other experimenters
  • Coordinated or negotiated reservations: reserving resources from aggregate B only if aggregate A can give you a complementary resource. For example, a VLAN tag. This is related to stitching, both network stitching and the more general form.
  • Two phase commit reservations (similar to the above)
  • Scheduled reservations in the future
  • Giving experimenters explicit control over when resources are started, stopped, and restarted (see the discussion on UpdateSliver above)

This change would introduce the concept of tickets, and new methods to use tickets. It requires:

  • a ticket schema
  • methods to get, redeem, split, and maybe return tickets
  • semantics for what promises a ticket provides, and what it means to redeem and return tickets
  • many other details.

Attachments (1)

Download all attachments as: .zip