wiki:UniformClearinghouseAPI

Version 14 (modified by jmccolga@bbn.com, 6 years ago) (diff)

--

GENI Clearinghouse API’s

Marshall Brinn, GPO

DRAFT: August 6, 2013

Introduction

This document proposes a set of standard API’s that any GENI Clearinghouse (CH) should or may provide. The document describes what is required and what is optional in the calls and responses to these APIs.

The GENI Software Architecture rests on the interaction between different entities:

Aggregates : Collections of resources managed and presented in accordance with the AM API

Members : Experimenters or other human consumers of aggregate resources

Authorities : Services that manage assertions about members and their permissions with respect to aggregate resources.

There are two fundamental authority types in GENI:

  • Member Authority [MA]: Manages and asserts attributes about particular members
  • Slice Authority [SA]: Manages slice objects and generates credentials for members with respect to slices.

A Federation is a collection of Authorities and Aggregates that establish mutual trust and common policies to facilitate the sharing of resources among members. A Clearinghouse (CH) is a set of software services representing a given Federation, providing lists of Slice Authorities, Member Authorities and aggregates associated with that federation, and providing a set of PKI certificates that any aggregate belonging to a given federation accepts as trust roots. The relationship of Federations to Clearinghouses is 1:1.

The Authorities of any given Federation are free to implement their own Authorization (AuthZ) scheme. The API’s allow for passing credentials to the calls, but an Authority may choose to allow or disallow calls using logic and policies that are internal to that Federation. There is no universal (cross-Federation) requirement for any particular policy regarding Authority AuthZ.

Authorities are fundamentally independent of one another. The objects defined at one Authority are not necessarily entitled to any services provided by another Authority. Each aggregate may choose to trust or not trust any particular Authority. Likewise, any Authority may chose to trust or not trust any other Authority. A CH may choose to advertise or not advertise any particular aggregate, regardless of whether that aggregate trusts the Authorities advertised by that CH. Similarly, a given Slice Authority or Member Authority may be advertised by a single CH or by multiple CH’s. CH API calls are unprotected: there is no notion of trust between CH’s or between CH’s and Authorities or Aggregates.

This document describes the APIs of the GENI CH as well as the MA and SA. It is expected that a well-behaved GENI tool will allow for interacting with any CH and Authority that implement the standard API’s described in this document.

API General Properties

The APIs described here share some common properties, which should be assumed for the rest of this document:

  • The wire-protocol is XML/RPC. It is thus language independent on both client and server side of the API calls.
  • Most calls are protected, running over SSL and thus requiring the caller to use its certificate and private key. Certain calls are unprotected and can be accessed with no certificate and private key.
  • Unprotected calls will be identified in API documentation with the annotation:

NB: This is an unprotected call , no client cert required.

  • Each call takes an ‘options’ argument, a dictionary allowing for passing specific non-standard/optional arguments
  • Each protected method takes a ‘credentials’ argument, a list of type/credential tuples that help the CH or Authority invoke whatever AuthZ logic it may choose. As noted above, the CH or Authority may choose to use or disregard these credentials. Unprotected methods do not take a ‘credentials’ argument.
  • Each CH or Authority provides a get_version method, which describes the version number of the API provided, credential types supported, supplementary object fields and other data for interpreting API call returns.
  • A CH or Authority is free to implement additional methods beyond those specified in this document.
  • The URN is the fundamental identifier in all GENI CH API’s. URN’s are globally unique at any given time, though not necessarily unique over time. Disambiguation for entities with the same URN over time may be provided by an optional UUID argument for certain API methods.

These API’s are provided in pseudo-code (i.e. language independent) format, with inputs and outputs (optional and required) described by comments, e.g.

/*

Perform method ‘ch_func’

Arguments

argl : ...

credentials : list of {type : credential} tuples representing credentials provided by caller to support AuthZ on method call. [NB: This argument will be omitted in descriptions below. ] options : … [ Recognized options: ….]

Return : E.g. a list of dictionaries with these fields mandatory (…) and these fields optional (…)

*/

function ch_func (arg1, arg2, credentials, options)

API 'get_version' methods

Different Clearinghouses will provide different sets of methods bundled into services. Further, they will manage different kinds of objects and support different details for these objects.

The CH API provides a ‘get_version’ method, which provides information to the caller (or a tool composing calls for a tool user) about versions and options supported by that API. The call takes no argument and is unguarded (anyone can call it). The return from the get_version call will be a dictionary including the following entries (by key):

  • VERSION: The version number of the API (e.g. “10.0.7”)
  • SERVICES: The set of services the given URL supports (only for Slice Authority)
  • CREDENTIAL_TYPES: A list of recognized credential types (e.g. [ABAC, SFA]) on protected API methods.
  • ROLES : A list of recognized roles for slice/project membership (optional for those Slice Authorities supporting membership)
  • FIELDS: A dictionary of supplementary object field names (i.e. in additional to the required fields) and associated attributes including:
    • “OBJECT” provides the object to which the field belongs (if not the default authority object, i.e. SLICE for Slice Authority, MEMBER for Member Authority, Service for Clearinghouse)
    • “TYPE” may be one of “URN”, “UID”, “STRING”, “DATETIME”, “EMAIL”, “KEY”,“BOOLEAN”, “CREDENTIAL”, “CERTIFICATE”. [NB. This set of types subject to change. See Appendix for more information on these data types.]
    • “CREATE” attributes may be specified as “REQUIRED”, “ALLOWED” or “NOT ALLOWED” (default = “NOT ALLOWED”). These indicate whether the given supplementary field is required, allowed or prohibited in create calls.
    • “UPDATE” attributes may be specified as TRUE or FALSE (default = FALSE). These indicate whether the given field may be specified in an update call.
    • “PROTECT” attributes may be labeled as “PUBLIC”, “PRIVATE” or “IDENTIFYING”. These are for the Member Authority Only to differentiate between public, identifying and private data fields on members.

The return from the get_version call will be used to construct and validate options to CH and Authority API calls, as described in subsequent sections.

The following page provides some example returns from different get_version calls.

Example get_version returns:

The following is an example of a return from a get_version for an SA. The responses are all dictionaries via XMLRPC into the native implementation. They are shown here in JSON-like syntax:

{

“VERSION”: “10.0.7”, “SERVICES”: [“SLICE”, “PROJECT”, “SLICE_MEMBER”, “PROJECT_MEMBER”] “CREDENTIAL_TYPES”: [“SFA”, “ABAC”], “ROLES” : [“LEAD”, “ADMIN”, “MEMBER”, “AUDITOR”, “OPERATOR” ], “FIELDS”:

“PROJECT_URN”: {“TYPE”: “URN”, “CREATE”: “REQUIRED”, “UPDATE”: FALSE},

“SLICE_EMAIL”: {“TYPE”: “EMAIL”, “CREATE”: “REQUIRED”, “UPDATE”: TRUE}

“PROJECT_EMAIL”: {“TYPE”: “EMAIL”, “CREATE”: “REQUIRED”, “UPDATE”: TRUE, “OBJECT”: “PROJECT”}

}

}

}

The following is an example of a return from a get_version for an MA, provided in JSON-like syntax:

{

“VERSION”: “10.0.7”,

“CREDENTIAL_TYPES”: [“SFA”, “ABAC”],

“FIELDS”: {

“MEMBER_DISPLAYNAME”: {“TYPE”: “STRING”, “CREATE”: “ALLOWED”, “UPDATE”,

TRUE, “PROTECT”: “IDENTIFYING”},

“MEMBER_AFFILIATION”: {“TYPE”: “STRING”, “CREATE”: “ALLOWED”, “UPDATE”:

TRUE, “PROTECT”: “IDENTIFTYING”},

“MEMBER_SPEAKS_FOR_CREDENTIAL”: {“TYPE”: “CREDENTIAL”},

“MEMBER_SSL_PUBLIC_KEY”: {“TYPE”: “SSL_KEY”},

“MEMBER_SSL_PRIVATE_KEY”: {“TYPE”: “SSL_KEY”, “PROTECT”: “PRIVATE”},

“MEMBER_SSH_PUBLIC_KEY”: {“TYPE”: “SSH_KEY”},

“MEMBER_SSH_PRIVATE_KEY”: {“TYPE”: “SSH_KEY”, “PROTECT”: “PRIVATE”},

“MEMBER_ENABLED”: {“TYPE”: “BOOLEAN”, “UPDATE”: TRUE}

}

}

The following is an example of a return from a get_version from a CH, provided in JSON-like syntax:

{

“VERSION”: “10.0.7”,

“FIELDS”: {

“SERVICE_DESCRIPTION”: {“TYPE”: “STRING”}}

}

}

API Error Handing

All method calls return a tuple [code, value, output]. What is described as ‘Return’ in the API’s described below is the ‘value’ of this tuple in case of a successful execution. ‘Code’ is the error code returned and ‘output’ is the returned text (e.g. descriptive error message).

Each CH and Authority is free to define and return its own specific error codes. However we suggest the following essential set of error codes to report on generic conditions:

CODE_NAME CODE_VALUE DESCRIPTION
NONE 0 No error encountered – the return value is a successful result. An empty list form a query should be interpreted as ‘nothing found matching criteria’.
AUTHENTICATION_ERROR 1 The invoking tool or member did not provide appropriate credentials indicating that they are known to the CH or that they possessed the private key of the entity they claimed to be
AUTHORIZATION_ERROR 2 The invoking tool or member does not have the authority to invoke the given call with the given arguments
ARGUMENT_ERROR 3 The arguments provided to the call were mal-formed or mutually inconsistent.
DATABASE_ERROR 4 An error from the underlying database was returned. (More info should be provided in the ‘output’ return value]
NOT_IMPLEMENTED_ERROR 100 The given method is not implemented on the server.
SERVER_ERROR 101 An error in the client/server connection

API Method Conventions

Each CH and Authority manages the state of or access to objects. Some conventions apply to similar methods across CH or Authority services.

Create_*Method : Creates a new instance of the given object with a ‘fields’ option specifying particular field values that are to be associated with the object. These may only include those fields specified as ‘ALLOWED or ‘REQUIRED’ in the ‘Creation’ column of the object descriptions below or in the “CREATE’ key in the supplemental fields in the get_version specification for that object. If successful, the call returns a dictionary of the fields associated with the newly created object.

Update_* Method : Updates an object instance specified by URN with a ‘fields’ option specifying the particular fields to update. Only a single object can be updated from a single update call. The fields may include those specified as ‘Yes’ in the ‘Update’ column of the object descriptions below, or ’TRUE’ in the ‘UPDATE’ key in the supplemental fields provided by the get_version call. Note: There may be more than one entity of a given URN at an authority, but only one ‘live’ one (any other is archived and cannot be updated).

Lookup_* Method : This call takes a set of ‘match’ criteria provided in the ‘options’ field, and returns a dictionary of dictionaries of object attributes keyed by object URN matching these criteria. If a ‘filter’ option is provided, only those attributes listed in the ‘filter’ options are returned.

Specifically, the options argument to the lookup_* call is a dictionary. It contains an entry with key ‘match’ that contains a dictionary of name/value pairs. The names are of fields listed in the get_version for that object. The values are values for those fields to be matched. The semantics of the match is to be an “AND” (all fields must match).

The value in the dictionary of a ‘match’ option can be a list of scalars, indicating an “OR”. For example, a list of URNs provided to the SLICE_URN key would match any slice with any of the listed URNs.

The options argument may include an additional dictionary keyed “filter” which is a list of fields associated with that object type (again, as specified in the get_version entry for that object). No “filter” provided means all fields are to be returned.

The return of the call will be a dictionary of dictionaries, one per matching object indexed by URN, of fields matching the filter criteria. If the query found no matches, an empty dictionary is returned (i.e. no error is reported, assuming no other error was encountered in processing).

If a lookup method asks for information about objects whose disclosure is prohibited to the requestor by policy, the method must not return the data. It is implementation specified as to whether the call should return an error, return a dictionary with the URN key pointing to an empty dictionary, or have no URN key in the returned dictionary.

API Method Examples:

A Member Authority (MA) manages information about member objects. The MA method lookup_member could take an options argument such as

{

"match”: {“MEMBER_LASTNAME”: “BROWN”},

"filter”: [“MEMBER_EMAIL”, “MEMBER_FIRSTNAME”]

Such a call would find any member with last name Brown and return a dictionary keyed by the member URN containing a dictionary with their email, and first name.

{

“urn:publicid:IDN+mych+user+abrown” :

{“MEMBER_EMAIL”: abrown@williams.edu,

“MEMBER_FIRSTNAME”: “Arlene”},

“urn:publicid:IDN+mych+user+mbrown” :

{“MEMBER_EMAIL”: mbrown@umass.edu,

“MEMBER_FIRSTNAME”: “Michael”},

“urn:publicid:IDN+mych+user+sbrown” :

{“MEMBER_EMAIL”: sbrown@stanford.edu,

“MEMBER_FIRSTNAME”: “Sam”}

}

A Slice Authority (SA) manages information about slice objects. The SA method update_slice could take an options ‘update’ argument such as to change the slice description and extend the slice expiration:

{

“fields” : { “SLICE_DESCRIPTION”: “Updated Description”,

“SLICE_EXPIRATION”: “2013-07-29T13:15:30Z” }

}

An example of lookup_slice that wanted to retrieve the slice names for a list of slice URNs could specify options:

{“match”: {

“SLICE_URN”: [

“urn:publicid+IDN+this_sa:myproject+slice+slice1”,

“urn:publicid+IDN+this_sa:myproject+slice+slice2”,

“urn:publicid+IDN+this_sa:myproject+slice+slice3”

]},

“filter”: [“SLICE_NAME”]

}

API Method Examples (cont.):

An example of create_slice call would specify required options e.g.:

{

‘fields’ : {

“SLICE_NAME”: “TEST_SLICE”,

“SLICE_DESCRIPTION”: “My Test Slice”,

“SLICE_EMAIL”: myemail@geni.net,

“PROJECT_URN”: “urn:publicid+IDN+this_sa+project+myproject”

}

}

and receive a return dictionary looking like:

{

“SLICE_URN”: “urn:publicid+IDN+this.sa+slice+TESTSLICE”,

“SLICE_UID”: “…”,

“SLICE_NAME”: “TESTSLICE”,

“SLICE_CREDENTIAL”: “.....”,

“SLICE_DESCRIPTION”: “My Test Slice”,

“PROJECT_URN”: “urn:publicid+IDN+this_sa+project+myproject”,

“SLICE_EXPIRATION”: “2013-08-29T13:15:30Z”,

“SLICE_EXPIRED”: “FALSE”,

“SLICE_CREATION”: “2013-07-29T13:15:30Z”,

“SLICE_EMAIL”: myemail@geni.net

}

API Authentication

This document suggests that the Authentication required for the CH is implicit in the SSL protocol: the invoker of the call must have its cert and private key to have a valid SSL connection. Moreover, the cert must be signed by a member of the trust chain recognized by the CH.

Support for Speaks-for API Invocations

Best practices dictate that individuals should speak as themselves: that is, the entity on the other side of an SSL connection is the one referred to by the certificate on the connection. Obviously, people typically use tools or software interfaces to create these connections. When a tool is acting directly on a user’s desktop using the user’s key and cert with the user’s explicit permission, it may be acceptable to consider the tool as speaking as the user. But for many tools, the tool is acting on behalf of the user in invoking CH or AM API calls. In this case, it is important for the tool to not speak as the user but to speak for the user, and to have the service to whom the tool is speaking handle the authorization and accountability of this request accordingly.

Accordingly, a Clearinghouse and associated Authorities should support speaks-for API transactions. These API transactions use the same signatures as the calls described in this document, with these enhancements:

  • The user (spoken-for) URN provided as a ‘speaking-for’ option on the call
  • The user’s certificate be provided as part of the credentials of the call
  • The speaks-for credential: a statement signed by the user indicating that the tool has the right to speak for the user, possibly limited to a particular scope (e.g. slice, project, API call, time window).

The CH is then required to determine if the call is being made in a speaks-for context or not (that is, the ‘speaking-for’ option provided). If so, the CH call must determine if the tool is allowed to speak for the user by checking for the presence of a valid speaks-for credential and the spoken-for user’s cert. If so, the CH should validate if the user is authorized to take the proposed API action. If so, the action is taken and accounted to the user, with identity of the speaking-for tool logged. If the call is ‘speaks-for’ but any of these additional criteria are not met, the call should fail with an authorization error. If the call is not a ‘speaks-for’, then the normal authorization is performed based on the identity (certificate) provided with the SSL connection.

Aggregates are also encouraged to support speaks-for authentication and authorization, but this is an aggregate-internal policy and implementation decision, and outside the scope of this document.

Clearinghouse API

The Clearinghouse provides a list of Slice Authorities, Member Authorities and Aggregates associated with a given Federation. The URL for accessing these methods (i.e. the URL of the Clearinghouse) is to be provided out-of-band (i.e. there is no global service for gaining access to CH addressees).

The following table describes the default fields for services (aggregates and authorities) provided by CH API calls:

Name Type DESCRIPTION
SERVICE_URN URN URN of given service
SERVICE_URL URL URL by which to contact the service
SERVICE_CERT Certificate Public certificate of service
SERVICE_NAME String Short name of service
SERVICE_DESCRIPTION String Descriptive name of service

/*

Provide a structure detailing the version information as well as details of accepted options s for CH API calls.

NB: This is an unprotected call, no client cert required.

Arguments:

None

Return:

get_version structure information as described above

*/

function get_version()

/*

Return information about all aggregates associated with the Federation

NB: This is an unprotected call, no client cert required.

Arguments:

options: List of field names (from get_version) to be provided for each AM

Return:

List of dictionaries of name/value pairs for each returned AM

*/

function get_aggregates(options)