GEC21 Developer Roundtable

This is an informal session for GENI developers to discuss details of software integration, and software issues that affect multiple control frameworks or tools. Note that this session is separate from the parallel experimenter and educator drop-in session.


Thursday, 9.00am - 10.30am & 11.00am - 12.30pm

Session Leaders

Tom Mitchell
Aaron Helsinger

Agenda / Details

This software development session provides an opportunity for GENI developers to collaborate informally. Topics are TBD based on topics raised by the GENI developers in attendance. Possible topics include Aggregate Authorization policy, Speaks For, Embedding Jacks, Openflow Stitching, Cross Slice Stitching, AM API implementation status, and tool use of the Common Federation API.

At 10:00 we will have a scheduled discussion topic: "Moving towards a mesoscale replacement". See Tim Upthegrove's post to for details.

Session Summary

At the Developer's Roundtable, around 40 people had a lively discussion on various issues of concern; we left with numerous action items, and kept discussing some issues over lunch. Our conversation covered Speaks For authorization, replacing the mesoscale, jFed, aggregate authorization policies, and cross-slice stitching.

Some highlights included:

  • The Speaks For signer tool supports optional authentication for tools that need it.
  • Max Ott described his slice service which exposes a REST API version of the AM API.
  • A large group including the operations community discussed replacing the mesoscale by enabling multipoint in aggregates, while we discuss how to do multipoint in the network between aggregates. The ExoGENI and InstaGENI teams agreed to pursue this.
  • Rob Ricci agreed to experiment with an aggregate authorization service implemented by Marshall Brinn to support cross-testbed federation policies.
  • We had an active discussion of how to enable cross-slice stitching, supporting a service in a slice or connecting two slices. This will involve specific requests and actions by the slices, with some details to be determined.

Action items

Speaks For

  • Jon Duerig: Publish description of new authentication mechanism in speaks for signing tool.
  • Jon Duerig: Augment signing tool to more readily support changing users on shared workstations.
  • Community: Design how speaks for credentials can be delegated, and accepted by services.

Replacing Mesoscale

  • Community: Define multipoint, what kind of multipoint in the network we want, and how we might accomplish this in AL2S or elsewhere.
  • Rob Ricci: Support a single stitched link connecting to multiple VMs within the aggregate.
  • Rob Ricci: Support the GENI OpenFlow extension for requesting OpenFlow control on a link.
  • Victor Orlikowski: Allow multiple GENI stitched links per slice per aggregate, whether OpenFlow controlled or not.
  • Victor Orlikowski: Explore allowing specifying on a per link basis whether a link is OpenFlow controlled
  • Xi Yang: Support Stitching Schema v2 at the SCS

jFed & AM API v3

  • Brecht Vermeulen / Rob Ricci: Resolve AM API v3 issues at ProtoGENI.

Aggregate Authorization

  • GPO: Run the aggregate authorization XMLRPC service for ProtoGENI to experiment.
  • Rob Ricci: Call the XMLRPC authorization service to experiment, report results.
  • Jim Griffoen: Provide aggregate authorization use cases to

Cross-Slice Stitching

  • Rob Ricci, Victor Orlikowski: Review cross-slice stitching design and write up details of how it could be supported in InstaGENI and ExoGENI. Circulate design on for community discussion and further design.

Update & Misc Issues

  • Hussam Nasir: Test using Allocate to add a node to an existing slice at InstaGENI.
  • Paul Ruth: Explore expanding node groups from RSpecs and single node node groups.
  • Rob Ricci: Ensure DeleteSliver returns without blocking.
  • Aaron Helsinger: Plan future roundtable agendas in a public forum, providing sufficient discussion time.

Session details

Attendees included representatives of InstaGENI, ExoGENI, Internet2, jFed, GENI Desktop, LabWiki, NSF Cloud projects, regionals, and experimenters.

Speaks For

  • Rob Ricci and Jon Duerig introduced a new mechanism in the speaks for signing tool allowing tools to receive an authentication token, for those tools that don't have their own authentication or use OpenID.
  • Hussam Nasir described his experience converting GENI Desktop to using Speaks For. Common APIs and existing hosted tools helped. Hussam would like the signer tool to more readily support multiple users sharing a browser. He also noted that more end user documentation and education is required, particularly to deal with pop-ups.
  • Max Ott described the 'slice service', which exposes a REST API authorized using Speaks For for calling the AM API and Federation API. He has an object model using users, slice membership, slices, slivers and resources, and uses a state model based on jFed. LabWiki and the 'slice service' are in fact two separate tools; they should get separate chained speaks for credentials to work correctly. As a group, we need to design how the user authorizes tool A that authorizes tool B, and then get aggregates to authorize based on that chain.

Replacing the Mesoscale


Tim Upthegrove introduced the problem: the mesoscale is going away, and we still want to support complex topologies that may be OpenFlow controlled. For now, we would like to support something soon, while allowing for future expansion to more complex capabilities. Specifically,

  1. Enable multipoint at the aggregates; create stitched links to the aggregate, allowing multipoint topologies within the aggregate.
  2. Allow OpenFlow control over stitched links plus any connected LAN within the aggregate.
  3. Allow future multipoint topologies in the network between aggregates.

Multipoint in the Core

We then had a side discussion about what more complex topologies we could create, what experimenters might want, what is supported by AL2S, what multipoint actually means, what is safe, and what we want.

We considered whether experimenters might want or we might want to allow multipoint involving multiple point to point connections, and agreed that we didn't want to allow this without extra effort.

We agreed that this is a critical discussion that deserves more time; therefore, we should aim to complete the easy steps now while we separately pursue this conversation.

Multipoint in an Aggregate

For supporting stitching to a multipoint network in an aggregate that might be OpenFlow controlled:

  • Xi Yang explained that the SCS today can group interfaces on a link into their aggregates, and determine that the link is point to point between aggregates.
  • We agreed that the representation therefore is accomplished with existing RSpecs - a link with multiple interfaces, and the existing stitching extension.
  • Rob Ricci agreed to try to implement this; it requires some changes to the 'mapper'.
  • Victor Orlikowski agreed to to remove the restriction to a single stitched link, but otherwise this may work already.

OpenFlow Control

For OpenFlow controlling these links:

  • Victor and Paul Ruth say that currently all links at an aggregate in a slice need to be OpenFlow controlled, or not; the switch is by slice not by link.
  • Rob agreed to change to using the standard GENI extension for marking a link as OpenFlow.
  • Nick noted that there are limits in the switches on how many VLAN tags can be OpenFlow controlled - we may need to explore how to expose this.
  • Jim Chen noted that calling a link non-OpenFlow controlled really just means that the experimenter is not running the controller.


Brecht Vermeulen described the latest on jFed (slides attached). jFed is a Java tool based on a library that supports AM API calls (v2 and v3), Federation API calls, connectivity testing, generating RSpecs, and service debugging. jFed was used for tutorials at this GEC, is used for monitoring status of aggregates as seen at Brecht then conveyed their experiences using AM API v3. They found a number of issues or surprises at the ProtoGENI implementation. Among other things, they found that the RSpec parser is more strict, they get some certificate errors intermittently, and found some returns unreliable. jFed works around these issues with a number of of special flags, which they track using an XML file of properties.

Aggregate Authorization

Brecht motivated a need for aggregate local authorization policies that support international federation, quotas, and scheduling. For example, classes want resources to definitely be there, but we want to avoid students reserving all resources long term. Aaron Helsinger described an ABAC-based XMLRPC service that allows federation and/or local aggregate policy covering whitelists and blacklists, quotas, scheduling, rules based on different clearinghouses and resource types. This service was built by Marshall Brinn, and is supported by the GRAM software that underlies OpenGENI. Rob Ricci agreed to try this in parallel within the XMLRPC service at ProtoGENI to see how it works.

Cross-Slice Stitching

Paul Ruth's plenary showed a need for combining two slices. To run a service in a slice, like Choice Net or other FIA architecture, or VTS, requires connecting multiple slices. Today, that requires using shared VLANs. Is there a better way?

Nick Bastin argued that GENI should provide an incremental approach, that may be unique to aggregate type. One slice tells the aggregate somehow that it is open to connections, and the 'client' slice specifies the slice it wants to connect to by URN.

Rob Ricci suggested that the 'service' slice specifies a node where it wants to receive connections. The AM probably requires a Perform Operational Action command for specifying this connection point and the desire to accept connections. Rob further suggested that this should then modify the Aggregate's advertisement RSpec to now include this 'service node' as a node that other slices can request a connection to. This would require an RSpec extension to specify fully the type of the node and the owning slice. Rob explained that in InstaGENI, this would be accomplished with a trunked interface on the node - so would need to be requested at reservation time.

Paul Ruth noted that in ExoGENI, they can add interfaces dynamically, and so could add these at runtime, one per client connection. In ExoGENI, they would use 'stitch ports', as they do for other things. So in ExoGENI: nothing is strictly required at creation time, a POA opens the slice for connections and adds the service slice node to the Ad. When a client wants a connection, they stitch to the aggregate, and the aggregate adds an interface to the service node, allowing the service to distinguish traffic by interface. In InstaGENI, the initial reservation request by the service slice must specify that a service slice is desired, to get a trunked port. The POA requests adding the service node to the Ad RSpec. When a client requests a connection with a link to the service slice node, the aggregate adds a new VLAN tag of traffic on that interface.

We then discussed authorization. Because the node you are sharing may be iSCSI device for example, we want the aggregate to provide some level of authorization. We agreed that this would just be an additional credential in the createsliver request by the client slice; presumably this credential would be signed by the owner of the service slice, binding the service and client slices together in some way - or something similar. However, this authorization step is not required for an initial implementation.

Update AM API call

We then briefly discussed updating an existing slice. Specifically, GENI Desktop would like to be able to add a node - even one without a link. Jon Duerig noted that at InstaGENI, Allocate permits this now. Paul Ruth noted that you can increase the size of an ExoGENI node group, but not yet from RSpecs and the GENI AM API.

Finally, over lunch we had an information discussion on the conference. We agreed that the Roundtable was a success, but rushed. We also agreed to try to plan the agenda for the roundtable in a more public way in future, and to try to extract clear action items and agreements. We briefly discussed a desire for InstaGENI to speed up booting of Xen VMs, though it may not be possible, and to ensure DeleteSliver returns without blocking.

Last modified 9 years ago Last modified on 10/27/14 16:41:53

Attachments (1)

Download all attachments as: .zip