Version 8 (modified by Aaron Helsinger, 7 years ago) (diff)


Coding Sprint, Experimenter Tutoring and Operations Hands-On

The joint software coding sprint, experimenter tutoring session, and operations hands-on-with-GENI session is an informal session where aggregate and tool developers will discuss software details, experimenters will run though GENI tutorials or ask questions of the GPO for help running experiments, and GPO operations staff will help site ops/admins folks with anything they're interested in.


Thursday, 1pm - 5pm

Session Leaders

Developers: Sarah Edwards, Aaron Helsinger, GENI Project Office

Experimenters: Niky Riga, GENI Project Office

Operations: Josh Smift, GENI Project Office

Agenda / Details

This software development session provides an opportunity for GENI engineers to collaborate in real time on a particular software or documentation issue. The topic will be selected well in advance of the conference, based on need and key party availability. At GEC12 and GEC13, the topic was AM API changes. At GEC14, discussion covered stitching, AM API changes, shared VLANs, and trust issues in the GENI Portal. Likely topics this time include stitching, minor proposed changes to the AM API, and adding Update() to the AM API.

The GPO will also hold a tutoring session for GENI experimenters, where experimenters can come and work on various online GENI tutorials. The list of available tutorials will be advertised in advance of the GEC. The GPO will answer questions and provide assistance. Experimenters with an idea they want to work on are encouraged to come do so in the same room, so that they can have support from the on site GPO staff and other experimenters.

GPO operations staff will also be available to help GENI ops/admin folks who'd like a hand getting set up with GENI credentials, the Omni command-line tool, etc. This can be useful if you're running a GENI aggregate, so you can create a slice and some slivers, and test for yourself whether the aggregate manager is working like you expect. We'll also be happy to discuss any other topics of interest to the ops/admin community.

Developers Coding Sprint Topics

The actual topics will be dictated by the people in the room and their concerns, but we anticipate discussing topics first raised at the AM Topics session.

  1. Stitching
  2. AM API v3 tweaks
  3. Update()
  4. Other miscellaneous topics


At the Coding Sprint, Experimenter Tutoring, and Operations Hand-On session, several productive conversations took place.

A group of experimenters worked through some of the exercises from tutorials, several operators came for one-on-one discussions, monitoring developers had discussions, and various GENI community members came for free-form discussions and to ask questions.

Coding Sprint - Developer Summary

Meanwhile at the developer coding sprint portion of the session, we discussed stitching, AM API changes, and various concerns with the portal specifically and persistent tools that experimenters use for GENI control actions in general.

First, we reviewed the overall plan to complete layer 2 network stitching across aggregates by GEC16. Then we talked in slightly more detail about the topology service, computation service, workflow computation, and negotiation.

We discussed the Topology Service being built by Ahmed El-Hassany of Indiana University in UNIS. Ahmed explained that the UNIS topology service reads in GENI aggregate advertisement RSpecs and DCN network topologies, and then makes those available in UNIS topology format. The UNIS representation and format is NML compatible and derived from perfSonar. This service is a helper service that tools are not required to use. Various community members expressed concern that the topology information is not in RSpec format, requiring tools to read another format. Rob Ricci also expressed a desire for helper functions as part of the topology service, like determining nodes that are all accessible by layer 2. We agreed to discuss modifications to the topology service in follow-on conversations to enhance its utility.

The computation service would find paths between nodes, providing another optional helper service.. We agreed that MAX and the GPO would develop the compute service API. The service would take RSpecs and constraints, and produce expanded RSpecs and workflow rules and instructions for how to pass tags from one aggregate to another. This could be in the form of XPath instructions, RSpec extensions, or something else.

We then discussed negotiation, which will be necessary to find a successful path between aggregates eventually. Rob Ricci argued strongly that tree mode (client driven as we are doing) is the wrong way to do negotiation, because it puts too much complexity in the tool. Nick Bastin argued negotiation could be bad in chain mode as well. Xi Yang of MAX mentioned that Allocate() in the OSCARS network is expensive - all the networks must coordinate. Tom Lehman asked if you can call Update() (AM API v4 addition) on allocated slivers (you can). Tom mentioned that for a single OSCARS domain, Update() would not be too slow. We agreed that in the near term, we will not implement negotiation, though we should not forget about it.

This led to a discussion of AM API changes. First, we discussed the proposed Update() method. This conversation picked up from the discussion at the AM Topics session. Nick Bastin asked again for an option to allow an aggregate to have only 1 sliver state machine. We agreed to provide this, with details to be discussed via email.

Rob Ricci asked whether PerformOperationalAction() could just say results are undefined while the sliver is updating. The group consensus was that disallowing this was better.

How should slivers being deleted be marked? Jon Duerig proposed adding a new geni_deleting state for such slivers. The group consensus was to not add another state, and instead return another value: geni_next_allocation_status, indicating whether the sliver is being deleted.

The group then agreed to adopt the `Update()` proposal with these changes for inclusion in AM API version 4. We also agreed that aggregates are free to implement Update() as part of their API version 3 implementations.

We then discussed the smaller changes to the AM API previously posted to the wiki. The group agreed to adopt these as part of AM API version 4 and allow aggregates to implement these as part of their AM API v3 implementations.

The group then began discussing several concerns that members of the community have with the GENI Portal and its handling of keys and certificates.

Issue 1: Services should 'Speak For' a user, not 'Speak As' as user

Rob Ricci wanted the Portal to use a 'Speaks For' assertion, where the portal has a signed statement showing that the experimenter has authorized the portal to speak on their behalf. But Rob did not want the portal to 'Speak As' an experimenter, pretending to be the experimenter. Leigh Stoller proposed using a delegated slice credential. Tom Mitchell pointed out that aggregates would think the portal was reserving all resources. Rob wants ProtoGENI to try something. He wants to distinguish between 'speaks for' assertions and a delegated credential. And he wants some way to do this before there is a slice, so experimenters can use this tool to create slices. Perhaps a delegated user credential.

Rob suggested that ProtoGENI aggregates would learn to accept a delegated user credential meaning 'Speaks For'. Once that happens, he said that he'd accept the portal using a checkbox authorizing generation of the 'speaks for' credential.

Nick Bastin noted that jury-rigging 'speaks for' into SFA style credentials is the kind of ugliness that Jeff Chase warned us about as a sign that we need to use ABAC. However ABAC is more work for aggregates and so less likely to be available soon.

Someone suggested that the portal use an intermediate CA certificate (identifying itself as the portal) and use that to sign experimenter certificates that the portal uses when it 'Speaks As' experimenters. That would allow aggregates to exclude all portal 'Speaks As' actions if they chose, but also (using the Issuer DN) identify experimenter certificates that are issued by the portal.

Rob Ricci noted that that the clearinghouse must issue Certificate Revocation Lists.

Issue 2: Keep keys local to the user's machine

Nick Bastin and Rob Ricci expressed the desire for experimenters to be able to use the GENI Portal without having to turn over their private key.

Nick Bastin expressed the desire to allow experimenters to keep their private key always local to their machine. The problem Nick has is that the portal lives on a server, and the server could be compromised. Nick proposed some kind of browser plugin. The experimenter could then make a decision whether they wanted to be prompted to sign each time, or have the browser plugin sign automatically for them. This plugin could sign everything - credentials, S/MIME messages, even SSL connections. Rob noted that SSL connections might be hard in such a plugin. Nick suggested this plugin would allow a 'Channels for Me' credential where the portal is merely passing messages already signed by the experimenter, to go along with Rob's desired 'Speaks for Me'.

Aaron Helsinger noted that with issues 1 and 2, these are various solutions that trade off various desirable attributes:

  • Accountability (can servers identify who they are talking to)
  • Ease of use (do not require experimenters to handle keys and certificates and signing)
  • Security: Protect private keys, allowing experimenters to retain control, preserving the PKI security model
  • Time: Users are coming and we must support them. Changes will take time, including at aggregates. But if we wait, it will get harder.

Issue 3: Rob wants ProtoGENI users to be able to use the portal

Rob Ricci asked that ProtoGENI users be able to use the portal using their existing account, key and certificate. Theoretically, this should work for all ProtoGENI authorities, as well as !PlanetLAb users. Aaron Helsinger expressed concerns about how this would work.

Rob expressed the priority that ProtoGENI users be able to create slices and add resources using the Portal. Less important is the ability to use existing ProtoGENI slices within the Portal. In both cases, using existing ProtoGENI SSH keys is desirable.

The following steps were proposed:

  • Add a Portal Intermediate CA certificate for issuing 'Speaks As' experimenter certificates
  • Support CRLs in the Clearinghouse
  • Then begin designing a 'Speaks For' credential (possibly a delegated user credential)
  • When aggregates are capable of accepting this credential, the portal will stop using certificates claiming to be the experimenter, and instead will use the new 'Speaks For' credentials.

Meanwhile, to keep keys protected, we proposed that the community would investigate a browser plugin for signing statements, that would allow experimenters to keep their keys local and secure.

Separately, we will investigate how to support ProtoGENI users or users of other control frameworks using the Portal as a tool, with their existing certificates and keys.

Coding Sprint - Experimenter Summary

At the Experimenter portion of the coding sprint, experimenters had a chance to:

  • Go through some of the tutorials, with specific emphasis in the use of Mesoscale and OpenFlow capabilities in GENI
  • Discuss research ideas with various members of the GPO getting feedback on GENI operations
  • Talk with developers and resource providers about capabilities.

Background Reading

Coding Sprint