Coding Sprint and Experimenter Tutoring
The joint software coding sprint and experimenter tutoring session is an informal session where aggregate and tool developers will discuss software details, and experimenters will run though GENI tutorials or ask questions of the GPO for help running experimenters.
Schedule
Wednesday, 1:30 pm - 5:30pm
Session Leaders
Sarah Edwards, Aaron Helsinger, and Niky Riga, GENI Project Office
Agenda / Details
This software development session provides an opportunity for GENI engineers to collaborate in real time on a particular software or documentation issue. The topic will be selected well in advance of the conference, based on need and key party availability. At GEC12 and GEC13, the topic was AM API changes. Likely topics this time include requesting resources connected to a shared VLAN (like the mesoscale), stitching by GEC16 and specifically stitching racks by GEC15, improving the GENI experimenter experience, minor revisions to AM APIv3 based on questions while implementing the API, UpdateSliver(), and RSpec schema changes.
The GPO will also hold a tutoring session for GENI experimenters, where experimenters can come and work on various online GENI tutorials. The list of available tutorials will be advertised in advance of the GEC. The GPO will answer questions and provide assistance. Experimenters with an idea they want to work on are encouraged to come do so in the same room, so that they can have support from the on site GPO staff and other experimenters.
GPO operations staff will also be available to help GENI ops/admin folks who'd like a hand getting set up with GENI credentials, the Omni command-line tool, etc. This can be useful if you're running a GENI aggregate, so you can create a slice and some slivers, and test for yourself whether the aggregate manager is working like you expect.
Developers Coding Sprint Topics
The actual topics will be dictated by the people in the room and their concerns, but we anticipate discussing these topics in particular:
- Shared VLANs
How do experimenters request resources on a shared VLAN like the GENI Mesoscale? How do aggregates define shared VLANs in RSpecs? How can we make the experimenter experience consistent? This topic may be quick. For reference, see the existing RSpec extension: http://www.geni.net/resources/rspec/ext/shared-vlan/1/
- Rack stitching demo at GEC15
By GEC16 our goal is to have GENI network stitching operational. As a near term goal, we anticipate doing some demos at GEC15 of operational GENI racks, showing the racks stitching amongst racks of a single flavor (InstaGENI or ExoGENI). We will discuss technical hurdles to making this a reality.
- Experimenter Experience
In coming months, we hope to have many new GENI experimenters. What can aggregate, tool and clearinghouse developers do to make it easier to get started with GENI? Of special interest is improvements to already existing tools and services.
- Other miscellaneous topics
- Update Sliver in the AM API
- Questions or concerns with the GENI Portal and Clearinghouse
Session Summary
The Coding Sprint session attracted around a dozen experimenters, several operators, and around 20 developers. Experimenters got help getting started or working with GENI, operators discussed primarily monitoring, and developers worked through several issues.
The developers had a productive discussion, working through AM API status and proposed changes, stitching, shared VLANs, and questions and concerns about the new GENI Clearinghouse and portal. Thank you to everyone who participated.
- AM API v3 implementations are in process,
- We agreed to move forward with Update() as proposed on the dev mailing list, and
- We're hoping for an inter-rack stitching demo this Fall.
- We worked out a plan for naming and working with shared VLANs (like the mesoscale), and
- People provided several good critiques of the GENI Clearinghouse and Portal as implemented.
AM API
Aaron solicited updates from the group on the status of AM API v3 implementations: Planetlab is well underway, ProtoGENI will be doing their implementation soon, and Orca will likely implement v3 soon after GEC15. Meanwhile, Omni and the GCF AM will likely be updated in the next few months.
Action
- All developers should work to complete their AM API v3 implementations, and keep the community updated.
Update()
We then discussed Jon Duerig's proposal for an Update() method in the AM API (http://lists.geni.net/pipermail/dev/2012-July/000823.html). After a brief discussion, we agreed in principle to Jon's overall proposal. We still need to work through the details, but Update() may be added to the AM API in version 4. In detail: Ilia pointed out that in many instances slivers cannot be modified without losing state. This is why Jon's proposal calls for a state guarantee in the manifest RSpec. We may also want some kind of default guarantee by sliver type in the Ad RSpec. Ilia removed his objection to Update() taking a full RSpec, as long as translating the RSpec into NDL does not require a stateful translator.
Actions
- Jon Duerig will flesh out his Update() proposal for posting on the GENI wiki
- All developers should read and comment on the Update() proposal
Stitching
We then moved on to stitching. First, we talked about the possibility of a GENI stitching demo between rack varieties. We agreed that this seems possible: probably starting with a rack-to-rack demo within the GPO lab, but then perhaps a demo going among RENCI, Starlight, MAX, and ProtoGENI Utah. The timing of this demo depends very much on other tasks for the rack teams, and will likely be after GEC15. ExoGENI expects to be able to first consume but then also produce a VLAN tag for stitching. Currently InstaGENI only produces tags (it cannot consume). So a demo where InstaGENI produces a tag and ExoGENI consumes it might be a first step. Rob believes it won't be hard for InstaGENI to also be a consumer of VLAN tags. Ilia warned that there are several layers of work at ExoGENI to make this work, pushing the timing for such a demo into December.
Actions
- InstaGENI will implement accepting a specified VLAN tag on a link using the stitching extension
- ExoGENI will implement both accepting and producing a VLAN tag for a link terminating on a particular port on some non-AM local switch
- We will plan a rack to rack demo within the GPO lab for November/December.
Rob then expressed concern that the stitching workflow logic is too complex to implement in every tool. We agreed there should be a library or service to encapsulate this logic. This service should not actually submit requests to aggregates, but only determine the order of requests, and help pass tags from earlier manifests into later requests in the workflow. We also noted that negotiation makes this component complex. We decided that the stitching extension in the Ad RSpec should be explicit about whether the aggregate can be a producer of labels, a consumer, or both. Rob commented that he has a path computation service he could share. Rob also suggested that such a service should have a function allowing tools to pre-cache information about the sets of nodes that can be connected with different kinds of connections, to simplify graphical tools.
Actions
- Rob will supply his path computation service to Tom Lehman and Aaron
- Tom Lehman and Xi will propose a modification to the stitching extension to advertise at each aggregate whether a given Link can produce or consume labels, or both
- ISI/MAX and the GPO will continue plans for a workflow library and service
Shared VLANs
There is an RSpec extension now for specifying that requested nodes
should be connected to a pre-existing shared VLAN
(http://www.geni.net/resources/rspec/ext/shared-vlan/1/). We noted that
the AMs do not yet support the geni.net schema location, nor the Ad
RSpec extension version. We then discussed common naming of shared
VLANs. We agreed these should be URNs, of the form
urn:publicid:IDN+<authority>+vlan+<label not specific tag>. This global
name may resolve into different tags at different aggregates. For
example, the mesoscale is not always 1750. Note that not all such shared
VLANs go across aggregates - some might be used to connect an aggregate
to the local campus. We also noted that some shared VLANs are Openflow
controlled in parts, but not at every switch or aggregate (like the
mesoscale), and others are not Openflow controlled at all.
We discussed the utility of a shared registration and lookup service, for finding these shared vlan labels, and for translating from the label to the AM-local VLAN tag. The tag has a description, plus perhaps other metadata. Then it has some per-aggregate data (local tag, FOAM URL if applicable, etc. We agreed that for now, a global wiki to track shared VLANs would be sufficient.
We also agreed that aggregates should advertise within the shared VLAN extension some extra information for each VLAN: the local FOAM URL, if FOAM is required, and maybe other items. We need to add to the extension attributes for that information.
Rob then asked about what special authorization might be needed for access to shared VLANs. Jeff argued that this is not necessary - this is an aggregate policy question. Aggregates may have special local policies for requesting such resources, but we will not try to describe that policy in RSpecs. An aggregate might choose to mark up the advertisement RSpec to indicate that certain resources have special access policies, with perhaps a URL for more information.
We then asked about how we allocate labels within these shared VLANs (IP addresses, ether types, etc). Currently, this is coordinated via a wiki page. Do we need something more formal? Rob commented that this is similar to Planetlab's port number reservation problem. Sarah suggested that a simple aggregate manager for allocating strings like this would be sufficient. Jeff noted that this becomes particularly important when a slice wants to offer a service to other slices. We also noted that if we go down the SDN path (as discussed at the SDN session), then this is not a problem. And currently, we don't have a problem. But if we grow a lot without the proposed SDN approach, then we will need something more formal For now, we agreed to table this.
Actions
- ExoGENI and InstaGENI will modify their AMs to support the new schema location for the shared VLANs extension
- ExoGENI and InstaGENI will support the shared VLANs Ad extension
- Jon Duerig will add to the shared VLAN Ad extension information on the FOAM URL, and perhaps other attributes
- GPO will stand up a Shared VLANs wiki page (or similar)
- GPO will publish a URN for the mesoscale shared VLAN on that page
- ExoGENI and InstaGENI will add to that wiki other shared VLANs, and AM-local information on the mesoscale VLAN
Portal/Clearinghouse
Several people then asked questions about how authorization works in the new portal and clearinghouse, particularly with respect to proxy aggregates. No specific actions came out of this discussion, but the GPO agreed to work on the concerns from the community.
The clearinghouse generates experimenter keys, and stores the experimenter's private keys. The portal is a privileged client of the clearinghouse, and is authorized to get your private key and act on your behalf when talking to aggregates. (Experimenters may also download their certificate and key, and never use the portal). In addition, the proxy aggregates that the GPO is building are also privileged clients of the clearinghouse, and are authorized to retrieve your keys from the clearinghouse, and act as the experimenter when talking to aggregates. Jeff, Rob, and Prateek expressed concerns with this approach.
- In the name of protecting naive users, all users are required to give up control of their private keys. There should be a way for more advanced users to retain control of their private keys - both for normal use of the Clearinghouse and AM APIs, and for use with proxy aggregates.
- Only tools that run local to the experimenter and are explicitly authorized by the experimenter should be able to identify themselves as the experimenter in the AM API. The portal and proxy are not local. A proxy should identify itself as a proxy.
- Tools that are not local to the experimenter should have their own keys.
- Only privileged proxies are currently supported. There should be a way for an experimenter to authorize a proxy aggregate, and for that proxy to identify the 'real' experimenter when talking to the aggregate (see http://groups.geni.net/geni/wiki/GAPI_AM_API_DRAFT?version=47#ChangeSetJ:Proxyaggregatemanagersaresupported). This requires something like a 'Speaks For' assertion, signed by the experimenter.
- Currently, the portal tool only works with experimenters with accounts at the GENI Clearinghouse identity provider. Rob would like it to be available to ProtoGENI users directly.
Several solutions to these critiques were discussed, but nothing was resolved. Jeff and Prateek suggested an ABAC style 'Speaks For' assertion, that an experimenter could sign and grant to a tool, particularly to a proxy aggregate. Rob suggested developing some Javascript that could run in a browser, and could retrieve an experimenters password protected private key (either from a server or a local file), and sign statements on behalf of the experimenter. In particular, this would allow an experimenter to sign a 'Speaks For' assertion with some short expiration, without having to locally manage their private key, if the experimenter doesn't want to do so. Jeff suggested a button in the portal where the experimenter explicitly grants permission for the proxy and/or portal to 'Speaks For' them (again, with some short expiration time.
Action
- The GPO will consider ways to address these critiques.
