GEC24 Developer Roundtable

This is an informal session for GENI developers to discuss topics of interest to the attendees. These topics may include details of software integration, software issues that affect multiple control frameworks, or common tools.

Schedule

There will be three developer roundtable sessions at GEC 24:

• Tuesday March 8 1:45 p.m. - 3:15 p.m.
• Tuesday March 8 3:30 p.m. - 5:00 p.m.
• Wednesday March 9 11:00 a.m. - 12:30 p.m.

Session Leaders

• Nick Bastin, Barnstormer Softworks
• Marshall Brinn, GENI Project Office
• Tom Mitchell, GENI Project Office
• Rob Ricci, University of Utah

Agenda / Details

This software development session provides an opportunity for GENI developers to collaborate informally.

Topics covered on Tuesday:

• GENI Developer collaboration
  • Is the ​geni-developers google group working?
  • Do we need a public ​GitHub repository to track proposals and discussions?
• ​geni-lib development
  • An opportunity for those using or developing geni-lib to coordinate
• Extensions to APIs
  • The ProtoGENI/CloudLab folks have added a number of ‘private’ extensions to the GENI APIs including the ability to dynamically resize slices. This is an opportunity for both users and developers to learn more about these extensions
• Experiment management
  • The ProtoGENI/CloudLab folks have a ​Chef-based system for automating experiments in GENI-like environments. This is an opportunity for both users and developers to learn more and to discuss other experiment management solutions.
  • ProtoGENI/CloudLab has also added features to get shared secrets onto nodes for better passwords (where they are necessary) and to set up passwordless ssh between nodes in a slice. They are interested in getting feedback and seeing who else might be interested in using them
  • The issue of getting some kind of keys onto nodes in a slice so that they can call GENI API calls (eg. to expand the slice) has come up many times over the years, and the ProtoGENI/CloudLab folks would like to revisit this discussion
• OpenFlow support
  • Discussions related to feedback received from the "OpenFlow Support for Experimenters in GENI" session on Tuesday
• Open issues from ​geni-developers
  • ​Proposal for updated site status/maintenance information via AM API (Nick Bastin, June 25, 2015)
  • ​Add control IP address in the manifest (Niky Riga, July 7, 2015)
  • ​Revealing trust root information in getversion (Nick Bastin, December 10, 2015)

Topics for Wednesday:

• Monitoring API
  • Follow up discussions from the Monitoring Support for Experimenters and Developers session, as needed, about the monitoring API.
• Increasing likelihood of experimenter resource reservations succeeding
  • major causes for reservations failing and what it will take to address them.
• Other ’third party’ services
  • The ProtoGENI/CloudLab folks are running a service to track disk images and want feedback on whether other developers would want to use it.
• Extending Jacks
  • How can Jacks be extended to handle new types of resources?

Summary

• GENI Developer collaboration

The sense of the room was that the geni-developers google group was working for collaboration and discussion. While some issues had been left unresolved, those who raised the topics felt the discussion was valuable and had chosen not to pursue resolution.

There was a desire to have a listing of GENI-related code repositories. It was pointed out that this came up at the GEC 23 Developer's Roundtable and that a wiki page was created to list code repositories.

• Extensions to APIs

The CloudLab team discussed a new capability to dynamically grow and shrink slices. This is a capability for slices in CloudLab that can have nodes added during light usage and nodes taken away during heavy usage. This capability is different from the slice update API call and different from the update capability demonstrated by the ExoGENI team via Flukes.

• Experiment management with Chef

Dmitry Duplyakin, of the CloudLab team gave an overview of his use of Chef to configure experiments. Chef provides a function similar to Ansible which is used in some GENI tutorials. The sense of the room was that sharing knowledge about multiple tools was useful and that there is no need to standardize on a single tool within the GENI community. Dmitry also pointed out the ​Emulab GitHub area which includes a ​Chef repository containing cookbooks he has written.

• Experiment management

Rob Ricci and Nick Bastin led a discussion about possible modifications to request RSpecs to allow passing shared secrets, like API keys and similar tokens, through the aggregate manager to be available within a slice via geni-get. The ExoGENI folks discussed how the NDL translator might handle the additional information. There was also a discussion about appropriate XML namespaces for this information. The CloudLab team will do some prototyping and coordinate with Nick and the ExoGENI team.

• geni-lib development

The CloudLab team is actively working with and contributing to geni-lib. There was a discussion of desirable features and what should go into a geni-lib 1.0 release. For those wishing to follow along, geni-lib development happens on ​BitBucket.

• OpenFlow support

There were no topics discussed related to the OpenFlow Support for Experimenters in GENI session.

• Open issues from geni-developers
  
  The Tuesday roundtables wrapped up with discussion about three outstanding threads on the geni-developers google group.
  
  • ​Proposal for updated site status/maintenance information via AM API - Rob Ricci would still like to see this happen in InstaGENI. He thinks it would be useful to broadcast expected outages and anticipated heavy load so that tools and users could see the information.
  • ​Add control IP address in the manifest - Rob Ricci said this is something they should be able to add to the manifest RSpecs without too much trouble.
  • ​Revealing trust root information in getversion - There was agreement that this was useful, but no concrete plans were developed to put it in place.

• Increasing likelihood of experimenter resource reservations succeeding

Vic Thomas led a discussion about common causes of failure when trying to reserve resources. Ilya Baldin presented information for the recent ExoGENI survey of experimenters. Of those who responded most were able to create their desired topology. Ilya enumerated various causes of failure that they have seen over the last 18 months as well.

There was discussion about the internal limits on InstaGENI racks and whether the maximum number of VMs should be adjusted downward and by how much. Rob Ricci and Hussam Nasir discussed issues with full disks on the InstaGENI racks, including preventative measures they could put in place to avoid certain failure modes.

• Monitoring API

There were no additional questions about the monitoring API that was presented at the Monitoring Support for Experimenters and Developers session

• CloudLab image service

Rob Ricci gave an overview of a VM image service that the CloudLab team is building. Victor Orlikowski is considering a rewrite of the ExoGENI image service. They discussed common features, how much overlap there was between the two services, and how they had to differ to accomodate the differences in the two control frameworks.

• Extending Jacks

There was a brief discussion about extending Jacks to work with new types of resources. It sounds like some types of resources might be relatively straightforward to add, but others that use a fairly different RSpec might be difficult to add inside Jacks proper and might be better added as a post-processing step.

• RSpec to geni-lib translation

There was a discussion about translating request RSpecs to geni-lib scripts via code generation. This would allow experimenters to design their own topology and then see how to build that topology using geni-lib. They could then use geni-lib to create new topologies based on the original. The concern was that the generated code for more complex topologies might not provide a good programming language to experimenters.