Changes between Version 13 and Version 14 of GEC22Agenda/DeveloperRoundtable

Timestamp:

04/02/15 13:55:31 (9 years ago)

Author:

Aaron Helsinger

Comment:

--

GEC22Agenda/DeveloperRoundtable

@@ -56 +56 @@
  - ?
 
-=== Listresources without authorization ===
+=== ListResources without authorization ===
 Currently the AM API !ListResources call requires both a valid certificate (authentication) and a user credential (authorization). To the extend to which Advertisements are not user specific, the credential is arguably not required. If we made the credential optional, then tools (with a valid and trusted GENI certificate) could pre-fetch Advertisements for use by multiple users.
 
@@ -63 +63 @@
 
 == Session Summary ==
+At the Developer’s Roundtable there was excellent representation and participation from developers associated with many of the core GENI projects. The first and largest discussion topic was cross slice stitching. Several options were discussed and approaches were narrowed. Both ExoGENI and InstaGENI developers have paths they will investigate to expose this capability to experimenters in the near future.
+
+Other discussion topics included cross testbed federation, regression testing, resource queries, public resource advertisements, and advertising available bandwidth.
+
+=== Detailed Summary ===
+
+Attendees included developers representing among others ProtoGENI/InsaGENI, ExoGENI, GENI Desktop, GIMI, jFed, FOAM/VTS, GPO, and the SCS (from the University of Utah, RENCI, University of Kentucky, University of Massachusetts-Amherst, iMinds, Barnstormer Softworks, BBN Technologies, and University of Maryland / MAX).
+
+==== Stitch to LAN ====
+A past topic of discussion has been the ability to stitch together LANs at 2 different aggregates into a single broadcast LAN, allowing more than one node at an aggregate to communicate out a single stitched VLAN. This capability works today in ExoGENI when reserving through the ExoSM or ExogENI APIs. '''Ilya Baldin''' of RENCI/ExoGENI says that this ability will be exposed through the GENI AM API soon.
+
+'''Rob Ricci''' of the University of Utah / InstaGENI sees no reason why this cannot be done at their aggregates, and will plan to make this enhancement.
+
+A related topic is the ability to stitch to an existing shared VLAN from outside the aggregate. However, this requires that the same VLAN tag number be available on the extrernal interface, so is unlikely to work at InstaGENI.
+
+Note however that you can attach nodes to an existing shared LAN.
+
+==== Cross Slice Stitching ====
+
+We next discussed cross slice stitching; connecting 2 slices at layer 2. As background, note that we have the notion of a 'shared VLAN'. This is a VLAN that gets a name and that is marked public, allowing anyone to connect to it. At InstaGENI, there is a !PerformOperationalAction to convert a newly allocated LAN into a shared VLAN.
+
+'''Brecht Vermeulen''' and '''Paul Ruth''' noted that currently you can allocate a VM to act as a bridge between a stitched link and a local LAN.
+
+'''Niky Riga''' notes that currently when you share a LAN, the bandwidth on the original LAN is not enforce on other slices that connect to the LAN. '''Rob Ricci''' replied that this is difficult to fix correctly, but they can set the bandwidth limit from the original LAN on the new connection. '''Nick Bastin''' noted that AL2S is best effort, so bandwidth settings are fairly meaningless. '''Ilya Baldin''' noted that at ExoGENI, you won't get more bandwidth than you request, but AL2S might give you less than you requested.
+
+We discussed support for shared VLANs at ExoGENI. Currently, ExoGENI does not offer this functionality. However, '''Ilya Baldin''' noted that ExoGENI is working on the sliver-modify function. ExoGENI will expose this as `PerformOperationalAction` (without implementing the rest of AM API version 3). With support for POA, ExoGENI can expose the ability to reboot a node (`geni_restart`), modify the bandwidth on a link, add an interface to a node, add a post boot script, etc. Additionally, this could be how ExoGENI adds support for sharing VLANs. 
+
+'''Rob Ricci''' and '''Nick Bastin''' of Barnstormer Networks agreed on a plan to support at InstaGENI/ProtoGENI a 'restricted shared VLAN'. In this model, a "server slice" would make a call on the local aggregate to tell the aggregate that it is willing to accept "client slice" connections on a local LAN that belongs to the server slice. This is in fact just the existing !PerformOperationalAction to make a LAN into a 'shared VLAN'. The change proposed here is that the POA would add an additional option through which the client could optionally ask for a 'restricted' shared VLAN, and then the client would be required to supply a credential in !CreateSliver requests in order to make a connection to the server.
+
+'''Ilya Baldin''' noted that connecting 2 slices is an operation at a single aggregate, where 1 slice asks to connect to a 'server' slice, which has to authorize the connection in advance. Mechanically, the connection could be node-to-LAN or LAN-to-LAN. We noted that this can be boiled down to the client/server model Rob and Nick proposed.
+
+'''Proposal''': Add to the existing POA `geni_sharelan` a new option `restricted` with default value `false` (old behavior). When true, the created shared VLAN requires a new credential when requesting a connection to this new Shared VLAN.
+The POA method will return in this case a GENI SFA credential with owner <user calling the method> and target <sliver  of the shared VLAN, or the shared VLAN in some way; contents are not specified but should be sufficient for the aggregate to authorize the call>.
+
+Note that shared vlan names are scoped within the AM and must be unique within the AM. 
+
+The server slice aggregate manager (the AM at which the shared VLAN was created) should include the shared vlan (whether restricted or not) in the advertisement RSpec for the aggregate, indicating if this LAN is shared or not. To do so, we need a new attribute on the `shared-vlan` RSpec extension (or a new extension).
+
+'''Proposal''': add a new optional attribute to the existing `shared-vlan` extension `restricted` with type `xml:boolean` and default value `false`.
+
+Slices desiring to connect to this restricted shared VLAN negotiate with the service slice. The service slice must delegate the shared VLAN credential to the client slice user. Then the client slice user must include this extra credential in the call to `createsliver` or `allocate`. The aggregate therefore receives 5 key pieces of information: 
+ 1. the client slice (URN is explicit argument to `createsliver`), 
+ 2. the place in the client slice where the connection should be made (in the request RSpec), 
+ 3. the server slice (may be in the additional credential, or else should be implicit given the owner of the shared VLAN), 
+ 4. the sliver ID of the shared VLAN (either explicitly in the credential or implicitly in the name of the shared VLAN in the request RSpec), plus 
+ 5. explicit authorization by the owner of the server slice that this client slice can connect to this restricted shared VLAN. 
+
+The aggregate can then create a LAN for the client slice that connects to the specified shared VLAN, allowing traffic to flow freely between the two slices.
+
+Note that there could also be an additional `PerformOperationalAction` command to modify an existing 'client' slice to connect a LAN belonging to that slice to one of these 'restricted shared VLANs'. We did not specify the syntax for such an operation, but Ilya suggested ExoGENI might support such an operation.
+
+'''Ezra Kissel''' of Indiana University and '''Niky Riga''' of the GENI Project Office requested that there be a way to request a shared VLAN as part of the initial request RSpec. The goal is to make it easier to get a shared VLAN, but also to avoid getting 2 VMs on the same host so that the link is trivial and not sharable.
+
+'''Rob Ricci''' replied that while this is possible, this is a larger change and therefore not something he is willing to commit to do at this time.
+
+However, '''Brecht Vermeulen''' of iMinds / jFed / Fed4Fire noted that there is an InstaGENI RSpec extension that allows you to request that a link be VLAN tagged, making it a non trivial link, spreading the VMs across 2 physical hosts, forcing the link to go via the hardware switch (and therefore it can be OpenFlow controlled), but also making the link sharable.
+
+To use this, include
+{{{
+<vlan_tagging xmlns="http://www.protogeni.net/resources/rspec/ext/emulab/1" enabled="true"/>
+}}}
+'''Actions''':
+ * Experiment Support will document this option in a wiki page.
+ * Jacks should add support for this option.
+
+'''Marshall Brinn''' asked how a 'server' slice could identify and distinguish among the traffic from different 'client' slices, when all traffic arrives on the same shared LAN?
+
+'''Rob Ricci''' proposed that the aggregate manager could sign the manifest RSpec of the 'client' slice, and this could be passed to the 'server' slice. Rob proposed that the hand-off of this signed manifest would be done directly between the two slices, not mediated by the aggregate. Additionally, Rob proposed that ProtoGENI / InstaGENI would begin signing all manifest RSpecs as returned by all AM API calls. (Note that this change will need to be tested to ensure compatibility with existing tools. The XML-DSIG signature is a new child element under `rspec`, so the resulting signed XML passes `rspeclint` and can be loaded in Jacks.)
+
+Since this discussion, ProtoGENI has begun implementing this feature, and a formal proposal has been posted to the [http://groups.geni.net/geni/wiki/GAPI_AM_API_DRAFT GENI AM API Draft changes wiki].
+
+==== Cross Federation Policy and Integration ====
+
+There are increasing movements to formalize the relationship among the various federations and testbeds. Specifically, this includes providing mechanisms for testbeds and aggregates to apply both federation level policies (differing by which federation a user comes from) and local policy.
+
+'''Rob Ricci''': Eventually !CloudLab will need federation level policies - using which MA a user comes from. For now, this will be a case-by-case thing.[[br]]
+'''Ilya Baldin''': Jeff Chase of Duke is working on more authorization policies for Orca and ExoGENI.[[br]]
+'''Rob''': GENI funding is dwindling. GENI should stick with fixing problems we have.[[br]]
+'''Jim Griffoen''': Local AMs should be allowed to have local policies. As the number of users grow, this may become an issue.
+
+'''Sarah Edwards''': There is no monitoring API to know where an experimenter has reservations.[[br]]
+'''Heidi Dempsey''': Monitoring does this. The University of Kentucky team has added some needed pieces. We'll be trying to get that out.
+
+'''Decision''': No action.
+
+==== !ListResources without a User Credential ====
+
+Tools would like to tell users what resources are available where. To do so currently, requires that the tool use the user's certificate to get a user credential, with which to query the aggregates. Tools cannot get their own credential, and aggregates requires a credential (per the AM API).
+
+'''Rob Ricci''': We will make the user credential optional. We may even try supporting `ListResources` without a certificate.[[br]]
+'''Ilya Baldin''': We can make the user credential optional.[[br]]
+'''Jim Griffoen''': What about a service to store all such information?[[br]]
+'''Ilya''': We built a service like that based on XMPP: you can subscribe to slices or AMs.
+
+Since then, this change to the AM API has been documented on the GENI
+Wiki:
+http://groups.geni.net/geni/wiki/GAPI_AM_API_DRAFT#ChangeSetAD:DonotrequireausercredentialinListResources
+
+'''Action''': Modify all aggregates such that `ListResources` does not require a user credential when called outside of a slice context.
+
+==== Advertising currently Available Bandwidth on Links ====
+
+One source of stitching failures is insufficient available bandwidth. If users and tools knew what bandwidth was currently available, they could tailor requests accordingly.
+
+'''Ilya Baldin''': Having this information in the Ad may be an issue with the RSpec converter. However, this information is relatively useless; it becomes stale almost immediately.[[br]]
+'''Niky Riga''': This would still be better than nothing.[[br]]
+'''Rob Ricci''': This is almost certainly something we can do.
+
+But the full answer is monitoring.
+
+'''Action''': ExoGENI and InstaGENI teams will make the actual current available bandwidth available in the Ad RSpec when the `geni_available` option is supplied to `ListResources`. This information should be updated in both the main body link and in the stitching extension.
+
+==== Shared VLAN Name in Manifest ====
+
+'''Thijs Walcarius''' of jFed/iMinds raised the issue that in the manifest for a slice whose LAN has been made shared using `PerformOperationalAction`, the shared LAN name is not present in the manifest. And this would be very useful. 
+
+'''Rob Ricci''' replied that they would discuss this internally at InstaGENI.
+
+==== Regression Testing at Aggregates ====
+
+'''Niky Riga''' introduced the topic: In the operations session, the rack teams indicated that often they rely on experimenter bug reports to identify problems. But experimenters aren't good testing.[[br]]
+'''Ilya Baldin''' noted that they could do good testing if they had more funding. [[br]]
+'''Niky''': What about doing regression testing for things that we've run into before? What about using jFed testing?[[br]]
+'''Ilya''': The # of scenarios to test is very large. We have a regression suite.[[br]]
+'''Sarah Edwards''': Start by testing the tutorial RSpecs.[[br]]
+'''Jim Griffoen''': Load is a problem, and is harder to test.[[br]]
+'''Tom Mitchell''': We test using Selenium Grid.[[br]]
+'''Heidi''': This isn't a GENI thing. And this isn't a developer specific problem. We should use jFed and the University of Kentucky. Tell us the use cases to test.[[br]]
+'''Jim G''': There is a verification function in GENI Desktop to check interfaces. There is no script to launch the verification, but could be.
+
+'''Thijs Walcarius''': In jFed, if an automatic test fails for a couple days in a row we want to notify users.
+
+'''Decision''': GPO Experiment Support support will gather use cases, starting with their standard tutorials, and work with the jFed team to have these regularly tested, notifying key players of failures.
+
+==== Query Aggregate Resources ====
+'''Aaron Helsinger''' introduced the topic. 
+
+You can ask an aggregate for all resources, and then parse the response. Or with APIv3, you can just try your reservation and see if it works. But it would be nice to be able to let the aggregate parse it's own resource listing to find for you the resource you want. '''Marshall Brinn''' of the GPO has prototyped doing this using RDF and shown that the AM API supports this.
+
+'''Marshall''' asks: Is there a demand?[[br]]
+'''Brecht''' noted: If you are dealing with simple resources, like nodes & links, then a query mechanisms is overkill. But if you start talking about different kinds of sensors, then parsing the Ad is a
+pain. European testbeds have more special devices. This is why we're trying to do this kind of research.
+
+The variety of resources available in GENI may increase with time.
+The group decided that GENI should follow what the Europeans do.
+
+==== Other Discussion Points ====
+'''Update''': ExoGENI could support this if an AM API interface was put on top of their existing slice_modify and sliver_modify functionality.
+
+'''Sarah''': no tool supports adding resources to an existing slice. That'd be nice.[[br]]
+'''Aaron''': Omni supports `Allocate` and `Update`, to the extent it is supported by aggregates.
+
+'''Hussam''': In GENI Desktop, users can add a little topology to an existing slice by doing allocate/provision/poa.
+
+'''Niky''': Can we update the !OpenFlow controller on a switch?[[br]]
+'''Ilya''': We could pass the information to Flowspace Firewall.[[br]]
+'''Heidi''': I doubt they support modifying the controller[[br]]
+'''Ilya''': It could work if you restart the slice; I don't know the size of the disruption.
+
+'''Ezra Kissel''' noted that it is still hard to have standard images cross rack types. InstaGENI image requirements are complicated. Without installing InstaGENI software in the image, you don't get InstaGENI functionality.