wiki:GeniRspec

Version 11 (modified by Aaron Helsinger, 12 years ago) (diff)

--

Resource Specification (RSpec)

In order to truly allow interoperability among multiple control frameworks and aggregates, GENI requires a common language for describing resources, resource requests, and reservations - a single, well defined RSpec schema. GENI has standardized now on XML documents following agreed schemas to represent resources, with aggregate or resource specific extensions supported. Ongoing work covers agreeing upon ontologies for other resource types.

This topic was the focus of one of the software engineering sessions at GEC10, and the agenda wiki page has detailed minutes from the meeting: GEC10 Meeting. A summary of that meeting is below, but this session agreed to use XML RSpecs starting with the ProtoGENI v2 schemas to represent resources in GENI. We agreed to work on common ontologies.

This topic was also the subject of a session at GEC11, and detailed minutes from that meeting are on the wiki. This session described progress on ontologies and AM API revisions to specify RSpec schemas.

Action items from GEC11:

  • Evolve ontologies as needed
  • Develop storage and wireless ontologies
  • Further rollout of support for GENI RSpec schema
  • Adopt AM API revisions
  • Move GENI RSpec schemas to the geni.net namespace

At GEC12 we heard an update on progress (slides here

  • GENI RSpec schemas are now in the geni.net namespace, using version 3 currently
  • Support for GENI schemas continues to roll out across the community, including further support at Orca (for requests and manifests) and OpenFlow (with a new Aggregate Manager using new extensions to be rolled out at the end of November 2011)
  • Ontology work continues

GEC10 RSpecs Engineering Meeting

At GEC10 there was a community discussion of how to talk in a common language about resources. For example, without a common way to describe what resources an aggregate provides, it is much harder for experimenters to reason about which resources to use.

Ilia Baldine of RENCI and the Orca control framework proposed a 'workable path forward'. Martin Swany, Jeroen van der Ham, Rob Ricci, Rob Sherwood, Max Ott, Jeff Chase, Andy Bavier, Aaron Falk and others all contributed to a lively debate. Thanks to everyone who has been participating. In the end we agreed to negotiate a common understanding of key concepts, and then use translators to support a single format on the wire as a path forward for GENI.

Detailed results of the meeting are on the meeting wiki page: GEC10 Meeting

Community Agreement

The session concluded with community agreement on three points:

  • The community will work to agree on common semantics
    • Ilia has proposals for layer 2 network resources and compute resources, and has recruited community members for other areas; thanks in advance to Max Ott, Hongwei Zhang, Martin Swany, Mike Zink, and Rob Sherwood. These individuals, and others who volunteer to help, will produce ontologies to represent the agreements.
  • GENI will use ProtoGENI RSpec V2 as the format on the wire
    • New concepts will be represented as PG RSpec extensions, allowing free addition and evolution of new concepts as needed.
  • Aggregates can use Translators to convert between formats
    • Ilia is building one between PG and Orca formats.

Next Steps

For all of these, community involvement is key. Email dev@geni.net or Aaron Helsinger (ahelsing at geni.net) to get involved. Results will be published to dev@geni.net and to the GENI wiki to keep everyone informed of progress.

Agree on Concepts

  • Align edge compute resource concepts, starting with Ilia's proposal.
  • 4 key semantic extensions
    • Wireless (Max Ott, Hongwei Zhang)
    • Measurement (Martin Swany)
    • Storage - physical & cloud (Mike Zink)
    • OpenFlow flowspace (Rob Sherwood)
  • Produce ontologies for possible ratification at GEC11
    • The same groups will produce and present these agreements

Adopt Schemas

  • Once the semantics of these things are agreed to, Jon Duerig, Ilia and the GPO will assist in representing these as PG extensions
  • Revisit PG schemas
    • Change more attributes to be elements as needed
    • Make some types less opaque as needed
    • Ensure base is extensible for the known needed extensions
    • Jon Duerig of ProtoGENI will handle this
  • Once adopted, RSpec schemas will be maintained by the community, along with the AM API.
  • Modify the AM API specification to require that RSpecs are in the PG RSpec V2 format
    • Aaron Helsinger and the GPO will handle this, when the support infrastructure is in place

Implement Translators

  • Write and deploy translators as necessary (coordinate with Ilia for a start - ibaldin at renci.org)
  • Enable conversion as a GENI-wide service
    • Replicating Ilia's converter as necessary
  • Test full PG/Orca interoperability by GEC11
  • Build a thin PL/PG translation layer at PL (Andy Bavier, Tony Mack, Jon Duerig)
  • Other translators as necessary

GEC11 RSpecs Session Summary

Meeting Summary

Aaron Helsinger described progress on deploying GENI RSpecs. The schema is published, and support has been added to ProtoGENI and SFA (PlanetLab). That support is rolling out to many sites, and the Omni and Flack tools support it. Support at Orca and OpenFlow is in process. AM API revisions to codify support for these RSpecs are available in draft form.

Ilia Baldine described the compute ontology. His next step is to encode it in NDL and merge that into his RSpec-to-NDL converter.

Comments included

  • PG makes the hardware type and the available images orthogonal. May need to define subclouds. Needs investigation.
  • Unknown shortcuts requested at an AM Should fail or be ignored.
  • Shortcuts could be organized by features.

Max Ott argued for several improvements:

  • use adaptations to make properties of the underlying fabric be constrains on the connecting AMs
  • model component lifecycle
  • keep the object model separate from its meaning
  • group resources
  • measurements must be in terms of same resources (coordinate with the I&M group)

Hongwei Zhang described LENS, a representation of wireless sensor networks that explicitly represents all properties, including observables.

Mike Zink described the problem of representing storage resources - both in the cloud and attached to compute resources. He plans to start by representing cloud resources (as in Amazon S3/EBS), building a simple model and an RSpec extension.

Comments indicated the importance of representing security, how the storage is accessed (iSCSI?), and including basic properties like capacity in the UML.

Proposed Next Steps

  • Evolve ontologies as needed
  • Develop storage and wireless ontologies
  • Further rollout of support for GENI RSpec schema
  • Adopt AM API revisions
  • Move GENI RSpec schemas to the geni.net namespace