wiki:GEC10RSpec

Version 14 (modified by Aaron Helsinger, 8 years ago) (diff)

--

RSpecs

Session Leaders

Ilia Baldine, RENCI
Aaron Helsinger, GPO

Time

Wed 1:00 - 3:00 pm

Dial In

866-453-5550 Participant pin: 6513886#

Description

This meeting will seek agreement on a proposed single common RSpec format for GENI. In order to truly allow interoperability among multiple control frameworks and aggregates, GENI requires a common language for describing resources, resource requests, and reservations - a single, well defined RSpec schema. Currently, each control framework has its own custom schema, often evolved incrementally as needs have changed. Over the years, the community has met to discuss the various alternative approaches, and their relative merits. But now it is time to make a decision and move forward. In recent weeks that discussion has become more focused. Several key stakeholders have debated various alternatives. In this meeting, they will present a proposal for a way forward resulting from this discussion, the alternatives, and open the meeting for discussion.

Proposal

The proposal for the RSpec workshop led by Ilia Baldine of RENCI is that all GENI AM API calls will be in terms of ProtoGENI RSpec V2 format RSpecs, with aggregates free to use translators to continue to talk their native formats internally. Orca is already building such a converter.

Agenda

  • Introduction and Background - Aaron Helsinger (5 mins) slides
  • Proposed Solution - Ilia Baldine (20 mins) slides
  • NML work at OGF, network resource abstractions; Measurement resource abstractions - Martin Swany, University of Delaware (20 mins) slides
  • Ongoing NDL work - Jeroen van der Ham, University of Amsterdam (20 mins) slides
  • Invited Discussion - Rob Ricci (10 mins) slides
  • Invited Discussion - Rob Sherwood (10 mins) slides
  • Invited Discussion - Aaron Helsinger for Max Ott (5 mins) slides
  • Discussion - All (20 mins)
  • Summary and Wrap Up - Aaron Helsinger (10 mins) slides

Community Agreement from the Meeting

The session concluded with community agreement on three points:

  • The community will work to agree on common semantics
    • Ilia has proposals for layer 2 network resources and compute resources, and has recruited community members for other areas; thanks in advance to Max Ott, Hongwei Zhang, Martin Swany, Mike Zink, and Rob Sherwood. These individuals, and others who volunteer to help, will produce ontologies to represent the agreements.
  • GENI will use ProtoGENI RSpec V2 as the format on the wire
    • New concepts will be represented as PG RSpec extensions, allowing free addition and evolution of new concepts as needed.
  • Aggregates can use Translators to convert between formats
    • Ilia is building one between PG and Orca formats.

Next Steps Agreed to at the Meeting

For all of these, community involvement is key. Email dev@geni.net or Aaron Helsinger (ahelsing at geni.net) to get involved. Results will be published to dev@geni.net and to the GENI wiki to keep everyone informed of progress.

Agree on Concepts
  • Align edge compute resource concepts, starting with Ilia's proposal.
  • 4 key semantic extensions
    • Wireless (Max Ott, Hongwei Zhang)
    • Measurement (Martin Swany)
    • Storage - physical & cloud (Mike Zink)
    • OpenFlow flowspace (Rob Sherwood)
  • Produce ontologies for possible ratification at GEC11
    • The same groups will produce and present these agreements
Adopt Schemas
  • Once the semantics of these things are agreed to, Jon Duerig, Ilia and the GPO will assist in representing these as PG extensions
  • Revisit PG schemas
    • Change more attributes to be elements as needed
    • Make some types less opaque as needed
    • Ensure base is extensible for the known needed extensions
    • Jon Duerig of ProtoGENI will handle this
  • Once adopted, RSpec schemas will be maintained by the community, along with the AM API.
  • Modify the AM API specification to require that RSpecs are in the PG RSpec V2 format
    • Aaron Helsinger and the GPO will handle this, when the support infrastructure is in place
Implement Translators
  • Write and deploy translators as necessary (coordinate with Ilia for a start - ibaldin at renci.org)
  • Enable conversion as a GENI-wide service
    • Replicating Ilia's converter as necessary
  • Test full PG/Orca interoperability by GEC11
  • Build a thin PL/PG translation layer at PL (Andy Bavier, Tony Mack, Jon Duerig)
  • Other translators as necessary

Selected Discussion Points from the Meeting

Ilia Baldine
  • Translators may not be a long term solution, but do provide a way to make progress.
  • The key requirement is that we agree on semantics.
  • For networking, NDL is a superset of PG, which is a superset of PL. GENI primarily works at the PG/PL level, and translators can work there.
  • Networking semantics will deal with ethernet only for now - leave lower levels like optical for Orca or others with interest.
  • Adaptations are a key concept - e.g. part of describing stitching capability.
  • RENCI will maintain and extend their converter for PG/Orca formats.
  • Ilia described a proposal for a compute resources ontology.
  • We should draw on work in the cloud community.
  • The Open Virtualization Format (OVF) standard from the Distributed Management Task Force (DMTF) may be a good source for VM representations.
  • IETF Common Information Model (CIM) standard is also relevant, but is very complicated.
Martin Swany
  • Uniform Network Information Schema (UNIS) from U Delaware leverages NML, perfSonar.
  • Philosophical points
    • Use the same schema for control & measurement
    • Express common stuff in a common way
    • Not everything is common
    • Non common stuff goes in an extension
    • Express complexity only as necessary
  • RFC6020 - YANG - provides an interesting new data modeling language.
  • Private or novel concepts should be kept separate with different namespaces.
  • N2 translators should be a concern: a good common internal representation inside the translator reduces this problem.
Rob Ricci
  • Maximize tool support: any programming language or environment should be supported.
  • Use an extensible format using namespaces
  • Add annotations as needed
  • The maintainer of the format must not be a bottleneck to innovation
  • The RSpec schema maintainer could be GENI
  • Projects can describe new stuff with a new extension without waiting for any kind of global consensus.
  • Format must have a defined base that evolves over time.
  • Bound the # of active versions of the schema at 2.
  • Services advertise which formats they support, and which extensions.
  • 'Consenting Adults' model for extensions
    • Those who want the extension agree at first & only they understand it.
    • Eventually the extension may get 'blessed', to avoid duplication of concepts.
  • PL vs PG format: PL has a 'site' tag, which is similar to OF 'network'.
    • Otherwise, it is quite similar to the PG format.
Rob Sherwood
  • OF extension would likely be very simple
  • May extend link types
  • Need to decide whether flowspace is defined relative to links, switches, or other.
Group
  • Ilia: first step must be a UML diagram or similar, not the schema. No one can read XSDs
  • Andy Bavier
    • the proposal is fine.
    • PL will translate.
  • Martin Swany
    • this is like old UNIS, so I support this
  • Use XML elements not attributes to make extension easier.
  • RSpec ontology & schema should be co-hosted with the GENI AM API
  • Community maintains the schema
  • Perhaps change the namespaces and URLs to a geni.net address as extensions and base are 'blessed'.

Background Reading

ProtoGENI RSpec V2: http://www.protogeni.net/trac/protogeni/wiki/RSpec2
Orca NDL-OWL RSpecs: https://geni-orca.renci.org/trac/wiki/NDL-OWL
Orca NDL-OWL Converter: http://geni-test.renci.org:11080/ndl-conversion/convert.jsp

Further Reading

XML vs RDF analysis by NDL team: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.nml-wg/docman.root.drafts/doc14257
ProtoGENI RSpecs description: https://www.protogeni.net/trac/protogeni/wiki/RSpec
Orca RSpecs: https://geni-orca.renci.org/trac/wiki/Semantic%20resource%20descriptions%20in%20ORCA
NDL (Orca RSpec schema): http://www.science.uva.nl/research/sne/ndl
NML Schema: http://forge.gridforum.org/sf/go/doc15674?nav=1

Attachments (10)