T5) Descriptors, Objects and Registries
Descriptor Schema and Registry Service
Object Names and Registry Service
Lookup Service
1) Goals
2) Tasks
a) Need to finalize MDOD schema, for XML file. References
b) Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)
c) Want to use MDOD schema to define Event Record schema. (NetKarma)
d) Do we need MDOD registry? Use UNIS lookup service? Use DOR registry? Include in iRODS? Consider IF-MAP server?
e) Need MDOD creation and editing service. (CNRI)
f) Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle
3) Team
LEAD Giridhar Manepalli (CNRI)
Jason Zurawski (Internet2)
Ezra Kissel (U Delaware)
Eric Boyd (Internet2)
Beth Plale (IU)
Chris Small (GEMINI, IU)
Scott Jensen (Indiana U)
Larry Lannom (CNRI)
Deniz Gurken (GIMI, UH)
Harry Mussman (GPO)
4) Meetings
Review with working team at GEC13
Summary of current status (Giridhar Manepalli):
Conclusions:
Good things:
Excellent start
Collaborative Specification
Great Coverage
Nicely broken down into elements
Mandatory vs. optional elements identified
Genuine Use Cases: Gathering, transferring, and sharing
Jensen's proposal (NetKarma):
Current: Identifiers, Descriptors, Holders
Proposed: Identification, Lineage/Provenance, Constraints/Security, MDO Description
Zurawski's comments:
Too many secondary identifiers
Descriptors should be contextualized
Variations based on the type of object
GENI specific descriptions should be clearly marked and separated
Slight changes to names & enclosing elements recommended
Comments/suggestions based on metadata practices:
Too many optional elements
Too many choices given to users
Users bound to take the path of least resistance
Keep the scope restricted to only mandatory elements – at least in the beginning
Try those out. Implement them.
One size fits all ---- No!
Capturing descriptions, formats, policies, transactions, etc. in a monolithic fashion
Register individual components separately
E.g., Capture legal formats & interpretations in their own records, and reference them here
E.g., Same with accepted policies
Identifiers cannot be semantic
Domain, sub-domain, and object-type are part of an ID
World view changes frequently
Non-semantic Ids are worth every penny
Search engines & registries mask the opaqueness
After all, IDs are just entities behind the scenes
Object Type controlled vocabulary enumerates apples and oranges
Collection, flow, directory, file, database, gui are not mutually exclusive
Doesn’t help the recipient make any decision looking at the descriptor
Bundle type & format into format interpretation method
Covers too many corner cases, e.g., flow-rate
Expects too many details, e.g., locator (type, access method, etc.)
5) Open Issues
6) Definition of Measurement Data Object Descriptor (MDOD)
1:05pm
Harry Mussman
1) Overview
Format (syntax)
Goal remains: one data model for GENI
Assume: transfer format is XML record (file)
Have example in XML-like format, and schema/types in RelaxNG
Need: first software to generate (and consume) MDODs
Contents (semantics)
Few required elements, many optional elements
To meet use cases in arch document, sec 4.5
Want one-for-one mapping to key elements in current tools (started: perfSONAR; GENI rspec)
Point-of-view
Often global; therefore includes locators for data objects; must be updated as objects are copied, moved
Sometimes associated; need to understand
Should we consider: storing all MDODs in global, MI srvc? this becomes a MD object registry
Structure
identifier(s)
descriptor(s)
holder(s)
2) Use Cases
Read the following sections of the Architecture document:
4.5 GATHERING, TRANSFERRING AND SHARING MD
4.5.1 Schema and Provenance of MD
schema for MD object contained in MDOD
provenance of MD object contained in MDOD
4.5.2 Gathering MD into a Slice
by first holder, the "owner"
starts MDOD
as owner, sets sharing policy, disposal policy
as gatherer, deals with privacy
4.5.3 Transferring MD between I&M Services in the Same Slice
holder updates MDOD as transfers MD object between locations
option: registers MDOD with MI service
4.5.4 Transferring MD between I&M Services in Different Slices
e.g., holder 1 to holder 2
holder 1 authorizes transfer, sends object and copy of MDOD
holder 1 records the transfer in their MDOD
holder 2 accepts MD object and MDOD, and adds their local location to their MDOD
holder 2 tracks further transfers in their MDOD
4.5.5 Sharing MD with Others
e.g., holder 1 offers MD object at a portal, and registers this with MI srvc
user 7 requests MD object
holder 1 checks their sharing policy, authorizes transfer, and records the transfer in their MDOD
e.g., holder 1 transfers MD object to DOA srvc
user 9 requests MD object from DOA srvc
DOA srvc checks sharing policy in associated MDOD, authorizes transfer, and records the transfer in their MDOD
Use these figures
A summary of the topic is presented in these slides
See: I&M in Experimenter's Slice
See: I&M in Operator's Infrastructure Measurement Slice
For example: a GENI operator could get resources, setup a long-running measurement slice, and make data available to other operators or even experimenters.
3) Content (Semantics)
Read the following sections of the Architecture document:
8 SCHEMA AND ELEMENTS FOR MEASUREMENT DATA OBJECT DESCRIPTORS (MDODS)
8.1 MEASUREMENT DATA OBJECTS (MDOS)
8.2 MEASUREMENT DATA (MD) OBJECT DESCRIPTOR
8.3 MD OBJECT DESCRIPTOR DATA MODEL
8.4 MD OBJECT DESCRIPTOR SCHEMAS
A summary of the topic is presented in these slides
DRAFT MDOD Data Models, Elements and Values
v0.2.x XML-like example (HEM)
v0.2 text
v0.2.1 text
v0.2.1 text, only mandatory elements
v0.3.x RelaxNG-Compact version (EK); includes RNC and converted XSD files, both schema and types
v0.3 schema RNC
v0.3 schema converted XSD
v0.3 types RNC
v0.3 types converted XSD
XML examples provided by Jason Zurawski:
example of proposed schema, with questions
modified schema, for further discussion
MDOD Discussion Points
For meeting on 072611 text
4) Supported Object Types and Formats
Instrumentation Tools (Kentucky)
OMF/OML (NICTA)
perSONAR/LAMP (Delaware, I2)
OnTimeMeasure (Ohio State)
Scalable Sensing Service? (Purdue, HP Labs)
GMOC? (Indiana)
NetCDF?
others?
5) Next Steps
First review of data model and schema (completed)
For each tool: map key elements into current data model; review expected use cases who? when?
Review suggested approach to recording provenance who? when?
Review choice for primary and secondary identifiers who? when?
Review formats for slice_id, etc. who? when?
Understand relationship to rspec? who? when?
Complete third pass at data model and schema who? when?
Complete first prototype software to create MDOD who? when?
Attachments (2)
- Visio-070111_UseCases_Services_MDOD_Page_01.jpg (447.3 KB) - added by 12 years ago.
- Visio-070111_UseCases_Services_MDOD_Page_02.jpg (492.5 KB) - added by 12 years ago.
Download all attachments as: .zip