wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService

T5) Descriptors, Objects and Registries

Descriptor Schema and Registry Service
Object Names and Registry Service
Lookup Service

Work in Progress

1) Goals

2) Tasks

a) Need to finalize MDOD schema, for XML file. References
b) Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)
c) Want to use MDOD schema to define Event Record schema. (NetKarma)
d) Do we need MDOD registry? Use UNIS lookup service? Use DOR registry? Include in iRODS? Consider IF-MAP server?
e) Need MDOD creation and editing service. (CNRI)
f) Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle

3) Team

LEAD Giridhar Manepalli (CNRI)
Jason Zurawski (Internet2)
Ezra Kissel (U Delaware)
Eric Boyd (Internet2)
Beth Plale (IU)
Chris Small (GEMINI, IU)
Scott Jensen (Indiana U)
Larry Lannom (CNRI)
Deniz Gurken (GIMI, UH)
Harry Mussman (GPO)

4) Meetings

Review with working team at GEC13

Summary of current status (Giridhar Manepalli):

slides

Conclusions:

Good things:

Excellent start
Collaborative Specification
Great Coverage
Nicely broken down into elements
Mandatory vs. optional elements identified
Genuine Use Cases: Gathering, transferring, and sharing

Jensen's proposal (NetKarma):

Current: Identifiers, Descriptors, Holders
Proposed: Identification, Lineage/Provenance, Constraints/Security, MDO Description

Zurawski's comments:

Too many secondary identifiers
Descriptors should be contextualized
Variations based on the type of object
GENI specific descriptions should be clearly marked and separated
Slight changes to names & enclosing elements recommended

Comments/suggestions based on metadata practices:

Too many optional elements

Too many choices given to users
Users bound to take the path of least resistance
Keep the scope restricted to only mandatory elements – at least in the beginning
Try those out. Implement them.

One size fits all ---- No!

Capturing descriptions, formats, policies, transactions, etc. in a monolithic fashion
Register individual components separately
E.g., Capture legal formats & interpretations in their own records, and reference them here
E.g., Same with accepted policies

Identifiers cannot be semantic

Domain, sub-domain, and object-type are part of an ID
World view changes frequently
Non-semantic Ids are worth every penny
Search engines & registries mask the opaqueness
After all, IDs are just entities behind the scenes

Object Type controlled vocabulary enumerates apples and oranges

Collection, flow, directory, file, database, gui are not mutually exclusive
Doesn’t help the recipient make any decision looking at the descriptor
Bundle type & format into format interpretation method
Covers too many corner cases, e.g., flow-rate
Expects too many details, e.g., locator (type, access method, etc.)

5) Open Issues

6) Definition of Measurement Data Object Descriptor (MDOD)

1:05pm
Harry Mussman

1) Overview

Format (syntax)

Goal remains: one data model for GENI
Assume: transfer format is XML record (file)
Have example in XML-like format, and schema/types in RelaxNG
Need: first software to generate (and consume) MDODs

Contents (semantics)

Few required elements, many optional elements
To meet use cases in arch document, sec 4.5
Want one-for-one mapping to key elements in current tools (started: perfSONAR; GENI rspec)

Point-of-view

Often global; therefore includes locators for data objects; must be updated as objects are copied, moved
Sometimes associated; need to understand
Should we consider: storing all MDODs in global, MI srvc? this becomes a MD object registry

Structure

identifier(s)
descriptor(s)
holder(s)

2) Use Cases

Read the following sections of the Architecture document:

4.5 GATHERING, TRANSFERRING AND SHARING MD

4.5.1 Schema and Provenance of MD

schema for MD object contained in MDOD
provenance of MD object contained in MDOD

4.5.2 Gathering MD into a Slice

by first holder, the "owner"
starts MDOD
as owner, sets sharing policy, disposal policy
as gatherer, deals with privacy

4.5.3 Transferring MD between I&M Services in the Same Slice

holder updates MDOD as transfers MD object between locations
option: registers MDOD with MI service

4.5.4 Transferring MD between I&M Services in Different Slices

e.g., holder 1 to holder 2
holder 1 authorizes transfer, sends object and copy of MDOD
holder 1 records the transfer in their MDOD
holder 2 accepts MD object and MDOD, and adds their local location to their MDOD
holder 2 tracks further transfers in their MDOD

4.5.5 Sharing MD with Others

e.g., holder 1 offers MD object at a portal, and registers this with MI srvc
user 7 requests MD object
holder 1 checks their sharing policy, authorizes transfer, and records the transfer in their MDOD

e.g., holder 1 transfers MD object to DOA srvc
user 9 requests MD object from DOA srvc
DOA srvc checks sharing policy in associated MDOD, authorizes transfer, and records the transfer in their MDOD

Use these figures

A summary of the topic is presented in these slides

See: I&M in Experimenter's Slice


See: I&M in Operator's Infrastructure Measurement Slice
For example: a GENI operator could get resources, setup a long-running measurement slice, and make data available to other operators or even experimenters.


3) Content (Semantics)

Read the following sections of the Architecture document:
8 SCHEMA AND ELEMENTS FOR MEASUREMENT DATA OBJECT DESCRIPTORS (MDODS)
8.1 MEASUREMENT DATA OBJECTS (MDOS)
8.2 MEASUREMENT DATA (MD) OBJECT DESCRIPTOR
8.3 MD OBJECT DESCRIPTOR DATA MODEL
8.4 MD OBJECT DESCRIPTOR SCHEMAS

A summary of the topic is presented in these slides

DRAFT MDOD Data Models, Elements and Values

v0.2.x XML-like example (HEM)
v0.2 text
v0.2.1 text
v0.2.1 text, only mandatory elements

v0.3.x RelaxNG-Compact version (EK); includes RNC and converted XSD files, both schema and types
v0.3 schema RNC
v0.3 schema converted XSD
v0.3 types RNC
v0.3 types converted XSD

XML examples provided by Jason Zurawski:
example of proposed schema, with questions
modified schema, for further discussion

MDOD Discussion Points

For meeting on 072611 text

4) Supported Object Types and Formats

Instrumentation Tools (Kentucky)
OMF/OML (NICTA)
perSONAR/LAMP (Delaware, I2)
OnTimeMeasure (Ohio State)
Scalable Sensing Service? (Purdue, HP Labs)
GMOC? (Indiana)
NetCDF?
others?

5) Next Steps

First review of data model and schema (completed)

For each tool: map key elements into current data model; review expected use cases who? when?
Review suggested approach to recording provenance who? when?
Review choice for primary and secondary identifiers who? when?
Review formats for slice_id, etc. who? when?

Understand relationship to rspec? who? when?

Complete third pass at data model and schema who? when?

Complete first prototype software to create MDOD who? when?

Last modified 7 years ago Last modified on 04/04/12 11:31:11

Attachments (2)

Download all attachments as: .zip