[[PageOutline]] == T5) Descriptors, Objects and Registries == Descriptor Schema and Registry Service [[BR]] Object Names and Registry Service [[BR]] Lookup Service [[BR]] [wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService Work in Progress] [[BR]] === 1) Goals === === 2) Tasks === a) Need to finalize MDOD schema, for XML file. [http://groups.geni.net/geni/wiki/GEC11InstMeasWorkingSession#a2MeasurementDataObjectDescriptorMDOD References] [[BR]] b) Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)[[BR]] c) Want to use MDOD schema to define Event Record schema. (NetKarma)[[BR]] d) Do we need MDOD registry? Use UNIS lookup service? Use DOR registry? Include in iRODS? Consider IF-MAP server? [[BR]] e) Need MDOD creation and editing service. (CNRI) [[BR]] f) Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle [[BR]] === 3) Team === LEAD Giridhar Manepalli (CNRI) [[BR]] Jason Zurawski (Internet2) [[BR]] Ezra Kissel (U Delaware) [[BR]] Eric Boyd (Internet2) [[BR]] Beth Plale (IU) [[BR]] Chris Small (GEMINI, IU) [[BR]] Scott Jensen (Indiana U) [[BR]] Larry Lannom (CNRI) [[BR]] Deniz Gurken (GIMI, UH) [[BR]] Harry Mussman (GPO) [[BR]] [[BR]] === 4) Meetings === Review with working team at GEC13 [[BR]] Summary of current status (Giridhar Manepalli): [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T5%29%20%20MDOD%20Status%20-%20CNRI.pptx slides] [[BR]] Conclusions: [[BR]] Good things: [[BR]] Excellent start [[BR]] Collaborative Specification [[BR]] Great Coverage [[BR]] Nicely broken down into elements [[BR]] Mandatory vs. optional elements identified [[BR]] Genuine Use Cases: Gathering, transferring, and sharing [[BR]] Jensen's proposal (NetKarma): [[BR]] Current: Identifiers, Descriptors, Holders [[BR]] Proposed: Identification, Lineage/Provenance, Constraints/Security, MDO Description [[BR]] Zurawski's comments: [[BR]] Too many secondary identifiers [[BR]] Descriptors should be contextualized [[BR]] Variations based on the type of object [[BR]] GENI specific descriptions should be clearly marked and separated [[BR]] Slight changes to names & enclosing elements recommended [[BR]] Comments/suggestions based on metadata practices: [[BR]] Too many optional elements [[BR]] Too many choices given to users [[BR]] Users bound to take the path of least resistance [[BR]] Keep the scope restricted to only mandatory elements – at least in the beginning [[BR]] Try those out. Implement them. [[BR]] One size fits all ---- No! [[BR]] Capturing descriptions, formats, policies, transactions, etc. in a monolithic fashion [[BR]] Register individual components separately [[BR]] E.g., Capture legal formats & interpretations in their own records, and reference them here [[BR]] E.g., Same with accepted policies [[BR]] Identifiers cannot be semantic [[BR]] Domain, sub-domain, and object-type are part of an ID [[BR]] World view changes frequently [[BR]] Non-semantic Ids are worth every penny [[BR]] Search engines & registries mask the opaqueness [[BR]] After all, IDs are just entities behind the scenes [[BR]] Object Type controlled vocabulary enumerates apples and oranges [[BR]] Collection, flow, directory, file, database, gui are not mutually exclusive [[BR]] Doesn’t help the recipient make any decision looking at the descriptor [[BR]] Bundle type & format into format interpretation method [[BR]] Covers too many corner cases, e.g., flow-rate [[BR]] Expects too many details, e.g., locator (type, access method, etc.) [[BR]] === 5) Open Issues === === 6) Definition of Measurement Data Object Descriptor (MDOD) === 1:05pm [[BR]] Harry Mussman[[BR]] ==== 1) Overview ==== Format (syntax) [[BR]] Goal remains: one data model for GENI [[BR]] Assume: transfer format is XML record (file) [[BR]] Have example in XML-like format, and schema/types in RelaxNG [[BR]] Need: first software to generate (and consume) MDODs [[BR]] Contents (semantics) [[BR]] Few required elements, many optional elements [[BR]] To meet use cases in arch document, sec 4.5 [[BR]] Want one-for-one mapping to key elements in current tools (started: perfSONAR; GENI rspec) [[BR]] Point-of-view [[BR]] Often global; therefore includes locators for data objects; must be updated as objects are copied, moved [[BR]] Sometimes associated; need to understand [[BR]] Should we consider: storing all MDODs in global, MI srvc? this becomes a MD object registry [[BR]] Structure [[BR]] identifier(s) [[BR]] descriptor(s) [[BR]] holder(s) [[BR]] ==== 2) Use Cases ==== Read the following sections of the [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/122010i%20GENI-SE-IM-ARCH-1.0.doc Architecture document:] [[BR]] 4.5 GATHERING, TRANSFERRING AND SHARING MD [[BR]] 4.5.1 Schema and Provenance of MD [[BR]] schema for MD object contained in MDOD [[BR]] provenance of MD object contained in MDOD [[BR]] 4.5.2 Gathering MD into a Slice [[BR]] by first holder, the "owner" [[BR]] starts MDOD [[BR]] as owner, sets sharing policy, disposal policy [[BR]] as gatherer, deals with privacy [[BR]] 4.5.3 Transferring MD between I&M Services in the Same Slice [[BR]] holder updates MDOD as transfers MD object between locations [[BR]] option: registers MDOD with MI service [[BR]] 4.5.4 Transferring MD between I&M Services in Different Slices [[BR]] e.g., holder 1 to holder 2 [[BR]] holder 1 authorizes transfer, sends object and copy of MDOD [[BR]] holder 1 records the transfer in their MDOD [[BR]] holder 2 accepts MD object and MDOD, and adds their local location to their MDOD [[BR]] holder 2 tracks further transfers in their MDOD [[BR]] 4.5.5 Sharing MD with Others [[BR]] e.g., holder 1 offers MD object at a portal, and registers this with MI srvc [[BR]] user 7 requests MD object [[BR]] holder 1 checks their sharing policy, authorizes transfer, and records the transfer in their MDOD [[BR]] e.g., holder 1 transfers MD object to DOA srvc [[BR]] user 9 requests MD object from DOA srvc [[BR]] DOA srvc checks sharing policy in associated MDOD, authorizes transfer, and records the transfer in their MDOD [[BR]] Use these [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/Visio-4.x%20%20030911%20_IM-ARCH-Figures.pdf figures][[BR]] A summary of the topic is presented in these [http://groups.geni.net/geni/attachment/wiki/GEC10InstMeasWorkingSession/031711_GatheringTransferringMD_I%26MArch_Overview.pptx slides][[BR]] See: I&M in Experimenter's Slice [[BR]] [[Image(Visio-070111_UseCases_Services_MDOD_Page_01.jpg, 90%)]] [[BR]] See: I&M in Operator's Infrastructure Measurement Slice [[BR]] For example: a GENI operator could get resources, setup a long-running measurement slice, and make data available to other operators or even experimenters. [[BR]] [[Image(Visio-070111_UseCases_Services_MDOD_Page_02.jpg, 90%)]] [[BR]] ==== 3) Content (Semantics) ==== Read the following sections of the [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/122010i%20GENI-SE-IM-ARCH-1.0.doc Architecture document:] [[BR]] 8 SCHEMA AND ELEMENTS FOR MEASUREMENT DATA OBJECT DESCRIPTORS (MDODS) [[BR]] 8.1 MEASUREMENT DATA OBJECTS (MDOS) [[BR]] 8.2 MEASUREMENT DATA (MD) OBJECT DESCRIPTOR [[BR]] 8.3 MD OBJECT DESCRIPTOR DATA MODEL [[BR]] 8.4 MD OBJECT DESCRIPTOR SCHEMAS [[BR]] A summary of the topic is presented in these [http://groups.geni.net/geni/attachment/wiki/GEC10InstMeasWorkingSession/031711_MDObjectsDescriptors%20_I%26MArch_Overview.pptx slides][[BR]] DRAFT MDOD Data Models, Elements and Values [[BR]] v0.2.x XML-like example (HEM) [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072011_ver0.2_MDOD_DataModel.txt v0.2 text][[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072511_ver0.2.1_MDOD_DataModel.txt v0.2.1 text][[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072511_ver0.2.1_mandatoryonly_MDOD_DataModel.txt v0.2.1 text, only mandatory elements][[BR]] v0.3.x RelaxNG-Compact version (EK); includes RNC and converted XSD files, both schema and types [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072111_v0.3_MDOD.rnc v0.3 schema RNC] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072111_v0.3_MDOD.xsd v0.3 schema converted XSD] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072111_v0.3_MDOD_types.rnc v0.3 types RNC] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072111_v0.3_MDOD_types.xsd v0.3 types converted XSD] [[BR]] XML examples provided by Jason Zurawski: [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072811Zurawski_instance1.xml example of proposed schema, with questions] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072811Zurawski_instance_proposed.xml modified schema, for further discussion] [[BR]] MDOD Discussion Points [[BR]] For meeting on 072611 [http://groups.geni.net/geni/attachment/wiki/GEC11InstMeasWorkingSession/072511_MDOD_Discussion.txt text] [[BR]] ==== 4) Supported Object Types and Formats ==== Instrumentation Tools (Kentucky) [[BR]] OMF/OML (NICTA) [[BR]] perSONAR/LAMP (Delaware, I2) [[BR]] OnTimeMeasure (Ohio State) [[BR]] Scalable Sensing Service? (Purdue, HP Labs) [[BR]] GMOC? (Indiana)[[BR]] NetCDF?[[BR]] others? [[BR]] ==== 5) Next Steps ==== First review of data model and schema (completed) [[BR]] For each tool: map key elements into current data model; review expected use cases who? when? [[BR]] Review suggested approach to recording provenance who? when? [[BR]] Review choice for primary and secondary identifiers who? when? [[BR]] Review formats for slice_id, etc. who? when? [[BR]] Understand relationship to rspec? who? when? [[BR]] Complete third pass at data model and schema who? when?[[BR]] Complete first prototype software to create MDOD who? when?[[BR]]