= GENI Instrumentation and Measurements Architecture, Version 0.1 = An effort is underway to define and document the GENI Instrumentation and Measurements Architecture. Plan:[[BR]] + Completed by GEC7: v0.1 DRAFT, by GPO [[BR]] + Completed by GEC8: v0.5 DRAFT, by GPO, with revised outline and contributions after 2nd workshop[[BR]] + In preparation: v1.0 DRAFT, including contributions from WG[[BR]] A first DRAFT (v0.1) was completed by the GPO: [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/030810%20%20GENI-SE-IM-ARCH-0.1.pdf GENI I&M Architecture (Document ID: GENI-SE-IM-ARCH-0.1, Date: March 8, 2010)] [[BR]] This DRAFT includes proposed I&M services and a proposed configuration [[BR]] Document outline:[[BR]] 1. Document Scope[[BR]] 2. Introduction[[BR]] 3. Definition and configuration of I&M services[[BR]] 4. Interfaces, protocols and schema for Measurement Data (MD)[[BR]] 5. Ownership of MD and privacy of owners[[BR]] 6. Interfaces, protocols and APIs for using I&M services[[BR]] 7. Basic GENI I&M use cases[[BR]] 8. MD transport via the GENI Measurement Plane[[BR]] 9. Discovery, authorization, assignment and binding of GENI I&M services[[BR]] 10. Measurement Orchestration (MO) service[[BR]] 11. Measurement Point (MP)[[BR]] 12. Time-stamping MD[[BR]] 13. Measurement Collection (MC) service[[BR]] 14. Measurement Analysis and Presentation (MAP) service[[BR]] 15. Measurement Data Archive (MDA) service[[BR]] 16. Additional GENI I&M use cases[[BR]] This document was reviewed at the Inst&Meas WG meeting at GEC7: [http://groups.geni.net/geni/attachment/wiki/Gec7InstMeasWGAgenda/031210%20%20IM-ARCH-GEC7Slides.pdf slides] [[BR]] A second DRAFT (v0.5) was completed by the GPO, with input from the 2nd I&M Workshop: [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/072210%20%20GENI-SE-IM-ARCH-0.5.pdf GENI I&M Architecture (Document ID: GENI-SE-IM-ARCH-0.5, Date: July 22, 2010)] [[BR]] This DRAFT has a revised outline: [[BR]] Document outline:[[BR]] 1. Document Scope[[BR]] 2. Introduction[[BR]] 3. Instrumentation and Measurement (I&M) Use Cases [[BR]] 4. Configuration of I&M services[[BR]] 5. Interfaces to I&M Services [[BR]] 6. Measurement Traffic Flows 7. Schema and Protocols for Measurement Data (MD) Flows/File Transfers[[BR]] 8. Ownership of MD and Privacy of Owners[[BR]] 9. APIs and Protocols for Using I&M Services[[BR]] 10. Discovery, Authorization, Assignment and Binding of GENI I&M Services[[BR]] 11. Measurement Orchestration (MO) service[[BR]] 12. Measurement Point (MP)Service[[BR]] 13. Time-stamping MD[[BR]] 14. Measurement Information (MI) Service [[BR]] 15. Measurement Collection (MC) Service[[BR]] 16. Measurement Analysis and Presentation (MAP) Service[[BR]] 17. Measurement Data Archive (MDA) Service[[BR]] 1 DOCUMENT SCOPE [[BR]] 2 INTRODUCTION [[BR]] 3 INSTRUMENTATION AND MEASUREMENT (I&M) USE CASES [[BR]] 3.1 FOR EXPERIMENT RESEARCHERS [[BR]] 3.2 FOR EXPERIMENT (OPT-IN) USERS [[BR]] 3.3 FOR CENTRAL (I.E., GMOC) OPERATORS [[BR]] 3.4 FOR AGGREGATE PROVIDERS AND OPERATORS [[BR]] 3.5 FOR ARCHIVE PROVIDERS AND OPERATORS [[BR]] 3.6 FOR RESEARCHERS THAT USE ARCHIVED MEASUREMENT DATA [[BR]] 4 CONFIGURATION OF I&M SERVICES [[BR]] 4.1 FUNCTIONS OF I&M SERVICES [[BR]] 4.1.1 Measurement Orchestration (MO) Service [[BR]] 4.1.2 Measurement Point (MP) Service [[BR]] 4.1.3 Measurement Information (MI) Service [[BR]] 4.1.4 Measurement Collection (MC) Service [[BR]] 4.1.5 Measurement Analysis and Presentation (MAP) Service [[BR]] 4.1.6 Measurement Data Archive (MDA) Service [[BR]] 4.2 TYPES OF I&M SERVICES [[BR]] 4.2.1 Type 1: Dedicated Service Platform for Customized Information 11 4.2.2 Type 2: Common Service Platform with Dedicated Slivers for Customized Information 11 4.2.3 Type 3: Common Service for Common or Customized Information 11 4.3 DATA FLOWS/TRANSFERS BETWEEN I&M SERVICES 11 4.3.1 Pull Data 11 4.3.2 Push Data 11 4.3.3 Publish and Subscribe to Data 11 4.4 USING I&M SERVICES 11 4.5 BASIC RESEARCHER I&M CONFIGURATION 11 4.6 BASIC OPERATOR I&M CONFIGURATION 12 4.7 RESEARCHER INCLUDING OPERATOR I&M CONFIGURATION 12 4.8 RANGE OF I&M SERVICE IMPLEMENTATIONS 12 5 INTERFACES TO I&M SERVICES 13 5.1 DISCOVER RESOURCES AND ASSIGN SLIVERS 13 5.2 CONFIGURE AND PROGRAM SLIVERS 13 5.3 MANAGE SERVICES 13 5.4 MEASUREMENT DATA FLOWS/TRANSFERS 13 5.5 REGISTER AVAILABILITY OF MD FROM I&M SERVICE 13 5.6 DISCOVER AVAILABILITY OF MD FROM I&M SERVICE AND START MD FLOW/TRANSFER 13 5.7 OBSERVE I&M SERVICE STATUS AND EXAMINE MD 13 6 MEASUREMENT TRAFFIC FLOWS 14 6.1 GENI ENVIRONMENT 14 6.1.1 Control Traffic 14 6.1.2 Experiment Traffic 14 6.2 MEASUREMENT TRAFFIC CARRIED WITH CONTROL TRAFFIC 15 6.2.1 When Server has a Public (reachable) IP Address 15 6.2.2 When Server has a Private IP Address 15 6.3 MEASUREMENT TRAFFIC CARRIED WITH EXPERIMENT TRAFFIC 15 7 SCHEMA AND PROTOCOLS FOR MEASUREMENT DATA (MD) FLOWS/TRANSFERS 16 7.1 RANGE OF SCHEMA AND PROTOCOLS 16 7.2 METADATA INCLUDED WITH MD 18 7.2.1 Identifiers, Annotation, Provenance, Privacy 18 7.2.2 Description, Collection, Description, Processing 18 7.3 OPTION 1: BYTES OF MD VIA SNMP (INSTRUMENTATION TOOLS, GMOC) 18 7.4 OPTION 2: FILE OF MD VIA MULTIPLE PROTOCOLS 18 7.5 OPTION 3: XML-FORMATTED MD VIA HTTP (PERFSONAR) 19 7.6 OPTION 4: TUPLES OF MD VIA CUSTOM OML PROTOCOL OVER TCP (OML) 19 7.7 OPTION 5: TUPLES OF MD VIA IPFIX OVER SCTP (PROPOSED FOR OML) 19 7.8 OPTION 6: METCDF-FORMATTED FILES OF DATA USING LDM OVER TCP (DICLOUD) 19 8 OWNERSHIP OF MD AND PRIVACY OF OWNERS 20 8.1 OWNERSHIP OF MD 20 8.2 PRIVACY OF RESEARCHERS/SLICES 20 8.3 PRIVACY OF APPLICATION USERS 20 9 API�S AND PROTOCOLS FOR USING I&M SERVICES 21 9.1 ASSIGN SLIVERS FOR I&M SERVICES 21 9.2 CONFIGURE AND PROGRAM SLIVERS FOR I&M SERVICES 21 9.3 MANAGE I&M SERVICES 21 9.4 REGISTER AVAILABILITY OF MD FROM I&M SERVICE 22 9.5 DISCOVER AVAILABILITY OF MD AND START MD FLOW/TRANSFER 22 9.6 OBSERVE I&M SERVICES AND EXAMINE MD 22 10 DISCOVERY, AUTHORIZATION, ASSIGNMENT AND BINDING OF GENI I&M SERVICES 23 11 MEASUREMENT ORCHESTRATION (MO) SERVICE 24 12 MEASUREMENT POINT (MP) SERVICE 25 12.1 FUNCTIONS 25 12.2 TYPES OF MP SERVICES 25 12.3 GATHERING MEASUREMENT DATA FROM INSTRUMENTATION (SENSORS) 25 12.4 TYPES OF INSTRUMENTATION (SENSORS) 26 13 TIME-STAMPING MD 27 14 MEASUREMENT INFORMATION (MI) SERVICE 28 15 MEASUREMENT COLLECTION (MC) SERVICE 29 16 MEASUREMENT ANALYSIS AND PRESENTATION (MAP) SERVICE 30 17 MEASUREMENT DATA ARCHIVE (MDA) SERVICE 31 17.1 FUNCTIONS 31 17.2 IMPLEMENTATION 31 17.3 USE 31 18 GLOSSARY 32 19 REFERENCES 33 A third DRAFT (v1.0) is in preparation, with input from the WG. [[BR]] The community is invited to help prepare and review this draft, and provide comments, additions or changes to the Inst&Meas WG SE, Harry Mussman, at hmussman@bbn.com