[[PageOutline]] = GENI Instrumentation and Measurement Work in Progress = == Charter == '''Scope:''' Discuss, develop and build consensus around the architectural framework for the instrumentation and measurement infrastructure that will be deployed and used in GENI. '''GPO System Engineer:''' [mailto:hmussman@bbn.com Harry Mussman] '''Mail list:''' [mailto:inst-meas-wg@geni.net mail to inst-meas-wg@geni.net] / [http://lists.geni.net/mailman/listinfo/inst-meas-wg subscribe] / [http://lists.geni.net/pipermail/inst-meas-wg/ mailman archive] '''Objectives:''' Create an architecture for measurement that enables GENI goals to be achieved, document and refine[[BR]] Facilitate dialog and coordination between teams focused on I&M[[BR]] Identify key challenges in I&M that could otherwise inhibit the infrastructure[[BR]] Solicit feedback from users[[BR]] Communicate via mailing list, wiki, GECs[[BR]] ---- == Summary == * [wiki:GEC10InstMeasSummary Instrumentation and Measurement Summary at the GEC-10, Tuesday, March 15, 12noon - 1pm, Agenda and Notes] ---- == Meeting Notes and Presentations == * [wiki:GENIMeasWS GENI Measurement Workshop, UW-Madison, 26 June, 2009] * [wiki:GEC6InstMeasWGAgenda Instrumentation and Measurement WG Meeting at the GEC-6, Wednesday, November 18, 2009, 9am - 11am, Agenda and Notes] * [http://groups.geni.net/geni/attachment/wiki/GEC6OutbriefsAgenda/111809%20report%20GEC%206%20WGInstAndMeas%20mtg.pdf Instrumentation and Measurement WG Meeting at the GEC-6, Summary Report] * [wiki:Gec7InstMeasWGAgenda Instrumentation and Measurement WG Meeting at the GEC-7, Wednesday, March 17, 3:30pm - 5:30pm, Agenda and Notes] * [http://groups.geni.net/geni/attachment/wiki/Gec7InstMeasWGAgenda/031710%20WGInstAndMeas_Report.ppt Instrumentation and Measurement WG Meeting at the GEC-7, Summary Report] * [wiki:2ndInstMeasWork 2nd GENI Instrumentation and Measurement Workshop, June 8 - 9, 2010] * [http://groups.geni.net/geni/attachment/wiki/Gec8InstMeasWGAgenda/070610%20%20IM-Workshop_%20ARCH-GEC8Slides.ppt 2nd GENI Instrumentation and Measurement Workshop, Summary Report and Priority Topics] * [wiki:Gec8InstMeasWGAgenda Instrumentation and Measurement WG Meeting at the GEC-8, Thursday, July 22, 9:00am - 10:30am, Agenda and Notes] * [http://groups.geni.net/geni/attachment/wiki/Gec8InstMeasWGAgenda/072210%20%20Wrapup%20GEC8%20WGInstAndMeas.ppt Instrumentation and Measurement WG Meeting at the GEC-8, Summary Report] * [wiki:Gec9InstMeasWorkshopAgenda 3rd Instrumentation and Measurement Workshop at the GEC-9, Tuesday, November 2, 9:00am - 12noon, Agenda and Notes] * [wiki:Gec9InstMeasWGAgenda Instrumentation and Measurement WG Meeting at the GEC-9, Thursday, November 4, 9:00am - 10:30am, Agenda and Notes] * [http://groups.geni.net/geni/attachment/wiki/GeniInstMeas/110410%20Wrapup%20%20GEC9_Agenda_WGMtgInstAndMeas.ppt Instrumentation and Measurement WG Meeting at the GEC-9, Summary Report] * [wiki:GEC10InstMeasSummary Instrumentation and Measurement Summary at the GEC-10, Tuesday, March 15, 12noon - 1pm, Agenda and Notes] * [wiki:GEC10InstMeasWorkingSession Instrumentation and Measurement Working Session at the GEC-10, Thursday, March 17, 1pm - 4pm, Agenda and Notes] * [ Instrumentation and Measurement Working Session at the GEC-10, Summary Report] * [wiki:GEC11InstMeasWorkingSession Instrumentation and Measurement Working Session at the GEC-11, Thursday, 28, 1pm - 3pm, Agenda and Notes] ---- == [wiki:GENIIandMCAPCAT GENI I&M Capabilities Catalog] == A v0.1 DRAFT has been completed by the GPO, and includes overview slides, a spreadsheet of the catalog, and drawings from referenced projects. [[BR]] This document reviews each current GENI I&M project, and other selected projects, and lists:[[BR]] Architecture components addressed or implemented[[BR]] Implementations in GENI, or elsewhere[[BR]] Uses in GENI, or elsewhere[[BR]] The purpose is to:[[BR]] Identify projects that can best contribute to architecture topics[[BR]] Identify projects that can implement enhanced GENI capabilities in Spiral 2[[BR]] ---- == [wiki:GeniInstrumentationandMeasurementsArchitecture GENII&M Architecture] == An effort is underway to define and document the GENI Instrumentation and Measurements Architecture. Plan:[[BR]] Completed by GEC7: v0.1 DRAFT, by GPO [[BR]] Completed by GEC8: v0.5 DRAFT, by GPO, with revised outline and contributions after 2nd workshop[[BR]] In preparation: v1.0 DRAFT, including contributions from WG[[BR]] A first DRAFT (v0.1) was completed by the GPO: [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/030810%20%20GENI-SE-IM-ARCH-0.1.pdf GENI I&M Architecture (Document ID: GENI-SE-IM-ARCH-0.1, Date: March 8, 2010)] [[BR]] This DRAFT includes proposed I&M services and a proposed configuration [[BR]] [[BR]] A second DRAFT (v0.5) was completed by the GPO, with input from the 2nd I&M Workshop: [http://groups.geni.net/geni/attachment/wiki/GeniInstrumentationandMeasurementsArchitecture/072210%20%20GENI-SE-IM-ARCH-0.5.pdf GENI I&M Architecture (Document ID: GENI-SE-IM-ARCH-0.5, Date: July 22, 2010)] [[BR]] This DRAFT has a revised outline. [[BR]] A third DRAFT (v1.0) is in preparation, with input from the WG. [[BR]] The community is invited to help prepare and review this draft, and provide comments, additions or changes to the Inst&Meas WG SE, Harry Mussman, at hmussman@bbn.com == Priority topics == The following priority topics were identified at the 2nd I&M workshop, and teams of attendees (and some non-attendees) were identified for each topic to discuss it, and write a text summary for review by the WG. === [wiki:Topic1 Topic 1 GENI I&M Use Cases] === === [wiki:Topic2 Topic 2 GENI I&M Services] === === [wiki:Topic3 Topic 3 GENI I&M Resources] === === [wiki:Topic4 Topic 4 GENI I&M Measurement Plane and Interfaces] === === [wiki:Topic5 Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services] === === [wiki:Topic6 Topic 6 GENI I&M Interfaces and Protocols (APIs): Data Flows and Data File Transfers] === === [wiki:Topic7 Topic 7 GENI I&M Interfaces and Protocols (APIs): Registration and Discovery of Services with Available Measurement Data] === === [wiki:Topic8 Topic 8 GENI I&M Interfaces and Protocols (APIs): GUIs] === === [wiki:Topic9 Topic 9 GENI Measurement Data Schema] === ---- == Available GENI I&M Tools == The GENI I&M Capabilities Catalog identified several available tools at the start of Spiral 2. Throughout Spiral 2, these tools have continued to mature, and several were ready for use by the beginning of Spiral 3. Tutorials covering four available tools were given in the 3rd I&M Workshop at GEC9, and one was given at GEC10. === INSTOOLS: Instrumenting and Monitoring Slices === Jim Griffioen and Hussamuddin Nasir (University of Kentucky) [http://groups.geni.net/geni/attachment/wiki/Gec9InstMeasWorkshopAgenda/INSTOOLS_IM_tutorial_gec9.pdf Slides] [[BR]] === OML Tutorial === Jolyon White (NICTA) [http://groups.geni.net/geni/attachment/wiki/Gec9InstMeasWorkshopAgenda/OML_tutorial.pptx Slides] [[BR]] === perfSONAR Tutorial === Martin Swany and Guilherme Fernandes (University of Delaware) [http://groups.geni.net/geni/wiki/LAMP/Tutorial Tutorial] [[BR]] === OnTimeMeasure Tutorial === Prasad Calyam (The Ohio State University) [http://groups.geni.net/geni/attachment/wiki/Gec9InstMeasWorkshopAgenda/OnTimeMeasure_Tutorial.pdf Slides] [[BR]] === Measurement Data Archive Tutorial === [http://groups.geni.net/geni/attachment/wiki/DigitalObjectRegistry/Measurement%20Data%20Archive%20Prototype.pdf MeasurementData Archive (MDA) prototype design][[BR]] [http://www.cnri.reston.va.us/GENI/docs/GENI_Poster_GEC10.pdf GENI Proposed Services Poster] CNRI displayed this poster during the GEC10 held in San Juan, PR[[BR]] [http://mda.doregistry.org/ Link to Measurement Data Archive Prototype][[BR]] [http://mda.doregistry.org/demo/ Measurement Data Archive Prototype Video] A video demonstration of the MDA Prototype[[BR]] The community is invited to start using these tools, and provide comments, additions or changes to their developers. ---- == Additional References == * [http://groups.geni.net/geni/attachment/wiki/OldGPGDesignDocuments/GDD-06-12.pdf Paul Barford (Ed), "GENI Instrumentation and Measurement Systems (GIMS) Specification," GENI Design Document 06-12, Facility Architecture Working Group, September 2006.] * [wiki:GIMS Early work on GENI Instrumentation and Measurement Service (GIMS)] * [http://groups.geni.net/geni/attachment/wiki/MeasurementSystem/MeasurementSysSpec.pdf "Requirements and Specification for an Instrumentation and Measurement System for GENI", May 11, 2009, Paul Barford et. al.] * Vision for GENI I&M (per Barford on 11/18/09)[[BR]] instrumentation and measurement systems provide broad data gathering, analysis and archieval capability[[BR]] sufficient for scientific mission[[BR]] sufficient for operations[[BR]] key for success of the infrastructure[[BR]] * Requirementsfor for GENI I&M (initial view) (per Barford on 11/18/09)[[BR]] measure details of GENI behavior with high precision and accuracy[[BR]] no impact on experiments[[BR]] ubiquitous[[BR]] extensible[[BR]] large capacity[[BR]] high availability[[BR]] resilient[[BR]] strong access control[[BR]] tight integration with CFs[[BR]] * Conceptual strawman (per Barford on 11/18/09)[[BR]] instrumentation - taps in the metwork and systems that provide basic signals[[BR]] collection and synthesis - programmable systems that collect, combine and transform basic signals[[BR]] archieve - measurement data index and repository[[BR]] * [http://www.datcat.org/ DatCat project], that provides a catalog that indexes Internet measurement data, which you can then find, annotate, and cite. * Contributions on [http://crawdad.cs.dartmouth.edu/ CRAWDAD project] at Dartmouth College , provided via email on March 30, 2009, by David Kotz – Dartmouth College at [wiki:CRAWDAD_Experience]. [[BR]] ---- == New to Instrumentation and Measurement == * [http://groups.geni.net/geni/attachment/wiki/GeniInstMeas/110210%20Day%201%20-%20Newcomers%20-%20WGInstAndMeas.ppt Introduction for Newcomers to GENI I&M WG] * [GeniWorkingGroups GENI Working Groups] * [GeniWikiGettingStarted Getting started on the GENI Wiki] * [GpoDoc Documents published by the GENI Project Office] * [SpiralOne GENI Spiral 1] * [SpiralTwo GENI Spiral 2] * [GeniGlossary GENI Glossary]