Changes between Version 44 and Version 45 of Gec7InstMeasWGAgenda


Ignore:
Timestamp:
03/24/10 18:24:57 (10 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Gec7InstMeasWGAgenda

    v44 v45  
    319319
    320320==  GENI I&M Architecture ==
    321 4:55pm  Harry Mussman  (GPO)
     3214:55pm  Harry Mussman  (GPO) [[BR]]
    322322[http://groups.geni.net/geni/attachment/wiki/Gec7InstMeasWGAgenda/031210%20%20IM-ARCH-GEC7Slides.pdf  slides] [[BR]]
    323323[http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture  GENI I&M Architecture document] (15min) [[BR]]
     
    326326
    327327
    328 Purpose:
    329 +  Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture
    330 +  Identify the priority topics that the WG needs to address first
    331 +  Pull together contributions by the WG though Spiral 2
    332 
    333 
    334 Plan:
    335 +  Now :  v0.1 DRAFT completed, by GPO;  see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture
    336 +  By GEC8:  v0.5 draft, by GPO, with contributions from WG
    337 +  By GEC9:  v1.0 draft, reviewed by WG
    338 
    339 Document outline:
    340 1.  Document Scope
    341 2.  Introduction
    342 3.  Definition and configuration of I&M services
    343 4.  Interfaces, protocols and schema for Measurement Data (MD)
    344 5.  Ownership of MD and privacy of owners
    345 6.  Interfaces, protocols and APIs for using I&M services
    346 7.  Basic GENI I&M use cases
    347 8.  MD transport via the GENI Measurement Plane
    348 9.  Discovery, authorization, assignment and binding of GENI I&M services
    349 10.  Measurement Orchestration (MO) service
    350 11.  Measurement Point (MP)
    351 12.  Time-stamping MD
    352 13. Measurement Collection (MC) service
    353 14.  Measurement Analysis and Presentation (MAP) service
    354 15.  Measurement Data Archive (MDA) service
    355 16.  Additional GENI I&M use cases
    356 
    357 Based on GENI I&M Capabilities Catalog (v0.1), these GENI projects have comprehensive, end-to-end capabilities:
    358 +  OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework) 
    359 -  (Ott, NICTA  and Gruteser, WINLAB/Rutgers, 1660)
    360 +  Instrumentation Tools
    361 -  (Griffioen, Univ Kentucky, 1642)
    362 +  perfSONAR for network measurements 
    363 -  (Zekauskas, I2 and Swany, Univ Delaware, 1788)
    364 +  Scalable Sensing Service
    365 -  (Fahmy, Purdue and Sharma, HP Labs, 1723)
    366 +  OnTimeMeasure
    367 -  (Calyam, Ohio Super Ctr, 1764)
    368 
    369 After considering projects with comprehensive, end-to-end capabilities, here are five services they have in common:
    370 +  Measurement Orchestration (MO) service
    371 -  (p/o Experiment Control service, uses a language to orchestrate I&M services)
    372 +  Measurement Point (MP) service
    373 -  (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema)
    374 +  Measurement Collection (MC) service
    375 -  (programmable systems that collect, combine, transform and cache measurement data)
    376 +  Measurement Analysis and Presentation (MAP) service 
    377 -  (programmable systems that analyze and then present measurement data)
    378 +  Measurement Data Archive (MDA) service 
    379 -  (measurement data repository, index and portal)
    380 
    381 Expected range of implementations: 
    382 +  Small-scale implementations might put all I&M services within one aggregate, and even in one server
    383 -  interfaces between services would be internal to the aggregate, or even internal to the server
    384 +  Large-scale implementations might have I&M services distributed over many aggregates
    385 -  with measurement data flowing between services
    386 -  with orchestration mechanisms based upon message exchanges
    387 
    388 
    389 Discussion topics:
    390 +  Are these five services a complete group of I&M services? 
    391 +  Are these good names for the five I&M services?
    392 +  Is this five the right granularity for I&M services?
    393 +  Is this a complete and flexible configuration for I&M services?
    394 +  Can this configuration accommodate the range from small-scale to large-scale implementations?
    395 +  How can we obtain a consensus, so that we can set a firm foundation for the other topics?
    396 
    397 Interfaces, protocols and schema for measurement data:
    398 +  Issues:
    399 -  This topic suggested at GEC6 meeting:  Common schema for MD
    400 -  Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types?
    401 -  What needs to be included in the MD schema?
    402 +  Approach:
    403 -  Assume all MD after MPs follows this common set of interfaces, protocols and schema
    404 -  Start with definition of MD schema
    405 -  Next, understand  [8.  MD Transport via GENI Measurement Plane]
    406 -  Then, complete first set of interfaces and protocols
    407 
    408 From GENI I&M Capabilities Catalog (v0.1), these GENI projects (and others) are working on data schema and/or data archives:
    409 +  perfSONAR for network measurements   (Swany, Univ Delaware, 1788)
    410 +  IMF project  (Dutta, NC State, 1718)
    411 +  Embedded Real-Time Measurements  (Bergman, Columbia, 1631)
    412 +  GENI Meta-Operations Center  (Herron, Indiana Univ, 1604)
    413 +  netKarma:  GENI Provenance Registry  (Pale and Small, Indiana Univ, 1706)
    414 +  DatCat project at http://www.datcat.org/   (Klaffy, CAIDA)
    415 +  Crawdad project at http://crawdad.cs.dartmouth.edu/  (Kotz, Dartmouth)
    416 +  Amazon Simple Storage Service
    417 +  Data-Intensive Cloud Control   (Zink and Cecchet, UMass Amherst, 1709    )
    418 +  Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663)
    419 +  others?
    420 -  What can we learn from these projects?
    421 
    422 Discussion topics:
    423 +  Standardized interfaces between measurement services
    424 -  Pt-to-pt vs pt-to-multipoint (e.g., pub/sub)
    425 -  Stream vs bulk transfer
    426 -  Disconnection operation expected, or not.
    427 +  Protocols for moving measurement data
    428 -  Streaming data
    429 -  Bulk-transfer of data
    430 +  Schema for measurement data
    431 -  Data record identifier
    432 -  Annotation, or meta data
    433 -  Data types and values, with timestamps
    434 +  How can we obtain a consensus on first set of intfc’s/protocols/schema for MD?
    435 +  What is the process for extending the set?
    436 
    437 
    438 GENI measurement plane:
    439 +  Issue:
    440 -  Need to understand how MD traffic flows are transported  by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined
    441 +  Approach:
    442 -  Understand current view of GENI Control Plane and Experiment Plane
    443 -  Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes
     328Purpose:[[BR]]
     329+  Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture[[BR]]
     330+  Identify the priority topics that the WG needs to address first[[BR]]
     331+  Pull together contributions by the WG though Spiral 2[[BR]]
     332
     333
     334Plan:[[BR]]
     335+  Now :  v0.1 DRAFT completed, by GPO;  see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture [[BR]]
     336+  By GEC8:  v0.5 draft, by GPO, with contributions from WG[[BR]]
     337+  By GEC9:  v1.0 draft, reviewed by WG[[BR]]
     338
     339Document outline:[[BR]]
     3401.  Document Scope[[BR]]
     3412.  Introduction[[BR]]
     3423.  Definition and configuration of I&M services[[BR]]
     3434.  Interfaces, protocols and schema for Measurement Data (MD)[[BR]]
     3445.  Ownership of MD and privacy of owners[[BR]]
     3456.  Interfaces, protocols and APIs for using I&M services[[BR]]
     3467.  Basic GENI I&M use cases[[BR]]
     3478.  MD transport via the GENI Measurement Plane[[BR]]
     3489.  Discovery, authorization, assignment and binding of GENI I&M services[[BR]]
     34910.  Measurement Orchestration (MO) service[[BR]]
     35011.  Measurement Point (MP)[[BR]]
     35112.  Time-stamping MD[[BR]]
     35213. Measurement Collection (MC) service[[BR]]
     35314.  Measurement Analysis and Presentation (MAP) service[[BR]]
     35415.  Measurement Data Archive (MDA) service[[BR]]
     35516.  Additional GENI I&M use cases[[BR]]
     356
     357Based on GENI I&M Capabilities Catalog (v0.1), these GENI projects have comprehensive, end-to-end capabilities:[[BR]]
     358+  OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework)[[BR]] 
     359-  (Ott, NICTA  and Gruteser, WINLAB/Rutgers, 1660)[[BR]]
     360+  Instrumentation Tools [[BR]]
     361-  (Griffioen, Univ Kentucky, 1642)[[BR]]
     362+  perfSONAR for network measurements  [[BR]]
     363-  (Zekauskas, I2 and Swany, Univ Delaware, 1788)[[BR]]
     364+  Scalable Sensing Service [[BR]]
     365-  (Fahmy, Purdue and Sharma, HP Labs, 1723)[[BR]]
     366+  OnTimeMeasure [[BR]]
     367-  (Calyam, Ohio Super Ctr, 1764)[[BR]]
     368
     369After considering projects with comprehensive, end-to-end capabilities, here are five services they have in common:[[BR]]
     370+  Measurement Orchestration (MO) service [[BR]]
     371-  (p/o Experiment Control service, uses a language to orchestrate I&M services)[[BR]]
     372+  Measurement Point (MP) service [[BR]]
     373-  (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema)[[BR]]
     374+  Measurement Collection (MC) service [[BR]]
     375-  (programmable systems that collect, combine, transform and cache measurement data)[[BR]]
     376+  Measurement Analysis and Presentation (MAP) service  [[BR]]
     377-  (programmable systems that analyze and then present measurement data)[[BR]]
     378+  Measurement Data Archive (MDA) service  [[BR]]
     379-  (measurement data repository, index and portal)[[BR]]
     380
     381Expected range of implementations:[[BR]] 
     382+  Small-scale implementations might put all I&M services within one aggregate, and even in one server[[BR]]
     383-  interfaces between services would be internal to the aggregate, or even internal to the server[[BR]]
     384+  Large-scale implementations might have I&M services distributed over many aggregates[[BR]]
     385-  with measurement data flowing between services[[BR]]
     386-  with orchestration mechanisms based upon message exchanges[[BR]]
     387
     388
     389Discussion topics:[[BR]]
     390+  Are these five services a complete group of I&M services?  [[BR]]
     391+  Are these good names for the five I&M services?[[BR]]
     392+  Is this five the right granularity for I&M services?[[BR]]
     393+  Is this a complete and flexible configuration for I&M services?[[BR]]
     394+  Can this configuration accommodate the range from small-scale to large-scale implementations?[[BR]]
     395+  How can we obtain a consensus, so that we can set a firm foundation for the other topics?[[BR]]
     396
     397Interfaces, protocols and schema for measurement data:[[BR]]
     398+  Issues:[[BR]]
     399-  This topic suggested at GEC6 meeting:  Common schema for MD[[BR]]
     400-  Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types?[[BR]]
     401-  What needs to be included in the MD schema?[[BR]]
     402+  Approach:[[BR]]
     403-  Assume all MD after MPs follows this common set of interfaces, protocols and schema[[BR]]
     404-  Start with definition of MD schema[[BR]]
     405-  Next, understand  [8.  MD Transport via GENI Measurement Plane][[BR]]
     406-  Then, complete first set of interfaces and protocols[[BR]]
     407
     408From GENI I&M Capabilities Catalog (v0.1), these GENI projects (and others) are working on data schema and/or data archives:[[BR]]
     409+  perfSONAR for network measurements   (Swany, Univ Delaware, 1788)[[BR]]
     410+  IMF project  (Dutta, NC State, 1718)[[BR]]
     411+  Embedded Real-Time Measurements  (Bergman, Columbia, 1631)[[BR]]
     412+  GENI Meta-Operations Center  (Herron, Indiana Univ, 1604)[[BR]]
     413+  netKarma:  GENI Provenance Registry  (Pale and Small, Indiana Univ, 1706)[[BR]]
     414+  DatCat project at http://www.datcat.org/   (Klaffy, CAIDA)[[BR]]
     415+  Crawdad project at http://crawdad.cs.dartmouth.edu/  (Kotz, Dartmouth)[[BR]]
     416+  Amazon Simple Storage Service[[BR]]
     417+  Data-Intensive Cloud Control   (Zink and Cecchet, UMass Amherst, 1709    )[[BR]]
     418+  Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663)[[BR]]
     419+  others?[[BR]]
     420-  What can we learn from these projects?[[BR]]
     421
     422Discussion topics:[[BR]]
     423+  Standardized interfaces between measurement services[[BR]]
     424-  Pt-to-pt vs pt-to-multipoint (e.g., pub/sub)[[BR]]
     425-  Stream vs bulk transfer[[BR]]
     426-  Disconnection operation expected, or not.[[BR]]
     427+  Protocols for moving measurement data[[BR]]
     428-  Streaming data[[BR]]
     429-  Bulk-transfer of data[[BR]]
     430+  Schema for measurement data[[BR]]
     431-  Data record identifier[[BR]]
     432-  Annotation, or meta data[[BR]]
     433-  Data types and values, with timestamps[[BR]]
     434+  How can we obtain a consensus on first set of intfc’s/protocols/schema for MD?[[BR]]
     435+  What is the process for extending the set?[[BR]]
     436
     437
     438GENI measurement plane:[[BR]]
     439+  Issue: [[BR]]
     440-  Need to understand how MD traffic flows are transported  by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined[[BR]]
     441+  Approach:[[BR]]
     442-  Understand current view of GENI Control Plane and Experiment Plane[[BR]]
     443-  Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes[[BR]]
    444444
    445445