Changes between Version 43 and Version 44 of Gec7InstMeasWGAgenda


Ignore:
Timestamp:
03/24/10 18:21:50 (14 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Gec7InstMeasWGAgenda

    v43 v44  
    327327
    328328Purpose:
    329 Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture
    330 Identify the priority topics that the WG needs to address first
    331 Pull together contributions by the WG though Spiral 2
     329Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture
     330Identify the priority topics that the WG needs to address first
     331Pull together contributions by the WG though Spiral 2
    332332
    333333
    334334Plan:
    335 Now :  v0.1 DRAFT completed, by GPO;  see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture
    336 By GEC8:  v0.5 draft, by GPO, with contributions from WG
    337 By GEC9:  v1.0 draft, reviewed by WG
     335Now :  v0.1 DRAFT completed, by GPO;  see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture
     336By GEC8:  v0.5 draft, by GPO, with contributions from WG
     337By GEC9:  v1.0 draft, reviewed by WG
    338338
    339339Document outline:
    340 Document Scope
     3401.  Document Scope
    3413412.  Introduction
    3423423.  Definition and configuration of I&M services
     
    356356
    357357Based on GENI I&M Capabilities Catalog (v0.1), these GENI projects have comprehensive, end-to-end capabilities:
    358 OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework) 
    359         (Ott, NICTA  and Gruteser, WINLAB/Rutgers, 1660)
    360 Instrumentation Tools
    361         (Griffioen, Univ Kentucky, 1642)
    362 perfSONAR for network measurements 
    363          (Zekauskas, I2 and Swany, Univ Delaware, 1788)
    364 Scalable Sensing Service
    365         (Fahmy, Purdue and Sharma, HP Labs, 1723)
    366 OnTimeMeasure
    367         (Calyam, Ohio Super Ctr, 1764)
     358OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework) 
     359-  (Ott, NICTA  and Gruteser, WINLAB/Rutgers, 1660)
     360Instrumentation Tools
     361-  (Griffioen, Univ Kentucky, 1642)
     362perfSONAR for network measurements 
     363-  (Zekauskas, I2 and Swany, Univ Delaware, 1788)
     364Scalable Sensing Service
     365-  (Fahmy, Purdue and Sharma, HP Labs, 1723)
     366OnTimeMeasure
     367-  (Calyam, Ohio Super Ctr, 1764)
    368368
    369369After considering projects with comprehensive, end-to-end capabilities, here are five services they have in common:
    370 
    371 Measurement Orchestration (MO) service
    372 (p/o Experiment Control service, uses a language to orchestrate I&M services)
    373 Measurement Point (MP) service 
    374 (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema)
    375 Measurement Collection (MC) service 
    376 (programmable systems that collect, combine, transform and cache measurement data)
    377 Measurement Analysis and Presentation (MAP) service 
    378 (programmable systems that analyze and then present measurement data)
    379 Measurement Data Archive (MDA) service 
    380 (measurement data repository, index and portal)
     370+  Measurement Orchestration (MO) service
     371-  (p/o Experiment Control service, uses a language to orchestrate I&M services)
     372+  Measurement Point (MP) service 
     373-  (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema)
     374+  Measurement Collection (MC) service 
     375-  (programmable systems that collect, combine, transform and cache measurement data)
     376+  Measurement Analysis and Presentation (MAP) service 
     377-  (programmable systems that analyze and then present measurement data)
     378+  Measurement Data Archive (MDA) service 
     379-  (measurement data repository, index and portal)
    381380
    382381Expected range of implementations: 
    383 
    384 Small-scale implementations might put all I&M services within one aggregate, and even in one server
    385 interfaces between services would be internal to the aggregate, or even internal to the server
    386 
    387 Large-scale implementations might have I&M services distributed over many aggregates
    388 with measurement data flowing between services
    389 with orchestration mechanisms based upon message exchanges
     382+  Small-scale implementations might put all I&M services within one aggregate, and even in one server
     383-  interfaces between services would be internal to the aggregate, or even internal to the server
     384+  Large-scale implementations might have I&M services distributed over many aggregates
     385-  with measurement data flowing between services
     386-  with orchestration mechanisms based upon message exchanges
    390387
    391388
    392389Discussion topics:
    393 Are these five services a complete group of I&M services? 
    394 
    395 Are these good names for the five I&M services?
    396 
    397 Is this five the right granularity for I&M services?
    398 
    399 Is this a complete and flexible configuration for I&M services?
    400 
    401 Can this configuration accommodate the range from small-scale to large-scale implementations?
    402 
    403 How can we obtain a consensus, so that we can set a firm foundation for the other topics?
     390+  Are these five services a complete group of I&M services? 
     391+  Are these good names for the five I&M services?
     392+  Is this five the right granularity for I&M services?
     393+  Is this a complete and flexible configuration for I&M services?
     394+  Can this configuration accommodate the range from small-scale to large-scale implementations?
     395+  How can we obtain a consensus, so that we can set a firm foundation for the other topics?
    404396
    405397Interfaces, protocols and schema for measurement data:
    406 
    407 Issues:
    408 This topic suggested at GEC6 meeting:  Common schema for MD
    409 Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types?
    410 What needs to be included in the MD schema?
    411 
    412 Approach:
    413 Assume all MD after MPs follows this common set of interfaces, protocols and schema
    414 Start with definition of MD schema
    415 Next, understand  [8.  MD Transport via GENI Measurement Plane]
    416 Then, complete first set of interfaces and protocols
     398+  Issues:
     399-  This topic suggested at GEC6 meeting:  Common schema for MD
     400-  Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types?
     401-  What needs to be included in the MD schema?
     402+  Approach:
     403-  Assume all MD after MPs follows this common set of interfaces, protocols and schema
     404-  Start with definition of MD schema
     405-  Next, understand  [8.  MD Transport via GENI Measurement Plane]
     406-  Then, complete first set of interfaces and protocols
    417407
    418408From GENI I&M Capabilities Catalog (v0.1), these GENI projects (and others) are working on data schema and/or data archives:
    419 
    420 perfSONAR for network measurements   (Swany, Univ Delaware, 1788)
    421 IMF project  (Dutta, NC State, 1718)
    422 Embedded Real-Time Measurements  (Bergman, Columbia, 1631)
    423 GENI Meta-Operations Center  (Herron, Indiana Univ, 1604)
    424 netKarma:  GENI Provenance Registry  (Pale and Small, Indiana Univ, 1706)
    425 DatCat project at http://www.datcat.org/   (Klaffy, CAIDA)
    426 Crawdad project at http://crawdad.cs.dartmouth.edu/  (Kotz, Dartmouth)
    427 Amazon Simple Storage Service
    428 Data-Intensive Cloud Control   (Zink and Cecchet, UMass Amherst, 1709    )
    429 Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663)
    430 others?
    431 
    432 What can we learn from these projects?
     409+  perfSONAR for network measurements   (Swany, Univ Delaware, 1788)
     410+  IMF project  (Dutta, NC State, 1718)
     411+  Embedded Real-Time Measurements  (Bergman, Columbia, 1631)
     412+  GENI Meta-Operations Center  (Herron, Indiana Univ, 1604)
     413+  netKarma:  GENI Provenance Registry  (Pale and Small, Indiana Univ, 1706)
     414+  DatCat project at http://www.datcat.org/   (Klaffy, CAIDA)
     415+  Crawdad project at http://crawdad.cs.dartmouth.edu/  (Kotz, Dartmouth)
     416+  Amazon Simple Storage Service
     417+  Data-Intensive Cloud Control   (Zink and Cecchet, UMass Amherst, 1709    )
     418+  Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663)
     419+  others?
     420-  What can we learn from these projects?
    433421
    434422Discussion topics:
    435 
    436 Standardized interfaces between measurement services
    437 Pt-to-pt vs pt-to-multipoint (e.g., pub/sub)
    438 Stream vs bulk transfer
    439 Disconnection operation expected, or not.
    440 
    441 Protocols for moving measurement data
    442 Streaming data
    443 Bulk-transfer of data
    444 
    445 Schema for measurement data
    446 Data record identifier
    447 Annotation, or meta data
    448 Data types and values, with timestamps
    449 
    450 How can we obtain a consensus on first set of intfc’s/protocols/schema for MD?
    451 
    452 What is the process for extending the set?
     423+  Standardized interfaces between measurement services
     424-  Pt-to-pt vs pt-to-multipoint (e.g., pub/sub)
     425-  Stream vs bulk transfer
     426-  Disconnection operation expected, or not.
     427+  Protocols for moving measurement data
     428-  Streaming data
     429-  Bulk-transfer of data
     430+  Schema for measurement data
     431-  Data record identifier
     432-  Annotation, or meta data
     433-  Data types and values, with timestamps
     434+  How can we obtain a consensus on first set of intfc’s/protocols/schema for MD?
     435+  What is the process for extending the set?
    453436
    454437
    455438GENI measurement plane:
    456 
    457 Issue:
    458 Need to understand how MD traffic flows are transported  by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined
    459 
    460 Approach:
    461 Understand current view of GENI Control Plane and Experiment Plane
    462 Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes
    463 
    464 
    465 
    466 
    467 
    468 
    469 
    470 
    471 Priority topics: [[BR]]
    472 +  Common terminology;  best granularity of functions  [[BR]]
    473 +  Measurement data schema;  common after MPs, before and within MCs, MDAs;  what is included in meta-data?  [[BR]]
    474 +  Measurement Plane;  options;  expect nodes with 3 or 2 NICs  [[BR]]
     439+  Issue:
     440-  Need to understand how MD traffic flows are transported  by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined
     441+  Approach:
     442-  Understand current view of GENI Control Plane and Experiment Plane
     443-  Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes
     444
     445
    475446
    476447== Next Steps for WG ==