| 328 | | Purpose: |
| 329 | | + Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture |
| 330 | | + Identify the priority topics that the WG needs to address first |
| 331 | | + Pull together contributions by the WG though Spiral 2 |
| 332 | | |
| 333 | | |
| 334 | | Plan: |
| 335 | | + Now : v0.1 DRAFT completed, by GPO; see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture |
| 336 | | + By GEC8: v0.5 draft, by GPO, with contributions from WG |
| 337 | | + By GEC9: v1.0 draft, reviewed by WG |
| 338 | | |
| 339 | | Document outline: |
| 340 | | 1. Document Scope |
| 341 | | 2. Introduction |
| 342 | | 3. Definition and configuration of I&M services |
| 343 | | 4. Interfaces, protocols and schema for Measurement Data (MD) |
| 344 | | 5. Ownership of MD and privacy of owners |
| 345 | | 6. Interfaces, protocols and APIs for using I&M services |
| 346 | | 7. Basic GENI I&M use cases |
| 347 | | 8. MD transport via the GENI Measurement Plane |
| 348 | | 9. Discovery, authorization, assignment and binding of GENI I&M services |
| 349 | | 10. Measurement Orchestration (MO) service |
| 350 | | 11. Measurement Point (MP) |
| 351 | | 12. Time-stamping MD |
| 352 | | 13. Measurement Collection (MC) service |
| 353 | | 14. Measurement Analysis and Presentation (MAP) service |
| 354 | | 15. Measurement Data Archive (MDA) service |
| 355 | | 16. Additional GENI I&M use cases |
| 356 | | |
| 357 | | Based on GENI I&M Capabilities Catalog (v0.1), these GENI projects have comprehensive, end-to-end capabilities: |
| 358 | | + OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework) |
| 359 | | - (Ott, NICTA and Gruteser, WINLAB/Rutgers, 1660) |
| 360 | | + Instrumentation Tools |
| 361 | | - (Griffioen, Univ Kentucky, 1642) |
| 362 | | + perfSONAR for network measurements |
| 363 | | - (Zekauskas, I2 and Swany, Univ Delaware, 1788) |
| 364 | | + Scalable Sensing Service |
| 365 | | - (Fahmy, Purdue and Sharma, HP Labs, 1723) |
| 366 | | + OnTimeMeasure |
| 367 | | - (Calyam, Ohio Super Ctr, 1764) |
| 368 | | |
| 369 | | After considering projects with comprehensive, end-to-end capabilities, here are five services they have in common: |
| 370 | | + Measurement Orchestration (MO) service |
| 371 | | - (p/o Experiment Control service, uses a language to orchestrate I&M services) |
| 372 | | + Measurement Point (MP) service |
| 373 | | - (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema) |
| 374 | | + Measurement Collection (MC) service |
| 375 | | - (programmable systems that collect, combine, transform and cache measurement data) |
| 376 | | + Measurement Analysis and Presentation (MAP) service |
| 377 | | - (programmable systems that analyze and then present measurement data) |
| 378 | | + Measurement Data Archive (MDA) service |
| 379 | | - (measurement data repository, index and portal) |
| 380 | | |
| 381 | | Expected range of implementations: |
| 382 | | + Small-scale implementations might put all I&M services within one aggregate, and even in one server |
| 383 | | - interfaces between services would be internal to the aggregate, or even internal to the server |
| 384 | | + Large-scale implementations might have I&M services distributed over many aggregates |
| 385 | | - with measurement data flowing between services |
| 386 | | - with orchestration mechanisms based upon message exchanges |
| 387 | | |
| 388 | | |
| 389 | | Discussion topics: |
| 390 | | + Are these five services a complete group of I&M services? |
| 391 | | + Are these good names for the five I&M services? |
| 392 | | + Is this five the right granularity for I&M services? |
| 393 | | + Is this a complete and flexible configuration for I&M services? |
| 394 | | + Can this configuration accommodate the range from small-scale to large-scale implementations? |
| 395 | | + How can we obtain a consensus, so that we can set a firm foundation for the other topics? |
| 396 | | |
| 397 | | Interfaces, protocols and schema for measurement data: |
| 398 | | + Issues: |
| 399 | | - This topic suggested at GEC6 meeting: Common schema for MD |
| 400 | | - Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types? |
| 401 | | - What needs to be included in the MD schema? |
| 402 | | + Approach: |
| 403 | | - Assume all MD after MPs follows this common set of interfaces, protocols and schema |
| 404 | | - Start with definition of MD schema |
| 405 | | - Next, understand [8. MD Transport via GENI Measurement Plane] |
| 406 | | - Then, complete first set of interfaces and protocols |
| 407 | | |
| 408 | | From GENI I&M Capabilities Catalog (v0.1), these GENI projects (and others) are working on data schema and/or data archives: |
| 409 | | + perfSONAR for network measurements (Swany, Univ Delaware, 1788) |
| 410 | | + IMF project (Dutta, NC State, 1718) |
| 411 | | + Embedded Real-Time Measurements (Bergman, Columbia, 1631) |
| 412 | | + GENI Meta-Operations Center (Herron, Indiana Univ, 1604) |
| 413 | | + netKarma: GENI Provenance Registry (Pale and Small, Indiana Univ, 1706) |
| 414 | | + DatCat project at http://www.datcat.org/ (Klaffy, CAIDA) |
| 415 | | + Crawdad project at http://crawdad.cs.dartmouth.edu/ (Kotz, Dartmouth) |
| 416 | | + Amazon Simple Storage Service |
| 417 | | + Data-Intensive Cloud Control (Zink and Cecchet, UMass Amherst, 1709 ) |
| 418 | | + Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663) |
| 419 | | + others? |
| 420 | | - What can we learn from these projects? |
| 421 | | |
| 422 | | Discussion topics: |
| 423 | | + Standardized interfaces between measurement services |
| 424 | | - Pt-to-pt vs pt-to-multipoint (e.g., pub/sub) |
| 425 | | - Stream vs bulk transfer |
| 426 | | - Disconnection operation expected, or not. |
| 427 | | + Protocols for moving measurement data |
| 428 | | - Streaming data |
| 429 | | - Bulk-transfer of data |
| 430 | | + Schema for measurement data |
| 431 | | - Data record identifier |
| 432 | | - Annotation, or meta data |
| 433 | | - Data types and values, with timestamps |
| 434 | | + How can we obtain a consensus on first set of intfc’s/protocols/schema for MD? |
| 435 | | + What is the process for extending the set? |
| 436 | | |
| 437 | | |
| 438 | | GENI measurement plane: |
| 439 | | + Issue: |
| 440 | | - Need to understand how MD traffic flows are transported by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined |
| 441 | | + Approach: |
| 442 | | - Understand current view of GENI Control Plane and Experiment Plane |
| 443 | | - Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes |
| | 328 | Purpose:[[BR]] |
| | 329 | + Provide a comprehensive and ordered list of topics that must be addressed for a complete architecture[[BR]] |
| | 330 | + Identify the priority topics that the WG needs to address first[[BR]] |
| | 331 | + Pull together contributions by the WG though Spiral 2[[BR]] |
| | 332 | |
| | 333 | |
| | 334 | Plan:[[BR]] |
| | 335 | + Now : v0.1 DRAFT completed, by GPO; see http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture [[BR]] |
| | 336 | + By GEC8: v0.5 draft, by GPO, with contributions from WG[[BR]] |
| | 337 | + By GEC9: v1.0 draft, reviewed by WG[[BR]] |
| | 338 | |
| | 339 | Document outline:[[BR]] |
| | 340 | 1. Document Scope[[BR]] |
| | 341 | 2. Introduction[[BR]] |
| | 342 | 3. Definition and configuration of I&M services[[BR]] |
| | 343 | 4. Interfaces, protocols and schema for Measurement Data (MD)[[BR]] |
| | 344 | 5. Ownership of MD and privacy of owners[[BR]] |
| | 345 | 6. Interfaces, protocols and APIs for using I&M services[[BR]] |
| | 346 | 7. Basic GENI I&M use cases[[BR]] |
| | 347 | 8. MD transport via the GENI Measurement Plane[[BR]] |
| | 348 | 9. Discovery, authorization, assignment and binding of GENI I&M services[[BR]] |
| | 349 | 10. Measurement Orchestration (MO) service[[BR]] |
| | 350 | 11. Measurement Point (MP)[[BR]] |
| | 351 | 12. Time-stamping MD[[BR]] |
| | 352 | 13. Measurement Collection (MC) service[[BR]] |
| | 353 | 14. Measurement Analysis and Presentation (MAP) service[[BR]] |
| | 354 | 15. Measurement Data Archive (MDA) service[[BR]] |
| | 355 | 16. Additional GENI I&M use cases[[BR]] |
| | 356 | |
| | 357 | Based on GENI I&M Capabilities Catalog (v0.1), these GENI projects have comprehensive, end-to-end capabilities:[[BR]] |
| | 358 | + OML (ORBIT Measure Library) in OMF (ORBIT Mgmt Framework)[[BR]] |
| | 359 | - (Ott, NICTA and Gruteser, WINLAB/Rutgers, 1660)[[BR]] |
| | 360 | + Instrumentation Tools [[BR]] |
| | 361 | - (Griffioen, Univ Kentucky, 1642)[[BR]] |
| | 362 | + perfSONAR for network measurements [[BR]] |
| | 363 | - (Zekauskas, I2 and Swany, Univ Delaware, 1788)[[BR]] |
| | 364 | + Scalable Sensing Service [[BR]] |
| | 365 | - (Fahmy, Purdue and Sharma, HP Labs, 1723)[[BR]] |
| | 366 | + OnTimeMeasure [[BR]] |
| | 367 | - (Calyam, Ohio Super Ctr, 1764)[[BR]] |
| | 368 | |
| | 369 | After considering projects with comprehensive, end-to-end capabilities, here are five services they have in common:[[BR]] |
| | 370 | + Measurement Orchestration (MO) service [[BR]] |
| | 371 | - (p/o Experiment Control service, uses a language to orchestrate I&M services)[[BR]] |
| | 372 | + Measurement Point (MP) service [[BR]] |
| | 373 | - (instrumentation that taps into a network and/or systems, links and/or nodes, to capture measurement data and format it using a standardized schema)[[BR]] |
| | 374 | + Measurement Collection (MC) service [[BR]] |
| | 375 | - (programmable systems that collect, combine, transform and cache measurement data)[[BR]] |
| | 376 | + Measurement Analysis and Presentation (MAP) service [[BR]] |
| | 377 | - (programmable systems that analyze and then present measurement data)[[BR]] |
| | 378 | + Measurement Data Archive (MDA) service [[BR]] |
| | 379 | - (measurement data repository, index and portal)[[BR]] |
| | 380 | |
| | 381 | Expected range of implementations:[[BR]] |
| | 382 | + Small-scale implementations might put all I&M services within one aggregate, and even in one server[[BR]] |
| | 383 | - interfaces between services would be internal to the aggregate, or even internal to the server[[BR]] |
| | 384 | + Large-scale implementations might have I&M services distributed over many aggregates[[BR]] |
| | 385 | - with measurement data flowing between services[[BR]] |
| | 386 | - with orchestration mechanisms based upon message exchanges[[BR]] |
| | 387 | |
| | 388 | |
| | 389 | Discussion topics:[[BR]] |
| | 390 | + Are these five services a complete group of I&M services? [[BR]] |
| | 391 | + Are these good names for the five I&M services?[[BR]] |
| | 392 | + Is this five the right granularity for I&M services?[[BR]] |
| | 393 | + Is this a complete and flexible configuration for I&M services?[[BR]] |
| | 394 | + Can this configuration accommodate the range from small-scale to large-scale implementations?[[BR]] |
| | 395 | + How can we obtain a consensus, so that we can set a firm foundation for the other topics?[[BR]] |
| | 396 | |
| | 397 | Interfaces, protocols and schema for measurement data:[[BR]] |
| | 398 | + Issues:[[BR]] |
| | 399 | - This topic suggested at GEC6 meeting: Common schema for MD[[BR]] |
| | 400 | - Can we identify a common set of interfaces, protocols and schema for MD, or at least a limited number of types?[[BR]] |
| | 401 | - What needs to be included in the MD schema?[[BR]] |
| | 402 | + Approach:[[BR]] |
| | 403 | - Assume all MD after MPs follows this common set of interfaces, protocols and schema[[BR]] |
| | 404 | - Start with definition of MD schema[[BR]] |
| | 405 | - Next, understand [8. MD Transport via GENI Measurement Plane][[BR]] |
| | 406 | - Then, complete first set of interfaces and protocols[[BR]] |
| | 407 | |
| | 408 | From GENI I&M Capabilities Catalog (v0.1), these GENI projects (and others) are working on data schema and/or data archives:[[BR]] |
| | 409 | + perfSONAR for network measurements (Swany, Univ Delaware, 1788)[[BR]] |
| | 410 | + IMF project (Dutta, NC State, 1718)[[BR]] |
| | 411 | + Embedded Real-Time Measurements (Bergman, Columbia, 1631)[[BR]] |
| | 412 | + GENI Meta-Operations Center (Herron, Indiana Univ, 1604)[[BR]] |
| | 413 | + netKarma: GENI Provenance Registry (Pale and Small, Indiana Univ, 1706)[[BR]] |
| | 414 | + DatCat project at http://www.datcat.org/ (Klaffy, CAIDA)[[BR]] |
| | 415 | + Crawdad project at http://crawdad.cs.dartmouth.edu/ (Kotz, Dartmouth)[[BR]] |
| | 416 | + Amazon Simple Storage Service[[BR]] |
| | 417 | + Data-Intensive Cloud Control (Zink and Cecchet, UMass Amherst, 1709 )[[BR]] |
| | 418 | + Experiment Mgmt System (Lannom and Manepalli, CNRI, 1663)[[BR]] |
| | 419 | + others?[[BR]] |
| | 420 | - What can we learn from these projects?[[BR]] |
| | 421 | |
| | 422 | Discussion topics:[[BR]] |
| | 423 | + Standardized interfaces between measurement services[[BR]] |
| | 424 | - Pt-to-pt vs pt-to-multipoint (e.g., pub/sub)[[BR]] |
| | 425 | - Stream vs bulk transfer[[BR]] |
| | 426 | - Disconnection operation expected, or not.[[BR]] |
| | 427 | + Protocols for moving measurement data[[BR]] |
| | 428 | - Streaming data[[BR]] |
| | 429 | - Bulk-transfer of data[[BR]] |
| | 430 | + Schema for measurement data[[BR]] |
| | 431 | - Data record identifier[[BR]] |
| | 432 | - Annotation, or meta data[[BR]] |
| | 433 | - Data types and values, with timestamps[[BR]] |
| | 434 | + How can we obtain a consensus on first set of intfc’s/protocols/schema for MD?[[BR]] |
| | 435 | + What is the process for extending the set?[[BR]] |
| | 436 | |
| | 437 | |
| | 438 | GENI measurement plane:[[BR]] |
| | 439 | + Issue: [[BR]] |
| | 440 | - Need to understand how MD traffic flows are transported by the GENI Measurement Plane before the interfaces and protocols for MD can be fully defined[[BR]] |
| | 441 | + Approach:[[BR]] |
| | 442 | - Understand current view of GENI Control Plane and Experiment Plane[[BR]] |
| | 443 | - Consider options for GENI Measurement Plane to transport MD flows, using networks that implement GENI Control and Experiment Planes[[BR]] |
