Version 19 (modified by 14 years ago) (diff) | ,
---|
-
2nd GENI Instrumentation and Measurement Workshop
- Announcement with goals, topics and reference material
- Attendees at workshop
- Announcement with notes
-
Priority topics
- Topic 1 GENI I&M Use Cases
- Topic 2 GENI I&M Services
- Topic 3 GENI I&M Resources
- Topic 4 GENI I&M Measurement Plane and Interfaces
- Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services
- Topic 6 GENI I&M Interfaces and Protocols (APIs): Data Flows and …
- Topic 7 GENI I&M Interfaces and Protocols (APIs): Registration and …
- Topic 8 GENI I&M Interfaces and Protocols (APIs): GUIs
- Topic 9 GENI Measurement Data Schema
- Text
- Figures
- References
2nd GENI Instrumentation and Measurement Workshop
Tuesday, June 8, 1pm - Wednesday, June 9, 2pm
Chicago O'Hare Hilton
NOTE: By invitation only
Announcement with goals, topics and reference material
The following announcement (with figures and references) was sent to the attendees to prepare for the workshop:
Announcement, v1.1 (060410)
Figures, v1.1 (060410)
Attendees at workshop
(Attended workshop: yes or no)
Paul Barford - University of Wisconsin – Madison (no)
Bruce Maggs – Duke University and Akamai (yes)
Harry Mussman – BBN/GPO (yes)
Vic Thomas - BBN/GPO (yes)
Evan Zhang – BBN/GPO (yes)
OML (ORBIT Measurement Library) OMF (ORBIT Management Framework)
Max Ott – NICTA (yes, by phone)
Ivan Seskar – Rutgers WINLAB (yes)
Instrumentation Tools
Jim Griffioen - Univ Kentucky (yes)
perfSONAR
Matt Zekauskas - Internet2 (no)
Jason Zurawski – Internet2 (yes)
Martin Swany - Univ Delaware (yes)
Guilherme Fernandes – Univ Delaware (yes)
Ezra Kissel – Univ Delaware (yes)
Scalable Sensing Service (S3)
Sonia Fahmy – Purdue (yes)
Puneet Sharma - HP Labs (yes)
OnTimeMeasure for network measurements
Prasad Calyam - Ohio Supercomputing Ctr (yes)
GENI Meta-Operations Center and NetKarma
Jon-Paul Herron - Indiana Univ
Camilo Viecco - Indiana Univ (yes)
Chris Small - Indiana Univ (yes)
Beth Plale - Indiana Univ (no)
Virtual Machine Introspection (VMI)
Brian Hay – Univ Alaska (yes)
Data-Intensive Cloud Control for GENI
Michael Zink - UMass Amherst (yes)
Experiment Management Service – Digital Object Registry
Jim French - CNRI (yes)
Giridhar Manepalli - CNRI (yes)
Larry Lannom – CNRI (no)
Announcement with notes
The following announcement now includes all notes from the workshop:
Announcement, v1.2 (062510) word file
[Figures, v1.2 (062510)]
Priority topics
The following priority topics were identified at the workshop, and teams of attendees (and some non-attendees0 were identified for each topic to discuss it, and write a text summary for review by the WG at GEC8.
Topic 1 GENI I&M Use Cases
Team members:
Paul Barford - University of Wisconsin – Madison (no)
Jim Griffioen - Univ Kentucky (yes)
Prasad Calyam* - Ohio Supercomputing Ctr (yes)
Camilo Viecco - Indiana Univ (yes)
Brian Hay – Univ Alaska (yes)
*agreed to organize first writing and discussion
Identify all user groups, and provide basic use cases:
- Experiment researchers
- Experiment (opt-in) users (see http://groups.geni.net/geni/attachment/wiki/041409NYCOptInWGAgenda/071509%20%20GENI-SE-OI-Overview-01.4.pdf for listing of opt-in issues, such as privacy)
- Central (i.e., GMOC) operators
- Cluster and aggregate providers and operators
- Archive service providers and operators
- Researchers that use archived measurement data, archived by other researchers (DatCat model)
Topic 2 GENI I&M Services
Team members:
Harry Mussman* – BBN/GPO (yes)
Evan Zhang – BBN/GPO
Giridhar Manepalli - CNRI (yes)
Chris Small - Indiana Univ (yes)
Beth Plale - Indiana Univ (yes)
*agreed to organize first writing and discussion
Summarize current view of:
Measurement Orchestration (MO) Service
Measurement Point (MP) Service
Measurement Information (MI) Service
Measurement Collection (MC) Service
Measurement Analysis and Presentation (MAP) Service
Measurement Data Archive (MDA) Service
Need: Basic definition of a Measurement Data Archive (MDA) Service
Identify different types of services:
Type 1: Dedicated service platform, with dedicated sliver, for customized information.
(Completely dedicated to an experiment)
Type 2: Common service platform, with dedicated slivers, for customized information.
(Common portion, plus parts associated with different experiments)
Type 3: Common service, for common or customized information.
(Common service, with data provided to multiple experiments)
Topic 3 GENI I&M Resources
Team members:
Vic Thomas - BBN/GPO (yes)
Jim Griffioen* - Univ Kentucky (yes)
Martin Swany - Univ Delaware (yes)
Camilo Viecco - Indiana Univ (yes)
Brian Hay – Univ Alaska (yes)
Giridhar Manepalli - CNRI (yes)
*agreed to organize first writing and discussion
Significant question uncovered at workshop:
Jim on 6/25 via email: We should involve Rob Ricci in the discussion.
Consider these resources for I&M capabilities:
- Hosts, VMs, etc.
- Network connectivity
- Software, e.g., I&M software that can be included in an experiment
- I&M services
- I&M data flows and file transfers
- I&M data files stored in archives
For each item, consider how to:
Create
Name
Register and discover
Authorize and assign
Does each item have:
Unique and persistent name?
Unique and persistent identifier?
Need to carefully consider this for all of GENI
For each item, consider:
Ownership
What sort of policies the owner may want to apply
How are each of these discovered, specified, authorized and assigned:
Always by mechanisms provided by the CF? With CF plus additional mechanisms?
Consider example of LS in perfSONAR
Consider example of data file stored in archive, owned by an experimenter Need to define and then compare these options
Need to understand interop with CF for each option
Does CF setup secondary authorization mechanisms in some cases? If so, how?
Topic 4 GENI I&M Measurement Plane and Interfaces
Team members:
Harry Mussman* – BBN/GPO (yes)
Ezra Kissel – Univ Delaware (yes)
Chris Small - Indiana Univ (yes)
*agreed to organize first writing and discussion
Consider:
IP network
Layer 2 (VLAN) connections
Discuss:
Which protocols are active
Access to resources in aggregates, even when resources are in private address space, via GWs or proxies
How to provide authentication and authorization
How to provide QoS to protect measurement traffic
How to provide QoS to protect other traffic when measurement traffic is large.
Reserve bandwidth?
Martin on 6/28: Consider XSP (extensible session protocol) to provide transport layer GW functions.
Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services
Team members:
Vic Thomas - BBN/GPO (yes)
Ivan Seskar – Rutgers WINLAB (yes)
Max Ott – NICTA (yes, by phone)
Sonia Fahmy* – Purdue (yes)
Giridhar Manepalli - CNRI (yes)
*agreed to organize first discussion and writing
Define an approach based on OMF/OML and S3:
HTTP(S)
REST vs SOAP
Authorization by credentials or ? If credentials, how to revoke?
Pass XML fragments
Define basic API
Topic 6 GENI I&M Interfaces and Protocols (APIs): Data Flows and Data File Transfers
Team members:
Harry Mussman* – BBN/GPO (yes)
Ivan Seskar – Rutgers WINLAB (yes)
Max Ott – NICTA (yes, by phone)
Ezra Kissel – Univ Delaware (yes)
Prasad Calyam - Ohio Supercomputing Ctr (yes)
Michael Zink - UMass Amherst (yes)
*agreed to organize first writing and discussion
Consider data flows and data file transfers between all services
Define range of options:
What:
Data flows
Data files transfers
Type:
Pull
Push
Pub/Sub
Protocol:
SNMP
SCP
FTP and gridFTP
HTTP
XMPP
TCP
SCTP
Consider:
Naming
Discovery
Connectivity
Authentication and authorization mechanisms
Map to current projects, giving examples:
Consider: Minimum set required for GENI
Topic 7 GENI I&M Interfaces and Protocols (APIs): Registration and Discovery of Services with Available Measurement Data
Team members:
Jason Zurawski* – Internet2 (yes)
Prasad Calyam - Ohio Supercomputing Ctr (yes)
*agreed to organize first writing and discussion
Consider approach used in perfSONAR
Summarize for:
Services with data flows
Also sources of file transfers?
Also GUIs?
Topic 8 GENI I&M Interfaces and Protocols (APIs): GUIs
Team members:
Jeremy Reed - Univ Kentucky (yes)
Guilherme Fernandes* – Univ Delaware (yes)
Puneet Sharma - HP Labs (yes)
*Agreed to organize team
Define overall goals for GENI GUIs
Types of GUIs:
Control experiments
Display I&M results
Report status
View archive service
Consider portal, for access to multiple GUIs
Consider need for authentication and authroization
Topic 9 GENI Measurement Data Schema
Team members:
Bruce Maggs – Duke University and Akamai (yes)
Max Ott – NICTA (yes, by phone)
Ivan Seskar – Rutgers WINLAB (yes)
Martin Swany* - Univ Delaware (yes)
Camilo Viecco - Indiana Univ (yes)
Michael Zink - UMass Amherst (yes)
Jim French - CNRI (yes)
*agreed to organize first discussion and writing
Consider:
Measurement data schema
Metadata schema
Metadata contents
Consider measurement data schema and/or metadata schema from:
perfSONAR
GMOC-provided
Current OML
Proposed using IPFIX
NetCDF (as used by DI Cloud)
Consider: Minimum set required for GENI
Provide overall template for GENI metadata, considering above.
Which items in GENI metadata template are:
Required?
Invariant?
Text
All text, word and PDF
Figures
060310 v1.2 All I&M figures
All figures, visio and PDF
Individual figures, visio and PDF
References
All references
Individual references
Ref GIMS_Design_UseCases: “Use-cases for GENI Instrumentation and Measurement Architecture Design”
Ref MeasPlane-1: “RESTful Web Services vs. “Big” Web Services: Making the Right Architectural Decision”
Ref OMF_OML-1: “XDR: External Data Representation Standard”
Ref OMF_OML-2: “ORBIT Measurements Framework and Library (OML): Motivations, Design, Implementation, and Features”
Ref OMF-OML-3: “OML Overview” slides
Ref OMF-OML-4: “Measurement Architectures for Network Experiments with Disconnected Mobile Nodes”
Ref InsTools-1: “Architectural Design and Specification of the INSTOOLS Measurement System”
Ref perfSONAR-1: “Scalable Framework for Representation and Exchange of Network Measurements”
Ref perfSONAR-2: “An Extensible Schema for Network Measurement and Performance Data”
Ref perfSONAR-3: “NM-WG/perfSONAR Topology Schema”
Ref GMOC-1: “GMOC Topology-Entity Data Exchange Format Specification”
Ref GMOC-2: “Proposal: Use of URN's as GENI Identifiers”
Attachments (78)
- 042810 InstMeasWorkshopAgenda.pdf (113.3 KB) - added by 15 years ago.
- 051810 InstMeasWorkshopAgenda.pdf (115.5 KB) - added by 15 years ago.
- 060410b InstMeasWorkshopAgenda.pdf (558.3 KB) - added by 14 years ago.
- 060410b InstMeasWorkshopAgenda.doc (2.7 MB) - added by 14 years ago.
- 060310 All Fig IM-ARCH-Figures.pdf (826.2 KB) - added by 14 years ago.
- 060310 IM-ARCH-Figures.vsd (5.4 MB) - added by 14 years ago.
- 062510 InstMeasWorkshopAgenda.doc (2.7 MB) - added by 14 years ago.
- 062510 InstMeasWorkshopAgenda.pdf (642.6 KB) - added by 14 years ago.
- 060310 IM-ARCH-Figures.2.vsd (5.4 MB) - added by 14 years ago.
- 060310Fig1-1_SrvcsRes_IM-ARCH-Figures.jpg (171.1 KB) - added by 14 years ago.
- 060310Fig1-1_SrvcsRes_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig1-2_SrvcsOps_IM-ARCH-Figures.jpg (166.0 KB) - added by 14 years ago.
- 060310Fig1-2_SrvcsOps_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig1-3_SrvcsBoth_ IM-ARCH-Figures.jpg (207.0 KB) - added by 14 years ago.
- 060310Fig1-3_SrvcsBoth_ IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-1_SrvcsOMFOML_IM-ARCH-Figures.jpg (168.7 KB) - added by 14 years ago.
- 060310Fig2-1_SrvcsOMFOML_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-2_SrvcsInsTools_IM-ARCH-Figures.jpg (171.1 KB) - added by 14 years ago.
- 060310Fig2-2_SrvcsInsTools_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-3_Srvcs_perfSONAR_IM-ARCH-Figures.jpg (166.0 KB) - added by 14 years ago.
- 060310Fig2-3_Srvcs_perfSONAR_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-4_SrvcsS3_IM-ARCH-Figures.jpg (166.0 KB) - added by 14 years ago.
- 060310Fig2-4_SrvcsS3_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-5_OnTimeMeas_IM-ARCH-Figures.jpg (166.0 KB) - added by 14 years ago.
- 060310Fig2-5_OnTimeMeas_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-6_DICloud_IM-ARCH-Figures.jpg (162.9 KB) - added by 14 years ago.
- 060310Fig2-6_DICloud_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig2-7_DOR_IM-ARCH-Figures.jpg (174.4 KB) - added by 14 years ago.
- 060310Fig2-7_DOR_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig3-1_MeasTrafficFlows_IM-ARCH-Figures.jpg (140.9 KB) - added by 14 years ago.
- 060310Fig3-1_MeasTrafficFlows_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig3-2_measTrafPorxies_IM-ARCH-Figures.jpg (129.3 KB) - added by 14 years ago.
- 060310Fig3-2_measTrafPorxies_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig4-1_OMLSrvcs_IM-ARCH-Figures.jpg (213.5 KB) - added by 14 years ago.
- 060310Fig4-1_OMLSrvcs_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig4-2_OMLArch_IM-ARCH-Figures.jpg (60.4 KB) - added by 14 years ago.
- 060310Fig4-2_OMLArch_IM-ARCH-Figures.vsd (1.5 MB) - added by 14 years ago.
- 060310Fig4-3_OMLOverview_IM-ARCH-Figures.jpg (168.1 KB) - added by 14 years ago.
- 060310Fig4-3_OMLOverview_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig4-4_OMLNtwk_IM-ARCH-Figures.jpg (149.2 KB) - added by 14 years ago.
- 060310Fig4-4_OMLNtwk_IM-ARCH-Figures.vsd (1.8 MB) - added by 14 years ago.
- 060310Fig5-1_InstToolsSrvcs_IM-ARCH-Figures.jpg (177.1 KB) - added by 14 years ago.
- 060310Fig5-1_InstToolsSrvcs_IM-ARCH-Figures.vsd (1.5 MB) - added by 14 years ago.
- 060310Fig5-2_InstToolsComps_IM-ARCH-Figures.jpg (115.0 KB) - added by 14 years ago.
- 060310Fig5-2_InstToolsComps_IM-ARCH-Figures.vsd (1.6 MB) - added by 14 years ago.
- 060310Fig5-3_InstToolsToplogy_IM-ARCH-Figures.jpg (88.5 KB) - added by 14 years ago.
- 060310Fig5-3_InstToolsToplogy_IM-ARCH-Figures.vsd (1.9 MB) - added by 14 years ago.
- 060310Fig6-1_perfSONARSrvcs_IM-ARCH-Figures.jpg (171.2 KB) - added by 14 years ago.
- 060310Fig6-1_perfSONARSrvcs_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig6-2_perfSONARSchema_IM-ARCH-Figures.jpg (83.3 KB) - added by 14 years ago.
- 060310Fig6-2_perfSONARSchema_IM-ARCH-Figures.vsd (1.6 MB) - added by 14 years ago.
- 060310Fig7-1_S3Srvcs_IM-ARCH-Figures.jpg (103.8 KB) - added by 14 years ago.
- 060310Fig7-1_S3Srvcs_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig8-1_OnTimeMeasSrvcs_IM-ARCH-Figures.jpg (87.4 KB) - added by 14 years ago.
- 060310Fig8-1_OnTimeMeasSrvcs_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig10-1_DORSrvcs_IM-ARCH-Figures.jpg (174.4 KB) - added by 14 years ago.
- 060310Fig10-1_DORSrvcs_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 060310Fig10-2_DORFileOrg_IM-ARCH-Figures.jpg (75.6 KB) - added by 14 years ago.
- 060310Fig10-2_DORFileOrg_IM-ARCH-Figures.vsd (1.4 MB) - added by 14 years ago.
- 062510_All_IM-ARCH-Figures.vsd (5.4 MB) - added by 14 years ago.
- 063010Fig9-1_DICloudSrvcs_IM-ARCH-Fig.jpg (181.5 KB) - added by 14 years ago.
- 063010Fig9-1_DICloudSrvcs_IM-ARCH-Fig.vsd (1.4 MB) - added by 14 years ago.
- Visio-062510_All_IM-ARCH-Figures.pdf (959.4 KB) - added by 14 years ago.
- 060410c All References InstMeasWorkshop.pdf (1.7 MB) - added by 14 years ago.
- GIMS_Design_UseCases_033110.pdf (36.6 KB) - added by 14 years ago.
- GIMS_Design_UseCases_033110.ppt (179.0 KB) - added by 14 years ago.
- ref GMOC-1 data_exchange_format-posted.pdf (106.9 KB) - added by 14 years ago.
- ref GMOC-2 urn-proposal.pdf (162.2 KB) - added by 14 years ago.
- ref InstTools-1 instools-design-doc.pdf (759.4 KB) - added by 14 years ago.
- ref MeasPlane-1 www2008-restws-pautasso-zimmermann-leymann.pdf (123.9 KB) - added by 14 years ago.
- ref OMF_OML-2 final-oml-paper.pdf (85.8 KB) - added by 14 years ago.
- ref OMF_OML-3 ott OML - GEC7 - march.pdf (456.9 KB) - added by 14 years ago.
- ref OMF_OML-4 oml_tridentcom_2009.pdf (1.9 MB) - added by 14 years ago.
- ref perfSONAR-2 extensible schema doc15649.pdf (133.2 KB) - added by 14 years ago.
- ref perfSONAR-3 topoSchema-OGF21.pdf (404.8 KB) - added by 14 years ago.
- ref OMF_OML-1 rfc4506.pdf (98.9 KB) - added by 14 years ago.
- ref OMF_OML-1 rfc4506.txt (54.2 KB) - added by 14 years ago.
- ref perfSONAR-1 trident.pdf (179.2 KB) - added by 14 years ago.