[[PageOutline]] = 2nd GENI Instrumentation and Measurement Workshop = Tuesday, June 8, 1pm - Wednesday, June 9, 2pm [[BR]] Chicago O'Hare Hilton [[BR]] NOTE: By invitation only [[BR]] == Announcement with goals, topics and reference material == The following announcement (with figures and references) was sent to the attendees to prepare for the workshop: [[BR]] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060410b%20%20InstMeasWorkshopAgenda.pdf Announcement, v1.1 (060410)] [[BR]] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060310%20%20All%20Fig%20%20IM-ARCH-Figures.pdf Figures, v1.1 (060410)] [[BR]] == Attendees at workshop == (Attended workshop: yes or no) Paul Barford - University of Wisconsin – Madison (no)[[BR]] Bruce Maggs – Duke University and Akamai (yes)[[BR]] Harry Mussman – BBN/GPO (yes)[[BR]] Vic Thomas - BBN/GPO (yes)[[BR]] Evan Zhang – BBN/GPO (yes)[[BR]] OML (ORBIT Measurement Library) OMF (ORBIT Management Framework) Max Ott – NICTA (yes, by phone)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Instrumentation Tools Jim Griffioen - Univ Kentucky (yes) perfSONAR Matt Zekauskas - Internet2 (no)[[BR]] Jason Zurawski – Internet2 (yes)[[BR]] Martin Swany - Univ Delaware (yes)[[BR]] Guilherme Fernandes – Univ Delaware (yes)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Scalable Sensing Service (S3) Sonia Fahmy – Purdue (yes)[[BR]] Puneet Sharma - HP Labs (yes)[[BR]] !OnTimeMeasure for network measurements Prasad Calyam - Ohio Supercomputing Ctr (yes) GENI Meta-Operations Center and !NetKarma Jon-Paul Herron - Indiana Univ[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] Beth Plale - Indiana Univ (no)[[BR]] Virtual Machine Introspection (VMI) Brian Hay – Univ Alaska (yes)[[BR]] Data-Intensive Cloud Control for GENI Michael Zink - UMass Amherst (yes)[[BR]] Experiment Management Service – Digital Object Registry Jim French - CNRI (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] Larry Lannom – CNRI (no)[[BR]] == Announcement with notes == The following announcement now includes all notes from the workshop: [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/062510%20InstMeasWorkshopAgenda.pdf Announcement, v1.2 (062510)] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/062510%20InstMeasWorkshopAgenda.doc word version][[BR]] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/Visio-062510_All_IM-ARCH-Figures.pdf Figures, v1.2 (062510)] [[BR]] == Priority topics == The following priority topics were identified at the workshop, and teams of attendees (and some non-attendees0 were identified for each topic to discuss it, and write a text summary for review by the WG at GEC8. === Topic 1 GENI I&M Use Cases === Team members: Paul Barford - University of Wisconsin – Madison (no)[[BR]] Jim Griffioen - Univ Kentucky (yes)[[BR]] Prasad Calyam* - Ohio Supercomputing Ctr (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Brian Hay – Univ Alaska (yes)[[BR]] *agreed to organize first writing and discussion Identify all user groups, and provide basic use cases: 1. Experiment researchers 2. Experiment (opt-in) users (see http://groups.geni.net/geni/attachment/wiki/041409NYCOptInWGAgenda/071509%20%20GENI-SE-OI-Overview-01.4.pdf for listing of opt-in issues, such as privacy) 3. Central (i.e., GMOC) operators 4. Cluster and aggregate providers and operators 5. Archive service providers and operators 6. Researchers that use archived measurement data, archived by other researchers (!DatCat model) === Topic 2 GENI I&M Services === Team members:[[BR]] Harry Mussman* – BBN/GPO (yes)[[BR]] Evan Zhang – BBN/GPO[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] Beth Plale - Indiana Univ (yes)[[BR]] *agreed to organize first writing and discussion Summarize current view of:[[BR]] Measurement Orchestration (MO) Service [[BR]] Measurement Point (MP) Service [[BR]] Measurement Information (MI) Service [[BR]] Measurement Collection (MC) Service [[BR]] Measurement Analysis and Presentation (MAP) Service[[BR]] Measurement Data Archive (MDA) Service[[BR]] Need: Basic definition of a Measurement Data Archive (MDA) Service[[BR]] Identify different types of services:[[BR]] Type 1: Dedicated service platform, with dedicated sliver, for customized information. [[BR]] (Completely dedicated to an experiment) [[BR]] Type 2: Common service platform, with dedicated slivers, for customized information. [[BR]] (Common portion, plus parts associated with different experiments)[[BR]] Type 3: Common service, for common or customized information. [[BR]] (Common service, with data provided to multiple experiments)[[BR]] === Topic 3 GENI I&M Resources === Team members: Vic Thomas - BBN/GPO (yes)[[BR]] Jim Griffioen* - Univ Kentucky (yes)[[BR]] Martin Swany - Univ Delaware (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Brian Hay – Univ Alaska (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] *agreed to organize first writing and discussion Significant question uncovered at workshop:[[BR]] Jim on 6/25 via email: We should involve Rob Ricci in the discussion.[[BR]] Consider these resources for I&M capabilities:[[BR]] 1. Hosts, VMs, etc. 2. Network connectivity 3. Software, e.g., I&M software that can be included in an experiment 4. I&M services 5. I&M data flows and file transfers 6. I&M data files stored in archives For each item, consider how to:[[BR]] Create[[BR]] Name[[BR]] Register and discover[[BR]] Authorize and assign[[BR]] Does each item have:[[BR]] Unique and persistent name?[[BR]] Unique and persistent identifier?[[BR]] Need to carefully consider this for all of GENI[[BR]] For each item, consider:[[BR]] Ownership[[BR]] What sort of policies the owner may want to apply[[BR]] How are each of these discovered, specified, authorized and assigned: Always by mechanisms provided by the CF? With CF plus additional mechanisms? Consider example of LS in perfSONAR [[BR]] Consider example of data file stored in archive, owned by an experimenter Need to define and then compare these options[[BR]] Need to understand interop with CF for each option[[BR]] Does CF setup secondary authorization mechanisms in some cases? If so, how?[[BR]] === Topic 4 GENI I&M Measurement Plane and Interfaces === Team members: Harry Mussman* – BBN/GPO (yes)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] *agreed to organize first writing and discussion Consider: IP network[[BR]] Layer 2 (VLAN) connections[[BR]] Discuss:[[BR]] Which protocols are active[[BR]] Access to resources in aggregates, even when resources are in private address space, via GWs or proxies[[BR]] How to provide authentication and authorization[[BR]] How to provide QoS to protect measurement traffic[[BR]] How to provide QoS to protect other traffic when measurement traffic is large.[[BR]] Reserve bandwidth?[[BR]] Martin on 6/28: Consider XSP (extensible session protocol) to provide transport layer GW functions. === Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services === Team members: Vic Thomas - BBN/GPO (yes)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Sonia Fahmy* – Purdue (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] *agreed to organize first discussion and writing Define an approach based on OMF/OML and S3: HTTP(S)[[BR]] REST vs SOAP[[BR]] Authorization by credentials or ? If credentials, how to revoke?[[BR]] Pass XML fragments[[BR]] Define basic API[[BR]] === Topic 6 GENI I&M Interfaces and Protocols (APIs): Data Flows and Data File Transfers === Team members:[[BR]] Harry Mussman* – BBN/GPO (yes)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Prasad Calyam - Ohio Supercomputing Ctr (yes)[[BR]] Michael Zink - UMass Amherst (yes)[[BR]] *agreed to organize first writing and discussion Consider data flows and data file transfers between all services Define range of options: What:[[BR]] Data flows[[BR]] Data files transfers[[BR]] Type:[[BR]] Pull[[BR]] Push[[BR]] Pub/Sub[[BR]] Protocol:[[BR]] SNMP[[BR]] SCP[[BR]] FTP and gridFTP[[BR]] HTTP[[BR]] XMPP[[BR]] TCP[[BR]] SCTP[[BR]] Consider:[[BR]] Naming[[BR]] Discovery[[BR]] Connectivity [[BR]] Authentication and authorization mechanisms[[BR]] Map to current projects, giving examples:[[BR]] Consider: Minimum set required for GENI[[BR]] === Topic 7 GENI I&M Interfaces and Protocols (APIs): Registration and Discovery of Services with Available Measurement Data === Team members:[[BR]] Jason Zurawski* – Internet2 (yes)[[BR]] Prasad Calyam - Ohio Supercomputing Ctr (yes)[[BR]] *agreed to organize first writing and discussion[[BR]] Consider approach used in perfSONAR Summarize for:[[BR]] Services with data flows[[BR]] Also sources of file transfers?[[BR]] Also GUIs?[[BR]] === Topic 8 GENI I&M Interfaces and Protocols (APIs): GUIs === Team members:[[BR]] Jeremy Reed - Univ Kentucky (yes)[[BR]] Guilherme Fernandes* – Univ Delaware (yes)[[BR]] Puneet Sharma - HP Labs (yes)[[BR]] *Agreed to organize team[[BR]] Define overall goals for GENI GUIs Types of GUIs:[[BR]] Control experiments[[BR]] Display I&M results[[BR]] Report status[[BR]] View archive service[[BR]] Consider portal, for access to multiple GUIs Consider need for authentication and authroization === Topic 9 GENI Measurement Data Schema === Team members:[[BR]] Bruce Maggs – Duke University and Akamai (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Martin Swany* - Univ Delaware (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Michael Zink - UMass Amherst (yes)[[BR]] Jim French - CNRI (yes)[[BR]] *agreed to organize first discussion and writing Consider:[[BR]] Measurement data schema[[BR]] Metadata schema[[BR]] Metadata contents[[BR]] Consider measurement data schema and/or metadata schema from:[[BR]] perfSONAR[[BR]] GMOC-provided[[BR]] Current OML[[BR]] Proposed using IPFIX[[BR]] NetCDF (as used by DI Cloud)[[BR]] Consider: Minimum set required for GENI Provide overall template for GENI metadata, considering above.[[BR]] Which items in GENI metadata template are: Required?[[BR]] Invariant?[[BR]] == Text == All text, word and PDF == Figures == [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060310%20%20IM-ARCH-Figures.2.vsd 060310 v1.2 All I&M figures][[BR]] All figures, visio and PDF Individual figures, visio and PDF == Project Summaries == === Instrumentation Tools === === OMF/OML === === perfSONAR === === Scalable Sensing Service === === On Time Measure === === Data Intensive Cloud === === Digital Object Registry === == References == All references Individual references Ref GIMS_Design_UseCases: “Use-cases for GENI Instrumentation and Measurement Architecture Design” Ref MeasPlane-1: “RESTful Web Services vs. “Big” Web Services: Making the Right Architectural Decision” Ref OMF_OML-1: “XDR: External Data Representation Standard” Ref OMF_OML-2: “ORBIT Measurements Framework and Library (OML): Motivations, Design, Implementation, and Features” Ref OMF-OML-3: “OML Overview” slides Ref OMF-OML-4: “Measurement Architectures for Network Experiments with Disconnected Mobile Nodes” Ref InsTools-1: “Architectural Design and Specification of the INSTOOLS Measurement System” Ref perfSONAR-1: “Scalable Framework for Representation and Exchange of Network Measurements” Ref perfSONAR-2: “An Extensible Schema for Network Measurement and Performance Data” Ref perfSONAR-3: “NM-WG/perfSONAR Topology Schema” Ref GMOC-1: “GMOC Topology-Entity Data Exchange Format Specification” Ref GMOC-2: “Proposal: Use of URN's as GENI Identifiers”