[[PageOutline]] = 2nd GENI Instrumentation and Measurement Workshop = Tuesday, June 8, 1pm - Wednesday, June 9, 2pm [[BR]] Chicago O'Hare Hilton [[BR]] NOTE: By invitation only [[BR]] == Announcement with goals, topics and reference material == The following announcement (with figures and references) was sent to the attendees to prepare for the workshop: [[BR]] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060410b%20%20InstMeasWorkshopAgenda.pdf Announcement, v1.1 (060410)] [[BR]] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060310%20%20All%20Fig%20%20IM-ARCH-Figures.pdf Figures, v1.1 (060410)] [[BR]] == Attendees at workshop == (Attended workshop: yes or no) Paul Barford - University of Wisconsin – Madison (no)[[BR]] Bruce Maggs – Duke University and Akamai (yes)[[BR]] Harry Mussman – BBN/GPO (yes)[[BR]] Vic Thomas - BBN/GPO (yes)[[BR]] Evan Zhang – BBN/GPO (yes)[[BR]] OML (ORBIT Measurement Library) OMF (ORBIT Management Framework) Max Ott – NICTA (yes, by phone)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Instrumentation Tools Jim Griffioen - Univ Kentucky (yes) perfSONAR Matt Zekauskas - Internet2 (no)[[BR]] Jason Zurawski – Internet2 (yes)[[BR]] Martin Swany - Univ Delaware (yes)[[BR]] Guilherme Fernandes – Univ Delaware (yes)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Scalable Sensing Service (S3) Sonia Fahmy – Purdue (yes)[[BR]] Puneet Sharma - HP Labs (yes)[[BR]] !OnTimeMeasure for network measurements Prasad Calyam - Ohio Supercomputing Ctr (yes) GENI Meta-Operations Center and !NetKarma Jon-Paul Herron - Indiana Univ[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] Beth Plale - Indiana Univ (no)[[BR]] Virtual Machine Introspection (VMI) Brian Hay – Univ Alaska (yes)[[BR]] Data-Intensive Cloud Control for GENI Michael Zink - UMass Amherst (yes)[[BR]] Experiment Management Service – Digital Object Registry Jim French - CNRI (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] Larry Lannom – CNRI (no)[[BR]] == Announcement with notes == The following announcement now includes all notes from the workshop: [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/062510%20InstMeasWorkshopAgenda.pdf Announcement, v1.2 (062510)] [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/062510%20InstMeasWorkshopAgenda.doc word file][[BR]] [Figures, v1.2 (062510)] [[BR]] == Priority topics == The following priority topics were identified at the workshop, and teams of attendees (and some non-attendees0 were identified for each topic to discuss it, and write a text summary for review by the WG at GEC8. === Topic 1 GENI I&M Use Cases === Team members: Paul Barford - University of Wisconsin – Madison (no)[[BR]] Jim Griffioen - Univ Kentucky (yes)[[BR]] Prasad Calyam* - Ohio Supercomputing Ctr (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Brian Hay – Univ Alaska (yes)[[BR]] *agreed to organize first writing and discussion Identify all user groups, and provide basic use cases: 1. GMOC operations group (NOC) 1. Cluster and aggregate operations groups (NOCs) 1. Archive service providers 1. Experiment researchers 1. Experiment (opt-in) users (see http://groups.geni.net/geni/attachment/wiki/041409NYCOptInWGAgenda/071509%20%20GENI-SE-OI-Overview-01.4.pdf for listing of opt-in issues, such as privacy) 1. Researchers that use measurement data archived by other researchers (!DatCat model) === Topic 2 GENI I&M Services === Team members:[[BR]] Harry Mussman* – BBN/GPO (yes)[[BR]] Evan Zhang – BBN/GPO[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] Beth Plale - Indiana Univ (yes)[[BR]] *agreed to organize first writing and discussion Summarize current view Identify different types of services:[[BR]] Completely dedicated to an experiment[[BR]] Common portion, plus parts associated with different experiments[[BR]] Common service, with data provided to multiple experiments[[BR]] Need: Basic definition of an archive service === Topic 3 GENI I&M Resources === Team members: Vic Thomas - BBN/GPO (yes)[[BR]] Jim Griffioen* - Univ Kentucky (yes)[[BR]] Martin Swany - Univ Delaware (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Brian Hay – Univ Alaska (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] *agreed to organize first writing and discussion Significant question uncovered at workshop:[[BR]] Jim on 6/25 via email: We should involve Rob Ricci in the discussion. What are resources: 1. Hosts, VMs, etc. 1. Network connectivity 1. Software, e.g., I&M software that can be included in an experiment 1. I&M services 1. I&M data flows and file transfers 1. I&M data files stored in archives How are each of these discovered, specified, authorized and assigned: a. Always by mechanisms provided by the CF? b. With CF plus additional mechanisms? Consider example of LS in perfSONAR [[BR]] Consider example of data file stored in archive, owned by an experimenter Goals:[[BR]] Need to define and then compare these options[[BR]] Need to understand interop with CF for each option[[BR]] Does CF setup secondary authorization mechanisms in some cases? If so, how?[[BR]] Does each item have:[[BR]] Unique and persistent name?[[BR]] Unique and persistent identifier?[[BR]] Need to carefully consider this for all of GENI[[BR]] For each item, consider how to:[[BR]] Create[[BR]] Name[[BR]] Register and discover[[BR]] Authorize and assign[[BR]] For each item, consider:[[BR]] Ownership[[BR]] What sort of policies the owner may want to apply[[BR]] === Topic 4 GENI I&M Measurement Plane and Interfaces === Team members: Harry Mussman* – BBN/GPO (yes)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Chris Small - Indiana Univ (yes)[[BR]] *agreed to organize first writing and discussion Consider: IP network[[BR]] Layer 2 (VLAN) connections[[BR]] Discuss:[[BR]] Which protocols are active[[BR]] Access to resources in aggregates, even when resources are in private address space, via GWs or proxies[[BR]] How to provide authentication and authorization[[BR]] How to provide QoS to protect measurement traffic[[BR]] How to provide QoS to protect other traffic when measurement traffic is large.[[BR]] Reserve bandwidth?[[BR]] Martin on 6/28: Consider XSP (extensible session protocol) to provide transport layer GW functions. === Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services === Team members: Vic Thomas - BBN/GPO (yes)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Sonia Fahmy* – Purdue (yes)[[BR]] Giridhar Manepalli - CNRI (yes)[[BR]] *agreed to organize first discussion and writing Define an approach based on OMF/OML and S3: HTTP(S)[[BR]] REST vs SOAP[[BR]] Authorization by credentials or ? If credentials, how to revoke?[[BR]] Pass XML fragments[[BR]] Define basic API[[BR]] === Topic 6 GENI I&M Interfaces and Protocols (APIs): Data Flows and Data File Transfers === Team members:[[BR]] Harry Mussman* – BBN/GPO (yes)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Ezra Kissel – Univ Delaware (yes)[[BR]] Prasad Calyam - Ohio Supercomputing Ctr (yes)[[BR]] Michael Zink - UMass Amherst (yes)[[BR]] *agreed to organize first writing and discussion Consider data flows and data file transfers between all services Define range of options: What:[[BR]] Data flows[[BR]] Data files transfers[[BR]] Type:[[BR]] Pull[[BR]] Push[[BR]] Pub/Sub[[BR]] Protocol:[[BR]] SNMP[[BR]] SCP[[BR]] FTP and gridFTP[[BR]] HTTP[[BR]] XMPP[[BR]] TCP[[BR]] SCTP[[BR]] Consider:[[BR]] Naming[[BR]] Discovery[[BR]] Connectivity [[BR]] Authentication and authorization mechanisms[[BR]] Map to current projects, giving examples:[[BR]] Consider: Minimum set required for GENI[[BR]] === Topic 7 GENI I&M Interfaces and Protocols (APIs): Service Registration and Discovery === Team members:[[BR]] Jason Zurawski* – Internet2 (yes)[[BR]] Prasad Calyam - Ohio Supercomputing Ctr (yes)[[BR]] *agreed to organize first writing and discussion[[BR]] Consider approach used in perfSONAR Summarize for:[[BR]] Services with data flows[[BR]] Also sources of file transfers?[[BR]] Also GUIs?[[BR]] === Topic 8 GENI I&M Interfaces and Protocols (APIs): GUIs === Team members:[[BR]] Jeremy Reed - Univ Kentucky (yes)[[BR]] Guilherme Fernandes* – Univ Delaware (yes)[[BR]] Puneet Sharma - HP Labs (yes)[[BR]] *Agreed to organize team[[BR]] Define overall goals for GENI GUIs Types of GUIs:[[BR]] Control experiments[[BR]] Display I&M results[[BR]] Report status[[BR]] View archive service[[BR]] Consider portal, for access to multiple GUIs Consider need for authentication and authroization === Topic 9 GENI Measurement Data Schema === Team members:[[BR]] Bruce Maggs – Duke University and Akamai (yes)[[BR]] Max Ott – NICTA (yes, by phone)[[BR]] Ivan Seskar – Rutgers WINLAB (yes)[[BR]] Martin Swany* - Univ Delaware (yes)[[BR]] Camilo Viecco - Indiana Univ (yes)[[BR]] Michael Zink - UMass Amherst (yes)[[BR]] Jim French - CNRI (yes)[[BR]] *agreed to organize first discussion and writing Consider:[[BR]] Measurement data schema[[BR]] Metadata schema[[BR]] Metadata contents[[BR]] Consider measurement data schema and/or metadata schema from:[[BR]] perfSONAR[[BR]] GMOC-provided[[BR]] Current OML[[BR]] Proposed using IPFIX[[BR]] NetCDF (as used by DI Cloud)[[BR]] Consider: Minimum set required for GENI Provide overall template for GENI metadata, considering above.[[BR]] Which items in GENI metadata template are: Required?[[BR]] Invariant?[[BR]] == Text == All text, word and PDF == Figures == [http://groups.geni.net/geni/attachment/wiki/2ndInstMeasWork/060310%20%20IM-ARCH-Figures.2.vsd 060310 v1.2 All I&M figures][[BR]] All figures, visio and PDF Individual figures, visio and PDF == References == All references Individual references Ref GIMS_Design_UseCases: “Use-cases for GENI Instrumentation and Measurement Architecture Design” Ref MeasPlane-1: “RESTful Web Services vs. “Big” Web Services: Making the Right Architectural Decision” Ref OMF_OML-1: “XDR: External Data Representation Standard” Ref OMF_OML-2: “ORBIT Measurements Framework and Library (OML): Motivations, Design, Implementation, and Features” Ref OMF-OML-3: “OML Overview” slides Ref OMF-OML-4: “Measurement Architectures for Network Experiments with Disconnected Mobile Nodes” Ref InsTools-1: “Architectural Design and Specification of the INSTOOLS Measurement System” Ref perfSONAR-1: “Scalable Framework for Representation and Exchange of Network Measurements” Ref perfSONAR-2: “An Extensible Schema for Network Measurement and Performance Data” Ref perfSONAR-3: “NM-WG/perfSONAR Topology Schema” Ref GMOC-1: “GMOC Topology-Entity Data Exchange Format Specification” Ref GMOC-2: “Proposal: Use of URN's as GENI Identifiers”