[[PageOutline]] = 4) GENI I&M Design Topics = == Description == These are the GENI I&M priority design topics that must be resolved to allow the current GENI I&M projects to proceed, particularly the large Sol 3 project (GIMI and GEMINI) [[BR]] These topics were identified and reviewed at the [http://groups.geni.net/geni/wiki/GEC12InstMeasWorkingSession GEC12 I&M meeting] [[BR]] == Goals == Resolve between now (GEC12) and GEC13. [[BR]] Simplify where possible! [[BR]] == Approach == Form teams for each topic, and identify a LEAD for each topic. [[BR] Teams should meet by confernce call, or in person, and complete a report for presentation at GEC13. [[BR]] == 4.1) List of All Identified Topics == List: [[BR]] a) I&M Use Cases for Experiments [[BR]] b) I&M Use Cases for Infrastructure Measurement [[BR]] c) Platform Targets [[BR]] d) Experiment and Measurement Orchestration Tools [[BR]] e) Interfaces/Protocols Between Tools and Services [[BR]] f) Descriptor Schema and Registry Service [[BR]] g) Object Names and Registry Service [[BR]] h) XML Messaging Service [[BR]] i) Lookup Service [[BR]] j) Portal Service [[BR]] k) User Workspace and Archive Services [[BR]] l) Operational Services [[BR]] m) Services Shared by GIMI and GEMINI projects [[BR]] n) Support for Experimenters [[BR]] o) Support for Operators [[BR]] These have been combined and summarized and shown in the sections below. [[BR]] == 4.2) I&M Use Cases for Experiments, and Support for Experimenters == I&M Use Cases for Experiments [[BR]] Support for Experimenters [[BR]] [wiki:InstMeasTopic_4.2UseCasesExperimenters Work in Progress] [[BR]] === 1) Goals === From Sec. 2 of the GENI I&M Architecture document: [[BR]] To conduct useful and repeatable experiments, GENI experimenters require extensive and reliable instrumentation and measurement capabilities to gather, analyze, present and archive Measurement Data (MD). [[BR]] Remove the burden on researcher to become a system and network measurement infrastructure expert so that researcher can better focus on the science in the experiments [[BR]] Work towards Max Ott's vision for experiment support [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC12InstMeasWorkingSession/Will%20RSpec%20help%20us%20to%20do%20better%20science%20-%20GEC11%20-%20July%202011.pdf slides] [[BR]] Define use cases for a wide range of experiments: complex (by skilled experimenters) to simple (by novice experimenter, i.e., a student) [[BR]] === 2) Tasks === Provide a concise but complete definition of I&M Use Cases for Experiments [[BR]] Identify the support that should be available to experimenters, i.e., on-line instructions, examples and help desks. [[BR]] Update the [http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture GENI I&M Architecture document]: [[BR]] Sec. 3.1. I&M Use cases for Experimenters [[BR]] Sec. 4.2.1 Typical Arrangements of I&M Services: For Experimenter Gathering MD from their Slice [[BR]] Sec. 4.3.1 Type 1 I&M Service: Service Contained within a Slice [[BR]] Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects [[BR]] === 3) Team === CO-LEAD Jim Griffioen (U Kentucky) [[BR]] Hussamuddin Nasir (U Kentucky) [[BR]] CO-LEAD Mike Zink (UMass Amherst) [[BR]] David Irwin (UMass Amherst) [[BR]] Max Ott (NICTA) [[BR]] Prasad Calyam (Ohio Super Center) [[BR]] Harry Mussman (GPO) [[BR]] == 4.3) I&M Use Cases for Infrastructure Measurement, and Support for Operators == I&M Use Cases for Infrastructure Measurement [[BR]] Support for Operators [[BR]] [wiki:InstMeasTopic_4.3UseCasesInfrastructure Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Delineate what infrastructure monitoring is and is not covered by this approach. [[BR]] 1) Yes: monitoring of clusters/racks [[BR]] 2) Yes: passive measurements of transport switches, etc. [[BR]] 3) Yes: active measurements of IP networks, of Layer 2 and OpenFlow paths [[BR]] b) Nominal infrastructure measurement process: [[BR]] 1) Setup persistent or on-demand infrastructure measurement slice. [[BR]] 2) Make passive measurements or make active measurements. [[BR]] 3) Gather MD, and observe as it is gathered; formulate MDOD. [[BR]] 4) Store MD in collector, describe with MDOD, and register MDOD so that MD can be shared. [[BR]] 5) Typically share MD with Aggregate Operator, GMOC and/or Experimenters, per policy written into MDOD. [[BR]] 6) Pull MD out of collector, analyze and visualize. [[BR]] 7) Archive MD with MDOD. [[BR]] 8) Share archived MD with others, per policy included within MDOD. [[BR]] 9) Pull MD out of archive, to analyze and/or visualize. [[BR]] c) What support must be provided for Operator? how? [[BR]] 2) Who: LEAD Martin Swany (Indiana U); Eric Boyd (Internet2); Jason Zurawski (Internet2); Prasad Calyam (Ohio Super Center); Chris Small (Indiana U); Ilia Baldine, for ExoGENI racks (RENCI); ?, for InstaGENI racks (HP); ?, for GMOC [[BR]] == 4.4) GENI Environment == Platform Targets [[BR]] Experiment and Measurement Orchestration Tools [[BR]] Interfaces/Protocols Between Tools and Services [[BR]] [wiki:InstMeasTopic_4.4GENIEnvironment Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Platform targets: [[BR]] 1) protoGENI servers/VMs [[BR]] 2) ORCA servers/VMs [[BR]] 3) WiMAX sites [[BR]] 4) ExoGENI (RENCI) racks; like ORCA [[BR]] 5) TransCloud (HP) racks; like protoGENI? [[BR]] 6) PlanetLab b) Experiment and Measurement Orchestration tools: [[BR]] 1) FLACK [[BR]] 2) GUSH (with Omni) [[BR]] 3) SFACE + RAVEN [[BR]] c) Interfaces/protocols between tools and services: [[BR]] 1) protoGENI API, with protoGENI RSpec v2, for resource assignment [[BR]] 2) GENI AM API, with protoGENI RSpec v2, for resource assignment [[BR]] 3) ns for topology [[BR]] 4) ssh for image loading [[BR]] 5) OMF messages for orchestration [[BR]] 2) Who: Martin Swany (IU); Jim Griffioen (U Kentucky); Ilia Baldine, for ExoGENI racks (RENCI); ?, for InstaGENI racks (?); David Irwin (UMass Amherst); Mike Zink (UMass Amherst); Christoph Dwertmann (NICTA); LEAD Harry Mussman (GPO); Tom Mitchell (GPO); Aaron Helsinger (GPO); Marshall Brinn (GPO); Vic Thomas (GPO) [[BR]] == 4.5) Descriptors, Objects and Registries == Descriptor Schema and Registry Service [[BR]] Object Names and Registry Service [[BR]] [wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Need to finalize MDOD schema, for XML file. [http://groups.geni.net/geni/wiki/GEC11InstMeasWorkingSession#a2MeasurementDataObjectDescriptorMDOD References] [[BR]] b) Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)[[BR]] c) Want to use MDOD schema to define Event Record schema. (NetKarma)[[BR]] d) MDOD registry. Use UNIS lookup service? Use DOR registry? Include in iRODS? Consider IF-MAP server? [[BR]] e) MDOD creation and editing service. [[BR]] f) Need object (e.g., Measurement Data Object or I&M Service) names; need a public reference; consider DataCite [[BR]] g) Do we need object registry, with names and public key? [[BR]] 2) Who: Jason Zurawski (Internet2); Ezra Kissel (U Delaware); Eric Boyd (Internet2); Beth Plale (IU); Chris Small (GEMINI, IU); Scott Jensen (Indiana U); Larry Lannom (CNRI); LEAD Giridhar Manepalli (CNRI); Deniz Gurken (GIMI, UH); Harry Mussman (GPO) [[BR]] == 4.6) Messaging Service == XML Messaging Service [[BR]] [wiki:InstMeasTopic_4.6MessagingService Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Use for messages, not generalized data; assume relatively low rate. [[BR]] b) Use for OMF messages. [[BR]] c) Use for "event record messages", that can be logged. [[BR]] d) XMPP server, in public IP space. [[BR]] e) Entities connect, and are authenticated. [[BR]] f) An entity may start a pub/sub node. [[BR]] g) When an entity subscribes, a message is sent to publisher requesting authorization. [[BR]] h) Operations plan for XML Messaging Service. [[BR]] 2) Who: Rudra Dutta (NCSU); LEAD Anirban Mandal (RENCI); Christoph Dwertmann (NICTA); Ahmed El-Hassany (IU); Harry Mussman (GPO) [[BR]] == 4.7) Lookup Service == Lookup Service [[BR]] [wiki:InstMeasTopic_4.7LookupService Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Home and global Lookup Service, as defined by perfSONAR [[BR]] b) Also Topology Service? c) Global UNIS? Also Local UNIS in slice? [[BR]] d) Map MDOD into metadata that is registered [[BR]] e) Can Lookup Service be used for finding web interfaces, and other types of portals? [[BR]] f) Operations plan for Global UNIS. 2) Who: ? (GEMINI); Giridhar Manepalli (CNRI); Harry Mussman (GPO) [[BR]] == 4.8) GENI User Workspace Service == Portal Service [[BR]] [wiki:InstMeasTopic_4.8PortalService Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Towards Max Ott's vision that provides a unified environment for experiment support [[BR]] b) GENI User Workspace Service is a container for a wide range tools including: experiment control tools; portal tools; measurement analysis and presentation tools [[BR]] c) Is a Linux OS environment with files system, certificate and credential stores [[BR]] d) Linux environment can be hosted on server dedicated to user; shared among multiple users in an organization; shared among multiple GENI users. [[BR]] e) Many tools interact with GENI aggregates using GENI APIs [[BR]] f) Many tools have a user interface, i.e.,a web interface; these are foten called portal tools [[BR]] g) Many tools call other tools through published APIs. [[BR]] h) Having all tools in the same container with a certificate store minimizes effort to gain authorizations. [[BR]] i) Would prefer a service useful with both GIMI and GEMINI tools [[BR]] j) Should compare with CNRI prototype User Workspace service [[BR]] k) Operations plan for GENI User Worksapce service [[BR]] 2) Who: Jim Griffioen and/or Charles Carpenter (U Kentucky); Max Ott (NICTA); Christoph Dwertmann (NICTA); Chris Small, for NetKarma (Internet2); Ahmed El-Hassany (IU); Jeanne Ohren (GPO) [[BR]] == 4.9) Archive Service == User Workspace and Archive Services [[BR]] [wiki:InstMeasTopic_4.9ArchiveService Work in Progress] [[BR]] 1) Definition of topic: [[BR]] a) Consider CNRI prototype user workspace and archive service [[BR]] b) Consider U Kentucky archive service [[BR]] c) Use iRODS service? [[BR]] d) How to move data to service: move file; move SQL DB; or? [[BR]] e) How are partitions for different users managed? [[BR]] f) How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] g) Operations plan for Archive Service. [[BR]] 2) Who: Giridhar Manepalli (CNRI); LEAD Shu Huang (RENCI) ; Larry Lannom (CNRI); Wesley Wu (U Kentucky); Hussam Nasir (U Kentucky); Chris Small (NetKarma, IU) [[BR]] == 4.10 List of Operational Services == Persistent Operational Services [[BR]] Services Shared by GIMI and GEMINI [[BR]] [wiki:InstMeasTopic_4.10OperationalServices Work in Progress] [[BR]] 1) What persistent operational services must be supported: [[BR]] a) Descriptor registry. use UNIS? DOR? iRODS? IF-MAP? [[BR]] b) Object registry. DOR? [[BR]] c) XML messaging service [[BR]] d) UNIS service [[BR]] e) GENI user workspace service [[BR]] f) iRODs archive service [[BR]] 2) Can all services can be shared by GIMI and GEMINI tools? [[BR]] 3) How do these relate to other GENI services/functions, including clearinghouse? 4) Who: Mike Zink (UMass Amherst); Martin Swany (Inidana U); Jim Griffioen (U Kentucky); Giridhar Manepalli (CNRI); Aaron Helsinger (GPO); Harry Mussman (GPO) [[BR]] [http://groups.geni.net/geni/wiki/GEC12InstMeasWorkingSession