[[PageOutline]] = Instrumentation and Measurement (I&M) Working Session at the GEC-13 = Tuesday, March 13, 10:00am - 12noon and 1:30pm - 3:00pm [[BR]] Room: [[BR]] Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies) [[BR]] == Description == This is the I&M working session that will review progress towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement. [[BR]] It will start with reports from several of the design topic teams formed at the GEC-12 working session; these will include the contributions of the I&M projects continuing from Solicitation 2. [[BR]] Then the two large Solicitation 3 I&M D&P projects, GIMI and GEMINI, will review their progress. [[BR]] Finally, we will discuss how to support experimenters and infrastructure measurement. [[BR]] == Summary == == Agenda == This is a tentative agenda, which may change. [[BR]] == 1) Introductions == 10:00am [[BR]] == 2) Reports by I&M design topic teams == 10:05am [[BR]] At GEC12, nine I&M priority design topics were identified, and nine teams were assigned to understand and resolve these topics by GEC13. [[BR]] See the [http://groups.geni.net/geni/wiki/InstMeasCurrentDesignTopicsAll list of topics], including a brief summary of each topic. [[BR]] === T2) I&M Use Cases for Experiments, and Support for Experimenters === 10:05am [[BR]] Team [[BR]] CO-LEAD Jim Griffioen (U Kentucky) [[BR]] CO-LEAD Mike Zink (UMass Amherst) [[BR]] Hussamuddin Nasir (U Kentucky) [[BR]] David Irwin (UMass Amherst) [[BR]] Max Ott (NICTA) [[BR]] Prasad Calyam (Ohio Super Center) [[BR]] Harry Mussman (GPO) [[BR]] Tasks [[BR]] Provide a concise but complete definition of I&M Use Cases for Experiments [[BR]] Define use cases for a wide range of experiments: complex (by skilled experimenters) to simple (by novice experimenter, i.e., a student) [[BR]] Identify the support that should be available to experimenters, i.e., on-line instructions, examples and help desks. [[BR]] Update the [http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture GENI I&M Architecture document]: [[BR]] Sec. 3.1. I&M Use cases for Experimenters [[BR]] Sec. 4.2.1 Typical Arrangements of I&M Services: For Experimenter Gathering MD from their Slice [[BR]] Sec. 4.3.1 Type 1 I&M Service: Service Contained within a Slice [[BR]] Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects [[BR]] [wiki:InstMeasTopic_4.2UseCasesExperimenters Work in Progress] [[BR]] Summary of current status: [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T2a%29%20%20IM%20Uses%20Cases.pdf slides] Discussion: Consider also: Compare experiments and their results. Do community benchmarking (Max) Do verifications (Mike) === T3) I&M Use Cases for Infrastructure Measurement, and Support for Operators === 10:20am [[BR]] Team [[BR]] LEAD Martin Swany (Indiana U) [[BR]] Guilherme Fernandes (?) [[BR]] Eric Boyd (Internet2) [[BR]] Jason Zurawski (Internet2) [[BR]] Prasad Calyam (Ohio Super Center) [[BR]] Chris Small, for NetKarma (Indiana U) [[BR]] Ilia Baldine, for ExoGENI racks (RENCI) [[BR]] Jonathan Mills (RENCI) [[BR]] ?, for InstaGENI racks (HP) [[BR]] ?, for GMOC [[BR]] Sarah Edwards (GPO) [[BR]] Chaos Golubitsky (GPO) [[BR]] Harry Mussman (GPO) [[BR]] Tasks [[BR]] Provide a concise but complete definition of I&M Use Cases for Infrastructure Measurement [[BR]] Identify the support that should be available to operators [[BR]] Update the [http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture GENI I&M Architecture document]: [[BR]] Sec. 3.3. I&M Use cases for Central Operators (i.e., GMOC) [[BR]] Sec. 3.4. I&M Use cases for Aggregate Providers and Operators [[BR]] Sec. 4.2.2 Typical Arrangements of I&M Services: For Operator Gathering MD from GENI Infrastructure [[BR]] Sec. 4.2.3 Typical Arrangements of I&M Services: For Experimenters Gathering MD from their Slice and from GENI Infrastructure [[BR]] Sec. 4.3.3 Type 3 I&M Service: Common Service with MD for Multiple Slices [[BR]] Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects [[BR]] [wiki:InstMeasTopic_4.3UseCasesInfrastructure Work in Progress] [[BR]] Summary of current status === T4) GENI I&M Tool Testing Environment === 10:35am [[BR]] Team [[BR]] LEAD Harry Mussman (GPO) [[BR]] Martin Swany (IU) [[BR]] Jim Griffioen (U Kentucky) [[BR]] Ilia Baldine, for ExoGENI racks (RENCI) [[BR]] ?, for InstaGENI racks (?) [[BR]] David Irwin (UMass Amherst) [[BR]] Mike Zink (UMass Amherst) [[BR]] Christoph Dwertmann (NICTA) [[BR]] Tom Mitchell (GPO) [[BR]] Aaron Helsinger (GPO) [[BR]] Vic Thomas (GPO) [[BR]] Jeanne Ohren (GPO) [[BR]] Luisa Nevers (GPO) [[BR]] Tasks [[BR]] Provide a concise definition of the environment where the GENI I&M tools will be tested, including: [[BR]] User tools for experiment and measurement setup and orchestration [[BR]] Interfaces/protocols between user tools and GENI aggregates [[BR]] Supported GENI aggregtaes [[BR]] [wiki:InstMeasTopic_4.4GENIEnvironment Work in Progress] [[BR]] Summary of current status [[BR]] === T5) Descriptors, Objects and Registries === 10:50am [[BR]] Team [[BR]] LEAD Giridhar Manepalli (CNRI) [[BR]] Jason Zurawski (Internet2) [[BR]] Ezra Kissel (U Delaware) [[BR]] Eric Boyd (Internet2) [[BR]] Beth Plale (IU) [[BR]] Chris Small (GEMINI, IU) [[BR]] Scott Jensen (Indiana U) [[BR]] Larry Lannom (CNRI) [[BR]] Deniz Gurken (GIMI, UH) [[BR]] Harry Mussman (GPO) [[BR]] [[BR]] Tasks [[BR]] Need to finalize MDOD schema, for XML file. [http://groups.geni.net/geni/wiki/GEC11InstMeasWorkingSession#a2MeasurementDataObjectDescriptorMDOD References] [[BR]] Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)[[BR]] Want to use MDOD schema to define Event Record schema. (NetKarma)[[BR]] Do we need MDOD registry? Use UNIS lookup service? Use DOR registry? Include in iRODS? Consider IF-MAP server? [[BR]] Need MDOD creation and editing service. (CNRI) [[BR]] Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle [[BR]] [wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService Work in Progress] [[BR]] Summary of current status [[BR]] === T6) XML Messaging Service === 11:05am [[BR]] Team [[BR]] LEAD Anirban Mandal (RENCI) [[BR]] Ilia Baldine (RENCI) [[BR]] Rudra Dutta (NCSU) [[BR]] Christoph Dwertmann (NICTA) [[BR]] Ahmed El-Hassany (IU) [[BR]] Harry Mussman (GPO) [[BR]] Tasks [[BR]] Define, prototype, demonstrate and operate a GENI Messaging service, starting at GEC13. [[BR]] The XML Mesaaging service will be built by the IMF project. For a detailed task list, see the Spiral 4 SOW of the IMF project. [[BR]] Support use in GENI by many tools, including GEMINI and GIMI I&M tools [[BR]] Define operations plan for XML Messaging service. [[BR]] [wiki:InstMeasTopic_4.6MessagingService Work in Progress] [[BR]] Summary of current status - [attachment:GEC13_IMF_overview.pptx Slides] [[BR]] === T7) Measurement Information (MI) Service === 11:15am [[BR]] Team [[BR]] LEAD Ahmed El-Hassany (IU) [[BR]] Martin Swany (IU) [[BR]] Harry Mussman (GPO) [[BR]] Tasks [[BR]] The MI service will be built by the GEMINI project using UNIS technology; see the Spiral 4 SOW of the GEMINI project for the detailed steps, which include: [[BR]] 1) Refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances: [[BR]] 2) Update topology model to Rspec version 3 and AM API (February 2012) [[BR]] 3) Modifications to allow local UNIS to register with global UNIS (March 2012) [[BR]] 4) Initial deployment of persistent GENI Global I&M Registry (GGR) service, based on UNIS. (May 2012) [[BR]] 5) Complete noSQL (MongoDB) backend (September 2012) [[BR]] 6) Unify Perl and Python implementations (September 2012) [[BR]] Define, prototype, demonstrate and operate a MI service, starting in May, 2012. [[BR]] Support use in GENI by many tools, including GEMINI and GIMI I&M tools [[BR]] Define operations plan for MI service. [[BR]] [wiki:InstMeasTopic_4.7LookupService Work in Progress] [[BR]] Summary of current status [[BR]] === T8) GENI User Tools and Services === 11:25am [[BR]] Team [[BR]] LEAD Jeanne Ohren (GPO)[[BR]] Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky) [[BR]] Max Ott and/or Christoph Dwertmann (NICTA) [[BR]] Chris Small (NetKarma, IU) [[BR]] Ahmed El-Hassany (IU) [[BR]] Giridhar Manepalli (CNRI) [[BR]] Harry Mussman (GPO) [[BR]] Vic Thomas (GPO) [[BR]] Niky Riga (GPO) [[BR]] Luisa Nevers (GPO) [[BR]] Goals [[BR]] Work towards Max Ott's vision for experiment support [[BR]] Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".[[BR]] Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data. [[BR]] These services should work together via APIs, etc., to streamline the experiment process. [[BR]] Tasks [[BR]] Based upon the configuration defined below, the implementation is split into: [[BR]] 1) A GENI User Workspace, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools [[BR]] 2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user. [[BR]] Define, prototype, deploy and operate a GENI User Workspace. It can be hosted on a server dedicated to the user, or on a server hosting multiple user workspaces for multiple users.[[BR]] Gather the various "user tools" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure. [[BR]] Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators). [[BR]] [wiki:InstMeasTopic_4.8PortalService Work in Progress] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T8%29%20GENIUserToolsAndServices.pdf Summary of current status] [[BR]] === T9) Digital Object Archive (DOA) Service === 11:40am [[BR]] Team [[BR]] Co-LEAD Giridhar Manepalli (CNRI) [[BR]] Co-LEAD Shu Huang (RENCI) [[BR]] Larry Lannom (CNRI) [[BR]] Wesley Wu (U Kentucky) [[BR]] Hussam Nasir (U Kentucky) [[BR]] Chris Small (NetKarma, IU) [[BR]] Jeannie Ohren (GPO) [[BR]] Tasks Start with CNRI prototype digital object archive service [[BR]] Plan transition to use of iRODS service [[BR]] How to move data to service: move file; move SQL DB; or? [[BR]] How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.[[BR]] How are partitions for different users managed? [[BR]] How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] Operations plans for Digital Object Archive (DOA) and Object Identifier (OI) services. [[BR]] [wiki:InstMeasTopic_4.9ArchiveService Work in Progress] [[BR]] Summary of current status [[BR]] === T10) List of Operational Services === (no report) [[BR]] Team [[BR]] Mike Zink (UMass Amherst) [[BR]] Martin Swany (Inidana U) [[BR]] Jim Griffioen (U Kentucky) [[BR]] Giridhar Manepalli (CNRI) [[BR]] Jeanne Ohren (GPO) [[BR]] Harry Mussman (GPO) [[BR]] Tasks: What persistent operational services must be supported: [[BR]] a) Descriptor registry required? use UNIS? DOR? iRODS? IF-MAP? [[BR]] b) Object registry required? use DOR? [[BR]] c) XML Messaging service [[BR]] d) Measurement Information (MI)service [[BR]] e) GENI User services [[BR]] f) Digital Object Archive (DOA) service [[BR]] Can all services can be shared by GIMI and GEMINI tools? [[BR]] How do these relate to other GENI services/functions, including clearinghouse? [wiki:InstMeasTopic_4.10OperationalServices Work in Progress] [[BR]] === Discussion of next steps === 11:55am [[BR]] == 3) Break for lunch == 12noon [[BR]] == 4) Update on GIMI project == 1:30pm [[BR]] '''PI''' Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu) [[BR]] '''Co-PI''' Max Ott, NICTA (mailto:max.ott@nicta.com.au) [[BR]] '''Key personnel''' Ilia Baldine, RENCI (mailto:ibaldin@renci.org) [[BR]] [wiki:GIMI GIMI wiki] [[BR]] [http://gimi.ecs.umass.edu/ GIMI website at UMass Amherst] [[BR]] Goals: [[BR]] **Starts with OML, and provides easy-to-use tools for GENI environment [[BR]] *First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf [[BR]] *Second focus: I&M use cases for infrastructure measurement [[BR]] *Later focuse: add sensor measurement data, carried in data network [[BR]] *Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] *Later, support: WiMAX sites [[BR]] *Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] *Introduces XML messaging service, with pub/sub, in public IP space [[BR]] *Introduces iRODS service for digital object archive. [[BR]] Members of the GIMI project will review: [[BR]] *Current architecture and design [[BR]] *Current status [[BR]] *Preview of tonight's GEC-13 demo [[BR]] *Plan for GEC-14 tutorial [[BR]] == 5) Update on GEMINI project == 2pm [[BR]] '''PI''' Martin Swany, Indiana University (mailto:swany@iu.edu) [[BR]] '''Co-PI''' Chris Small, Indiana University (mailto:chsmall@indiana.edu) [[BR]] '''Co-PI''' James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu) [[BR]] '''Co-PI''' Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu) [[BR]] [wiki:GEMINI GEMINI wiki] [[BR]] Goals: [[BR]] **Starts with LAMP, which provides flexible tools based on perfSONAR for active infrastructure measurements [[BR]] ++Fold INSTOOLS features into LAMP to form GEMINI tools, and then discontinue INSTOOLS [[BR]] *First focus: I&M use cases for infrastructure measurement *Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential [[BR]] *Show how to monitor clusters/racks [[BR]] *Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/OnTimeMeasure_GEC13-I%26M-Talk.pdf Slides]) [[BR]] *Second focus: I&M use cases for experiments [[BR]] *Consider how to gather basic host measurements with BLiPP service (as was done by INSTOOLS using SNMP) [[BR]] *Consider how to gather customized data from applications (as provided by OML Client) [[BR]] ++Extending LAMP with INSTOOLS features: [[BR]] *Consider how to load tools on top of basic image [[BR]] *Consider how to extend Periscope to include druple content management service [[BR]] *Consider how to add xserver to basic node image, to login from portal using vnc protocol [[BR]] *Continue separate portal service, which is really a gateway service, within a user workspace service [[BR]] *Consider how to add interface to archive service, e.g., iRODs [[BR]] *Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] *Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] *Continue INSTOOLS portal service, in user workspace, to find GUIs [[BR]] *Continue LAMP local and global UNIS service, to register available measurement data, and network topology [[BR]] Members of the GEMINI project will review: [[BR]] *Current architecture and design [[BR]] *Current status [[BR]] *Preview of tonight's GEC-13 demo [[BR]] *Plan for GEC-14 tutorial [[BR]] == 6) Support for experimenters == 2:30pm [[BR]] Round-table discussion: [[BR]] What use cases must be supported by GEC14? by GEC15? [[BR]] What tools must be available by GEC14? by GEC15? [[BR]] How can we best verify that available tools meet the needs of experimenters? process for fixing bugs? process for making extensions? [[BR]] How can we best reach out to experimenters? tutorials? workshops? on-line instructions? examples? help desks? [[BR]] == 7) Support for infrastructure measurement == 2:45pm [[BR]] Round-table discussion: [[BR]] What use cases must be supported by GEC14? by GEC15? [[BR]] What tools must be available by GEC14? by GEC15? [[BR]] What use cases/tools will support experimenters? [[BR]] What use cases/tools will support operators? [[BR]] How can we best verify that available tools meet the needs of experimenters and operators? process for fixing bugs? process for making extensions? [[BR]] How can we best reach out to experimenters? tutorials? workshops? on-line instructions? examples? help desks? [[BR]] == 8) Adjourn == 3pm [[BR]]