wiki:GEC14Agenda/IMDesignTopics

Version 27 (modified by hmussman@bbn.com, 12 years ago) (diff)

--

Instrumentation and Measurement (I&M) Design Topics

Monday, July 9, 8am - 10am
Room:
Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies)

Description

This I&M session will review progress towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.

Since the I&M projects continuing from Solicitation 2 are ending, it will focus on the two large Solicitation 3 I&M D&P projects, GIMI and GEMINI, starting with a review their design and status.

Next, we will review the current status of three ongoing I&M design topics, and five operational I&M services that are being introduced by the GIMI and GEMINI projects.

Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

Summary

Agenda

This is a tentative agenda, which may change.

1) Introductions

8:00am

2) Goals for this meeting

8:05am

Harry Mussman (GPO)

Goals

  • Review the progress of the GIMI and GEMINI I&M projects towards providing a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.
  • Review the current status of three ongoing I&M design topics.
  • Review five persistent multi-user I&M services that are being introduced by the GIMI and GEMINI projects, to see which can be used by both projects and/or by other parts of GENI.
  • Discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

3) Summary of GIMI design and status

8:10am
PI Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu)
Co-PI Max Ott, NICTA (mailto:max.ott@nicta.com.au)
Key personnel Ilia Baldine, RENCI (mailto:ibaldin@renci.org)

GIMI wiki
GIMI website at UMass Amherst

Goals

  • Starts with OML, and provides easy-to-use tools for GENI environment
  • First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf
  • Second focus: I&M use cases for infrastructure measurement
  • Later focus: add sensor measurement data, carried in data network
  • Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Later, support: WiMAX sites
  • Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Introduces XML messaging service, with pub/sub, in public IP space
  • Introduces iRODS service, as a measurement data archive.
  • Introduces IREEL portal service, for measurement data analysys and presentation.

Summary:

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

4) Summary of GEMINI design and plans

8:25am
PI Martin Swany, Indiana University (mailto:swany@iu.edu)
Co-PI Chris Small, Indiana University (mailto:chsmall@indiana.edu)
Co-PI James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu)
Co-PI Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu)

GEMINI wiki

Goals

  • Includes LAMP code and features in GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements
  • Includes INSTOOLS code and features in GEMINI tools, to gather basic host measurements
  • Later, integrates LAMP and INSTOOLS code and features into new code base for GEMINI tools.
  • First focus: I&M use cases for infrastructure measurement
  • Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential
  • Show how to monitor clusters/racks
  • Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project)
  • Second focus: I&M use cases for experiments
  • Gather basic host measurements (as was done by INSTOOLS using SNMP)
  • Show how to gather customized data from applications (as provided by OML Client)
  • Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Continues INSTOOLS portal service, to find GUIs
  • Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology

Summary

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

5) Reports on I&M design topics

8:40m

T4) GENI I&M Tool Testing Environment

8:40am

Harry Mussman (GPO)

Goals

  • Provide a concise definition of the environment where the GENI I&M tools will be tested.

Tasks

  • Include:
  • User tools for experiment and measurement setup and orchestration
  • Interfaces/protocols between user tools and GENI aggregates
  • Supported GENI aggregates

Summary

T8) GENI User Tools and Services

8:45am

Jeanne Ohren (GPO)

Goals

  • Work towards Max Ott's vision for experiment support
  • Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".
  • Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present their measurement data.
  • These services should work together via APIs, etc., to streamline the experiment process.

Tasks

  • Based upon the configuration defined below, the implementation is split into:
    1) A GENI User Workspace, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools
    2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.
  • Define, prototype, deploy and operate a GENI User Workspace. It can be hosted on a server dedicated to the user (even the user's laptop), or on a server hosting multiple user workspaces for multiple users.
  • Gather the various "user tools" that have been implemented to date, and fit them into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.
  • Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).
  • Use the "GENI User Workspace" to test the GIMI and GEMINI tools, and during their tutorials.

Summary

T5) Descriptors, Objects and Registries

8:50am

Harry Mussman (GPO)

Tasks

  • Need to finalize MDOD schema, for XML file. References
  • Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)
  • Want to use MDOD schema to define Event Record schema. (NetKarma)
  • Do we need MDOD registry? Include in iRODS?
  • Need MDOD creation and editing service. (who?)
  • Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle

Summary

  • work in progress
  • v0.1 design reviewed at GEC13 slides (Giridhar Manepalli) Conclusion: too complex
  • DRAFT simplified v0.2 design [ slides] (Harry Mussman)

Discussion

  • Who will review? Shall we have a breakout meeting on Wed, 7/11?
  • Who will implement MDOD creation and editing service?
  • How will MDOD be included in iRODS?

6) Reports on persistent multi-user I&M services, being introduced by GIMI and GEMINI projects

9:00am

Goals

  • Use by both GIMI and GEMINI tools sets
  • Use by other parts of GENI
  • Identify need for further reviews and discussions, in breakout meetings on Wed, 7/11

S1) XML Messaging Service

9:00am

Anirban Mandal (RENCI)
Tasks

  • Define, prototype, demonstrate and operate a GENI XML Messaging service, starting at GEC13.
  • Support use in GENI by many tools, starting with GIMI I&M tools, who will use it to exchange OMF messages.
  • Define operations plan for XML Messaging service.

Summary

  • work in progress
  • An XMPP server is operational at Rutgers WINLAB (and other GENI wireless sites) for OMF messages.
  • An XML messaging service has been prototyped by the IMF project. [ slides] (see also I&M and Monitoring session at GEC14)
  • An XMPP server has been setup at UMass Amherst.
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Where will XMPP servers be located? will they be fully federated?
  • How do we set "topics" in XMPP servers to support OMF messages?
  • Will these XMPP servers provide a generalized GENI XML messaging service? If so, what functions will be included?
  • What authorization mechanisms will be required?
  • Will the GEMINI project want to use the XML messaging service?
  • Will other parts of GENI (e.g., monitoring) want to use the XML messaging service?
  • Do we need a breakout session for further discussion? when?

S2) iRODS Service

9:10am

Shu Huang (RENCI)

Tasks

  • Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions.

Summary

  • Work in Progress
  • slides
  • iRODS service has been established at RENCI
  • iRODS service has been established at Univ of Kentucky
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Will all iRODS services be federated, for redundancy?
  • How to move data to service: move file; move SQL DB; or?
  • How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.
  • How are partitions for different users managed?
  • How is data in MDOD used to set archive sharing policy? what is required in MDOD?
  • Will the GEMINI project want to use the iRODS service?
  • Will other parts of GENI want to use the iRODS service?
  • Do we need a breakout session for further discussion? when?

S3) IREEL Portal Service

9:20am

Christoph Dwertmann (NICTA)

Tasks

  • Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project

Summary

  • IREEL service has been established at NICTA
  • IREEL service has been established at UMassAmherst
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • How are partitions for different users managed?
  • How is authorization for a user done?
  • Can IREEL be used to provide other functions, e.g., experiment meangement? if so, how?
  • Will the GEMINI project want to use the IREEL service?
  • Will other parts of GENI want to use the IREEL service?
  • Do we need a breakout session for further discussion? when?

S4) INSTOOLS portal-to-GUIs Service

9:30am

Charles Carpenter (U Kentucky)

Tasks:

  • Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools

Summary

  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • How are partitions for different users managed?
  • How is authorization for a user done?
  • What additional features are under consideration?
  • Will the GIMI project want to use the INSTOOLS portal-to-GUIs service?
  • Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service?
  • Do we need a breakout session for further discussion? when?

S5) UNIS Service

9:40am

Ahmed El-Hassany (IU)

Tasks

  • Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set.

Summary

  • Work in Progress
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances?
  • What additional features are under consideration?
  • Will the GIMI project want to use the UNIS service?
  • Will other parts of GENI want to use the UNIS service, e.g., stitching?
  • Do we need a breakout session for further discussion? when?

7) Goals for the next 4 months

9:50am

Harry Mussman (GPO)

Goals

  • Working and supported v1.0 GIMI tools should be readily available to users of ExoGENI racks (who is lead?)
  • Working and supported v1.0 GEMINI tools should be readily available to users of InstaGENI racks (and ProtoGENI/Emulab clusters) (who is lead?)
  • Tutorial for GIMI tool set must be kept up-to-date, so that users can teach themselves (who is lead?)
  • Tutorial for GEMINI tool set must be kept up-to-date, so that users can teach themselves (who is lead?)
  • Bugs must be gathered, tracked and fixed by GIMI team; final acceptance test will be performed by the GPO (who is lead?)
  • Bugs must be gathered, tracked and fixed by GEMINI team; final acceptance test will be performed by the GPO (who is lead?)
  • Feature requests must be gathered, tracked and evaluated by GIMI and GEMINI team (who is lead?)
  • Feature requests must be gathered, tracked and evaluated by GIMI and GEMINI team (who is lead?)
  • v1.1 of GIMI tools must be released by GEC15, including bug fixes and new features, fully tested and ready for day-to-day use (who is lead?)
  • v1.1 of GEMINI tools must be released by GEC15, including bug fixes and new features, fully tested and ready for day-to-day use (who is lead?)
  • XML messaging service should provide agreed-upon features, and be reliably operational (who is lead?)
  • iRODS archive service should provide agreed-upon features, and be reliably operational (who is lead?)
  • IREEL analysis and presentation service should provide agreed-upon features, and be reliably operational (who is lead?)
  • INSTOOLS portal-to-GUIs service should provide agreed-upon features, and be reliably operational (who is lead?)
  • Unified Network Information Service (UNIS) should provide agreed-upon features, and be reliably operational (who is lead?)

Discussion

  • Do we have leads identified for all items?
  • How should user workspace be supported? on a VM? on a dedicated server?

8) Adjourn

10:00am

Attachments (13)