wiki:GEC14Agenda/IMDesignTopics

Version 34 (modified by hmussman@bbn.com, 7 years ago) (diff)

--

Instrumentation and Measurement (I&M) Design Topics at GEC14

Monday, July 9, 8am - 10am
Room:
Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies)

Description

This I&M session will review progress towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.

Since the I&M projects continuing from Solicitation 2 are ending, it will focus on the two large Solicitation 3 I&M D&P projects, GIMI and GEMINI, starting with a review their design and status.

Next, we will review the current status of three ongoing I&M design topics, and five operational I&M services that are being introduced by the GIMI and GEMINI projects.

Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

Summary

Agenda

This is a tentative agenda, which may change.

1) Introductions

8:00am

2) Goals for this meeting

8:05am

Harry Mussman (GPO)

Goals

  • Review the progress of the GIMI and GEMINI I&M projects towards providing a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.
  • Review the current status of three ongoing I&M design topics.
  • Review five persistent multi-user I&M services that are being introduced by the GIMI and GEMINI projects, to see which can be used by both projects and/or by other parts of GENI.
  • Discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

3) Summary of GIMI design and status

8:10am
PI Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu)
Co-PI Max Ott, NICTA (mailto:max.ott@nicta.com.au)
Key personnel Ilia Baldine, RENCI (mailto:ibaldin@renci.org)

GIMI wiki
GIMI website at UMass Amherst

Goals

  • Starts with OML, and provides easy-to-use tools for GENI environment
  • First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf
  • Second focus: I&M use cases for infrastructure measurement
  • Later focus: add sensor measurement data, carried in data network
  • Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Later, support: WiMAX sites
  • Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Introduces XML messaging service, with pub/sub, in public IP space
  • Introduces iRODS service, as a measurement data archive.
  • Introduces IREEL portal service, for measurement data analysys and presentation.

Summary:

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

4) Summary of GEMINI design and plans

8:25am
PI Martin Swany, Indiana University (mailto:swany@iu.edu)
Co-PI Chris Small, Indiana University (mailto:chsmall@indiana.edu)
Co-PI James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu)
Co-PI Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu)

GEMINI wiki

Goals

  • Includes LAMP code and features in GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements
  • Includes INSTOOLS code and features in GEMINI tools, to gather basic host measurements
  • Later, integrates LAMP and INSTOOLS code and features into new code base for GEMINI tools.
  • First focus: I&M use cases for infrastructure measurement
  • Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential
  • Show how to monitor clusters/racks
  • Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project)
  • Second focus: I&M use cases for experiments
  • Gather basic host measurements (as was done by INSTOOLS using SNMP)
  • Show how to gather customized data from applications (as provided by OML Client)
  • Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Continues INSTOOLS portal service, to find GUIs
  • Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology

Summary

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

5) Reports on I&M design topics

8:40m

T4) GENI I&M Tool Testing Environment

8:40am

Harry Mussman (GPO)

Goals

  • Define a step-by-step process for the GEMINI and GIMI tools sets over Spirals 4, 5 and 6 such that:
    • They deliver useful tools in each spiral
    • They eventually provide tools that can be included in slices built with any type of GENI hosts
    • GENI hosts should include those provided by ExoGENI and InstaGENI racks, as soon as they are available
  • Include:
    • User tools for experiment and measurement setup and orchestration
    • Interfaces/protocols between user tools and GENI aggregates
    • Supported GENI aggregates

No image "Visio-030212_UseCases_Projects_Figures_Page_04.jpg" attached to GEC14Agenda/IMDesignTopics

Goals for Spiral 4:

  • By the end of Spiral 4, GIMI tools will test/operate in slices built in ORCA/ExoGENI environment
  • By the end of Spiral 4, GEMINI tools will test/operate in slices built in Emulab/protoGENI/InstaGENI environment
  • As much as possible, the two groups will use common user tools and common interfaces/protocols/APIs
  • As much as possible, the two groups will avoid arrangements customized to one environment
  • Both groups will be ready to provide tutorials on their capabilities starting at GEC-14.
  • Not yet defined: support for PlanetLab/InstaGENI environment

No image "Visio-030212_UseCases_Projects_Figures_Page_04.jpg" attached to GEC14Agenda/IMDesignTopics

Goals for Spiral 5:

  • By the end of Spiral 5, GIMI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not together in one slice)
  • By the end of Spiral 5, GIMI tools will also support measurements at WiMAX sites.
  • By the end of Spiral 5, GEMINI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not together in one slice)
  • Not yet defined: support for PlanetLab/InstaGENI environment

No image "Visio-030212_UseCases_Projects_Figures_Page_04.jpg" attached to GEC14Agenda/IMDesignTopics

Goals for later:

  • Both GEMINI and GIMI tools will test/operate in slices built with both Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments. However, this requires stitching functions that are not currently available.

T8) GENI User Tools and Services

8:45am

Jeanne Ohren (GPO)

Goals

  • Work towards Max Ott's vision for experiment support
  • Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".
  • Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present their measurement data.
  • These services should work together via APIs, etc., to streamline the experiment process.

Tasks

  • Based upon the configuration defined below, the implementation is split into:
    1) A GENI User Workspace, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools
    2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.
  • Define, prototype, deploy and operate a GENI User Workspace.
  • Gather the various "user tools" that have been implemented to date, and fit them into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.
  • So that the GENI User Workspacet can be hosted on a server dedicated to the user (even the user's laptop), provide on a Virtual Box "portable VM".
  • Use the "GENI User Workspace" to test the GIMI and GEMINI tools, and during their tutorials.

Summary

Discussion

  • Should the GENI User Workspace be supported on a server hosting multiple user workspaces for multiple users?
  • Which tools should be added to the GENI User Workspace?
  • How can we best optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).

T5) Descriptors, Objects and Registries

8:50am

Harry Mussman (GPO)

Tasks

  • Need to finalize MDOD schema, for XML file. References
  • Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)
  • Want to use MDOD schema to define Event Record schema. (NetKarma)
  • Do we need MDOD registry? Include in iRODS?
  • Need MDOD creation and editing service. (who?)
  • Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle

Summary

  • work in progress
  • v0.1 design reviewed at GEC13 slides (Giridhar Manepalli) Conclusion: too complex
  • DRAFT simplified v0.2 design [ slides] (Harry Mussman)

Discussion

  • Who will review? Shall we have a breakout meeting on Wed, 7/11?
  • Who will implement MDOD creation and editing service?
  • How will MDOD be included in iRODS?

6) Reports on persistent multi-user I&M services, being introduced by GIMI and GEMINI projects

9:00am

Goals

  • Use by both GIMI and GEMINI tools sets
  • Use by other parts of GENI
  • Identify need for further reviews and discussions, in breakout meetings on Wed, 7/11

S1) XML Messaging Service

9:00am

Anirban Mandal (RENCI)
Tasks

  • Define, prototype, demonstrate and operate a GENI XML Messaging service, starting at GEC13.
  • Support use in GENI by many tools, starting with GIMI I&M tools, who will use it to exchange OMF messages.
  • Define operations plan for XML Messaging service.

Summary

  • work in progress
  • An XMPP server is operational at Rutgers WINLAB (and other GENI wireless sites) for OMF messages.
  • An XML messaging service has been prototyped by the IMF project. [ slides]
  • An XMPP server has been setup at UMass Amherst.
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Where will XMPP servers be located? will they be fully federated?
  • How do we set "topics" in XMPP servers to support OMF messages?
  • Will these XMPP servers provide a generalized GENI XML messaging service? If so, what functions will be included?
  • What authorization mechanisms will be required?
  • Will the GEMINI project want to use the XML messaging service?
  • Will other parts of GENI (e.g., monitoring) want to use the XML messaging service?
  • Do we need a breakout session for further discussion? when?

S2) iRODS Service

9:10am

Shu Huang (RENCI)

Tasks

  • Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions.

Summary

  • Work in Progress
  • slides
  • iRODS service has been established at RENCI
  • iRODS service has been established at Univ of Kentucky
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Will all iRODS services be federated, for redundancy?
  • How to move data to service: move file; move SQL DB; or?
  • How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.
  • How are partitions for different users managed?
  • How is authorization for a user done?
  • How is data in MDOD used to set archive sharing policy? what is required in MDOD?
  • How will the GEMINI project use the iRODS service?
  • Will other parts of GENI want to use the iRODS service?
  • Do we need a breakout session for further discussion? when?

S3) IREEL Portal Service

9:20am

Christoph Dwertmann (NICTA) and Cong Wang (UMass Amherst)

Tasks

  • Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project

Summary

  • IREEL service has been established at NICTA
  • IREEL service has been established at UMassAmherst
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • How are partitions for different users managed?
  • How is authorization for a user done?
  • Can IREEL be used to provide other functions, e.g., experiment meangement? if so, how?
  • Will the GEMINI project want to use the IREEL service?
  • Will other parts of GENI want to use the IREEL service?
  • Do we need a breakout session for further discussion? when?

S4) INSTOOLS portal-to-GUIs Service

9:30am

Charles Carpenter (U Kentucky)

Tasks:

  • Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools

Summary

  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • How are partitions for different users managed?
  • How is authorization for a user done?
  • What additional features are under consideration?
  • Will the GIMI project want to use the INSTOOLS portal-to-GUIs service?
  • Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service?
  • Do we need a breakout session for further discussion? when?

S5) UNIS Service

9:40am

Ahmed El-Hassany (IU)

Tasks

  • Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set.

Summary

  • Work in Progress
  • Current configuration [ slides]
  • Proposed configuration [ slides]
  • Operations plan [ slides]

Discussion

  • Who is lead?
  • Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances?
  • What additional features are under consideration?
  • What new interfaces are under consideration?
  • Will the GIMI project want to use the UNIS service?
  • Will other parts of GENI want to use the UNIS service, e.g., stitching?
  • Do we need a breakout session for further discussion? when?

7) Goals for the next 4 months

9:50am

Harry Mussman (GPO)

Goals

  • Both GIMI and GEMINI tools should be available to Users during this period
    • Acceptance testing by the GPO (Jeanne Ohren)
    • Bug tracking system, plus periodic bug fixes (who are leads?)
    • How can we facilitate feedback from Users, after EVERY use?
  • Both GIMI and GEMINI must provide a high level of support to Users
    • Tutorials must be kept up-to-date and operational, so that Users can teach themselves. (who are leads?)
    • Instructions must be continuously refined, to make the tools easier to use. (who are leads?)
    • Requested tool extensions must be gathered from users, based upon their experience. (who are leads?)
    • Users with advanced requirements must be found, better identify gaps in the tools. How? (who are leads?)
    • Tool extensions must be proposed by the teams, and reviewed with the users. How? (who are leads?)
    • A list of tool extensions needs to be developed for inclusion into v1.1, due at GEC15. (who are leads?)
  • Persistent multi-user services must be reliably available to all Users
    • User Worksapce (Jeanne Ohren)
    • XML messaging service (who is lead?)
    • iRODS archive service (who is lead?)
    • IREEL analysis and presentation service (who is lead?)
    • INSTOOLS portal-to-GUIs service (who is lead?)
    • Unified Network Information Service (UNIS) (who is lead?)

Discussion

  • Do we have leads identified for all items?

8) Adjourn

10:00am

Attachments (13)