wiki:GEC14Agenda/IMDesignTopics

Version 23 (modified by hmussman@bbn.com, 12 years ago) (diff)

--

Instrumentation and Measurement (I&M) Design Topics

Monday, July 9, 8am - 10am
Room:
Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies)

Description

This I&M session will review progress towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.

Since the I&M projects continuing from Solicitation 2 are ending, it will focus on the two large Solicitation 3 I&M D&P projects, GIMI and GEMINI, starting with a review their design and status.

Next, we will review the current status of three ongoing I&M design topics, and five operational I&M services that are being introduced by the GIMI and GEMINI projects.

Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

Summary

Agenda

This is a tentative agenda, which may change.

1) Introductions

8:00am

2) Goals for this meeting

8:05am

Harry Mussman (GPO)

  • Review the progress of the GIMI and GEMINI I&M projects towards providing a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.
  • Review the current status of three ongoing I&M design topics.
  • Review five operational I&M services that are being introduced by the GIMI and GEMINI projects, to see which can be used by both projects and/or by other parts of GENI.
  • Discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.

3) Summary of GIMI design and status

8:10am
PI Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu)
Co-PI Max Ott, NICTA (mailto:max.ott@nicta.com.au)
Key personnel Ilia Baldine, RENCI (mailto:ibaldin@renci.org)

GIMI wiki
GIMI website at UMass Amherst

Goals:

  • Starts with OML, and provides easy-to-use tools for GENI environment
  • First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf
  • Second focus: I&M use cases for infrastructure measurement
  • Later focus: add sensor measurement data, carried in data network
  • Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Later, support: WiMAX sites
  • Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Introduces XML messaging service, with pub/sub, in public IP space
  • Introduces iRODS service, as a measurement data archive.
  • Introduces IREEL portal service, for measurement data analysys and presentation.

Mike Zink will review:

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

4) Summary of GEMINI design and plans

8:25am
PI Martin Swany, Indiana University (mailto:swany@iu.edu)
Co-PI Chris Small, Indiana University (mailto:chsmall@indiana.edu)
Co-PI James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu)
Co-PI Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu)

GEMINI wiki

Goals:

  • Includes LAMP code and features into GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements
  • Includes INSTOOLS code and features into GEMINI tools, to gather basic host measurements
  • Later, integrate LAMP and INSTOOLS code and features into new code base for GEMINI tools.
  • First focus: I&M use cases for infrastructure measurement
  • Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential
  • Show how to monitor clusters/racks
  • Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project)
  • Second focus: I&M use cases for experiments
  • Gather basic host measurements (as was done by INSTOOLS using SNMP)
  • Show how to gather customized data from applications (as provided by OML Client)
  • Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
  • Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
  • Continues INSTOOLS portal service, to find GUIs
  • Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology

Martin Swany and Jim Griffioen will review:

  • Current architecture, design and status.
  • Plan for GEC-14 tutorial
  • Next steps, during Spiral 5

5) Reports on I&M design topics

8:40m

T4) GENI I&M Tool Testing Environment

8:40am

Harry Mussman (GPO)

Goals

Provide a concise definition of the environment where the GENI I&M tools will be tested.

Tasks

Include:
User tools for experiment and measurement setup and orchestration
Interfaces/protocols between user tools and GENI aggregates
Supported GENI aggregates

Summary

work in progress
Current configuration (Spiral 4)
Next configuration (Spiral 5)

T8) GENI User Tools and Services

8:45am

Jeanne Ohren (GPO)

Goals

Work towards Max Ott's vision for experiment support
Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".
Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present their measurement data.
These services should work together via APIs, etc., to streamline the experiment process.

Tasks

Based upon the configuration defined below, the implementation is split into:
1) A GENI User Workspace, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools
2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.
Define, prototype, deploy and operate a GENI User Workspace. It can be hosted on a server dedicated to the user (even the user's laptop), or on a server hosting multiple user workspaces for multiple users.
Gather the various "user tools" that have been implemented to date, and fit them into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.
Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).
Use the "GENI User Workspace" to test the GIMI and GEMINI tools, and during their tutorials.

Summary

work in progress
Current configuration (Spiral 4)
Next configuration (Spiral 5)

T5) Descriptors, Objects and Registries

8:50am

Harry Mussman (GPO)

Tasks

Need to finalize MDOD schema, for XML file. References
Want to extend MDOD to cover all types of objects, i.e., software images. (NetKarma)
Want to use MDOD schema to define Event Record schema. (NetKarma)
Do we need MDOD registry? Use UNIS lookup service? Include in iRODS?
Need MDOD creation and editing service. (who?)
Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle

Summary

work in progress
v0.1 design reviewed at GEC13 slides (Giridhar Manepalli) Conclusion: too complex
DRAFT simplified v0.2 design [ slides] (Harry Mussman)

Discussion

Who will review? Shall we have a breakout meeting on Wed, 7/11?
Who will implement?
How will this be included into iRODS?

6) Reports on persistent multi-user I&M services, being introduced by GIMI and GEMINI projects

9:00am

Goals

Use by both GIMI and GEMINI tools sets
Use by other parts of GENI
Identify need for further reviews and discussions in a breakout meeting on Wed, 7/11

S1) XML Messaging Service

9:00am

Anirban Mandal (RENCI)
Tasks

Define, prototype, demonstrate and operate a GENI XML Messaging service, starting at GEC13.
Support use in GENI by many tools, starting with GIMI I&M tools, who will use it to exchange OMF messages.
Define operations plan for XML Messaging service.

Summary

work in progress
An XMPP server is operational at Rutgers WINLAB (and other GENI wireless sites) for OMF messages.
An XML messaging service has been prototyped by the IMF project. [ slides] (see also I&M and Monitoring session at GEC14)
An XMPP server has been setup at UMass Amherst.
Current configuration [ slides]
Proposed configuration [ slides]
Operations plan [ slides]

Discussion

Who is lead?
Where will XMPP servers be located? will they be fully federated?
How do we set "topics" in XMPP servers to support OMF messages?
Will these XMPP servers provide a generalized GENI XML messaging service? If so, what functions will be included? What authorization mechanisms will be required?
Will the GEMINI project want to use the XML messaging service?
Will other parts of GENI (e.g., monitoring) want to use the XML messaging service?
Do we need a breakout session for further discussion? when?

S2) iRODS Service

9:10am

Shu Huang (RENCI)

Tasks

Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions.

Summary

Work in Progress
slides
iRODS service has been established at RENCI
iRODS service has been established at Univ of Kentucky
Current configuration [ slides]
Proposed configuration [ slides]
Operations plan [ slides]

Discussion

Who is lead?
Will all iRODS services be federated, for redundancy? How to move data to service: move file; move SQL DB; or?
How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.
How are partitions for different users managed?
How is data in MDOD used to set archive sharing policy? what is required in MDOD?
Will the GEMINI project want to use the iRODS service?
Will other parts of GENI want to use the iRODS service?
Do we need a breakout session for further discussion? when?

S3) IREEL Portal Service

9:20am

Christoph Dwertmann (NICTA)

Tasks

Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project

Summary

IREEL service has been established at NICTA
IREEL service has been established at UMassAmherst
Current configuration [ slides]
Proposed configuration [ slides]
Operations plan [ slides]

Discussion

Who is lead?
How are partitions for different users managed?
How is authorization for a user done?
Can IREEL be sued to provide other functions, e.g., experiment meangement? if so, how?
Will the GEMINI project want to use the IREEL service?
Will other parts of GENI want to use the IREEL service?
Do we need a breakout session for further discussion? when?

S4) INSTOOLS portal-to-GUIs Service

9:30am

Charles Carpenter (U Kentucky)

Tasks:

Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools

Summary

Current configuration [ slides]
Proposed configuration [ slides]
Operations plan [ slides]

Discussion

Who is lead?
How are partitions for different users managed?
How is authorization for a user done?
What additional features are under consideration? Will the GIMI project want to use the INSTOOLS portal-to-GUIs service?
Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service?
Do we need a breakout session for further discussion? when?

S5) UNIS Service

9:40am

Ahmed El-Hassany (IU)

Tasks

Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set.

Summary

Work in Progress
Current configuration [ slides]
Proposed configuration [ slides]
Operations plan [ slides]

Discussion

Who is lead?
Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances?
What additional features are under consideration? Will the GIMI project want to use the UNIS service?
Will other parts of GENI want to use the UNIS service, e.g., stitching?
Do we need a breakout session for further discussion? when?

7) Goals for the next 4 months

9:50am

Harry Mussman (GPO)

Goals

Working and supported v1.0 GIMI tools should be available to users of ExoGENI racks
Working and supported v1.0 GEMINI tools should be available to users of InstaGENI racks (and ProtoGENI/Emulab clusters)
Tutorials for using both GIMI and GEMINI tool sets must be kept up-to-date, so that users can teach themselves
Bugs must be gathered, tracked and fixed by GIMI and GEMINI teams; final acceptance test will be performed by the GPO
Feature requests must be gathered, tracked and evaluated by GIMI and GEMINI teams

v1.1 of GIMI tools must be released by GEC15, including bug fixes and new features, fully tested and ready for day-to-day use
v1.1 of GEMINI tools must be released by GEC15, including bug fixes and new features, fully tested and ready for day-to-day use

XML messaging service should provide agreed-upon features, and be reliably operational
iRODS archive service should provide agreed-upon features, and be reliably operational
IREEL analysis and presentation service should provide agreed-upon features, and be reliably operational
INSTOOLS portal-to-GUIs service should provide agreed-upon features, and be reliably operational
Unified Network Information Service (UNIS) should provide agreed-upon features, and be reliably operational

Discussion

Do we have leads identified for all items?
How should user workspace be supported? on a VM? on a dedicated server?

8) Adjourn

10:00am

Attachments (13)