Instrumentation and Measurement (I&M) Working Session

Wednesday, November 2, 2011, 8:3oam - 12noon
Room:
Session Leader: Harry Mussman, (GPO, Raytheon BBN Technologies)

Description

This session will review the GENI I&M design and prototyping projects planned for Spiral 4 (this year), including the two large Solicitation 3 I&M D&P projects and those continuing projects from Solicitation 2. Then, it will define and discuss the priority topics that must be resolved between now and GEC13.

Summary

Agenda

This is a tentative agenda, which may change.

1) Introductions

8:30am

2) GENI I&M D&P Projects for Spiral 4

8:35am
Harry Mussman

2.1) New Sol 3 Projects

GIMI
PI: Mike Zink (UMass Amherst)
Co-PI/Key: Max Ott (NICTA); Ilia Baldine (RENCI)

Goals:

Starts with OML, and provides easy-to-use tools in GENI environment

First focus on I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS
First supported aggregates: ORCA servers/VMs and RENCI racks
First utilized experiment control tools: GUSH, with additions
First utilized interfaces: GENI AM API for resources; ns3 for topology; ssh; add OMF for orchestration

Requires XML messaging service, with pub/sub, in public IP space

Introduces portal service, with: OML Server, for collection; Results Service, for presentation; measurement orchestration service using OMF; documentation service to create and edit MDOD. Can this be shared with GEMINI project?

Introduces iRODS service for user workspace and archive. Start with CNRI prototype? Can GEMINI use iRODS? Can iRODS serve as registry for MDOD?

Begin: support for WiMAX sites
Consider: adding OML server with perfSONAR interface, for sharing data
Later: add sensor measurement data, carried in data network

GEMINI
PI: Martin Swany (Indiana U)
Co-PI/Key: Chris Small (Indiana U); Eric Boyd (Internet2); Jim Griffioen (U Kentucky); Zongming Fei (U Kentucky)

Goals:

Starts with INSTOOLS, and continues to provide easy-to-use tools in GENI environment
Consider: how to gather customized data from application

First focus on I&M use cases for experiments
First supported aggregates: protoGENI servers/VMs; can these be used for HP racks?
First utilized experiment control tools: FLACK, with additions
First utilized interfaces: protoGENI API for resources; ns3 for topology; ssh;

Continues portal to presentation service; need to add documentation service to create and edit MDOD. Can portal be shared with GIMI project?

Continues use of Kentucky and CNRI services for user workspace and archive. Consider move to iRODS?

Starts with LAMP/perfSONAR, and continues to provide for infrastructure measurement, and for sharing of data
Continue to setup within resources assigned by slice mechanism
Consider: monitoring of clusters/racks
Consider: measurements of Layer 2 and OpenFlow paths

First focus on I&M use cases for infrastructure measurement
First supported aggregates: protoGENI servers/VMs; can these be used for HP racks?
First utilized experiment control tools: FLACK, with additions
First utilized interfaces: protoGENI API for resources; ns3 for topology; ssh;

Continues home and global Lookup Services
Create and map MDOD into metadata that is registered

Continue to use perfSONAR clients to present data; can these be integrated into portal?

Consider how INSTOOLS and LAMP/perfSONAR tools might be shared or merged

2.2) Continuing Sol 2 Projects

LAMP/perfSONAR
PI: Martin Swany (Indiana U)
Co-PI/Key: Eric Boyd (Internet2); Jason Zurawski (Internet 2); Guilherme Fernandes (U Delaware); Ezra Kissel (U Delaware)

Goals:

Provides operational home and global Lookup Services
Creates and maps MDOD into metadata that is registered in Lookup Service

Expand authorization at interface where data is shared; introduce use of GENI credentials

Continue to participate in the GENI I&M effort, particularly in defining the schema for the “measurement data object descriptor”, and mapping it to the perfSONAR metadata that is registered with the Lookup Service.

Provide LAMP images for creating “infrastructure measurement slices” in both protoGENI and ORCA environments.

Install and operate a perfSONAR Lookup Service, for use with “infrastructure measurement slices”.

Demonstrate and/or help others to demonstrate “infrastructure measurement slices” in both protoGENI and ORCA environments, with the data available at the perfSONAR MP is registered with the operational Lookup Service.

Demonstrate using GENI credentials to authorize access to the data available at the perfSONAR MP.

Define a plan for using ABAC to replace simple credentials to authorize the access to the data available at a perfSONAR MP.

OnTimeMeasure
PI: Prasad Calyam (Ohio Super Center)
Co-PI: Paul Schopis (OARnet)

Goals:

Continue to participate in the GENI I&M effort, particularly in defining the schema for the “measurement data object descriptor”.

Work with the LAMP project to understand the LAMP/perfSONAR tools, and demonstrate using them to implement “infrastructure measurement slices” in today’s GENI backbone and access networks.

Assuming LAMP/perfSONAR tools, develop a plan to evaluate the performance of today’s GENI IP backbone and access networks, using persistent or temporary “infrastructure measurement slices”, and utilizing existing perSONAR nodes in I2 and/or other networks.

Define a procedure for evaluating the performance of GENI’s backbone and access networks when carrying L2 VLANs; start with the existing IP measurement tools including in the LAMP/perfSONAR tools, but consider tools that are specialized for evaluating the performance of L2 networks.

Define a procedure for evaluating the performance of GENI’s backbone and access networks when using OpenFlow (e.g., “Tango GENI”); consider how to monitor the performance of OF networks, and introducing new specialized tools.

NetKarma
PI: Beth Plale (Indiana U)
Co-PI/Key: Chris Small (Indiana U); Scott Jensen (Indiana U)

Goals:

Participate in the GENI I&M effort, particularly in defining the schema for the “measurement data object descriptor”, and uses of “descriptors” in tracking provenance.

Work with the GENI I&M community to define extensions to the “descriptor” that would locate and describe objects beyond “measurement data objects” associated with an experiment, such as scripts, images, etc.

Modify your tools to create, gather and forward “descriptors”, that could locate and describe all objects associated with an experiment; demonstrate how “descriptors” can be automatically created.

Work with the GENI I&M community to define a modified “descriptor” that would provide a standardized GENI “resource event record”; these records could describe events such as assignments, faults or errors.

Add modules to your tools to create, gather and forward “resource event records”, that when logged could fully describe the sequence of an experiment.
Demonstrate forwarding your “records” using pub/sub servers and to repositories provided by the IMF project.

Demonstrate your tools, and support experimenters in using them.

Participate in the GENI I&M effort, in fitting your provenance tools into the “experimenter’s portal”.

IMF
PI: Rudra Dutta (NCSU)
Co-PI/Key: George Rouskas (NCSU); Ilia Baldine (RENCI); Keren Bergman (Columbia U)

Goals:

Participate in the GENI I&M effort, focusing on providing basic capabilities, and on extending the architecture to include a pub/sub mechanism.

Prototype and demonstrate a GENI I&M capability to manage I&M services/slivers within a slice using a “measurement orchestration service”; use ORBIT Management Framework (OMF) software modules provided by NICTA, including the Experiment Controller (EC) and the Resource Controller (RC), that communicate using a pub/sub mechanism utilizing an XMPP server.

Define and prototype an authorization mechanism for your sliver management service; consider message signing being proposed by NICTA.

Demonstrate your capability to manage a sliver.

Prototype and demonstrate a GENI I&M capability to transport event (or measurement) records; include software modules that collect the records, communicate using a pub/sub mechanism utilizing an XMPP server, archive the records in a repository, search the records, and display the records.

Define and prototype an authorization mechanism for your event record transport service.

Demonstrate your capability by transporting and distributing standardized GENI “resource event records”, as defined by the NetKarma project; these records could describe events such as assignments, faults or errors.

Prototype and operate a GENI I&M service to provide pub/sub services utilizing an XMPP server, for use by your sliver management and event record transport services.

Define and prototype a resource assignment mechanism (using the GENI AM API, if possible) for users (experimenters and operators) to define nodes within your GENI pub/sub service, as they establish modules that publish or subscribe.

Prototype and operate a data repository associated with your pub/sub service, that could subscribe to a node and archive all event messages.

DOR
PI: Larry Lannom (CNRI)
Co-PI/Key: Giridhar Manepalli (CNRI); Jim French (CNRI); Christophe Blanchi (CNRI)

Goals:

Continue to deploy a prototype Measurement Data Archive (MDA) service, and support integration with GENI I&M tools.

Participate in the GENI I&M effort, particularly in the MDA specification effort that includes defining metadata model for archived entities, defining access-control layer, and integrating control framework authorization scheme with the MDA.

Enforce metadata schema on archived objects, and enable access control rights as defined by the MDA specification.

Integrate GENI identity management and authorization scheme if feasible and applicable, resources permitting.

Evangelize and demonstrate how the MDA/DOR service can be used to archive published data.

2.3 GIMI Project Overview

8:45am
PI: Mike Zink (UMass Amherst)
co-PI/Key: Max Ott (NICTA); Ilia Baldine (RENCI)
[ slides]

2.4 GEMINI Project Overviews

9:05am
PI: Martin Swany (Indiana U)
Co-PI/Key: Chris Small (Indiana U); Eric Boyd (Internet2); Jim Griffioen (U Kentucky); Zongming Fei (U Kentucky)
[ slides]

2.4 LAMP/perfSONAR Project Overviews

9:25am PI: Martin Swany (Indiana U)
Co-PI/Key: Eric Boyd (Internet2); Jason Zurawski (Internet 2); Guilherme Fernandes (U Delaware); Ezra Kissel (U Delaware)
[ slides]

2.6 OnTimeMeasure Project Overview

9:30am PI: Prasad Calyam (Ohio Super Center)
Co-PI: Paul Schopis (OARnet)
[ slides]

2.7 NetKarma Project Overview

9:35am PI: Beth Plale (Indiana U)
Co-PI/Key: Chris Small (Indiana U); Scott Jensen (Indiana U)
[ slides]

2.8 IMF Project Overview

9:40am PI: Rudra Dutta (NCSU)
Co-PI/Key: George Rouskas (NCSU); Ilia Baldine (RENCI); Keren Bergman (Columbia U)
[ slides]

2.9 DOR Project Overview

9:45am PI: Larry Lannom (CNRI)
Co-PI/Key: Giridhar Manepalli (CNRI); Jim French (CNRI); Christophe Blanchi (CNRI)
[ slides]

2.10 Discussion

9:50am

3) Break

10:00am

4) GENI I&M Operational Environment at End of Spiral 4

10:30am

5) Priority Topics and Next Steps

10:35am

5.1) Overview

10:35am

I&M Use Cases for Experiments

I&M Use Cases for Infrastructure Measurement

Platform Targets

Experiment and Measurement Orchestration Tools

Interfaces/Protocols Between Tools and Services

Descriptor Schema and Registry Service

Object Names and Registry Service

XML Messaging Service

Lookup Service

Portal Service

User Workspace and Archive Services

Operational Services

Services Shared by GIMI and GEMINI

Support for Experimenters

Support for Operators

5.2) I&M for Experiments

10:40am I&M Use Cases for Experiments
Support for Experimenters

5.3) I&M Use Cases for Infrastructure Measurement

10:50am I&M Use Cases for Infrastructure Measurement
Support for Operators

5.4) GENI Environment

11:00am Platform Targets
Experiment and Measurement Orchestration Tools
Interfaces/Protocols Between Tools and Services

5.5) Descriptors, Objects and Registries

11:10am Descriptor Schema and Registry Service
Object Names and Registry Service

5.6) Messaging Service

11:20am XML Messaging Service

5.7) Lookup Service

11:25am Lookup Service

5.8) Portal Service

11:30am Portal Service

5.9) User Workspace and Archive Services

11:40am User Workspace and Archive Services

5.10 Operational Services

11:50am Operational Services
Services Shared by GIMI and GEMINI

6) Adjourn

12noon

Background Reading

​I&M integration wiki
​GENI I&M Capabilities Catalog
​GENI I&M Architecture document