Version 1 (modified by 12 years ago) (diff) | ,
---|
- Instrumentation and Measurement (I&M): GIMI and GEMINI Design Topics …
Instrumentation and Measurement (I&M): GIMI and GEMINI Design Topics at GEC15
Tuesday, October 23, 8:30am - 10am
Room:
Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies)
Description
This I&M session will review progress towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.
It will focus on the two large Solicitation 3 I&M D&P projects, GIMI and GEMINI, starting with a review their design and status.
Next, we will review the current status of outstanding I&M design topics, and the operational I&M services that are being introduced by the GIMI and GEMINI projects. We will consider how GIMI might be able to use the GEMINI services, and vice-versa.
Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.
Summary
- GIMI I&M tool set design
- Based on OML, provides basic host and active network measurements
- Ready for use on ExoGENI racks; successful tutorial
- Introduces XML messaging service, iRODS archive service, IREEL(lab wiki) portal service; use by others? iRODS by GEMINI
- Spiral 5 goals: also work with InstaGENI racks, WiMAX sites
- Based on OML, provides basic host and active network measurements
- GEMINI I&M tool set design
- Based on INSTOOLS and perfSONAR/LAMP, provides basic host and active network measurements
- Ready for use on InstaGENI/protoGENI racks ; successful tutorial
- Introduces portal to GUIs service, UNIS unified ntwk info service; use by others?
- Spiral 5 goals: also work with ExoGENI racks
- Based on INSTOOLS and perfSONAR/LAMP, provides basic host and active network measurements
- Need simplified MDOD measurement data object descriptor schema
- Also service for creation and editing of MDOD
- Possible extensions to other GENI objects, events
- Also service for creation and editing of MDOD
Agenda
This is a tentative agenda, which may change.
1) Introductions
8:30am
2) Goals for this meeting
8:35am
Harry Mussman (GPO)
Goals
- Review the progress of the GIMI and GEMINI I&M projects towards providing a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.
- Review the current status of outstanding I&M design topics.
- Review five persistent multi-user I&M services that are being introduced by the GIMI and GEMINI projects, to see which can be used by both projects and/or by other parts of GENI.
- Discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement.
3) Summary of GIMI design and status
8:40am
PI Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu)
Co-PI Max Ott, NICTA (mailto:max.ott@nicta.com.au)
Key personnel Ilia Baldine, RENCI (mailto:ibaldin@renci.org)
GIMI wiki
GIMI website at UMass Amherst
Goals
- Starts with OML, and provides easy-to-use tools for GENI environment
- First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf
- Second focus: I&M use cases for infrastructure measurement
- Later focus: add sensor measurement data, carried in data network
- Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
- Later, support: WiMAX sites
- Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
- Introduces XML messaging service, with pub/sub, in public IP space
- Introduces iRODS service, as a measurement data archive.
- Introduces IREEL portal service, for measurement data analysys and presentation.
Summary:
- GIMI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5
- For more information, attend or audit GIMI v1.0 tutorial
4) Summary of GEMINI design and plans
8:55am
PI Martin Swany, Indiana University (mailto:swany@iu.edu)
Co-PI Chris Small, Indiana University (mailto:chsmall@indiana.edu)
Co-PI James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu)
Co-PI Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu)
Goals
- Includes LAMP code and features in GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements
- Includes INSTOOLS code and features in GEMINI tools, to gather basic host measurements
- Later, integrates LAMP and INSTOOLS code and features into new code base for GEMINI tools.
- First focus: I&M use cases for infrastructure measurement
- Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential
- Show how to monitor clusters/racks
- Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project)
- Second focus: I&M use cases for experiments
- Gather basic host measurements (as was done by INSTOOLS using SNMP)
- Show how to gather customized data from applications (as provided by OML Client)
- Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks
- Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks
- Continues INSTOOLS portal service, to find GUIs
- Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology
Summary
- GEMINI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5
- For more information, attend or audit GEMINI v1.0 tutorial
5) Reports on I&M design topics
9:10m
T4) GENI I&M Tool Testing Environment
9:10am
Harry Mussman (GPO)
Goals
- Define a step-by-step process for the GEMINI and GIMI tools sets over Spirals 4, 5 and 6 such that:
- They deliver useful tools in each spiral
- They eventually provide tools that can be included in slices built with any type of GENI hosts
- GENI hosts should include those provided by ExoGENI and InstaGENI racks, as soon as they are available
- They deliver useful tools in each spiral
- Include:
- User tools for experiment and measurement setup and orchestration
- Interfaces/protocols between user tools and GENI aggregates
- Supported GENI aggregates
- User tools for experiment and measurement setup and orchestration
Goals for Spiral 4:
- By the end of Spiral 4, GIMI tools will test/operate in slices built in ORCA/ExoGENI environment
- By the end of Spiral 4, GEMINI tools will test/operate in slices built in Emulab/protoGENI/InstaGENI environment
- As much as possible, the two groups will use common user tools and common interfaces/protocols/APIs
- As much as possible, the two groups will avoid arrangements customized to one environment
- Both groups will be ready to provide tutorials on their capabilities starting at GEC-14.
- Not yet defined: support for PlanetLab/InstaGENI environment
Goals for Spiral 5:
- By the end of Spiral 5, GIMI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not together in one slice)
- By the end of Spiral 5, GIMI tools will also support measurements at WiMAX sites.
- By the end of Spiral 5, GEMINI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not together in one slice)
- Not yet defined: support for PlanetLab/InstaGENI environment
Goals for later:
- Both GEMINI and GIMI tools will test/operate in slices built with both Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments. However, this requires stitching functions that are not currently available.
T8) GENI User Tools and Services
9:20am
Jeanne Ohren (GPO)
Goals
- Work towards Max Ott's vision for experiment support
- Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".
- Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present their measurement data.
- These services should work together via APIs, etc., to streamline the experiment process.
Tasks
- Based upon the configuration defined below, the implementation is split into:
1) A GENI User Workspace, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools
2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.
- Define, prototype, deploy and operate a GENI User Workspace.
- Gather the various "user tools" that have been implemented to date, and fit them into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.
- So that the GENI User Workspacet can be hosted on a server dedicated to the user (even the user's laptop), provide on a Virtual Box "portable VM".
- Use the "GENI User Workspace" to test the GIMI and GEMINI tools, and during their tutorials.
Summary
Discussion
- Should the GENI User Workspace be supported on a server hosting multiple user workspaces for multiple users?
- Which tools should be added to the GENI User Workspace?
- How can we best optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).
T5) Descriptors, Objects and Registries
9:30am
Harry Mussman (GPO)
Summary
- Current v0.2 design references
- status at GEC13
Review of v0.2 design at GEC13 (Giridhar Manepalli):
Conclusions:
Good things:
Excellent start
Collaborative Specification
Great Coverage
Nicely broken down into elements
Mandatory vs. optional elements identified
Genuine Use Cases: Gathering, transferring, and sharing
Jensen's proposal (NetKarma):
Current: Identifiers, Descriptors, Holders
Proposed: Identification, Lineage/Provenance, Constraints/Security, MDO Description
Zurawski's comments:
Too many secondary identifiers
Descriptors should be contextualized
Variations based on the type of object
GENI specific descriptions should be clearly marked and separated
Slight changes to names & enclosing elements recommended
Comments/suggestions based on metadata practices:
Too many optional elements
Too many choices given to users
Users bound to take the path of least resistance
Keep the scope restricted to only mandatory elements – at least in the beginning
Try those out. Implement them.
One size fits all ---- No!
Capturing descriptions, formats, policies, transactions, etc. in a monolithic fashion
Register individual components separately
E.g., Capture legal formats & interpretations in their own records, and reference them here
E.g., Same with accepted policies
Identifiers cannot be semantic
Domain, sub-domain, and object-type are part of an ID
World view changes frequently
Non-semantic Ids are worth every penny
Search engines & registries mask the opaqueness
After all, IDs are just entities behind the scenes
Object Type controlled vocabulary enumerates apples and oranges
Collection, flow, directory, file, database, gui are not mutually exclusive
Doesn’t help the recipient make any decision looking at the descriptor
Bundle type & format into format interpretation method
Covers too many corner cases, e.g., flow-rate
Expects too many details, e.g., locator (type, access method, etc.)
Tasks
- Need to DRAFT simplified v1.0 MDOD schema, for XML file; then review and finalize.
- Optional: extend MDOD to cover all types of objects, i.e., software images.
- Optional: extend MDOD schema to define Event Record schema.
- Do we need MDOD registry? Include in iRODS?
- Need MDOD creation and editing service.
- Need Measurement Data Object identifiers (names); sometimes need a persistent, public reference; consider DataCite approach, which uses handle
Discussion
- Who will help DRAFT simplified v1.0 MDOD schema, for XML file?
- GIMI rep?
- GEMINI rep?
- Shall we have a breakout meeting on Wed, 7/11?
- GIMI rep?
- Who will implement MDOD creation and editing service?
- How will MDOD be included in iRODS?
6) Reports on persistent multi-user I&M services, being introduced by GIMI and GEMINI projects
9:00am
Goals
- Use by both GIMI and GEMINI tools sets
- Use by other parts of GENI
- Identify need for further reviews and discussions, in breakout meetings on Wed, 7/11
S1) XML Messaging Service
9:00am
Mike Zink (UMass Amherst) and Anirban Mandal (RENCI)
Tasks
- Define, prototype, demonstrate and operate a GENI XML Messaging service, starting at GEC13.
- Support use in GENI by many tools, starting with GIMI I&M tools, who will use it to exchange OMF messages.
- Define operations plan for XML Messaging service.
Summary
- status at GEC13
- An XMPP server has been setup at UMass Amherst for OMF messages.
- An XMPP server is operational at Rutgers WINLAB (and other GENI wireless sites) for OMF messages.
- An XML messaging service has been prototyped by the IMF project, that includes authorization.
- status at GEC14
Discussion
- Who is lead/leads?
- Where will XMPP servers be located? will they be fully federated?
- How do we set "topics" in XMPP servers to support OMF messages?
- Will these XMPP servers provide a generalized GENI XML messaging service? If so, what functions will be included?
- What authorization mechanisms will be required?
- Will the GEMINI project want to use the XML messaging service?
- Will other parts of GENI (e.g., monitoring) want to use the XML messaging service?
- Do we need a breakout session for further discussion? when?
- We need an operations plan for reliable service; when? who?
S2) iRODS Service
9:10am
Shu Huang (RENCI)
Tasks
- Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions.
Summary
- status at GEC13
- slides at GEC13
- iRODS service has been established at RENCI
- iRODS service has been established at Univ of Kentucky
- iRODS configuration at GEC14
- Proposed configuration?
- Operations plan?
Discussion
- Who is lead?
- Will all iRODS services be federated, for redundancy?
- How to move data to service: move file; move SQL DB; or?
- How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.
- How are partitions for different users managed?
- How is authorization for a user done?
- How is data in MDOD used to set archive sharing policy? what is required in MDOD?
- How will the GEMINI project use the iRODS service?
- Will other parts of GENI want to use the iRODS service?
- Do we need a breakout session for further discussion? when?
S3) IREEL/Lab wiki Portal Service
9:20am
Max Ott and Christoph Dwertmann (NICTA); Mike Zink and Cong Wang (UMass Amherst)
Tasks
- Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project
- Extend/change to become Lab wiki service
Summary
- IREEL service has been established at NICTA
- IREEL service has been established at UMassAmherst
- Current configuration [ slides]
- Proposed configuration [ slides]
- Operations plan [ slides]
Discussion
- Who is lead?
- How are partitions for different users managed?
- How is authorization for a user done?
- Can IREEL be used to provide other functions, e.g., experiment meangement? if so, how?
- Will the GEMINI project want to use the IREEL service?
- Will other parts of GENI want to use the IREEL service?
- Do we need a breakout session for further discussion? when?
S4) GEMINI portal-to-GUIs Service
9:30am
Charles Carpenter (U Kentucky)
Tasks:
- Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools
Summary
Discussion
- Who is lead?
- How are partitions for different users managed?
- How is authorization for a user done?
- What additional features are under consideration?
- Will the GIMI project want to use the INSTOOLS portal-to-GUIs service?
- Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service?
- Do we need a breakout session for further discussion? when?
S5) UNIS Service
9:40am
Ahmed El-Hassany (IU)
Tasks
- Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set.
Summary
Discussion
- Who is lead?
- Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances?
- What additional features are under consideration?
- What new interfaces are under consideration?
- Will the GIMI project want to use the UNIS service?
- Will other parts of GENI want to use the UNIS service, e.g., stitching?
- Do we need a breakout session for further discussion? when?
7) Goals for the next 4 months
9:50am
Harry Mussman (GPO)
Goals
- Both GIMI and GEMINI tools should be available to Users during this period
- Acceptance testing continued by the GPO (Jeanne Ohren)
- Bug tracking system, plus periodic bug fixes, continued (who are leads?)
- How can we facilitate feedback from Users, after EVERY use?
- Acceptance testing continued by the GPO (Jeanne Ohren)
- Both GIMI and GEMINI must provide a high level of support to Users
- Tutorials must be kept up-to-date and operational, so that Users can teach themselves. (who are leads?)
- Instructions must be continuously refined, to make the tools easier to use. (who are leads?)
- Requested tool extensions must be gathered from users, based upon their experience. (who are leads?)
- Users with advanced requirements must be found, better identify gaps in the tools. How? (who are leads?)
- Tool extensions must be proposed by the teams, and reviewed with the users. How? (who are leads?)
- A list of tool extensions needs to be developed for inclusion into v1.1, due at GEC15. (who are leads?)
- Tutorials must be kept up-to-date and operational, so that Users can teach themselves. (who are leads?)
- Persistent multi-user services must be reliably available to all Users
- User Worksapce (Jeanne Ohren)
- XML messaging service (who is lead?)
- iRODS archive service (who is lead?)
- IREEL analysis and presentation service (who is lead?)
- INSTOOLS portal-to-GUIs service (who is lead?)
- Unified Network Information Service (UNIS) (who is lead?)
- User Worksapce (Jeanne Ohren)
Discussion
- Do we have leads identified for all items?
8) Adjourn
10:00am
Attachments (3)
- iRODS GENI plan.docx (132.6 KB) - added by 12 years ago.
- MD_Workspace_And_Archive_GEC15.pptx (468.1 KB) - added by 12 years ago.
- IM_AA.pptx (213.1 KB) - added by 12 years ago.
Download all attachments as: .zip