wiki:GEC16Agenda/IMDesignTopics

I&M Design Topics at GEC16

Tuesday, March 19, 10:30am - 12noon
Room: City Creek
Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies)

Description

This I&M session will focus on three key issues of interest to both the GIMI and GEMINI projects, as they continue to work towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement.

We will first review a goal to move towards a "unified experiment environment".

Then, we will discuss these topics:

  • Support for GIMI and GEMINI I&M tools on both ExoGENI and InstaGENI racks
  • Use of an instrumentize script, driven by rspec extension, to configure/init all I&M tools
  • Options for further measurement data analysis and presentation

For each topic, we will have a review of current practice, and a discussion of how to move forward.

It is hoped that some of the discussions at this session may inspire proposals that will be submitted to GENI Solicitation 4.

Finally, we will form teams to continue the discussions.

Pre-Requisites

None

Agenda

This is a tentative agenda, which may change.

1) Introductions

8:30am

2) Towards a Unified Experiment Environemnt

8:35am
Marshall Brinn (GPO)
10min

slides: What does it mean to move towards a "unified experiment environment"?

3) Support GIMI and GEMINI I&M tools on both ExoGENI and InstaGENI racks

8:45am

Introduction

Harry Mussman (GPO)

GENI experiment management Step 3: Obtain slice, install software

  • Now: GIMI loaded (and tested) on ExoGENI, with Flukes
  • Now: GEMINI loaded (and tested) on InstaGENI, with FLACK
  • Next: be able to use GEMINI on ExoGENI, and GIMI on InstaGENI
  • Suggest: set goal to begin testing all combinations by 5/1; good support by GEC17

Lessons learned from using GIMI tools on ExoGENI racks

Cong Wang (UMass Amherst)
5min

slides: current config; key techniques; known challenges

Configuration:

  • No extra node
  • Persistent XMPP server, with reachable IP address
  • Presistent OML server (p/o GIMI portal), with reachable IP address

Lessons learned from using GEMINI tools on InstaGENI racks

Hussam Nasir (Univ Kentucky)
5min

slides: current config; key techniques; known challenges

Configuration:

  • Only support for VMs; largest to date 20 nodes; to date, all on one aggregate
  • One extra VM for "Global Node"
  • Add desired measurements to request rspec
  • Compatible image for quick setup

Discussion

Cong Wang (UMass Amherst; Hussam Nasir (Univ Kentucky); Louisa Nevers (GPO); Jeanne Ohren (GPO)
15min

How do we move towards using GEMINI on ExoGENI, and GIMI on InstaGENI?

Discuss:

  • Suggested approaches
  • Key techniques
  • Likely challenges
  • Goal to begin testing all combinations by 5/1; good support by GEC17; are there any roadblocks?

1) Issues:

  • images
  • rspecs, since slight differences between ExoGENI and InstaGENI
  • workflow differences

2) Helpful:

  • list of reference experiments
  • repository of rspecs

3) Start testing soon!

Team to continue discussion:

Cong Wang (UMass Amherst; Hussam Nasir (Univ Kentucky); Ilia Baldine (RENCI); Rob Ricci (Univ Utah); Louisa Nevers (GPO); Jeanne Ohren (GPO)

4) Use of instrumentize script, driven by rspec extension, to configure/initialize I&M tools

9:10am

Overview

Harry Mussman (GPO)

GENI experiment management Step 4: Configure/init I&M and exp app

  • Now: GIMI using post boot scripts, OMF
  • Now: GEMINI using instrumentize script, with choices based on rspec extensions (ignored by AggMgr's); also used to add GN (an extra node, one per Agg, for common GEMINI services)
  • Next: move towards common approach, to better unify experiment environment
  • Suggest: both move towards script, driven by rspec ext

Current GIMI use of post boot scripts, OMF, to configure/inti services

Cong Wang (UMass Amherst)
5min

slides: key techniques; known challenges

Post-boot scripts can do anything a shell script can do

Prime use: connect OMF RC on node into messaging system, so that OMF EC can talk to node

Current GEMINI use of instrumentize script, to configure/init services

Hussam Nasir (Univ Kentucky)
5min

slides: key techniques; known challenges

Now: Python scripts, that ssh into nodes

Discussion

Cong Wang (Umass Amherst) ; Hussam Nasir (Univ Kentucky); Marshall Brinn (GPO); Jeanne Ohren (GPO)
15min

How do we move towards a common approach to configure and initialize I&M tools, to better unify experiment environment?

Strawman:

  • move towards instrumentize script, driven by rspec ext

Conclusions:

1) Can do equivalent of post-boot script in InstaGENI

2) Instrumentize script uses ssh, checks config, then sets config

3) Would like to be able to easily setup a basic set of measurements

4) Not clear how to resolve; next:

  • draw out both flows
  • discuss
  • try to unify

Team for further discussion:

Cong Wang (UMass Amherst); Mike Zink (UMass Amherst); Hussam Nasir (Univ Kentucky); Harry Mussman (GPO); Jeanne Ohren (GPO)

5) Options for further measurement data analysis and presentation

9:35am

Introduction

Harry Mussman (GPO)

GENI experiment management Step 6: Analyze

  • Now: Both GEMINI and GIMI push measurement data objects (files) to GENI Storage Service (iRODS)
  • Now: GEMINI provides dump of mongo DB, plus separate file of metadata
  • Now: GIMI (will) provide SQLdump of PostgreSQL DB, which includes metadata
  • Now: Both use portal on GENI Storage Service (iRODS) to curate files (objects)

Both: Pull measurement data files from GENI Storage Service (iRODS) back to user workspace, where DBs can be reconstituted, and data pulled for further analysis and presentation.

Both: Many options for further analysis and presentation, not being developed as part of tools.

Next: Consider possible ways to unify data formats, analysis and presentation; what commonly available tools might be useful?

Current GIMI measurement data object schema

Stephen Glass (NICTA)
10min

GIMI measurement data object schema:

  • OML moving to OML 6.0, which includes metadata
  • OML server has DB backend, SQlite or PostgreSGL
  • Version of DB stored in iRODS

Current GEMINI measurement data object schema

Ezra Kissel (Indiana Univ)
10min

slides: GEMINI measurement data object schema

GEMINI measurement data object schema:

  • maesurement data stored in Mongo DB
  • topology and metadata stored in UNIS
  • versions of DB, topology and metadata stored in iRODS

Discussion

Stephen Glass (NICTA); Ezra Kissel (Indiana Univ); Marshall Brinn (GPO); Fraida Fund (NYU Poly)
10min

How could we unify further analysis and presentation?

Conclusions:

1) Both GIMI and GEMINI store measurement data into iRODS as a DB, but with different metadata formats

2) Want an open format; do we need to merge into one open format, or jst accept two open formats?

3) Earlier effort: Emulab workbench; ref?

4) Current effort: Research Data Alliance, formed to unify stored results; ref?

Team to continue discussion:

Stephen Glass (NICTA); Max Ott (NICTA); Ezra Kissel (Indiana Univ); Marshall Brinn (GPO); Fraida Fund (NYU Poly)

6) Adjourn

12noon

I&M BoF Dinner

Please join us for a I&M BoF dinner on Wednesday evening. Please indicate on this poll, if you will be attending.

Last modified 11 years ago Last modified on 03/28/13 11:33:02

Attachments (6)

Download all attachments as: .zip