wiki:GIMIv2tasks

Version 3 (modified by hmussman@bbn.com, 6 years ago) (diff)

--

GIMI v2 Plan, Tasks and Status

Plan

Team

Mike Zink (UMass Amherst)

Cong Wang (UMass Amherst)

Max Ott (NICTA)

Ilia Baldine (RENCI)

Shu Huang (RENCI)

Harry Mussman (BBN)

Jeanne Ohren (BBN)

Planning and Status Meetings/Conference Calls

040113

040413

041513

041913

042213

043013

051413

052813

061113

062513

070913

072113 - 072313 is GEC17

Tasks

GIMI Tasks Through GEC17

Started 032013 Revised 032913 Revised 040213 after 040113 call, and after GEMINI call on 040213 Revised 040513 after discussion with Mike on 040413

1) Meet biweekly starting 4/1, with 3 demo sessions: 5/13, 6/10 and 7/8

2) By 4/15: /cleanup/checkin/test/document current code

What needs work?

3) Agree on goals:

a) GIMI (also GEMINI) to work with both ExoGENI and InstaGENI b) Consistent user experience, both racks, both sets of I&M tools c) Implement basic workflow per http://groups.geni.net/geni/wiki/GeniExperiments#a5StepsinBasicGENIExperimentTutorialTestWorkflowHarry and http://groups.geni.net/geni/wiki/TestTutorialExperimentWorkflow

Issue: what range of workflows must be supported? small (classroom exercise); medium (research experiment); large (long-running service, with opt-in) Answer: eventually all of these, but with an emphasis on small and medium at this time; should have a way to extend medium to large; Sol 4 calls for shakedown experiments that are large

d) Easy to setup basic set of measurements (towards unified experiment environment)

Issue: what are basic set of measurements? Answer: at least "passive host measurements", and "mesh of pings between nodes"

e) Easy to mix-and-match tools, as go from step to step, with automatic transfer of info between steps/tools f) Easy to configure and then visualize (graph) measurements, starting from a graphical (or tabular) view of the topology of resources

Issue: But, starting from a starting from a graphical (or tabular) view of the topology of resources to configure desired measurements is not always best. For example, for a classroom exercise, may want a pre-defined experiment, with a pre-define measurement configuration. For example, for a large experiment with 100s of nodes, it is not practical or useful to drive measurements from a graphical view of the topology; want to do everything with a script. Answer: Should be easy to configure and then visualize (graph) measurements, but the mechanism will likely differ depending on the type of experiment: small (predefined), medium, large. Nonetheless, should provide a method to serve all three that is not cumbersome for the experimenter. For a large experiment, will need to be able to use a script.

h) Ability to simultaneously run multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific), each with multiple measurements and multiple graphs, during an experiment i) Stay consistent with Spiral 5 GIMI SOW

4) Build workflow for GEMINI to work on both InstaGENI and ExoGENI

a) Start with browser, GENI Desktop, FLACK and OMNI, for InstaGENI b) Start with User Workspace, Flukes and OMNI for ExoGENI c) When can InstaGENI be used with ExoGENI? also GENI Desktop? d) Which browsers are supported? e) Which credentials are supported? f) Need detailed plan from team: Jeanne (Lead), Luisa, Ezra, Hussam, Ilia (or rep) and Cong g) Current InstaGENI configuration per attached figure; changes? h) Current ExoGENI configuration?

i) By 4/15: Jeanne will have met with team, and provide report of DRAFT plan

5) Decide on configuration of GIMI Portal service, and complete development.

a) On 4/2: Reconfirm that will continue as a multi-user persistent service, with account for each user, to retain advantages of a persistent sink for measurement data. However, encourage users to move completed measurements to iRODS, so that we do not end up with long-term storage of measurement data within GIMI Portal Service.

b) Continue to include OML Server and Lab wiki (for orchestration, results, and documentation) c) Separate OML results, by experiment and user, per the defined method d) For each user, introduce multiple "windows" to support multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific) e) For each user, be able to include session with Lab wiki to use OMF script to orchestrate experiment services (independent of any measurements) f) Store results in iRODS, by user, following defined structure g) Realize workable authentication/authorization to iRODS; start with current proxycert arrangement h) Do we want to support link from GENI Desktop? or somehow include Lab wiki as a plug-in in the GENI Desktop? These could be a step towards a "consistent user interface".

i) On 4/4: We need a workable solution, not the “best” solution. We can’t wait for future authentication/authorization mechanisms.

j) On 4/4: We need to have the bulk of the work done in 6-8 weeks (no later than 6/11), then cleanup towards GEC17.

k) On 4/4: A small team will be organized to work on the problem, define multiple solutions, and pick the best we can do in the time allotted: Jeanne, Cong, Mike (lead), Shu, (also web services expert from RENCI?), Max.

l) By 4/15, Mike will have met with team, and will provide a report, including: evaluation of current Lab wiki code; available options for authentication of user to iRODS (starting with current proxycert arrangement); overview of current structure, and early view of possible options

6) Add features to iRODS

a) Add rules to process incoming descriptors in XML files, ans store info in iCAT b) Add rules to move object to public archive directory, and manage any changes c) Integrate archive service with handle service

d) By 4/15, Shu needs to summarize planned extensions, per his earlier document.

7) Include easy way to configure measurements, for all types of experiments (small, medium and large)

Issue: Currently, involves: resource request (with ORCA or AM API); post-boot script; OMF script. But these are not linked. So, all is easy for a pre-defined (small) experiment, but script writing is reuired for a medium experiment, and info must be manually transferred from resource request/post boot script to the OMF script.

a) Consider best way to "add GIMI extensions" (modify request to add software image or packages, and add any necessary nodes) b) Consider best way to "instrumentize with GIMI" (use script to setup proxy certs, configure services, connect via XMPP, and then start services) c) Consider best way to drive OMF script (on a medium experiment) from the slice information, e.g., number of nodes, names of nodes, topology of nodes.

Option 1: consider use of GENI Desktop, with plug-ins to "add GIMI extensions" and "instrumentize with GIMI"; this would be a step towards a "consistent user interface". Option 2: consider some way to automatically write OMF script based on parameters forwarded from resource assignment. Maybe, manifest info could be transferred via iRODS. If this could be done for a medium experiment, perhaps an extension to a large experiment would be identified. Option 3: particularly for a large experiment, perhaps could automatically discover the nodes that are up, and the measurements that have been configured in each node. This could verify an OMF script. Option 4: although we do not need to drive measurement setup from graphical view of topology, we could discover the measurements that have been setup, and present a graphical view of the measurements, along with a graphical view of the resource topology. This could be used to verify the OMF script mathces the slioce resources, etc.

d) On 040413, put off discussion of this topic, until other topics are understood; may have BBN summer interns work on some of these topics.

8) Overall schedule:

a) By 6/11: We need to have the bulk of the work done on GIMI Portal no later than 6/11 b) By 7/8: v2 ready for testing c) On 7/21 - 7/23 at GEC17: v2 software released after testing; integrated tutorials presented

Key Task List

ID Description Assignee Due Status Notes
1
1
1
1
1
1

Attachments (6)