[[PageOutline]] = GIMI v2 Plan, Tasks and Status = == Plan == == Team == Mike Zink (UMass Amherst) Cong Wang (UMass Amherst) Max Ott (NICTA) Ilia Baldine (RENCI) Shu Huang (RENCI) Harry Mussman (BBN) Jeanne Ohren (BBN) == Planning and Status Meetings/Conference Calls == 040113 040413 041513 042213 discuss iticket for iRODS 043013 051413 demo 052813 061113 demo; bulk of work done on v2 GIMI Portal 062513 070913 demo; v2 ready for testing 072113 - 072313 is GEC17; v2 software released after testing; integrated tutorials presented == Tasks == GIMI Tasks Through GEC17 * Started 032013 * Revised 032913 * Revised 040213 after 040113 call, and after GEMINI call on 040213 * Revised 040513 after discussion with Mike on 040413 1) Meet biweekly starting 4/1, with 3 demo sessions: 5/14, 6/11 and 7/9 [[BR]] 2) By 4/15: /cleanup/checkin/test/document current code [[BR]] What needs work? [[BR]] 3) Agree on goals: [[BR]] a) GIMI (also GEMINI) to work with both ExoGENI and InstaGENI [[BR]] b) Consistent user experience, both racks, both sets of I&M tools [[BR]] c) Implement basic workflow per http://groups.geni.net/geni/wiki/GeniExperiments#a5StepsinBasicGENIExperimentTutorialTestWorkflowHarry and http://groups.geni.net/geni/wiki/TestTutorialExperimentWorkflow [[BR]] Issue: what range of workflows must be supported? small (classroom exercise); medium (research experiment); large (long-running service, with opt-in) [[BR]] Answer: eventually all of these, but with an emphasis on small and medium at this time; should have a way to extend medium to large; Sol 4 calls for shakedown experiments that are large [[BR]] d) Easy to setup basic set of measurements (towards unified experiment environment) [[BR]] Issue: what are basic set of measurements? [[BR]] Answer: at least "passive host measurements", and "mesh of pings between nodes" [[BR]] e) Easy to mix-and-match tools, as go from step to step, with automatic transfer of info between steps/tools [[BR]] f) Easy to configure and then visualize (graph) measurements, starting from a graphical (or tabular) view of the topology of resources [[BR]] Issue: But, starting from a starting from a graphical (or tabular) view of the topology of resources to configure desired measurements is not always best. For example, for a classroom exercise, may want a pre-defined experiment, with a pre-define measurement configuration. For example, for a large experiment with 100s of nodes, it is not practical or useful to drive measurements from a graphical view of the topology; want to do everything with a script. [[BR]] Answer: Should be easy to configure and then visualize (graph) measurements, but the mechanism will likely differ depending on the type of experiment: small (predefined), medium, large. Nonetheless, should provide a method to serve all three that is not cumbersome for the experimenter. For a large experiment, will need to be able to use a script. [[BR]] h) Ability to simultaneously run multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific), each with multiple measurements and multiple graphs, during an experiment [[BR]] i) Stay consistent with Spiral 5 GIMI SOW [[BR]] 4) Build workflow for GEMINI to work on both InstaGENI and ExoGENI [[BR]] a) Start with browser, GENI Desktop, FLACK and OMNI, for InstaGENI [[BR]] b) Start with User Workspace, Flukes and OMNI for ExoGENI [[BR]] c) When can InstaGENI be used with ExoGENI? also GENI Desktop? [[BR]] d) Which browsers are supported? [[BR]] e) Which credentials are supported? [[BR]] f) Need detailed plan from team: Jeanne (Lead), Luisa, Ezra, Hussam, Ilia (or rep) and Cong [[BR]] g) Current InstaGENI configuration per attached figure; changes? [[BR]] h) Current ExoGENI configuration? [[BR]] i) By 4/15: Jeanne will have met with team, and provide report of DRAFT plan [[BR]] 5) Decide on configuration of GIMI Portal service, and complete development. [[BR]] a) On 4/2: Reconfirm that will continue as a multi-user persistent service, with account for each user, to retain advantages of a persistent sink for measurement data. However, encourage users to move completed measurements to iRODS, so that we do not end up with long-term storage of measurement data within GIMI Portal Service. [[BR]] b) Continue to include OML Server and Lab wiki (for orchestration, results, and documentation) [[BR]] c) Separate OML results, by experiment and user, per the defined method [[BR]] d) For each user, introduce multiple "windows" to support multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific) [[BR]] e) For each user, be able to include session with Lab wiki to use OMF script to orchestrate experiment services (independent of any measurements) [[BR]] f) Store results in iRODS, by user, following defined structure [[BR]] g) Realize workable authentication/authorization to iRODS; start with current proxycert arrangement [[BR]] h) Do we want to support link from GENI Desktop? or somehow include Lab wiki as a plug-in in the GENI Desktop? These could be a step towards a "consistent user interface". [[BR]] i) On 4/4: We need a workable solution, not the “best” solution. We can’t wait for future authentication/authorization mechanisms. [[BR]] j) On 4/4: We need to have the bulk of the work done in 6-8 weeks (no later than 6/11), then cleanup towards GEC17. [[BR]] k) On 4/4: A small team will be organized to work on the problem, define multiple solutions, and pick the best we can do in the time allotted: Jeanne, Cong, Mike (lead), Shu, (also web services expert from RENCI?), Max. [[BR]] l) By 4/15, Mike will have met with team, and will provide a report, including: evaluation of current Lab wiki code; available options for authentication of user to iRODS (starting with current proxycert arrangement); overview of current structure, and early view of possible options [[BR]] 6) Add features to iRODS [[BR]] a) Add rules to process incoming descriptors in XML files, ans store info in iCAT [[BR]] b) Add rules to move object to public archive directory, and manage any changes [[BR]] c) Integrate archive service with handle service [[BR]] d) By 4/15, Shu needs to summarize planned extensions, per his earlier document. [[BR]] 7) Include easy way to configure measurements, for all types of experiments (small, medium and large) [[BR]] Issue: Currently, involves: resource request (with ORCA or AM API); post-boot script; OMF script. But these are not linked. So, all is easy for a pre-defined (small) experiment, but script writing is reuired for a medium experiment, and info must be manually transferred from resource request/post boot script to the OMF script. [[BR]] a) Consider best way to "add GIMI extensions" (modify request to add software image or packages, and add any necessary nodes) [[BR]] b) Consider best way to "instrumentize with GIMI" (use script to setup proxy certs, configure services, connect via XMPP, and then start services) [[BR]] c) Consider best way to drive OMF script (on a medium experiment) from the slice information, e.g., number of nodes, names of nodes, topology of nodes. [[BR]] Option 1: consider use of GENI Desktop, with plug-ins to "add GIMI extensions" and "instrumentize with GIMI"; this would be a step towards a "consistent user interface". [[BR]] Option 2: consider some way to automatically write OMF script based on parameters forwarded from resource assignment. Maybe, manifest info could be transferred via iRODS. If this could be done for a medium experiment, perhaps an extension to a large experiment would be identified. [[BR]] Option 3: particularly for a large experiment, perhaps could automatically discover the nodes that are up, and the measurements that have been configured in each node. This could verify an OMF script. [[BR]] Option 4: although we do not need to drive measurement setup from graphical view of topology, we could discover the measurements that have been setup, and present a graphical view of the measurements, along with a graphical view of the resource topology. This could be used to verify the OMF script mathces the slioce resources, etc. [[BR]] d) On 040413, put off discussion of this topic, until other topics are understood; may have BBN summer interns work on some of these topics. [[BR]] 8) Overall schedule: [[BR]] a) By 6/11: We need to have the bulk of the work done on GIMI Portal no later than 6/11 [[BR]] b) By 7/8: v2 ready for testing [[BR]] c) On 7/21 - 7/23 at GEC17: v2 software released after testing; integrated tutorials presented [[BR]] == Key Task List == || '''ID''' || '''Description''' || '''Assignee''' || '''Due''' || '''Status''' || '''Notes''' || || 1 || || || || || || || 1 || || || || || || || 1 || || || || || || || 1 || || || || || || || 1 || || || || || || || 1 || || || || || ||