Changes between Initial Version and Version 1 of _GIMI_NextSteps

Timestamp:

04/27/12 11:59:34 (12 years ago)

Author:

hmussman@bbn.com

Comment:

--

_GIMI_NextSteps

@@ +1 @@
+[[pageoutline]]
+
+
+= GIMI tasks, issues, next steps =
+ After discussion on 4/5/12:
+ After iRODS review with Shu on 4/12/12:
+ After status call with Mike and Max on 4/18/12:
+ After status call on 4/25/12:
+
+== 1)  Status calls and reviews ==
+ on 4/25:  when is next status call?  5/2
+ on 4/25:  when is design review?  not 5/2;  week of 5/7?  week of 5/14?
+ ***  marks items that will be important to review on the next call
+   3)  *** Architecture of GIMI tools for GEC14 tutorial
+   4)  *** Introduce XML messaging service:  required for OMF messages, used to orchestrate services
+   7)  *** Steps towards GIMI testing:  start by forwarding all code and plans for GEC13 demo;  by 4/25?
+   8)  *** Steps towards GIMI tutorial at GEC14
+
+
+== 2)  Target aggregates for GEC14 tutorial ==
+
+a)  (Ilia) exoGENI racks;  all VMs  (Ilia)
+
+b)  ( Ilia)  access to shell on all nodes by public IP address;  authentication/authorization using keys
+
+c)  ( Ilia)  access to web interfaces (GUIs) on all nodes by public IP address;    authentication/authorization using keys?  certificates?  
+
+
+
+== 3)  *** Architecture of GIMI tools for GEC14 tutorial  ==
+
+a)  (Mike)  need to arrive at nominal configuration ASAP
+ on 4/18:  good discussion (see below);  still need a clear and complete description soon;  when? 
+ on 4/25:  discussion as noted below;  still need a clear and complete description soon, and a full design review;  when?
+ 
+a')  on 4/20:  issue:  how to configure MPs/MCs?  from EC to RC in  MP/MC?
+
+a'')  on 4/20:  issue:  is there a MAP on common node?  at least to preview data?  is there a MAP in user workspace?  what?
+ on 4/25:  for 1st, no MAP on common node
+
+b)  MPs with multiple possible measurements:  host info;  basic network performance with iperf, ping;  data from app
+ on 4/25:  need to list baseline measurements, and configurable parameters
+ 
+c)  MC (OML server) in common node (node n+1)
+
+d)  where is MAP service installed?  in common node?    in user workspace?    what is planned for this service?  how can analysis/presentation be configured?  is there an r script?  is there a GUI?  how is GUI accessed?
+ on 4/25:  for 1st, expect MAP only bin user workspace
+ on 4/25:  what is 1st MAP service?  what functions are provided?  can these be done using iREEL and r?
+
+e)  movement of data from MC (OML server) to MAP service:   direct?   how?   via iRODS, i.e., MC to iRODS to MAP?  how?
+ on 4/18,  per Shu, and discussion:  could login to MC node;  establish first transfer from iRODS client on MC to to iRODS with iput;  use  iinit, so iRODS client remembers password;  then periodic transfers with no human intervention;  another push protocol, from MC to iRODS.
+ on 4/18,  per Shu, and discussion: next, can install iRODS client on user workspace, then pull data back to user workspace for analysis and presentation, and then push back to iRODS for archive
+ 
+f)  1st:  common node in slice;  2nd:  common node in user workspace (out of slice)  (is this beyond GEC14?)
+ on 4/25:  for 1st, do common node in slice, but plan one per aggregate
+ 
+g)  1st:  nodes on VMs, one aggregate;  2nd:  nodes on VMs, multiple aggregates
+
+h)  1st:  Load image for common node, MP nodes;  2nd:  load MP code as package before/after app
+ on 4/25:  what is test app?  need to have something visible, e.g., video to a web page?
+
+i)  Is there authentication/authorization when data pushed fom MP to MC?  1st:  none;  considered OK  2nd:  add ssl?
+
+j)  Is there authentication/authorization when accessing GUI (MAP) on common node?  how?   does this use portal to GUIs?
+ on 4/25:  not in 1st
+
+== 4)  *** Introduce XML messaging service:  required for OMF messages, used to orchestrate services ==
+
+a)  (Max, Christoph)  public XMPP server must be available;  what?  where?
+
+b)  (Max, Christoph)  signed OMF messages:  need summary of arrangement from NICTA.    what are names?  who configures pub/sub nodes?    what is structure?    how to generate and distribute keys?  is a public key registry required?    how would it be done?    
+ on 4/18:  Max explains plan to use OMF6.0 approach;  limited messages;  can use XMPP or another transport;  signed messages;  can start with no authentication/authorization, and then turn it on later;  
+ on 4/18:  need clear and complete description soon, particularly naming and establsihment of XMPP nodes. so that others can consider for use and Jeanne can begin testing.
+ on 4/18:  expect release of first OMF6.0 by mid-May, with firm RC and skeleton EC;  expect updated EC by mid-June;  is this too late for GEC14?
+  on 4/25:  need to start with an earlier version of OMF, e.g., OMF 5.3
+
+== 5)  Tools in user workspace  ==
+
+a)  includes stores for keys, certificates, credentials, rspecs
+
+b)  (Jeanne A, David) Use Gush with OMNI to acquire resources, load images/packages, setup connectivity, manage processes
+
+c)  (Christoph) Use OMF EC to configure/orchestrate services;  can EC drive scripts?
+
+d)  (Ilia) FLUKES optional to create (or observe) rspec
+
+e)  (Ilia)  what is best way to load images?  packages?
+
+f)  (Christoph)  Add iREEL (experimenter portal service);  
+ on 4/25:  what can this do?  can it provide MAP functions?  can it drive scripts?  (need copy to run locally)  
+
+g)  (From GEMINI;  under discussion)  Include portal service to find other GUIs?
+ on 4/25:  for 1st, not necessary  
+
+h)  (Giridhar, Jeanne, Shu)  Include Directory Archive (DA) service ,which can push data to DOA service, using OI service  (all from CNRI, also GPO )  
+ on 4/25:  now replaced with iRODS client
+ 
+i)   (Giridhar, Jeanne, Shu) Include MDOD creator/editor  How does this interface with iRODS, ot push MDOD to iRODS?  
+  on 4/18:  who on GIMI project will help define final MDOD schema?  Shu.
+  on 4/25:  need to get group together, to revise MDOD schema, and consider implementation of creator/editor
+
+  
+
+
+== 6)  Introduce iRODS service ==
+
+After review with Shu on 4/12/12:
+
+a)  (Shu)  define configuration of iRODS servers, etc.  also logins for users
+
+b)  (Shu)  define functions, interfaces to iRODS, and iRODS client    can a host mount a file system in iRODS?
+
+c)  (Shu)  define naming in iRODS
+
+d) (Shu)  define use case for:  moving data from MC to iRODS, then from iRODS to user workspace    
+use iRODS cleint in MC to push to iRODS?  how is authentication/authorization handled?
+use iRODS client in user workspace to pull to user workspace?  how is authentication/authorization handled?
+
+On 4/18:  use cases received:
+
+GIMI iRODS Use Cases - Preparation
+
+ Create an account for user: shuang
+  iadmin mkuser shuang user_type
+ Assign user to a group
+  iadmin atg groupName shuang
+ Admin can assign password to user (also supports GSI/Kerberos)
+  iadmin moduser shuang password xxx
+ A lot of other things can be done:
+  iadmin –h
+ Create an iRODS directory for the user OML measurement data
+  imkdir results
+  
+GIMI iRODS Use Case 1
+ Perform an OML-based measurement, result is saved in a file (a.sqlite3) under /home/shuang/results
+ Put the file into iRODS
+  iput a.sqlite3 (suppose ipwd is /geniRenci/home/shuang/results)
+ Work on the local copy under /home/shuang/results
+ Push the modified file back into iRODS (think it as a version control system)
+  Irsync –r /home/shuang/result i:/geniRenci/home/shuang/results
+  
+GIMI iRODS Use Case 2
+ Check out a file in iRODS to my local directory
+  iget a.sqlite3
+ Work on the local copy
+ Push the modified file back into iRODS
+  irsync a.sqlite3 i:a.sqlite3
+
+GIMI iRODS Use Case 3
+ Suppose there are many files under /home/shuang/data that need to be copied into iRODS, we can mount the local directory directly into iRODS
+  imkdir /geniRenci/home/shuang/mountpoint
+  imcoll –m filesystem /home/shuang/data /geniRenci/home/shuang/mountpoint
+ Work on the local files
+ Sync with iRODS
+  Irsync –r /home/shuang/data i:/geniRenci/home/shuang/mountpoint
+
+GIMI iRODS Use Case 4
+ Data processing – e.g. filtering files under /geniRenci/home/shuang
+  Write a rule called filter.r which filters files and save them under /geniRenci/home/shuang/filtered
+  irule –F filter.r “*Path=‘/geniRenci/home/shuang’”
+
+e)  (Shu)  consider interface to get unique handle;  need to get current handle minting service from CNRI
+
+f)  (Shu)  consider storing and possibly formulating MDOD in iRODS    how?
+
+g)  (Shu)  review above with GEMINI team
+
+
+== 7)  *** Steps towards GIMI testing ==
+
+a)  (Jeanne) Arrange user workspace 
+
+c)  (Mike)  What is first configuration of tools code?  when?  who?
+ on 4/18:  plan to forward all code and plans for GEC13 to Jeanne
+ on 4/18:  code leads are David and Delip;  will join future calls
+
+d)  (Jeanne)  What are first test scripts?  when?
+ on 4/25:  need to outline process for using tools in user workspace
+ on 4/25:  need to provide first test plan 
+
+e)  (Jeanne)  When are first tests?
+
+
+
+== 8)  *** Steps towards GIMI tutorial at GEC14 ==
+
+a)  (Ilia) Start with GEC13 ORCA turorial? 
+
+b)  (Mike)  Define covered use cases, and review with experiment support team (Mark, Niky)
+
+c)  (Mike)  When is first demo?
+
+d)  (Mike)  Need to establish high-level schedule:  freeze code by end of June?  then only testing, tutorial prep? 
+
+
+
+== 9)  Use GIMI in WiMAX site  ==
+
+a)  (Mike, Cong)  Can we do something by GEC14?  what?
+ on 4/25:  agree UMass Amherst WiMAX site should be up by GEC14
+ on 4/25:  agree should have plan to integrate GIMI with WiMAX by GEC14, but does not need to be done