Changes between Version 1 and Version 2 of GIMIv2tasks


Ignore:
Timestamp:
04/22/13 15:24:17 (7 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GIMIv2tasks

    v1 v2  
     1[[PageOutline]]
     2
     3=  GIMI v2 Plan, Tasks and Status =
     4
     5
     6
     7==  Plan ==
     8
     9
     10
     11
     12
    113== Team ==
     14
     15 Mike Zink  (UMass Amherst)
     16
     17 Cong Wang  (UMass Amherst)
     18
     19 Max Ott (NICTA)
     20
     21 Ilia Baldine  (RENCI)
     22
     23 Shu Huang (RENCI)
     24
     25 Harry Mussman  (BBN)
    226
    327 Jeanne Ohren (BBN)
    428
    5  Luisa Nevers  (BBN)
    629
    7  Ezra Kissel (IU)
     30== Planning and Status Meetings/Conference Calls ==
    831
    9  Hussman Nasir  (Kentucky)
     32 040113
    1033
    11  Ilia Baldine  (RENCI)
     34 040413
    1235
    13  Cong Wang  (UMass Amherst)
     36 041513
    1437
    15 -----
     38 041913
    1639
    17 == Planning and Status Meetings ==
     40 042213
    1841
    19  041113
     42 043013
    2043
    21  042513
     44 051413
    2245
    23  050913
     46 052813
    2447
    25  052313
     48 061113
    2649
    27  061313
     50 062513
    2851
    29  062713
    30 
    31  071113
     52 070913
    3253
    3354 072113 - 072313 is GEC17
    3455
    35 -----
    3656
    37 == Task List ==
     57== Tasks ==
     58
     59GIMI Tasks Through GEC17
     60Started 032013
     61Revised 032913
     62Revised 040213 after 040113 call, and after GEMINI call on 040213
     63Revised 040513 after discussion with Mike on 040413
     64
     651)  Meet biweekly starting 4/1, with 3 demo sessions: 5/13, 6/10 and 7/8
     66
     672)  By 4/15:  /cleanup/checkin/test/document current code
     68        What needs work?
     69
     703)  Agree on goals:
     71        a)  GIMI (also GEMINI) to work with both ExoGENI and InstaGENI
     72        b)  Consistent user experience, both racks, both sets of I&M tools
     73        c)  Implement basic workflow per        http://groups.geni.net/geni/wiki/GeniExperiments#a5StepsinBasicGENIExperimentTutorialTestWorkflowHarry  and http://groups.geni.net/geni/wiki/TestTutorialExperimentWorkflow
     74                Issue:   what range of workflows must be supported?  small (classroom exercise);  medium  (research experiment);  large  (long-running service, with opt-in)
     75                Answer:  eventually all of these, but with an emphasis on small and medium at this time;  should have a way to extend medium to large;  Sol 4 calls for shakedown experiments that are large
     76        d)  Easy to setup basic set of measurements (towards unified experiment environment)
     77                Issue:  what are basic set of measurements?
     78                Answer:  at least "passive host measurements", and "mesh of pings between nodes"
     79        e)  Easy to mix-and-match tools, as go from step to step, with automatic transfer of info between steps/tools
     80        f)  Easy to configure and then visualize (graph) measurements, starting from a graphical (or tabular) view of the topology of resources
     81                Issue:  But, starting from a starting from a graphical (or tabular) view of the topology of resources to configure desired measurements is not always best.  For example, for a classroom exercise, may want a pre-defined experiment, with a pre-define measurement configuration.  For example, for a large experiment with 100s of nodes, it is not practical or useful to drive measurements from a graphical view of the topology;  want to do everything with a script.
     82                Answer:  Should be easy to configure and then visualize (graph) measurements, but the mechanism will likely differ depending on the type of experiment:  small (predefined), medium, large.  Nonetheless, should provide a method to serve all three that is not cumbersome for the experimenter.  For a large experiment, will need to be able to use a script.   
     83        h)  Ability to simultaneously run multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific), each with multiple measurements and multiple graphs, during an experiment
     84        i)  Stay consistent with Spiral 5 GIMI SOW
     85
     864)  Build workflow for GEMINI to work on both InstaGENI and ExoGENI
     87    a)  Start with browser, GENI Desktop, FLACK and OMNI, for InstaGENI
     88    b)  Start with User Workspace, Flukes and OMNI for ExoGENI
     89    c)  When can InstaGENI be used with ExoGENI?  also GENI Desktop?
     90    d)  Which browsers are supported?
     91    e)  Which credentials are supported?
     92    f)  Need detailed plan from team:  Jeanne (Lead), Luisa, Ezra, Hussam, Ilia (or rep) and Cong
     93    g)  Current InstaGENI configuration per attached figure;  changes?
     94    h)  Current ExoGENI configuration?
     95
     96        i)  By 4/15:  Jeanne will have met with team, and provide report of DRAFT plan
     97       
     985)  Decide on configuration of GIMI Portal service, and complete development.
     99        a)  On 4/2:  Reconfirm that will continue as a multi-user persistent service, with account for each user, to retain advantages of a persistent sink for measurement data.  However, encourage users to move completed measurements to iRODS, so that we do not end up with long-term storage of measurement data within GIMI Portal Service.
     100       
     101        b)  Continue to include OML Server and Lab wiki (for orchestration, results, and documentation)
     102        c)  Separate OML results, by experiment and user, per the defined method
     103        d)  For each user, introduce multiple "windows" to support multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific)
     104        e)  For each user, be able to include session with Lab wiki to use OMF script to orchestrate experiment services (independent of any measurements)
     105        f)  Store results in iRODS, by user, following defined structure
     106        g)  Realize workable authentication/authorization to iRODS;  start with current proxycert arrangement
     107        h)  Do we want to support link from GENI Desktop?  or somehow include Lab wiki as a plug-in in the GENI Desktop? These could be a step towards a "consistent user interface".
     108
     109        i)  On 4/4:  We need a workable solution, not the “best” solution.  We can’t wait for future authentication/authorization mechanisms.
     110       
     111        j)  On 4/4:  We need to have the bulk of the work done in 6-8 weeks (no later than 6/11), then cleanup towards GEC17. 
     112       
     113        k)  On 4/4:  A small team will be organized to work on the problem, define multiple solutions, and pick the best we can do in the time allotted:  Jeanne, Cong, Mike (lead), Shu, (also web services expert from RENCI?), Max.
     114
     115        l)  By 4/15, Mike will have met with team, and will provide a report, including:  evaluation of current Lab wiki code;  available options for authentication of user to iRODS (starting with current proxycert arrangement);  overview of current structure, and early view of possible options
     116
     1176)  Add features to iRODS
     118        a)  Add rules to process incoming descriptors in XML files, ans store info in iCAT
     119        b)  Add rules to move object to public archive directory, and manage any changes
     120        c)  Integrate archive service with handle service
     121       
     122        d)  By 4/15, Shu needs to summarize planned extensions, per his earlier document.
     123       
     1247)  Include easy way to configure measurements, for all types of experiments (small, medium and large)
     125                Issue:  Currently, involves:  resource request (with ORCA or AM API);  post-boot script;  OMF script.  But these are not linked.  So, all is easy for a pre-defined (small) experiment, but script writing is reuired for a medium experiment, and info must be manually transferred from resource request/post boot script to the OMF script.   
     126        a)  Consider best way to "add GIMI extensions"  (modify request to add software image or packages, and add any necessary nodes)
     127        b)  Consider best way to "instrumentize with GIMI"  (use script to setup proxy certs, configure services, connect via XMPP, and then start services)
     128        c)  Consider best way to drive OMF script (on a medium experiment) from the slice information, e.g., number of nodes, names of nodes, topology of nodes.
     129                Option 1:  consider use of GENI Desktop, with plug-ins to "add GIMI extensions" and "instrumentize with GIMI";  this would be a step towards a "consistent user interface".
     130                Option 2:  consider some way to automatically write OMF script based on parameters forwarded from resource assignment.  Maybe, manifest info could be transferred via iRODS. If this could be done for a medium experiment, perhaps an extension to a large experiment would be identified.
     131                Option 3:  particularly for a large experiment, perhaps could automatically discover the nodes that are up, and the measurements that have been configured in each node.  This could verify an OMF script.
     132                Option 4:  although we do not need to drive measurement setup from graphical view of topology, we could discover the measurements that have been setup, and present a graphical view of the measurements, along with a graphical view of the resource topology.  This could be used to verify the OMF script mathces the slioce resources, etc.
     133               
     134        d)  On 040413, put off discussion of this topic, until other topics are understood;  may have BBN summer interns work on some of these topics.
     135       
     1368)  Overall schedule:
     137        a)  By 6/11:  We need to have the bulk of the work done on GIMI Portal no later than 6/11
     138        b)  By 7/8:  v2 ready for testing
     139        c)  On 7/21 - 7/23  at GEC17:  v2 software released after testing;  integrated tutorials presented
     140
     141== Key Task List ==
    38142
    39143|| '''ID''' || '''Description''' || '''Assignee''' || '''Due'''  || '''Status''' || '''Notes''' ||
    40 || 1 || Create basic Fedora image for ExoGENI  ||  Ezra and Hussam  ||  || [[Color(#B0E0E6, In Progress)]]  ||  ||
    41 || 2 || Create custom GIMI image for InstaGENI  || Mike and Cong ||  || [[Color(#B0E0E6, In Progress)]] ||  ||
    42 || 3 || Convert current GIMI tutorial RDF to rspec || Jeanne ||  ||  [[Color(#B0E0E6, In Progress)]] ||  ||
    43 || 4 || Test the GIMI rspec on InstaGENI using both Flack and omni || Jeanne and Luisa || ||  ||  ||
    44 || 5 || Convert postboot functionality into execute service script || Mike and Cong ||  ||   [[Color(#B0E0E6, In Progress)]] ||  ||
    45 || 6 || Determine basic use cases for testing both sets of tools ||  ||  ||  ||  ||
    46 || 7 || Test the GEMINI rspec on ExoGENI using omni || Ezra ||  ||  [[Color(#B0E0E6, In Progress)]] ||  ||
    47 || 8 || Test the rspec on ExoGENI using Flack || Ezra ||  ||  [[Color(#B0E0E6, In Progress)]] ||  ||
    48 ||  9 || Determine if initialize and instrumentize can get everything it needs from the ExoGENI manifest  || Hussam ||   || ||  ||
    49 ||  10 || Look into possible common measurement extensions for ExoGENI || Ilia, Ezra, Hussam, and Mike ||  ||  [[Color(#B0E0E6, In Progress)]] ||  ||
    50 ||  11  ||  Add support for GENI Portal certificates in GENI Desktop || Hussam ||  ||  [[Color(#B0E0E6, In Progress)]] ||  ||
    51 || 12 || Look into adding location (lat/lon) tags to the ExoGENI manifest || Ilia ||  ||  [[Color(#B0E0E6, In Progress)]] || ||
    52 || 13 || Create custom GIMI image for InstaGENI ||  ||  ||   ||  ||
     144|| 1 ||   ||   ||  ||   ||  ||
     145|| 1 ||   ||   ||  ||   ||  ||
     146|| 1 ||   ||   ||  ||   ||  ||
     147|| 1 ||   ||   ||  ||   ||  ||
     148|| 1 ||   ||   ||  ||   ||  ||
     149|| 1 ||   ||   ||  ||   ||  ||