Changes between Version 6 and Version 7 of GIMIv2tasks


Ignore:
Timestamp:
04/22/13 15:35:08 (7 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GIMIv2tasks

    v6 v7  
    6464 * Revised 040513 after discussion with Mike on 040413
    6565
    66 1)  Meet biweekly starting 4/1, with 3 demo sessions: 5/13, 6/10 and 7/8
     661)  Meet biweekly starting 4/1, with 3 demo sessions: 5/14, 6/11 and 7/9  [[BR]]
    6767
    68 2)  By 4/15:  /cleanup/checkin/test/document current code
    69         What needs work?
     682)  By 4/15:  /cleanup/checkin/test/document current code [[BR]]
     69        What needs work? [[BR]]
    7070
    71 3)  Agree on goals:
    72   a)  GIMI (also GEMINI) to work with both ExoGENI and InstaGENI
    73   b)  Consistent user experience, both racks, both sets of I&M tools
    74   c)  Implement basic workflow per      http://groups.geni.net/geni/wiki/GeniExperiments#a5StepsinBasicGENIExperimentTutorialTestWorkflowHarry  and http://groups.geni.net/geni/wiki/TestTutorialExperimentWorkflow
    75                 Issue:   what range of workflows must be supported?  small (classroom exercise);  medium  (research experiment);  large  (long-running service, with opt-in)
    76                 Answer:  eventually all of these, but with an emphasis on small and medium at this time;  should have a way to extend medium to large;  Sol 4 calls for shakedown experiments that are large
    77         d)  Easy to setup basic set of measurements (towards unified experiment environment)
    78                 Issue:  what are basic set of measurements?
    79                 Answer:  at least "passive host measurements", and "mesh of pings between nodes"
    80         e)  Easy to mix-and-match tools, as go from step to step, with automatic transfer of info between steps/tools
    81         f)  Easy to configure and then visualize (graph) measurements, starting from a graphical (or tabular) view of the topology of resources
    82                 Issue:  But, starting from a starting from a graphical (or tabular) view of the topology of resources to configure desired measurements is not always best.  For example, for a classroom exercise, may want a pre-defined experiment, with a pre-define measurement configuration.  For example, for a large experiment with 100s of nodes, it is not practical or useful to drive measurements from a graphical view of the topology;  want to do everything with a script.
    83                 Answer:  Should be easy to configure and then visualize (graph) measurements, but the mechanism will likely differ depending on the type of experiment:  small (predefined), medium, large.  Nonetheless, should provide a method to serve all three that is not cumbersome for the experimenter.  For a large experiment, will need to be able to use a script.   
    84         h)  Ability to simultaneously run multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific), each with multiple measurements and multiple graphs, during an experiment
    85         i)  Stay consistent with Spiral 5 GIMI SOW
     713)  Agree on goals: [[BR]]
     72        a)  GIMI (also GEMINI) to work with both ExoGENI and InstaGENI  [[BR]]
     73        b)  Consistent user experience, both racks, both sets of I&M tools [[BR]]
     74        c)  Implement basic workflow per        http://groups.geni.net/geni/wiki/GeniExperiments#a5StepsinBasicGENIExperimentTutorialTestWorkflowHarry  and http://groups.geni.net/geni/wiki/TestTutorialExperimentWorkflow  [[BR]]
     75                Issue:   what range of workflows must be supported?  small (classroom exercise);  medium  (research experiment);  large  (long-running service, with opt-in)   [[BR]]
     76                Answer:  eventually all of these, but with an emphasis on small and medium at this time;  should have a way to extend medium to large;  Sol 4 calls for shakedown experiments that are large [[BR]]
     77        d)  Easy to setup basic set of measurements (towards unified experiment environment) [[BR]]
     78                Issue:  what are basic set of measurements?   [[BR]]
     79                Answer:  at least "passive host measurements", and "mesh of pings between nodes" [[BR]]
     80        e)  Easy to mix-and-match tools, as go from step to step, with automatic transfer of info between steps/tools [[BR]]
     81        f)  Easy to configure and then visualize (graph) measurements, starting from a graphical (or tabular) view of the topology of resources [[BR]]
     82                Issue:  But, starting from a starting from a graphical (or tabular) view of the topology of resources to configure desired measurements is not always best.  For example, for a classroom exercise, may want a pre-defined experiment, with a pre-define measurement configuration.  For example, for a large experiment with 100s of nodes, it is not practical or useful to drive measurements from a graphical view of the topology;  want to do everything with a script. [[BR]]
     83                Answer:  Should be easy to configure and then visualize (graph) measurements, but the mechanism will likely differ depending on the type of experiment:  small (predefined), medium, large.  Nonetheless, should provide a method to serve all three that is not cumbersome for the experimenter.  For a large experiment, will need to be able to use a script.   [[BR]]
     84        h)  Ability to simultaneously run multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific), each with multiple measurements and multiple graphs, during an experiment [[BR]]
     85        i)  Stay consistent with Spiral 5 GIMI SOW [[BR]]
    8686
    87 4)  Build workflow for GEMINI to work on both InstaGENI and ExoGENI
    88     a)  Start with browser, GENI Desktop, FLACK and OMNI, for InstaGENI
    89     b)  Start with User Workspace, Flukes and OMNI for ExoGENI
    90     c)  When can InstaGENI be used with ExoGENI?  also GENI Desktop?
    91     d)  Which browsers are supported?
    92     e)  Which credentials are supported?
    93     f)  Need detailed plan from team:  Jeanne (Lead), Luisa, Ezra, Hussam, Ilia (or rep) and Cong
    94     g)  Current InstaGENI configuration per attached figure;  changes?
    95     h)  Current ExoGENI configuration?
     874)  Build workflow for GEMINI to work on both InstaGENI and ExoGENI [[BR]]
     88    a)  Start with browser, GENI Desktop, FLACK and OMNI, for InstaGENI [[BR]]
     89    b)  Start with User Workspace, Flukes and OMNI for ExoGENI [[BR]]
     90    c)  When can InstaGENI be used with ExoGENI?  also GENI Desktop? [[BR]]
     91    d)  Which browsers are supported? [[BR]]
     92    e)  Which credentials are supported? [[BR]]
     93    f)  Need detailed plan from team:  Jeanne (Lead), Luisa, Ezra, Hussam, Ilia (or rep) and Cong   [[BR]]
     94    g)  Current InstaGENI configuration per attached figure;  changes? [[BR]]
     95    h)  Current ExoGENI configuration? [[BR]]
    9696
    97         i)  By 4/15:  Jeanne will have met with team, and provide report of DRAFT plan
     97        i)  By 4/15:  Jeanne will have met with team, and provide report of DRAFT plan [[BR]]
    9898       
    99 5)  Decide on configuration of GIMI Portal service, and complete development.
    100         a)  On 4/2:  Reconfirm that will continue as a multi-user persistent service, with account for each user, to retain advantages of a persistent sink for measurement data.  However, encourage users to move completed measurements to iRODS, so that we do not end up with long-term storage of measurement data within GIMI Portal Service.
     995)  Decide on configuration of GIMI Portal service, and complete development. [[BR]]
     100        a)  On 4/2:  Reconfirm that will continue as a multi-user persistent service, with account for each user, to retain advantages of a persistent sink for measurement data.  However, encourage users to move completed measurements to iRODS, so that we do not end up with long-term storage of measurement data within GIMI Portal Service. [[BR]]
    101101       
    102         b)  Continue to include OML Server and Lab wiki (for orchestration, results, and documentation)
    103         c)  Separate OML results, by experiment and user, per the defined method
    104         d)  For each user, introduce multiple "windows" to support multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific)
    105         e)  For each user, be able to include session with Lab wiki to use OMF script to orchestrate experiment services (independent of any measurements)
    106         f)  Store results in iRODS, by user, following defined structure
    107         g)  Realize workable authentication/authorization to iRODS;  start with current proxycert arrangement
    108         h)  Do we want to support link from GENI Desktop?  or somehow include Lab wiki as a plug-in in the GENI Desktop? These could be a step towards a "consistent user interface".
     102        b)  Continue to include OML Server and Lab wiki (for orchestration, results, and documentation) [[BR]]
     103        c)  Separate OML results, by experiment and user, per the defined method [[BR]]
     104        d)  For each user, introduce multiple "windows" to support multiple measurement sessions (e.g., passive host, mesh of pings, intermittent iperf, and experiment specific) [[BR]]
     105        e)  For each user, be able to include session with Lab wiki to use OMF script to orchestrate experiment services (independent of any measurements) [[BR]]
     106        f)  Store results in iRODS, by user, following defined structure [[BR]]
     107        g)  Realize workable authentication/authorization to iRODS;  start with current proxycert arrangement [[BR]]
     108        h)  Do we want to support link from GENI Desktop?  or somehow include Lab wiki as a plug-in in the GENI Desktop? These could be a step towards a "consistent user interface". [[BR]]
    109109
    110         i)  On 4/4:  We need a workable solution, not the “best” solution.  We can’t wait for future authentication/authorization mechanisms.
     110        i)  On 4/4:  We need a workable solution, not the “best” solution.  We can’t wait for future authentication/authorization mechanisms. [[BR]]
    111111       
    112         j)  On 4/4:  We need to have the bulk of the work done in 6-8 weeks (no later than 6/11), then cleanup towards GEC17. 
     112        j)  On 4/4:  We need to have the bulk of the work done in 6-8 weeks (no later than 6/11), then cleanup towards GEC17.   [[BR]]
    113113       
    114         k)  On 4/4:  A small team will be organized to work on the problem, define multiple solutions, and pick the best we can do in the time allotted:  Jeanne, Cong, Mike (lead), Shu, (also web services expert from RENCI?), Max.
     114        k)  On 4/4:  A small team will be organized to work on the problem, define multiple solutions, and pick the best we can do in the time allotted:  Jeanne, Cong, Mike (lead), Shu, (also web services expert from RENCI?), Max.  [[BR]]
    115115
    116         l)  By 4/15, Mike will have met with team, and will provide a report, including:  evaluation of current Lab wiki code;  available options for authentication of user to iRODS (starting with current proxycert arrangement);  overview of current structure, and early view of possible options
     116        l)  By 4/15, Mike will have met with team, and will provide a report, including:  evaluation of current Lab wiki code;  available options for authentication of user to iRODS (starting with current proxycert arrangement);  overview of current structure, and early view of possible options [[BR]]
    117117
    118 6)  Add features to iRODS
    119         a)  Add rules to process incoming descriptors in XML files, ans store info in iCAT
    120         b)  Add rules to move object to public archive directory, and manage any changes
    121         c)  Integrate archive service with handle service
     1186)  Add features to iRODS [[BR]]
     119        a)  Add rules to process incoming descriptors in XML files, ans store info in iCAT [[BR]]
     120        b)  Add rules to move object to public archive directory, and manage any changes [[BR]]
     121        c)  Integrate archive service with handle service [[BR]]
    122122       
    123         d)  By 4/15, Shu needs to summarize planned extensions, per his earlier document.
     123        d)  By 4/15, Shu needs to summarize planned extensions, per his earlier document. [[BR]]
    124124       
    125 7)  Include easy way to configure measurements, for all types of experiments (small, medium and large)
    126                 Issue:  Currently, involves:  resource request (with ORCA or AM API);  post-boot script;  OMF script.  But these are not linked.  So, all is easy for a pre-defined (small) experiment, but script writing is reuired for a medium experiment, and info must be manually transferred from resource request/post boot script to the OMF script.   
    127         a)  Consider best way to "add GIMI extensions"  (modify request to add software image or packages, and add any necessary nodes)
    128         b)  Consider best way to "instrumentize with GIMI"  (use script to setup proxy certs, configure services, connect via XMPP, and then start services)
    129         c)  Consider best way to drive OMF script (on a medium experiment) from the slice information, e.g., number of nodes, names of nodes, topology of nodes.
    130                 Option 1:  consider use of GENI Desktop, with plug-ins to "add GIMI extensions" and "instrumentize with GIMI";  this would be a step towards a "consistent user interface".
    131                 Option 2:  consider some way to automatically write OMF script based on parameters forwarded from resource assignment.  Maybe, manifest info could be transferred via iRODS. If this could be done for a medium experiment, perhaps an extension to a large experiment would be identified.
    132                 Option 3:  particularly for a large experiment, perhaps could automatically discover the nodes that are up, and the measurements that have been configured in each node.  This could verify an OMF script.
    133                 Option 4:  although we do not need to drive measurement setup from graphical view of topology, we could discover the measurements that have been setup, and present a graphical view of the measurements, along with a graphical view of the resource topology.  This could be used to verify the OMF script mathces the slioce resources, etc.
     1257)  Include easy way to configure measurements, for all types of experiments (small, medium and large) [[BR]]
     126                Issue:  Currently, involves:  resource request (with ORCA or AM API);  post-boot script;  OMF script.  But these are not linked.  So, all is easy for a pre-defined (small) experiment, but script writing is reuired for a medium experiment, and info must be manually transferred from resource request/post boot script to the OMF script.   [[BR]]
     127        a)  Consider best way to "add GIMI extensions"  (modify request to add software image or packages, and add any necessary nodes) [[BR]]
     128        b)  Consider best way to "instrumentize with GIMI"  (use script to setup proxy certs, configure services, connect via XMPP, and then start services) [[BR]]
     129        c)  Consider best way to drive OMF script (on a medium experiment) from the slice information, e.g., number of nodes, names of nodes, topology of nodes. [[BR]]
     130                Option 1:  consider use of GENI Desktop, with plug-ins to "add GIMI extensions" and "instrumentize with GIMI";  this would be a step towards a "consistent user interface". [[BR]]
     131                Option 2:  consider some way to automatically write OMF script based on parameters forwarded from resource assignment.  Maybe, manifest info could be transferred via iRODS. If this could be done for a medium experiment, perhaps an extension to a large experiment would be identified.  [[BR]]
     132                Option 3:  particularly for a large experiment, perhaps could automatically discover the nodes that are up, and the measurements that have been configured in each node.  This could verify an OMF script. [[BR]]
     133                Option 4:  although we do not need to drive measurement setup from graphical view of topology, we could discover the measurements that have been setup, and present a graphical view of the measurements, along with a graphical view of the resource topology.  This could be used to verify the OMF script mathces the slioce resources, etc. [[BR]]
    134134               
    135         d)  On 040413, put off discussion of this topic, until other topics are understood;  may have BBN summer interns work on some of these topics.
     135        d)  On 040413, put off discussion of this topic, until other topics are understood;  may have BBN summer interns work on some of these topics. [[BR]]
    136136       
    137 8)  Overall schedule:
    138         a)  By 6/11:  We need to have the bulk of the work done on GIMI Portal no later than 6/11
    139         b)  By 7/8:  v2 ready for testing
    140         c)  On 7/21 - 7/23  at GEC17:  v2 software released after testing;  integrated tutorials presented
     1378)  Overall schedule: [[BR]]
     138        a)  By 6/11:  We need to have the bulk of the work done on GIMI Portal no later than 6/11 [[BR]]
     139        b)  By 7/8:  v2 ready for testing [[BR]]
     140        c)  On 7/21 - 7/23  at GEC17:  v2 software released after testing;  integrated tutorials presented  [[BR]]
    141141
    142142== Key Task List ==