Changes between Version 31 and Version 32 of GEMINI_TopicsIssuesTasks


Ignore:
Timestamp:
04/24/12 10:28:06 (12 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GEMINI_TopicsIssuesTasks

    v31 v32  
    99Addtions and changes after call with Jim and Charles on 4/17/12:  [[BR]]
    1010Additions and changes after call with Martin on 4/18/12:  [[BR]]
     11Additions and changes after team call 4/19/12:   (sections are rearranged) [[BR]]
    1112
    1213* 4/5/12 topics :  [[BR]]
    … …  
    2829 4d)  Old UNIS vs new UNIS: understand transition plan  (?) [[BR]]
    2930 5i)  who will help formulate final MDOD schema? (Martin on4/18:  Ezra and Omer)  [[BR]]
    30  5j')  report on Gush  (Jeanne) [[BR]]
     31 5j')  report on Gush  (Jeanne)   (delayed) [[BR]]
    3132 6d)  provide a more complete view of GEMINI portal service  (Jim, Charles on 4/18/12)  [[BR]]
    3233 7h)  review current status of baseline configuration to gather host metrics (Guilherme, and all) [[BR]]
    … …  
    3536 9l)  review updated test plan for making active network measurements  (Jeanne)  [[BR]]
    3637
    37 
     38+ 4/26/12 topics suggested by Harry: [[BR]]
     39 3g)  review access to http on VM (Hussam+) [[BR]]
     40 5j')  report on Gush  (Jeanne)   (delayed) [[BR]]
     41 6d)  provide a more complete view of GEMINI portal service  (Jim, Charles on 4/18/12)  [[BR]]
     42 7l)  review updated test plan for making active network measurements  (Jeanne)  [[BR]]
     43 8a)  review current status of baseline configuration to gather host metrics (Guilherme, and all) [[BR]]
     44 9)  agree on steps towards GEMINI tutorials at GEC14  (all) [[BR]]
     45 9a)  agree on plan to coordinate configurations to make active network measurements  and to collect basic host metrics in one slice (all) [[BR]]
     46 9d)  agree on schedule  [[BR]]
     47
     48 
     49 
    3850== 1)  Authentication and authorization:  multiple actor options: ==
    3951
    … …  
    6880
    6981c')  [in IMF, GENI credentials included with XML messages, for authorization?  how?  reuse?]
    70 
    71 *4/5/12 topic:  [[BR]]
    7282
    7383d)  ABAC  [Harry:  GPO believes that ABAC may eventually be used for resource assignment, but not soon]  [What code is available from ISI?  Jim is checking with Teb Faber;  waiting for a response]
    … …  
    106116f)  Task:  can ssh into public host name (or public IP) , with special 5+ digit port number (from manifest) from port map
    107117
    108 + 4/12/12 topics suggested by Harry: [[BR]]
    109 3g) and beyond  review access to http on VM (Hussam+) [[BR]]
    110 
    111 + 4/19/12 topics suggested by Harry: [[BR]]
    112 3g) and beyond  review access to http on VM (Hussam+) [[BR]]
    113 
    114118g)  Task:  (see e) above) how to access http interface?  tunnel through ssh?  port map, like ssh?  perhaps could "piggyback" on opening ssh port?  setup a separate proxy?   [[BR]]
     119
    115120 See [http://groups.geni.net/geni/wiki/RspecExtensionProposal_PortMapping  Port mapping proposal from Hussam]  [[BR]]
    116  Jim on 4/12:  Talked with Rob Ricci.  He is willing to work on it, since many needs to proxy to VMs. 
    117  (In INSTOOLS, used VNC) [[BR]]
    118 
    119 g')  New task:  Review possible tunnel through ssh (or use fo ssh to forward http port), to reuse available ssh port mapping.  (who?) [[BR]]
    120  On 4/12:  Too complicated;  simpler to just punch through ports 22 and 80 [[BR]]
    121 
    122 g!'')  New task:  Review port mapping for http, like ssh, with protoGENI, to see how it might be done (Nasir/Jim) [[BR]]
    123 
    124 g!''') New task:  Review need within GENI/GPO to open ports, and implications for rspec  (Harry)
    125 
     121 Jim on 4/12:  Talked with Rob Ricci.  He is willing to work on it, since many needs to proxy to VMs.   (In INSTOOLS, used VNC) [[BR]]
     122 On 4/12:  protoGENI considering using OPS server to provide persistent proxy. [[BR]]
     123 Jim on 4/19:  Long term solution:  Gary Wong at Utah is considering a proxy service per testbed, advertised in manifest.  [[BR]]
     124 Jim of 4/19:  Short term solution:  user can get to http port via ssh and port forwarding;  Hussam has script.  [[BR]]
     125 
    126126h)  Task:  what about vnc tunnels?  how were they done in INSTOOLS?   which port on host?  (who?)
    127127
    128 i)  Task:  what happens when VMs are on multiple aggregates?   (who?)
    129 
    130 j)  Task:  consider separate host for managing communications?  VM?  server?  centralized?  include pub/sub?  is this GENI Event Messaging Service?     (who?) [[BR]]
    131  On 4/12:  protoGENI considering using OPS server to provide persistent proxy. [[BR]]
     128i)  Task:  what happens when VMs are on one aggregate?  can reach each other via private IP addresses
     129
     130i')  Task:  what happens when VMs are on multiple aggregates?   (who?)
     131
    132132
    133133
    … …  
    139139Per call with Martin on 3/27/12:
    140140
    141 a)  Question:  In LAMP, is there a local UNIS, or not??   (Martin) Not yet;  needs to be, with push from local UNIS to global UNIS.
    142 
    143 b)  How does UNIS authenticate/authorize when receiving data?   (Martin)  [in LAMP, service to UNIS, https with protoGENI cert]
     141a)  Question:  In LAMP, is there a local UNIS, or not??   
     142 (Martin on 3/27:) Not yet;  needs to be, with push from local UNIS to global UNIS.
     143
     144b)  How does UNIS authenticate/authorize when receiving data?   
     145 (Martin on 3/27:)  [in LAMP, service to UNIS, https with protoGENI cert]
    144146
    145147c)  Question:  Use web interface on common node to configure services, tests;  how does this push config to UNIS?  What authentication/authorization steps are included?
    146148
    147 
    148 *topic for 4/5/12, 4/18/12, 4/19/12:  [[BR]]
    149 
    150149d)  Old UNIS vs new UNIS: [[BR]]
    151150
    152 *4/5/12 notes from Jeanne:  [[BR]]
    153 What is the transition plan?    Both can run in parallel until full functionality is available with new UNIS.  Then turn down old UNIS.  [[BR]]
    154 Local vs. global UNIS hierarchy:  Will new UNIS have local and global configuration?  Yes, probably not by GEC14. [[BR]]
    155 
    156 *4/18/12 notes from Harry:  Martin indicates that Old UNIS and New Unis can run in parallel, and in sync, until Old UNIS is eventually turned down.  [[BR]]
    157 
    158 *4/19/12:  explain more;  provide definition for other uses.  [[BR]]
     151 4/5/12 notes from Jeanne:  What is the transition plan?   
     152 Both can run in parallel until full functionality is available with new UNIS.  Then turn down old UNIS.  [[BR]]
     153 Local vs. global UNIS hierarchy:  Will new UNIS have local and global configuration?  Yes, probably not by GEC14. [[BR]]
     154  4/18/12 notes from Harry:  Martin indicates that Old UNIS and New UNIS can run in parallel, and in sync, until Old UNIS is eventually turned down.  [[BR]]
     155 Guilherme on 4/19:  Expect Old UNIS and New UNIS to contain smae information, but no 1:1 mapping  [[BR]]
     156 Guilherme on 4/19:  Could write a wrapper for perfSONAR services to talk with New UNIS, but significant effort.  [[BR]]
    159157
    160158== 5)  User workspace service ==
    … …  
    180178[Have iRODS at IU for NetKarma;  Jim and Wesley talking with Ilia and Shu] [[BR]]
    181179
    182 Or, do we use IRODS client to push/pull date to/from IRODS?  [[BR]]
     180g')  Or, do we use IRODS client to push/pull date to/from IRODS?  [[BR]]
    183181
    184182h)  Include MDOD creator/editor  (CNRI, GPO)
    185183
    186 + 4/12/12 topics suggested by Harry: [[BR]]
    187  5i)  who will help formulate final MDOD schema?   [[BR]]
    188 
    189 + 4/19/12 topics suggested by Harry: [[BR]]
    190  5i)  who will help formulate final MDOD schema? (Martin>?)  [[BR]]
    191 
    192 i)  Task:  Need help with final formulation of MDOD   (Ezra and Omer)  (Shu from GIMI)
     184i)  Task:  Need help with final formulation of MDOD   (Ezra and Omer from GEMINI)  (Shu from GIMI)
    193185
    194186j)  Task:  Define view of user workspace service (Jeanne, Matt, Harry, Jim, Martin, Niky)  [[BR]]
    195 [Jeanne to add security policy into view]  [[BR]]
    196 
    197 *topic on 4/5/12:  [[BR]]
    198 
    199 + 4/12/12 topics suggested by Harry: [[BR]]
    200  5j')  report on GUSH  (Jeanne) [[BR]]
    201 
    202 + 4/19/12 topics suggested by Harry: [[BR]]
    203  5j')  more on GUSH  (Jeanne) [[BR]]
     187 [Jeanne to add security policy into view]  [[BR]]
    204188
    205189j')  Use of GUSH:  [[BR]]
    … …  
    234218c)  Task:  understand options for authentication and authorization at a web interface.  (who?)
    235219
    236 + 4/19/12 topics suggested by Harry: [[BR]]
    237  6d)  provide a more complete view of GEMINI portal service  (Jim, Charles)  [[BR]]
    238 
    239 d)  Task: provide a more complete view of GEMINI portal service   (Harry, Jim and Charles) [[BR]]
     220d)  Task: provide a more complete view of GEMINI portal service   (Jim and Charles) [[BR]]
    240221
    241222Task:  Jim and Charles plan to provide in a week or two. [[BR]]
    … …  
    264245See updated drawing.  [[BR]]
    265246
    266 Jim and Charles on 4/17:  showed early demo;  portal is very thin, and runs in a server;  user logs into portal with browser, typically using HTML5 that runs in browser;  portal includes gadgets that interface to various parts of the system, parts of slice and/or tools;  tools can run in user workspace;  portal remembers parameters, but is NOT persistent;  common denominator is  a web GUI, even when interfacing to a CLI;  expect user's identity to pull credentials from store  ("ABAC" approach);  expect various rspecs to be stored in UNIS.
     247Jim and Charles on 4/17:  showed early demo;  portal is very thin, and runs in a server;  user logs into portal with browser, typically using HTML5 that runs in browser;  portal includes plugins that interface to various parts of the system, parts of slice and/or tools;  tools can run in user workspace;  portal remembers parameters, but is NOT persistent;  common denominator is  a web GUI, even when interfacing to a CLI;  expect user's identity to pull credentials from store  ("ABAC" approach);  expect various rspecs to be stored in UNIS.
     248
     249Guiherme on 4/19:  saw demo of portal
    267250
    268251Jim and Charles on 4/17/12:  next step is to further define portal, and write it up.
    … …  
    275258
    276259
    277 
    278 == 7)  New configuration to gather basic host metrics  ==
    279 
    280 + 4/12/12 topics suggested by Harry: [[BR]]
    281 7h)  review current status of baseline configuration to gather host metrics (Guilherme, and all) [[BR]]
    282 
    283 + 4/19/12 topics suggested by Harry: [[BR]]
    284 a)  review current status of baseline configuration to gather host metrics (Guilherme, and all) [[BR]]
    285 
    286 a)  Need baseline configuration ASAP   (Guilherme) [[BR]]
    287 4/12:  (Guilherme)  considering overall framework, working on interfaces, considering UNIS functions and schema.   [[BR]]
    288 
    289 a')  Need to define which host metrics to gather
    290 
     260== 7)  Configuration 1:  make active network measurements  ==
     261
     262a)  follows p15 Operator A slice, like LAMP project. 
     263
     264a') for GEC14, consider: use LAMP code with perhaps minor modifications
     265
     266a!'')  Start:  each node on a server, with an available public IP address  single aggregate or multiple aggregates
     267
     268a!''')  Second:  all nodes on VMs, or all nodes on VMs except common node on server
     269
     270b)  One common node (e.g., node n+1) to:  configure measurements, collect data, present data
     271
     272c)  Multiple measurement nodes (e.g., nodes 1, 2, ..., n)
     273
     274d)  Start:  one slice, classic pS interface, with data pulled from measurement node, authorized with GENI credential or ?
     275
     276e)  Global old UNIS as shown;  include local UNIS on common node?
     277
     278f)  Load software onto common node with image;  as option, load package
     279
     280g)  Load software onto measurement node with image;  as option, load package after app
     281
     282g')  Use Gush to manage processes, so that can easily switch between app and active network measurements;  good for establishing continuity and available bandwidth at beginning of experiment.
     283
     284h)  Use web interface on common node to configure services, tests, like LAMP;  how does this push config to UNIS??  How do we let only user do this with keys, etc.
     285
     286i)  Use web interface on common node to present/observe data, like LAMP  How do we let only user do this with keys, etc.
     287
     288j)  Demo all of the perfSONAR network performance tools, tests, as was done in LAMP project
     289
     290k)  Extension:  pull data from one slice to another, as shown in p15 from Operator A to Operator B;  authorize using GENI credentials;  how is this done?
     291
     292l)  Provide regression tests of various configurations, features, etc., driven by scripts  (Jeanne)
     293
     294m)  Formulate tutorial for users at GEC14.  What is content of the tutorial, i.e., what are the use cases?  what is different between the test plan and the tutorial?
     295
     296n)  Coordinate tutorial content with GPO (Mark, Niky)  when? starting in mid-May   how?
     297
     298
     299
     300== 8)  Configuration 2:  gather basic host metrics  ==
     301
     302a)  follows p15 Experimenter C slice 
     303
     304a')  Need baseline configuration ASAP   [[BR]]
     305  4/12:  (Guilherme)  considering overall framework, working on interfaces, considering UNIS functions and schema.   [[BR]]
     306  4/19:  (Guilherme) Need baseline config for GEC14 tutorials ASAP  [[BR]]
     307 
     308a!'')  Need MP to gather host metrics   (Guilherme)
     309  - easy for raw servers;  hard for VMs
     310  - considering BLiPP (Matt) to gather all utilization metrics;  extend via libvirt to gather metrics from VMs
     311  - Could still use SNMP daemon from INSTOOLS  (Jim, Hussam)
     312  -  Use netflow to gather flow metrics;  need to be able to turn on/off since uses many resources
     313  -  Could associate ntflow with both hosts and switches.
     314 
     315a!''')  Need to define which host metrics to gather
    291316 - 4/12:  For those gathered by INSTOOLS, see [http://groups.geni.net/geni/attachment/wiki/GEMINI_TopicsIssuesTasks/012312_INSTOOLS_Measurements.txt   list] by Hussam
    292317 - Talking to Dan about use cases for gathering host metrics.
    293  
    294 a!'')  Need MP to gather host metrics   (Guilherme)
    295 
    296  - considering BLiPP (Matt)  via libvirt? via Shinken?
    297  - easy for raw servers;  hard for VMs
    298  - Could still use SNMP daemon from INSTOOLS  (Jim, Hussam)
     318 - 4/19:  see list on github at: 
     319 - 4/19:  start with utilization metrics
     320 - 4/19:  later, add netflow metrics
     321
    299322
    300323b)  MP pushes to Measurement Store (MS)
    301 
    302324  - Use http?  POST to port?  what about authentication and authorization?
    303325  - Use XSP, for streaming?
    304326       
    305 c)  Need to realize MS
    306 
     327c)  Need to realize new MS
    307328  - How many options?
    308329  - One per Aggregate?
    309330  - Can also be arranged for publish/subscribe;  how?
    310331
    311 d)  Need to realize MAP service
    312 
    313   - Based on Periscope?
    314   - Include druple from INSTOOLS?
    315   - Integrate with MS?
    316   - Integrate with protal?
    317        
    318 e)  Uses UNIS (new version)
    319        
    320   - Uses RESTful interface, replaces older UNIS with SOAP interface
    321   - Allows drawing topology
    322   - Used to configure services?
    323   - Prototype underway (Ahmed)
    324        
    325 e') Concern:  new UNIS incompatible with earlier UNIS, which will still be required;  see 4d)
     332d)  Uses new UNIS
     333 - Uses RESTful interface, replaces older UNIS with SOAP interface
     334 - Allows drawing topology
     335 - Used to configure services?
     336 - Prototype underway (Ahmed)
     337 
     338e)  Need to realize MAP service
     339
     340 - Start with modified Periscope service
     341 - Later:  include druple from INSTOOLS?
     342 - Uses new MS
     343 - Uses new UNIS
    326344
    327345f)  Later:  Extend to gathering data from an application
    … …  
    331349h)  Backup option:  Use what capabilities are in LAMP?
    332350
    333 
    334 == 8)  Steps towards GEMINI tutorial at GEC14 ==
    335 
    336 + 4/12/12 topics suggested by Harry: [[BR]]
    337  agree on steps towards GEMINI tutorials at GEC14  (all) [[BR]]
    338 
    339 + 4/18/12 with Martin:  [[BR]]
    340  agree to review plan in mid-May.  [[BR]]
    341  
    342 + 4/19/12 topics suggested by Harry: [[BR]]
    343  agree on steps towards GEMINI tutorials at GEC14  (all) [[BR]]
    344 
    345 a)  Which aggregates, servers, hosts, etc.?
    346 
    347 b)  Start with protoGENI tutorial?  LAMP tutorial?  INSTOOLS tutorial?
    348 
    349 c)  Arrange user workspace (GPO, Jeanne)
    350 
    351 d)  What is first configuration of tools to make active network measurements(see below) ?  LAMP on servers/VMs?  (who provides?)   test scripts?  (Jeanne)
    352 
    353 e)  What is second configuration of tools to gather basic host metrics(see below)?  BLiPP to measurement store, with presentation?  (Guilherme?)  when?  test scripts?  (Jeanne)
    354 
    355 
    356 
    357 == 9)  1st configuration, to make active network measurements  ==
    358 
    359 + 4/12/12 topics suggested by Harry: [[BR]]
    360  9)  agree on configuration to collect network measurements (all) [[BR]]
    361 
    362 + 4/19/12 topics suggested by Harry: [[BR]]
    363  9)  agree on configuration to make active network measurements (all) [[BR]]
    364 
    365 a)  follows p15 Operator A slice, like LAMP project. 
    366 
    367 a') for GEC14, consider: use LAMP code with perhaps minor modifications
    368 
    369 a!'')  Start:  each node on a server, with an available public IP address  single aggregate or multiple aggregates
    370 
    371 a!''')  Second:  all nodes on VMs, or all nodes on VMs except common node on server
    372 
    373 b)  One common node (e.g., node n+1) to:  configure measurements, collect data, present data
    374 
    375 c)  Multiple measurement nodes (e.g., nodes 1, 2, ..., n)
    376 
    377 d)  Start:  one slice, classic pS interface, with data pulled from measurement node, authorized with GENI credential or ?
    378 
    379 e)  Global UNIS as shown;  include local UNIS on common node?
    380 
    381 f)  Load software onto common node with image;  as option, load package
    382 
    383 g)  Load software onto measurement node with image;  as option, load package after app
    384 
    385 g')  Use Gush to manage processes, so that can easily switch between app and active network measurements;  good for establishing continuity and available bandwidth at beginning of experiment.
    386 
    387 h)  Use web interface on common node to configure services, tests, like LAMP;  how does this push config to UNIS??  How do we let only user do this with keys, etc.
    388 
    389 i)  Use web interface on common node to present/observe data, like LAMP  How do we let only user do this with keys, etc.
    390 
    391 j)  Demo all of the perfSONAR network performance tools, tests, as was done in LAMP project
    392 
    393 k)  Extension:  pull data from one slice to another, as shown in p15 from Operator A to Operator B;  authorize using GENI credentials;  how is this done?
    394 
    395 + 4/12/12 topics suggested by Harry: [[BR]]
    396  9l)  review preliminary test plan for collecting network measurements  (Jeanne)  [[BR]]
    397 
    398 + 4/19/12 topics suggested by Harry: [[BR]]
    399  9l)  review updated test plan for collecting network measurements  (Jeanne)  [[BR]]
    400 
    401 l)  Provide regression tests of various configurations, features, etc., driven by scripts  (Jeanne)
    402 
    403 m)  Formulate tutorial for users at GEC14.  What is content of the tutorial, i.e., what are the use cases?  what is different between the test plan and the tutorial?
    404 
    405 n)  Coordinate tutorial content with GPO (Mark, Niky)  when? starting in mid-May   how?
    406 
    407 
    408 
    409 == 10)  2nd configuration to gather basic host metrics  ==
    410 
    411 a)  follows p15 Experimenter C slice 
    412 
    413 a')  for GEC14, need firm plan from 7) ASAP
    414 
    415 a!'')  Introduces push of data to common node;  what protocol?  http?  XSP?  (is this GENI Event Messaging Service?)
    416 
    417 a!''')  Backup option:  Use what capabilities are in LAMP
    418 
    419 b)  Need to organize presentation of data at a web interface;  like INSTOOLS?  Introduce DRUPLE into periscope? (plan Dec 2012)
    420 
    421 c)  Extend:  gather data from user's application (like OML client)
    422 
    423 d)  Provide regression tests of various configurations, features, etc., driven by scripts  (Jeanne)
    424 
    425 e)  Formulate tutorial for users at GEC14.  What is content of the tutorial, i.e., what are the use cases?  what is different between the test plan and the tutorial?
    426 
    427 f)  Coordinate tutorial content with GPO (Mark, Niky)  when?  how?
    428 
    429 
    430 == 11)  Strategy to support multiple configurations in one slice ==
     351i)  Provide regression tests of various configurations, features, etc., driven by scripts  (Jeanne)
     352
     353j)  Formulate tutorial for users at GEC14.  What is content of the tutorial, i.e., what are the use cases?  what is different between the test plan and the tutorial?
     354
     355k)  Coordinate tutorial content with GPO (Mark, Niky)  when?  how?
     356
     357
     358== 9)  Steps toward GEC14 tutorials ==
    431359
    432360a)  Need plan to coordinate configurations to provide make active network measurements and to collect basic host metrics in the same slice
    433 
    434 b)  For GEC13, LAMP code and INSTOOLS code were both loaded;  portal was pointed to both GUIs
    435 
    436 c)  Need to agree on strategy for GEC14:  load both types of code?  is there any chance of a unified set of tools?
    437 
    438 d)  Long term strategy:  unified set of tools
    439 
    440 
    441 
    442 == 12)  GEMINI project documentation ==
     361 - 4/19:  For GEC13, LAMP code and INSTOOLS code were both loaded;  portal was pointed to both GUIs
     362 - 4/19:  Plan to load both types of code
     363 - 4/19:  Perhaps UK could work on unified deployment plan.  Note:  Cannot use UpdateSlice.
     364 
     365b)  Long term strategy:  unified set of tools
     366
     367c)  Start with protoGENI tutorial?  LAMP tutorial?  INSTOOLS tutorial?
     368
     369d)  Arrange user workspace (GPO, Jeanne)
     370
     371e)  Need to agree on schedule for GEC14 tools:
     372 - Design review week of 5/7?  includes arch, configuration, use cases
     373 - Development complete (freeze) on 6/15
     374 - Testing and bug fixes:  +2 weeks
     375 - Prepare and test turotrial:  +1 week
     376
     377
     378
     379== 10)  GEMINI project documentation ==
    443380
    444381a)  Code on IU github  Good:  all relevant code appears to be here, including Kentuck code