wiki:GEMINI_TopicsIssuesTasks

Version 28 (modified by hmussman@bbn.com, 8 years ago) (diff)

--

GEMINI Topics, Issues and Tasks

Notes and tasks from 3/22/12 GEMINI status call:
Additions and corrections after call on 3/27/12 with Martin:
Additions and corrections after team call on 3/29/12, and call on 3/30/12 with Jim:
Additions and changes after call with Martin and Jim on 4/4/12:
Addtions and changes after call with Jim and Charles on 4/17/12:
Additions and changes after call with Martin on 4/18/12:

  • 4/5/12 topics :
    2d) ABAC
    4d) Old UNIS vs new UNIS
    5j') Use of Gush

+ 4/12/12 topics suggested by Harry:

3g) review access to http on VM (Hussam+)
5i) who will help formulate final MDOD schema?
5j') report on Gush (Jeanne)
7h) review current status of baseline configuration to gather host metrics (Guilherme, and all)
8) agree on steps towards GEMINI tutorials at GEC14 (all)
9) agree on configuration to collect network measurements (all)
9l) review preliminary test plan for colelcting network measurements (Jeanne)

+ 4/19/12 topics suggested by Harry:

3g) review access to http on VM (Hussam+)
5i) who will help formulate final MDOD schema? (Martin>?)
5j') report on Gush (Jeanne)
6d) provide a more complete view of GEMINI portal service (Jim, Charles)
7h) review current status of baseline configuration to gather host metrics (Guilherme, and all)
8) agree on steps towards GEMINI tutorials at GEC14 (all)
9) agree on configuration to make active network measurements (all)
9l) review updated test plan for making active network measurements (Jeanne)

1) Authentication and authorization: multiple actor options:

a) tool (outside slice) to AggMgr srvc; AM API; XMl-RPC + ssl [protoGENI cert + GENI credential]

b) tool (outside slice) to host (Slice A); ssh, scp [private/public keys]

c) tool (outside slice) to I&M srvc (Slice A); http(s) [in LAMP, browser to GUI, https with protoGENI cert] [can private/public keys be used for access to a GUI?] [in OMF, signed messages using private/public keys; more details?]

d) I&M srvc (Slice A) to I&M srvc (Slice A); http(s) [in LAMP, service to service, https with LAMP cert, from LAMP CA] [in GIMI/OML, not using http; what is done there?]

e) I&M srvc (Slice A) to I&M srvc (Slice B); http(s) [in European perfSONAR, SOAP interface with security tokens] [can delegated GENI credentials be used?] [can credentials based on ABAC be used?]

f) I&M srvc (Slice A) to UNIS srvc; http(s) [in LAMP, service to UNIS, https with protoGENI cert]

g) tool (outside slice) to iRODS archive srvc; what is interface to iRODS? ftp(s)? can it be http(s)? how is authentication/authorization handled? [need info from Shu] Note; iRODS review call on 4/12, 9:30am.

h) option: I&M srvc (Slice A) to iRODS archive srvc; is there any way to move data from MC direct to iRODS? perhaps mount iRODS on node with MC? [need info from Shu]

2) Authentication and authorization: multiple methods:

a) [for ssh, ssl, etc.] private/public keys

a') [in OMF] signed messages using private/public keys

b) user certificates

c) GENI credentials (user and slice)

c') [in IMF, GENI credentials included with XML messages, for authorization? how? reuse?]

*4/5/12 topic:

d) ABAC [Harry: GPO believes that ABAC may eventually be used for resource assignment, but not soon] [What code is available from ISI? Jim is checking with Teb Faber; waiting for a response]

ABAC references:
Deter web site: http://abac.deterlab.net/
Authorization storyboard from Jeff Chase: http://groups.geni.net/geni/wiki/AuthStoryBoard
Slides on credential store from Jeff Chase: http://groups.geni.net/geni/attachment/wiki/AuthStoryBoard/certstore.ppt
Slides on future of authorization in GENI from Tom Mitchell: http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/Authorization/AuthFuture.pdf [note options without and with credential store]
Summary of GENI authorization discussion at GEC13 (and before): http://groups.geni.net/geni/wiki/GeniAuthorization

*4/5/12 notes from Jeanne:
Jim got code from Ted Faber. Looking through it. Looking at example code.
ABAC is not currently implemented (at ISI?) as a service. This needs to be done. Ted thinks this should be trivial. Looks like (via papers) ORCA has implemented as a server (ORCA Pod?) with RESTful interface.
Jim contacted Jeff Chase to get code. Making some progress, still some unknowns.
Martin: Thinks we perhaps can use UNIS for source of constraints or reference (URL) to constraints. Use libabac to prove the chain of assertions.
All agree that we should have a central location for rules.
Guilherme: Don’t want the rules to be exposed.
Is the proving done at the service or at the authenticating application?
Task: Harry suggests drawing up a proposal for using ABAC. Jim: Jim and Martin to discuss, learn more, and come up with a proposal.

3) Target protoGENI environments:

a) servers: relatively few; public IP available

b) VMs: OpenVZ; expect move to LXC; internal to an aggregate, private host name, private IP addresses, need more details

c) To date, all LAMP/periscope has been on servers

c') Task: try to run all LAMP nodes (or just measurement nodes) on VMs (Matt Jaffe)

d) To date, all INSTOOLS has been with MC on server, and MPs on VMs

e) Task: Try to run INSTOOLS MC in a VM; Nasir on 3/30: still runs, although might need some small code changes; but would need to open http port, perhaps with extension to rspec; need to discuss with protoGENI (Jonathan Deurig) about adding to mapping agent; Jim had discussed with Jonathan and Rob earlier, quite doable, but would have to restart mapping agent; perhaps could "piggyback" on opening ssh port?

f) Task: can ssh into public host name (or public IP) , with special 5+ digit port number (from manifest) from port map

+ 4/12/12 topics suggested by Harry:
3g) and beyond review access to http on VM (Hussam+)

+ 4/19/12 topics suggested by Harry:
3g) and beyond review access to http on VM (Hussam+)

g) Task: (see e) above) how to access http interface? tunnel through ssh? port map, like ssh? perhaps could "piggyback" on opening ssh port? setup a separate proxy?

See Port mapping proposal from Hussam
Jim on 4/12: Talked with Rob Ricci. He is willing to work on it, since many needs to proxy to VMs. (In INSTOOLS, used VNC)

g') New task: Review possible tunnel through ssh (or use fo ssh to forward http port), to reuse available ssh port mapping. (who?)

On 4/12: Too complicated; simpler to just punch through ports 22 and 80

g'') New task: Review port mapping for http, like ssh, with protoGENI, to see how it might be done (Nasir/Jim)

g''') New task: Review need within GENI/GPO to open ports, and implications for rspec (Harry)

h) Task: what about vnc tunnels? how were they done in INSTOOLS? which port on host? (who?)

i) Task: what happens when VMs are on multiple aggregates? (who?)

j) Task: consider separate host for managing communications? VM? server? centralized? include pub/sub? is this GENI Event Messaging Service? (who?)

On 4/12: protoGENI considering using OPS server to provide persistent proxy.

4) UNIS service

Per call with Martin on 3/27/12:

a) Question: In LAMP, is there a local UNIS, or not?? (Martin) Not yet; needs to be, with push from local UNIS to global UNIS.

b) How does UNIS authenticate/authorize when receiving data? (Martin) [in LAMP, service to UNIS, https with protoGENI cert]

c) Question: Use web interface on common node to configure services, tests; how does this push config to UNIS? What authentication/authorization steps are included?

*topic on 4/5/12:
d) Old UNIS vs new UNIS:
*4/5/12 notes from Jeanne:
What is the transition plan? Both can run in parallel until full functionality is available with new UNIS. Then turn down old UNIS.
Local vs. global UNIS hierarchy: Will new UNIS have local and global configuration? Yes, probably not by GEC14.

5) User workspace service

Current view: (Harry)

a) Persistent Linux environment, with file system, key/certificate/credential store, dedicated to the user; could also have rspec store, etc.

b) Place for tools, e.g., Gush and OMNI, and scripts; can easily call one another; not in slice; could deal with multiple slices

c) Place for "portals"; but what are they? (see below)

d) Task: Setup user workspace using server (or VM) in BBN Cambridge lab; begin to include tools, etc . (Jeanne) On 3/30/12: Done on VM in BBN Cambridge lab, ubuntu 10.04, internal to BBN. Next: external to BBN

e) Task: Consider VM to distribute user workspace (Matt); e.g, ubuntu on virtual box [similar to what has been done at GEC tutorials]

f) Task: What is required to secure keys/certificates/credentials? passphrase? other? [Per Tom Mitchell, OMNI does not require passphrase, but FLACK does currently require passpharase] [Per Jim protoGENI cert does require passphrase] [Vic to check with Steve Schwab; need to balance security and ability ot use scripts.]

g) Start with CNRI: Directory Archive (DA) service, which can push data to DOA service, using OI service
Then replace DOA with iRODS
[Have iRODS at IU for NetKarma; Jim and Wesley talking with Ilia and Shu]

h) Include MDOD creator/editor (CNRI, GPO)

+ 4/12/12 topics suggested by Harry:

5i) who will help formulate final MDOD schema?

+ 4/19/12 topics suggested by Harry:

5i) who will help formulate final MDOD schema? (Martin>?)

i) Task: Need help with final formulation of MDOD (Ezra?) (Shu from GIMI)

j) Task: Define view of user workspace service (Jeanne, Matt, Harry, Jim, Martin, Niky)
[Jeanne to add security policy into view]

*topic on 4/5/12:

+ 4/12/12 topics suggested by Harry:

5j') report on GUSH (Jeanne)

+ 4/19/12 topics suggested by Harry:

5j') more on GUSH (Jeanne)

j') Use of GUSH:
*4/5/12 notes from Jeanne:
What does Gush provide vs. Flack? Why would user use Gush?
Working with VMs. According to Vic, Jeannie A. says Gush will work with anything that allows SSH.
Jeanne O. has experienced some issues with VMs in Gush. Investigate further.
Issues with hostnames? Need to investigate this further.
Harry: Suggest Jeanne talk with Luisa about Gush information. She has worked with it a lot.
Jim asks Martin: How does Gush integrate with UNIS?
Discussion of using UNIS to store/access information about the slices for the experiment rather than passing around rspecs.
How do we keep this UNIS information up-to-date?
Guilherme suggests things that are outside of slice introspection, user needs to push to UNIS.
What types of changes can we make to the slice in Gush/Omni/other that I&M and others need to discover from UNIS?
Task: Things to investigate regarding Gush (Jeanne will report next week):

  1. Tridentcom paper says gush has ability to add and remove nodes from a slice. How is this done? Under what circumstances does this work? [GENI AM API does not support updateSliver]
  2. How does Gush work with protogeni VMs?

4/12: Jeanne reports:
Certain commands (slice add, renew, update) apply only to PlanetLab.
Issue: Gush takes hostname and goes to physical node, not VMs; considering how to rectify.
In ORCA, this works. hostname in manifest is not the same as the hostname reported for the node.

4/12: Jim asks: How would be run shell scripts on nodes? scp between nodes?

6) Portal services

a) Option 1: "portal to UIs". [Is this close to Jim's proposal?]

b) Option 2: a more complete tool for managing I&M services, as is implied in the SOW, so that it can view, orchestrate, edit MDOD, manage archiving, etc. [Is this close to Max's proposal]

c) Task: understand options for authentication and authorization at a web interface. (who?)

+ 4/19/12 topics suggested by Harry:

6d) provide a more complete view of GEMINI portal service (Jim, Charles)

d) Task: provide a more complete view of GEMINI portal service (Harry, Jim and Charles)

Task: Jim and Charles plan to provide in a week or two.
Task: Charles needs to find a name for the service

After discussion on 3/31/12 with Jim, Harry feels that this is very close to Option 1: "portal to UIs".
Jim expects User to have a capable browser, e.g., one that runs HTML-5
Jim expects portal to manage windowing to various GUIs.
Jim expect all interactions to be via browser, so there are window(s) to login to shell(s), etc.
Jim does not specify whether browser is looking at GUI in slice, or a tool; tools are not in a specified place. Harry feels that portal and other tools are in a "user workspace", in a persistent Linux environment, with file system, key/certificate/credential store, dedicated to the user; could also have rspec store, etc. ; then, all tools have ready access to required info, and can readily call one another.
Harry thinks of "persistent Linux environment" on infrastructure, e.g., a server under your desk or in the lab; not your laptop; Jim agrees, and has thought portal would be hosted on infrastructure at Kentucky
Harry feels that this is just a strucutre, that there is much more work to define tools, interfaces, etc.; Jim agrees, was concerned it was the final configuration.
Task: Harry will modify drawing to reflect discussion with Jim, and then two perspectives can be compared.
Done on 4/4; agree that portal can be in user workspace, or somewhere else.
See updated drawing.

e) Task: Understand NICTA's iREEL portal service; is this a more complete tool for managing I&M services?

  • Get login, and survey (Jeanne)

  • Provide more info (NICTA, e.g., Christoph)

7) New configuration to gather basic host metrics

+ 4/12/12 topics suggested by Harry:
7h) review current status of baseline configuration to gather host metrics (Guilherme, and all)

+ 4/19/12 topics suggested by Harry:
a) review current status of baseline configuration to gather host metrics (Guilherme, and all)

a) Need baseline configuration ASAP (Guilherme)
4/12: (Guilherme) considering overall framework, working on interfaces, considering UNIS functions and schema.

a') Need to define which host metrics to gather

  • 4/12: For those gathered by INSTOOLS, see list by Hussam
  • Talking to Dan about use cases for gathering host metrics.

a'') Need MP to gather host metrics (Guilherme)

  • considering BLiPP (Matt) via libvirt? via Shinken?
  • easy for raw servers; hard for VMs
  • Could still use SNMP daemon from INSTOOLS (Jim, Hussam)

b) MP pushes to Measurement Store (MS)

  • Use http? POST to port? what about authentication and authorization?
  • Use XSP, for streaming?

c) Need to realize MS

  • How many options?
  • One per Aggregate?

d) Need to realize MAP service

  • Based on Periscope?
  • Include druple from INSTOOLS?
  • Integrate with MS?
  • Integrate with protal?

e) Uses UNIS (new version)

  • Uses RESTful interface, replaces older UNIS with SOAP interface
  • Allows drawing topology
  • Used to configure services?
  • Prototype underway (Ahmed)

e') Concern: new UNIS incompatible with earlier UNIS, which will still be required (see 9) below.

f) Later: Extend to gathering data from an application

g) Task: Prototype soon when? (Guilherme)

h) Backup option: Use what capabilities are in LAMP

8) Steps towards GEMINI tutorial at GEC14

+ 4/12/12 topics suggested by Harry:

8) agree on steps towards GEMINI tutorials at GEC14 (all)

+ 4/19/12 topics suggested by Harry:

8) agree on steps towards GEMINI tutorials at GEC14 (all)

a) Which aggregates, servers, hosts, etc.?

b) Start with protoGENI tutorial? LAMP tutorial? INSTOOLS tutorial?

c) Arrange user workspace (GPO, Jeanne)

d) What is first configuration of tools to make active network measurements(see below) ? LAMP on servers/VMs? (who provides?) test scripts? (Jeanne)

e) What is second configuration of tools (see below)? BLiPP to measurement store, with presentation? (Guilherme?) when? test scripts? (Jeanne)

9) 1st configuration, to make active network measurements

+ 4/12/12 topics suggested by Harry:

9) agree on configuration to collect network measurements (all)

+ 4/19/12 topics suggested by Harry:

9) agree on configuration to make active network measurements (all)

a) follows p15 Operator A slice, like LAMP project.

a') for GEC14, consider: use LAMP code with perhaps minor modifications

a'') Start: each node on a server, with an available public IP address single aggregate or multiple aggregates

a''') Second: all nodes on VMs, or all nodes on VMs except common node on server

b) One common node (e.g., node n+1) to: configure measurements, collect data, present data

c) Multiple measurement nodes (e.g., nodes 1, 2, ..., n)

d) Start: one slice, classic pS interface, with data pulled from measurement node, authorized with GENI credential or ?

e) Global UNIS as shown; include local UNIS on common node?

f) Load software onto common node with image; as option, load package

g) Load software onto measurement node with image; as option, load package after app

g') Use Gush to manage processes, so that can easily switch between app and active network measurements; good for establishing continuity and available bandwidth at beginning of experiment.

h) Use web interface on common node to configure services, tests, like LAMP; how does this push config to UNIS?? How do we let only user do this with keys, etc.

i) Use web interface on common node to present/observe data, like LAMP How do we let only user do this with keys, etc.

j) Demo all of the perfSONAR network performance tools, tests, as was done in LAMP project

k) Extension: pull data from one slice to another, as shown in p15 from Operator A to Operator B; authorize using GENI credentials; how is this done?

+ 4/12/12 topics suggested by Harry:

9l) review preliminary test plan for collecting network measurements (Jeanne)

+ 4/19/12 topics suggested by Harry:

9l) review updated test plan for collecting network measurements (Jeanne)

l) Provide regression tests of various configurations, features, etc., driven by scripts (Jeanne)

m) Formulate tutorial for users at GEC14. What is content of the tutorial, i.e., what are the use cases? what is different between the test plan and the tutorial?

n) Coordinate tutorial content with GPO (Mark, Niky) when? how?

10) 2nd configuration to gather basic host metrics

a) follows p15 Experimenter C slice

a') for GEC14, need firm plan from 7) ASAP

a'') Introduces push of data to common node; what protocol? http? XSP? (is this GENI Event Messaging Service?)

a''') Backup option: Use what capabilities are in LAMP

b) Need to organize presentation of data at a web interface; like INSTOOLS? Introduce DRUPLE into periscope? (plan Dec 2012)

c) Extend: gather data from user's application (like OML client)

d) Provide regression tests of various configurations, features, etc., driven by scripts (Jeanne)

e) Formulate tutorial for users at GEC14. What is content of the tutorial, i.e., what are the use cases? what is different between the test plan and the tutorial?

f) Coordinate tutorial content with GPO (Mark, Niky) when? how?

11) Strategy to support multiple configurations in one slice

a) Need plan to coordinate configurations to provide make active network measurements and to collect basic host metrics in the same slice

b) For GEC13, LAMP code and INSTOOLS code were both loaded; portal was pointed to both GUIs

c) Need to agree on strategy for GEC14: load both types of code? is there any chance of a unified set of tools?

d) Long term strategy: unified set of tools

12) GEMINI project documentation

a) Code on IU github Good: all relevant code appears to be here, including Kentuck code

b) Jira Good: being used by IU to track project Concern: Kentucky effort not reflected here

c) GENI trac for GEMINI Include TopicsTasksIssues Include drawings

Attachments (3)

Download all attachments as: .zip