GIMI I&M Tools: Basic Experimenter Use Case

For discussion on 053012:

1) Establish test/experiment environment

2) Obtain slice of GENI resources, install and configure GIMI I&M tools and reference/actual experiment

3) Run and orchestrate GIMI I&M tools and reference/actual experiment

4) Observe I&M measurement results on GIMI presentation service inside of the slice

5) Push I&M measurement results to iRODS archive service

6) Pull I&M measurement results from iRODS archive service to IREEL portal service, then analyze/process/present results, and push back to iRODS archive service

1) Establish test/experiment environment

1.1) Establish user workspace service

1.2) Verify availability of desired aggregates

1.3) Verify availability of desired software images/packages

1.4) Verify avaialbility of necessary oeprational services

• XMPP server
• IREEL portal service
• iRODS archive service

2) Obtain slice of GENI resources, install and configure GIMI I&M tools and reference/actual experiment

2.1) Select target aggregates:

• a) ExoGENI rack VMs, one aggregate
  • MC on VM
  • MPs and ExperApp on VMs (2 or more)

• b) ExoGENI rack VMs, multiple aggregates (2 or more)
  • MC on VM; each aggregate or one aggregate?
  • MPs and ExperApp on VMs, each aggregate (2 or more)

2.2) Select desired software images/packages:

• a) VM image including OS plus all GIMI I&M code
  • GIMI I&M code includes both MC and MP code
• if Gush is used, Gush client can be loaded in image

• b) Reference/Active Experiment pkg(s)
  • need: experiment topology
    • option: one package to be installed on all nodes
    • option: multiple packages, to be installed on different nodes

2.3) Obtain certs, credentials, passwords and keys

• required services: GENI CA; GENI SA; GENI key generation service

• tools: shell scripts (later: clearinghouse portal)

• at completion: certs, credentials, passwords (to iRODS) and keys in local store, in user workspace

• typically, do not have to repeat

2.4) Formulate slice topology for test/experiment, and build request rspec

• tools: how to visualize topology of experiment and I&M tools? use FLUKES?

• tools: how to build request rspec? edit respec file? use FLUKES?

• at completion: request rspec in local store, in user workspace

2.5) Acquire resources and load images

• tools: OMNI, to acquire resources and load images

• option: also loads software packages for reference/actual experiment

• at completion: completed slice/slivers of hosts, with images, with VLAN connections

• at completion: manifest rspec in local store, in user workspace

• verification: shell scripts that use keys to ssh into hosts; also ping other hosts

• note: 2.1) through 2.5) parallels rack acceptance tests

2.6) Load software packages for reference/actual experiment (as needed)

• tools: shell scripts

• at completion: completed slice/slivers of hosts, with images, with VLAN connections, with all software packages installed

• verification: shell scripts that ssh into hosts, also ping other hosts; also verify all desired software installed

2.7) Start software processes (as selected)

• Select options for each node:
  • I&M Measurement Collector (MC), including analysis and presentation (yes or no)
  • I&M Measurement Point (MP) for basic host measurements, e.g., cpu and memory usage (yes or no)
  • I&M Measurement Point (MP) for active network measurements, e.g., ping and iperf (yes or no)
  • Reference/Actual Experiment (yes or no) option: differnet processes on different nodes

• Typical selections:
  • One node with MC; multiple nodes with MP + Ref/Actual Exper

• tools: shell scripts or use Gush?

• at completion: completed slice/slivers of hosts, with images, with VLAN connections, with all software packages installed, with selected software processes running

• verification: shell scripts that ssh into hosts, ping other hosts, verify all software installed, verify selected processes running or Gush that verifies selected processes running

2.8) Configure selected I&M services

• objectives:
  • binds services together, e.g., between each MP and the MC
  • configures baseline services
  • verifies that they are ready to run

• tools: use what tool to assign MPs to interfaces? use what tool to point data from MPs to MC?

• tools: use what tool to point browser at MC? use what tool to configure presentation service in MC?

• tools: use what tool (OMF?) to start, stop, reset I&M services; and to observe the transition

• at completion: configured I&M services, ready to run and collect data

2.9) Configure selected reference/actual experiment services

• objectives:
  • binds services together to realize overall reference/actual experiment
  • configures baseline services
  • verifies that they are ready to run

• tools: use what tool to select options in reference/actual experiment? use what tool to bind services together?

• tools: use what tool (OMF?) to start, stop, reset reference/actual experiment services; and to observe the transition

• at completion: configured reference/actual experiment services, ready to run and process data, which can be observed by I&M services

3) Run and orchestrate GIMI I&M tools and reference/actual experiment

3.1) Initial setup: start basic host measurement and ping active network measurement I&M services, to verify functionality of I&M services and topology of slice

• tools: use what tool (OMF?) to start basic host measurement and ping active network measurement I&M services, and observe the transition

• tools: use what to find GUI on MC, then use GUI to observe presentation of measurements by MAP (is there a way to script this?)

• verify getting expected measurements from each MP, and collection at MC (is there a way to script this?)

• tools: stop I&M services

• at completion: I&M services have collected basic host measurements and ping active network measurements from each MP, measured data is in MC, and measured data has been visualized by MAP service; functionality of I&M services and topology of slice has been verified

3.2) Continuity test: start basic host measurement and iperf active network measurement I&M services, to verify continuity of slice

• tools: use what tool (OMF?) to start basic host measurement and iperf active network measurement I&M services, and observe the transition

• tools: use what to find GUI on MC, then use GUI to observe presentation of measurements by MAP (is there a way to script this?)

• verify getting expected measurements from each MP, and collection at MC (is there a way to script this?)

• tools: stop I&M services

• at completion: I&M services have collected basic host measurements and iperf active network measurements from each MP, measured data is in MC, and measured data has been visualized by MAP service; functionality of I&M services and continuity of slice has been verified

3.3) Instrument reference/actual experiment: start basic host measurement and ping active network measurement I&M services; then start reference/actual experiment, and gather measurments to verify functionality of refrence/actual experiment in slice

• tools: use what tool (OMF?) to start basic host measurement and ping active network measurement I&M services, and observe the transition

• tools: use what tool (OMF?) to start reference/actual experiment services; and to observe the transition

• tools: use what to find GUI on MC, then use GUI to observe presentation of measurements by MAP (is there a way to script this?)

• verify getting expected measurements from each MP, and collection at MC (is there a way to script this?)

• tools: stop I&M services and reference/actual experiment services

• at completion: I&M services have collected basic host measurements and ping active network measurements from each MP while reference/actual experiment is running, measured data is in MC, and measured data has been visualized by MAP service; functionality of of refrence/actual experiment in slice has been verified

4) Observe I&M measurement results on GIMI presentation service inside of the slice

• use GUI on MC to observe measurement results in real time (as noted above)

• use GUI on MC to observe measurement results after experiment has been completed

• use what tool to configure presentation of measurment results?

• can measurement results be keyed to a topology map?

5) Push I&M measurement results to iRODS archive service

• use what tool to format group measurement results?

• use what tool to push measurement results to iRODS archive service?

• use what tool to formulate Measurement Data Object Descriptor (MDOD) for measurement results? to add annotation?

• use what tool to push MDOD to iRODS archive service?

6) Pull I&M measurement results from iRODS archive service to IREEL portal service, then analyze/process/present results, and push back to iRODS archive service

• use what tool to pull measurement results from iRODS archive service to IREEL portal service?

• what are options to analyze/process/present results in IREEL service? how are these configured?

• use what tool to push measurement results from IREEL portal service back to iRODS archive service, after they have been modified?