GIMI_v1.0_BasicUseCase: 080312_I&M_GIMI_BasicUseCase.txt

[[PageOutline]]

== GIMI I&M Tools:  Basic Experimenter Use Case ==

For discussion on 053012:

1)  Establish test/experiment environment

2)  Obtain slice of GENI resources, install and configure GIMI I&M tools and reference/actual experiment 

3)  Run and orchestrate GIMI I&M tools and reference/actual experiment 

4)  Observe I&M measurement results on GIMI presentation service inside of the slice

5)  Push I&M measurement results to iRODS archive service

6)  Pull I&M measurement results from iRODS archive service to IREEL portal service, then analyze/process/present results, and push back to iRODS archive service

7)  Release experiment resources

=== 1)  Establish test/experiment environment ===

1.1)  Establish user workspace service

1.2)  Gather necessary keys, certificates and credentials

1.3)  Verify availability of desired aggregates

1.4)  Verify availability of desired software images/packages

1.5)  Verify avaialbility of necessary oeprational services

 -  XMPP server
 -  IREEL portal service
 -  iRODS archive service

=== 2)  Obtain slice of GENI resources, install and configure GIMI I&M tools and reference/actual experiment ===

2.1)  Select target aggregates:

 - a)  ExoGENI rack VMs, one aggregate
   - MC on VM
   - MPs and ExperApp on VMs  (2 or more)
   
 - b)  ExoGENI rack VMs, multiple aggregates  (2 or more)
   - MC on VM; each aggregate or one aggregate?
   - MPs and ExperApp on VMs, each aggregate  (2 or more)   
        
2.2)  Select desired software images/packages: 

 - a)  VM image including OS plus all GIMI I&M code
        - GIMI I&M code includes both MC and MP code
        - if Gush is used, Gush client can be loaded in image

 - b)  Reference/Active Experiment pkg(s)
    - need:  experiment topology
        - option:  one package to be installed on all nodes
        - option:  multiple packages, to be installed on different nodes 
        
2.3)  Obtain certs, credentials, passwords and keys

 - required services:  GENI CA;  GENI SA;  GENI key generation service
 
 - tools:  shell scripts   (later:  clearinghouse portal)
 
 - at completion:    certs, credentials, passwords (to iRODS) and keys in local store, in user workspace
 
 - typically, do not have to repeat


2.4)  Formulate slice topology for test/experiment, and build request rspec

 - tools:   how to visualize topology of experiment and I&M tools?    use  FLUKES?
 
 - tools:  how to build request rspec?  edit respec file?   use  FLUKES?
 
 - at completion:    request rspec in local store, in user workspace


2.5)  Acquire resources and load images

 - tools:   OMNI, to acquire resources and load images
 
 - option:  also loads software packages for reference/actual experiment
 
 - at completion:    completed slice/slivers of hosts, with images, with VLAN connections
 
 - at completion:  manifest rspec in local store, in user workspace
 
 - verification:  shell scripts that use keys to ssh into hosts;  also ping other hosts
 
 - note:  2.1) through 2.5) parallels rack acceptance tests


2.6)  Load software packages for reference/actual experiment (as needed)

 - tools:  shell scripts
 
 - at completion:    completed slice/slivers of hosts, with images, with VLAN connections, with all software packages installed
 
 - verification:  shell scripts that ssh into hosts, also ping other hosts;  also verify all desired software installed



2.7)  Start software processes  (as selected)  

 - Select options for each node: 
   - I&M Measurement Collector (MC), including  analysis and presentation (yes or no)
   - I&M Measurement Point (MP) for basic host measurements, e.g., cpu and memory usage  (yes or no)
   - I&M Measurement Point (MP) for active network measurements, e.g., ping and iperf (yes or no)
   - Reference/Actual Experiment  (yes or no)  option:  differnet processes on different nodes
        
 - Typical selections:
    -  One node with MC;  multiple nodes with MP + Ref/Actual Exper 

 - tools:  shell scripts  or use Gush?
 
 - at completion:    completed slice/slivers of hosts, with images, with VLAN connections, with all software packages installed, with selected software processes running
 
 - verification:  shell scripts that ssh into hosts, ping other hosts, verify all software installed, verify selected processes running or Gush that verifies selected processes running
 
 
2.8)   Configure selected I&M services

 - objectives:
   - binds services together, e.g., between each MP and the MC
   - configures baseline services
   - verifies that they are ready to run

 - tools:  use what tool to assign MPs to interfaces?   use what tool to point data from MPs to MC?
 
 - tools:  use what tool to point browser at MC?  use what tool to configure presentation service in MC?
 
 - tools:  use what tool (OMF?) to start, stop, reset I&M services; and to observe the transition  
 
 - at completion:    configured I&M services, ready to run and collect data
 

2.9)  Configure selected reference/actual experiment services

 - objectives:
   - binds services together to realize overall reference/actual experiment
   - configures baseline services
   - verifies that they are ready to run   
   
 - tools:  use what tool to select options in reference/actual experiment?  use what tool to bind services together?  
   
 - tools:  use what tool (OMF?) to start, stop, reset reference/actual experiment services; and to observe the transition  
 
 - at completion:    configured reference/actual experiment services, ready to run and process data, which can be observed by I&M services
 
 
 
 
=== 3)  Run and orchestrate GIMI I&M tools and reference/actual experiment  ===


3.1)  Initial setup:  start basic host measurement and ping active network measurement I&M services, to verify functionality of I&M services and topology of slice 

 - tools:  use what tool (OMF?) to start basic host measurement and ping active network measurement I&M services, and observe the transition 

 - tools:  use what to find GUI on MC, then use GUI to observe  presentation of measurements by MAP (is there a way to script this?)
 
 - verify getting expected measurements from each MP, and collection at MC  (is there a way to script this?)
 
 - tools:  stop I&M services
 
 - at completion:    I&M services have collected basic host measurements and ping active network measurements from each MP, measured data is in MC, and measured data has been visualized by MAP service;  functionality of I&M services and topology of slice has been verified
 
 
3.2)  Continuity test:  start basic host measurement and iperf active network measurement I&M services, to verify continuity of slice 

 - tools:  use what tool (OMF?) to start basic host measurement and iperf active network measurement I&M services, and observe the transition 

 - tools:  use what to find GUI on MC, then use GUI to observe  presentation of measurements by MAP (is there a way to script this?)
 
 - verify getting expected measurements from each MP, and collection at MC  (is there a way to script this?)
 
 - tools:  stop I&M services
 
 - at completion:    I&M services have collected basic host measurements and iperf active network measurements from each MP, measured data is in MC, and measured data has been visualized by MAP service;  functionality of I&M services and continuity of  slice has been verified
 
3.3)  Instrument reference/actual experiment:  start basic host measurement and ping active network measurement I&M services;  then start reference/actual experiment, and gather measurments to  verify functionality of refrence/actual experiment in slice 

 - tools:  use what tool (OMF?) to start basic host measurement and ping active network measurement I&M services, and observe the transition 
 
 - tools:  use what tool (OMF?) to start reference/actual experiment services; and to observe the transition

 - tools:  use what to find GUI on MC, then use GUI to observe  presentation of measurements by MAP (is there a way to script this?)
 
 - verify getting expected measurements from each MP, and collection at MC  (is there a way to script this?)
 
 - tools:  stop I&M services and reference/actual experiment services
 
 - at completion:    I&M services have collected basic host measurements and ping active network measurements from each MP while reference/actual experiment is running, measured data is in MC, and measured data has been visualized by MAP service;  functionality of of refrence/actual experiment in slice has been verified
  
=== 4)  Observe I&M measurement results on GIMI presentation service inside of the slice ===

 - use GUI on MC to observe measurement results in real time  (as noted above)
 
 - use GUI on MC to observe measurement results after experiment has been completed
 
 - use what tool to configure presentation of measurment results?
 
 - can measurement results be keyed to a topology map?

=== 5)  Push I&M measurement results to iRODS archive service ===

 - use what tool to format group measurement results?
 
 - use what tool to push measurement results to iRODS archive service?
 
 - use what tool to formulate Measurement Data Object Descriptor (MDOD) for measurement results?  to add annotation?
 
 - use what tool to push MDOD to iRODS archive service?

=== 6)  Pull I&M measurement results from iRODS archive service to IREEL portal service, then analyze/process/present results, and push back to iRODS archive service ===

 - use what tool to pull measurement results from iRODS archive service to IREEL portal service?
 
 - what are options to analyze/process/present results in IREEL service?  how are these configured?
 
 - use what tool to push measurement results from  IREEL portal service back to iRODS archive service, after they have been modified?


=== 7)  Release experiment resources  ===