Changes between Version 13 and Version 14 of TestTutorialExperimentWorkflow


Ignore:
Timestamp:
01/02/13 11:41:39 (7 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • TestTutorialExperimentWorkflow

    v13 v14  
    550)  Overview of experiment
    66
    7 1)  Design experiment  [Design Experiment]
    8 
    9 2)  Establish experiment environment   [Establish Environment]
    10 
    11 3)  Obtain slice of GENI resources; install I&M and experiment software  [Obtain Resources and Load Software]
    12 
    13 4)  Configure and initialize I&M and experiment services  [Configure Services]
    14 
    15 5)  Run and orchestrate I&M and experiment services, to complete run of experiment; move selected measurements and other artifacts to storage service  [Run Experiment and Store Artifacts]
    16 
    17 6)  Analyze and visualize measurement results  [Analyze Results]
    18 
    19 7)  Release experiment resources  [Release Resources]
    20 
    21 8)  Move selected measurements and other artifacts from storage service to long-term archive service  [Archive Artifacts]
    22 
     71)  Design experiment
     8
     92)  Establish experiment management environment
     10
     113)  Obtain slice of GENI resources, and then install I&M and experiment software
     12
     134)  Configure and initialize I&M and experiment services
     14
     155)  Run and orchestrate I&M and experiment services, to complete run of experiment
     16
     176)  Analyze experiment results, and decide on next step(s) 
     18
     197)  When no longer needed, release all experiment resources 
     20
     218)  Optionally, archive selected measurement and/or experiment artifacts 
    2322
    2423
     
    28270.1)  Title and abstract
    2928
    30 0.2)  Goals
     290.2)  Goals and strategy for results
    3130
    32310.3)  Configuration
    3332
    34 0.4)   Process  (flow chart)
     330.4)  Process  (flow chart)
     34
     35
    3536
    3637=== 1)  Design experiment ===
    3738
    38 1.1)  Establish experiment goals and overall plan
    39 
    40 1.2)  Design configuration of experiment, including topology, desired resources, desired tools and desired services;  consider previous, or reference, experiments as guides
     391.1)  Establish experiment goals and strategy for results
     40
     411.2)  Design configuration of experiment to meet goals, including topology, desired resources, desired tools and desired services;  consider previous, or reference, experiments as guides
    4142
    42431.3)  Map desired resources to aggregates that are likely to be available
     
    48491.6)  Find, or design and code, software images/packages/applications, rspecs and/or scripts necessary to realize experiment, and put into storage service
    4950
    50 
    51 === 2)  Establish experiment environment ===
    52 
    53 2.1)  Establish experiment managment (user workspace) service
     511.7)  Outline process for the experiment, and capture in a high-level flowchart
     52
     531.8)  Optionally, prototype the experiment using "GENI-in-a-box"
     54
     55
     56=== 2)  Establish experiment management environment ===
     57
     582.1)  Establish experiment management (user workspace) service
    5459
    55602.2)  Gather necessary keys, passwords, certificates and credentials
    5661
    57 2.3)  Verify availability of desired resources in aggregates
     622.3)  Verify availability of desired resources in aggregates, including nodes and networking resources, by querying aggregates and/or GMOC;  optionally, reserve scarce resources
    5863
    59642.4)  Verify availability of necessary operational services, and access to those services
     
    6469
    6570
    66 === 3)  Obtain slice of GENI resources; install I&M and experiment software ===
     71=== 3)  Obtain slice of GENI resources, and then install I&M and experiment software ===
    6772
    68733.1)  Select target aggregates
     
    74793.4)  Acquire resources and load images/packages for I&M and experiment services
    7580
    76 3.5)  Confirm at completion:  slice with installed images/packages for I&M and experiment services
     813.5)  Verify slice with desired topology and installed images/packages/software for I&M and experiment services on nodes and on networking resources
     82
     83Status at completion of step 3):
     84
     85 - obtained slice of GENI resources, with desired topology
     86 
     87 - installed images/packages/software for I&M and experiment services on nodes and on networking resources
    7788
    7889
    7990=== 4) Configure and initialize I&M and experiment services ===
    8091
    81 4.1)   Configure selected I&M services
    82 
    83  - objectives:
    84    - binds services together, e.g., between each MP and the MC
    85    - configures baseline services
    86    - initializes them, and verifies that they are ready to run
    87  
    88  - at completion:    configured I&M services, ready to run and collect data
    89  
    90 
    91 4.2)  Configure selected experiment services
    92 
    93  - objectives:
    94    - binds services together to realize overall reference/actual experiment
    95    - configures baseline services
    96    - initializes them, and verifies that they are ready to run   
     924.1)  Configure I&M services
     93
     94 - include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
     95 
     96 - bind I&M services together (e.g., between each MP and the MC)
     97 
     98 - initialize I&M services
     99
     100 - verify that I&M services are ready to run and collect measurement data
     101
     1024.2)  Configure experiment services
     103
     104 - bind experiment services together to realize desired topology and functionality
     105 
     106 - initialize experiment services
     107
     108 - verify that experiment services are ready to run and collect data, and can be observed by I&M services
     109
     110 
     111Status at completion of step 4):
     112
     113 - I&M services have been configured to include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
     114 
     115 - I&M services have been bound together and initialized, so that they are ready to run and collect measurement data
     116 
     117 - experiment services have been configured to realize desired topology and functionality
     118 
     119 - experiment services have been bound together and initialized, so that they are ready to run and collect measurement data
     120 
     121 
     122=== 5)  Run and orchestrate I&M and experiment services, to complete run of experiment  ===
     123
     1245.1)  Inititate basic measurements
     125
     126 - start basic host measurements to verify functionality of hosts
     127 
     128 - start basic ping active network measurements to verify topology of slice
     129
     130 - observe measurements with a real-time presentation service
     131 
     132 - continue measurements throughout duration of the experiment, to verify functionality of hosts and topology of slice
     133 
     1345.2)  Verify topology and bandwidth/performance of networking resources (continuity test)
     135
     136 - configure and start active network measurements (i.e., iperf) in a configuration designed to verify ability of slice to carry traffic expected during experiment
     137
     138 - observe measurements with a real-time presentation service
     139 
     140 - once satisfactory measurements have been observed, stop active network measurments
    97141   
    98  - at completion:    configured experiment services, ready to run and process data, which can be observed by I&M services
    99  
     1425.3)  Instrument and run experiment
     143
     144 - begin to run and orchestrate I&M services
     145 
     146 - begin to run and orchestrate experiment services
     147 
     148 - observe measurements with a real-time presentation service, to verify expected operation of experiment
     149 
     150 - collect all measurements for duration of experiment, and save for further analysis
     151 
     152 - stop experiment services, when this run of the experiment has been completed
     153 
     154 - stop I&M services
     155   
     1565.4)  Save collected measurements and other artifacts
     157
     158 - save collected measurements and other artifacts from experiment in storage service, for further analysis and reference
     159
     160 - then, slice resources can then be released at any time, without loss of any measurements or artifacts
     161
     162 
     163Status at completion of step 5):
     164
     165 - one run of experiment has been completed
     166
     167 - real-time look at measurements has verfied expected operation of experiment (or not)
     168
     169 - a full set of measurements has been collected, for later analysis 
     170
     171 - collected measurments have been saved to storage service, for further analysis
     172
     173 - other exepriment artifacts have been saved to storage service, for further reference
     174 
     175 - slice resources can now be released at any time, without loss of any measurements or artifacts
     176
     177 
     178=== 6)  Analyze experiment results, and decide on next step(s)  ===
     179
     180 - if necessary, retrieve measurement results from storage service
     181 
     182 - analyze measurement results to evaluate experiment results
     183 
     184 - as desired, format to facilitate analysis
     185 
     186 - as desired, visualize using a presentation serivce to facilitate analysis
     187 
     188 - decide whether experiment goals have been met or not
     189 
     190 - decide whether additional experiment runs are required or not, or whether other changes need to be made in experiment before additional runs can be made
     191
     192 - decide whether slice resources can now be released 
     193 
     194 - as appropriate, save analyzed results and/or visualization in storage service
     195
     196
     197Status at completion of step 6):
     198
     199 - current set of experiment measurement results have been analyzed, formatted and/or visaualized
     200 
     201 - given current set of experiment measurement results, have decided whether experiemnt goals have been met or not, whether additional experiment runs are required or not, and whether slice resources can now be released
     202 
     203 - as appropriate, analyzed results and/or visualizations have been saved in storage service
     204
     205 
     206=== 7)  When no longer needed, release all experiment resources  ===
     207
     208 - release all experiment resources as soon as possible, to free up them for use by others, but after being certain that they are not needed for another run of the experiment
     209 
     210 - release slice of GENI resources
     211 
     212 - erase entries in operational services that are no longer needed
     213
     214 - update any software images/packages with changes identifed during experiment
     215
     216 - update software applications, rspecs and/or scripts with changes identifed during experiment
     217
     218Status at completion of step 7):
     219
     220 - all experiment resouerces have been released, so that they can be used by others
     221 
     222 - entries in operational services that are no longer needed, have been erased
     223 
     224 - software images/packages/appications, and rspecs, scritps, etc., have been updated with changes identified during experiment
     225 
     226 
     227=== 8)  Optionally, archive selected measurement and/or experiment artifacts ===
     228 
     229 - within storage service, gather selected components (measurement and/or experiment artifacts) into bundle to form an object to be archived, and move this object into long-term archive service
    100230 
    101 === 5)  Run and orchestrate I&M and experiment services, to complete run of experiment; move selected measurements and other artifacts to storage service  ===
    102 
    103 
    104 5.1)  Initial setup:  start basic host measurements and basic ping active network measurements
    105 
    106  - objectives:
    107    - verify functionality of hosts
    108    - verify topology of slice
    109 
    110  - observe measurements with a real-time presentation service
    111  
    112  - continue measurements throughout duration of the test/tutorial/experiment
    113  
    114  - at completion:
    115    - functionality of hosts and topology of slice has been verified throughout duration of the experiment
    116  
    117  
    118 5.2)  Continuity test:  for a limited time, run iperf active network measurements
    119 
    120  - objectives:
    121    - verify ability of slice to carry traffic expected from experiment
    122 
    123  - observe measurements with a real-time presentation service
    124  
    125  - once satisfactory measurements have been observed, stop continuity test
    126  
    127  - at completion:
    128    - capability of slice to carry traffic expected from experiment has been verified
    129 
    130    
    131 5.3)  Instrument and run experiment: 
    132 
    133  - objectives:
    134    - gather measurements during experiment that allow experiment goals to be met
    135    
    136  - include desired measurement points within hosts and/or experiment services to instrument test/tutorial/experiment
    137  
    138  - begin to run and orchestrate measurement services
    139  
    140  - begin to run and orchestrate experiment services
    141  
    142  - observe measurements with a real-time presentation service, to verify expected operation of experiment
    143  
    144  - collect all measurements for duration of experiment
    145  
    146  - stop experiment services, when this run of the experiment has been completed
    147  
    148  - stop measurement services
    149  
    150  - at completion:
    151    - one run of experiment has been completed
    152    - real-time look at measurements has verfied expected operation of experiment
    153    - a full set of measurements has been collected, for later analysis and presentation
    154    - collected measurments have been transfered to storage service, so that slice resources can be released (if desired)
    155    
    156 5.4)  Store collected measurements and other artifacts from test/tutorial/experiment in storage service
    157 
    158  - at completion:
    159    - collected measurements and other artifacts have been transfered to storage service
    160    - collected measurements and other artifacts are available for later analysis
    161    - slice resources can then be released at any time, without loss of any measurements or artifacts
    162    
    163 === 6)  Analyze and visualize measurement results  ===
    164 
    165  - as necessary, retrieve measurement results from storage service
    166  
    167  - analyze and format results as desired, for visualization with presentation service
    168  
    169  - as appropriate, move analyzed results and/or visualization to storage service
    170  
    171 
    172 === 7)  Release experiment resources  ===
    173 
    174  - as soon as possible, to free up resources for use by others
    175  
    176  - but, after the experimenter is reasonably certain that they do not need to do another run of the experiment
    177 
    178 === 8)  Move selected collected measurements and other artifacts from storage service to long-term archive service ===
    179 
    180231 - identify object to be archived with a peristent identifier (handle)
    181232 
    182  - within storage service, gather components into bundle to form object to be archived, and move this object into long-term archive service
    183  
    184  - include policy for sharing with others  (e.g., share with all others, or share with no one)
    185  
    186  - allow retrieval for further analysis and visualization, or for use as a reference
    187 
     233 - include metadata to identify owner, and facilitate searching
     234 
     235 - allow retrieval by owner for further analysis and visualization, or for use as a reference
     236 
     237 - include policy for sharing with others or not (e.g., share with all others, or share with no one)
     238 
     239 - when object can be shared with others, allow searching from the Internet, and retrieval using the handle identifier
     240