TestTutorialExperimentWorkflow: 011813_v0.8_TestTutorialExperiment_Workflow.txt

File 011813_v0.8_TestTutorialExperiment_Workflow.txt, 16.6 KB (added by hmussman@bbn.com, 11 years ago)
Line 
1[[PageOutline]]
2
3== Basic Experiment Workflow ==
4
5Version 0.8, January 18, 2013
6(after review with Vic and Jeanne on 010913)
7
8
9=== 0)  Overview of experiment ===
10
110.1)  Title and abstract
12
130.2)  Goals and strategy for results
14
150.3)  Configuration  (e.g, topology of slice resources for experiment)
16
170.4)  Process  (i.e., flow chart of experiment)
18
19
20
21=== 1)  Design experiment ===
22
231.1)  Establish experiment goals and strategy for results
24
251.2)  Design configuration of experiment to meet goals, including topology, desired resources, desired tools and desired experiment application services;  consider previous, or reference, experiments as guides;  capture in a high-level configuration drawing or list
26
271.3)  Map desired resources (i.e., computing and networking resources) to aggregates that are likely to be available
28
291.4)  Map desired tools (i.e., I&M tools) to tools that are available, and identify necessary persistent services.
30
311.5)  Map desired experiment application services to software that is likely to be available, and note gaps.
32
331.6)  Find, or design and code, software images/packages/applications, rspecs and/or scripts necessary to realize remaining experiment appication services, identifying repositories and/or gathering into storage service
34
351.7)  Outline process for the experiment, and capture in a high-level flowchart
36
371.8)  Optionally, prototype the experiment using "GENI-in-a-box"
38
39Experiment management tools used:
40
41 - Gap:  Workflow description creation tool;  consider on-line entry or script
42 
43 - Option:  "GENI-in-a-box"
44
45 
46Persistent services required: 
47
48 - Repositories for images/packages/software
49
50 - GENI Storage Service (admin OOB, web, API)
51
52Artifacts involved:
53
54 - Gap:  Workflow description;  consider text, DB entries and/or scripts;  consider putting into GENI Storage Service
55
56 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
57
58 - Images/packages/software for tools (i.e., GIMI and/or GEMINI I&M tools)
59 
60 - Images/packages/software for experiment application services
61 
62Status at completion:
63
64 - Experiment goals and strategy for results extablished
65 
66 - Experiment configuration identified, and mapped into avaialble resources, etc.
67 
68 - Found or designed and coded software, rspecs and scripts necessary for experiment application services
69
70 - Experiment process outlined
71
72 
73=== 2)  Establish experiment management environment ===
74
752.1)  Establish user workspace environment, as repository for experiment management tools and local copy of some artifacts
76
772.2)  Gather necessary keys, passwords, certificates and credentials
78
792.3)  Verify availability of necessary persistent services, and access to those services
80
812.4)  Verify availability of desired resources in aggregates, including nodes and networking resources, by querying aggregates and/or GMOC;  optionally, reserve scarce resources
82
83Experiment management tools used:
84
85 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
86
87 - Option:  use GENI CH Portal to obtain certificates, etc.
88 
89 - Gap:  Persistent services availability verification tool
90 
91 - Gap:  Aggregate resources availability verification tool(s)
92 
93
94Persistent services required: 
95
96 - GENI CA (or ?)
97 
98 - GENI SA
99 
100 - GENI Storage Service (admin OOB, web, API)
101 
102 - If using GIMI I&M tools:  GENI Messaging Service  (API)
103 
104 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
105 
106 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
107 
108 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
109 
110
111Artifacts involved:
112
113 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
114 
115 - Experimenter usernames/passwords, certificates, keys, credentials
116
117Status at completion:
118
119 - Established user workspace environment, as repository for experiment management tools and local copy of some artifacts
120 
121 - Gathered necessary keys, passwords, certificates and credentials for experimenter
122
123 - Verified availability of necessary persistent services, and access to those services
124
125 - Verified availability of desired resources in aggregates
126 
127 - Optionally, reserved scarce resources
128
129
130
131
132=== 3)  Obtain slice of GENI resources, and then install I&M and experiment software ===
133
1343.1)  Select target aggregates
135
1363.2)  Select desired software images/packages for I&M tools and experiment application services
137
1383.3)  Formulate slice topology for experiment, and build request rspec
139
1403.4)  Acquire slice of resources and load images/packages for I&M and experiment application services
141
1423.5)  Verify expected slice topology and installed images/packages/software for I&M and experiment services, on computing and on networking resources, based on manifest rspec returned from aggregates
143
144 
145 
146Experiment management tools used:
147
148 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
149 
150 - If using InstaGENI rack(s):  FLACK with OMNI
151 
152 - If using ExoGENI rack(s):  Flukes
153 
154 - Towards one tool to use for both InstaGENI and ExoGENI racks:  FLACK with OMNI
155 
156 - Option:  Use of GENI CH Portal, plus FLACK with OMNI
157
158 - Gap:  Topology and installation verification tool(s);  could use features in FLACK;  could push manifest rspec to UNIS, and use a GUI on UNIS to visualize topology;  could verify correct softwar loaded using ssh queries or features currently in GUSH
159 
160
161Persistent services required: 
162
163 - Repositories for images/packages/software
164 
165 - Unified Network Information Service (UNIS) (stores experiment topology, based on manifest rspec)
166
167 - Gap:  UNIS provides web interface to display topology
168
169
170Artifacts involved:
171
172 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
173 
174 - Images/packages/software for I&M tools and experiment application services
175 
176 - Request rspec (build and then submit)
177 
178 - Manifest rspec (received)
179
180
181Status at completion:
182
183 - Obtained slice of GENI resources, with desired topology
184 
185 - Installed images/packages/software for I&M and experiment services on computing and on networking resources
186
187
188=== 4) Configure and initialize I&M and experiment services ===
189
1904.1)  Configure I&M services
191
192 - include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
193 
194 - bind I&M services together (e.g., between each MP and the MC)
195 
196 - initialize I&M services
197
198 - verify that I&M services are ready to run and collect measurement data
199
2004.2)  Configure experiment services
201
202 - bind experiment services together to realize desired topology and functionality
203 
204 - initialize experiment services
205
206 - verify that experiment services are ready to run and collect data, and can be observed by I&M services
207
208 
209Experiment management tools used:
210
211 - If using GEMINI I&M tools:  include configuration in request rspec (p/o Step 3), then run instrumentize script
212 
213 - I using GIMI I&M tools:  p/o configuration is done in post-boot scripts (p/o Step 3), then completed using OMF EC with ruby scripts
214 
215 - Gap:  Experiment application configuration tool/scripts;  consider using OMF EC and OEDL
216
217
218Persistent services required: 
219
220 - GENI Storage Service (admin OOB, web, API)
221 
222 - If using GIMI I&M tools:  GENI Messaging Service  (API)
223 
224 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
225 
226 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
227 
228 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
229
230Artifacts involved:
231
232 - Manifest rspec (received)
233
234 
235Status at completion:
236
237 - I&M services have been configured to include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
238 
239 - I&M services have been bound together and initialized, so that they are ready to run and collect measurement data
240 
241 - Experiment application services have been configured to realize desired topology and functionality
242 
243 - Experiment application services have been bound together and initialized, so that they are ready to run and collect measurement data
244 
245 
246=== 5)  Run and orchestrate I&M and experiment services, to complete run of experiment  ===
247
2485.1)  Inititate basic measurements
249
250 - start basic host measurements to verify functionality of all hosts
251 
252 - start a set of ping active network measurements to verify topology of slice
253
254 - observe measurements with a real-time presentation service
255 
256 - continue measurements throughout duration of the experiment, to verify functionality of hosts and topology of slice
257 
2585.2)  Verify topology and bandwidth/performance of networking resources using active networking measurements (do a "continuity test")
259
260 - configure and start active network measurements (i.e., iperf) in a configuration designed to verify ability of slice to carry traffic expected during experiment
261
262 - observe measurements with a real-time presentation service
263 
264 - once satisfactory measurements have been observed, stop active network measurments
265   
2665.3)  Instrument and run experiment
267
268 - begin to run and orchestrate I&M services
269 
270 - begin to run and orchestrate experiment services
271 
272 - observe measurements with a real-time presentation service, to verify expected operation of experiment
273 
274 - collect all measurements for duration of experiment, and save for further analysis
275 
276 - stop experiment services, when this run of the experiment has been completed
277 
278 - stop I&M services
279   
2805.4)  Save collected measurements and other artifacts
281
282 - save collected measurements and other artifacts from experiment in storage service, for further analysis and reference
283
284 - then, slice resources can then be released at any time, without loss of any measurements or artifacts
285
286
287Experiment management tools used:
288
289 - User workspace, including windows with scripts and browsers
290 
291 - Gap:  metadata creation and editing service
292 
293 - If using GEMINI I&M tools:  browser;  Gap: consider adding OMF EC with ruby script   
294 
295 - GIMI:  OMF EC with ruby script
296 
297 - Experiment:  Gap: consider adding OMF EC with ruby script
298
299
300Persistent services required: 
301
302 - GENI Storage Service (admin OOB, web, API)
303 
304 - If using GIMI I&M tools:  GENI Messaging Service  (API)
305 
306 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
307 
308 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
309 
310 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
311
312
313
314Artifacts involved:
315
316 - Ruby scripts
317 
318 - Measurement data files, with associated metadata files
319 
320
321Status at completion:
322
323 - one run of experiment has been completed
324
325 - real-time look at measurements has verfied expected operation of experiment (or not)
326
327 - a full set of measurements has been collected, for later analysis 
328
329 - collected measurments have been saved to storage service, for further analysis
330
331 - other exepriment artifacts have been saved to storage service, for further reference
332 
333 - slice resources can now be released at any time, without loss of any measurements or artifacts
334
335 
336=== 6)  Analyze experiment results, and decide on next step(s)  ===
337
338 - if necessary, retrieve measurement results from storage service, in a standardized format
339 
340 - as desired, format to facilitate analysis
341 
342 - as desired, visualize using a presentation serivce to facilitate analysis
343 
344 - analyze measurement results to evaluate experiment results
345 
346 - decide whether experiment goals have been met or not
347 
348 - decide whether additional experiment runs are required or not, or whether other changes need to be made in experiment before additional runs can be made
349
350 - decide whether slice resources can now be released 
351 
352 - as appropriate, save analyzed results and/or visualization in storage service
353
354
355Experiment management tools used:
356
357 - User workspace, including windows with scripts and browsers
358 
359 - Option if using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
360 
361 - Option if using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
362 
363 - Option if using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
364
365 - Gap:  presentation tools;  consider r and gnuplot;  others?
366
367 - Gap:  analysis tools;  which?
368 
369 - Option:  a different workspace, better configured for analyzing results
370
371 - Gap:  metadata creation and editing service
372
373Persistent services required: 
374
375 - GENI Storage Service (admin OOB, web, API)
376
377
378Artifacts involved:
379
380 - Measurement data files, with associated metadata files
381
382Status at completion:
383
384 - Current set of experiment measurement results have been analyzed, formatted and/or visualized
385 
386 - given current set of experiment measurement results, have decided whether experiemnt goals have been met or not, whether additional experiment runs are required or not, and whether slice resources can now be released
387 
388 - as appropriate, analyzed results and/or visualizations have been saved in storage service
389
390 
391=== 7)  When no longer needed, release all experiment resources  ===
392
393 - release all experiment resources as soon as possible, to free up them for use by others, but after being certain that they are not needed for another run of the experiment
394 
395 - release slice of GENI resources
396 
397 - erase entries in operational services that are no longer needed
398
399 - update any software images/packages with changes identifed during experiment
400
401 - update software applications, rspecs and/or scripts with changes identifed during experiment
402
403 
404 
405Experiment management tools used:
406
407 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
408 
409 - If using InstaGENI rack(s):  FLACK with OMNI
410 
411 - If using ExoGENI rack(s):  Flukes
412 
413 - Towards one tool to use for both InstaGENI and ExoGENI racks:  FLACK with OMNI
414 
415 - Option:  Use of GENI CH Portal, plus FLACK with OMNI
416
417 - Gap:  Topology and installation verification tool(s);  could use features in FLACK;  could push manifest rspec to UNIS, and use a GUI on UNIS to visualize topology;  could verify correct softwar loaded using ssh queries or features currently in GUSH
418
419Persistent services required: 
420
421Artifacts involved:
422
423
424Status at completion:
425
426 - all experiment resources have been released, so that they can be used by others
427 
428 - entries in operational services that are no longer needed, have been erased
429 
430 - software images/packages/appications, and rspecs, scritps, etc., have been updated with changes identified during experiment
431 
432 
433=== 8)  Optionally, archive selected measurement and/or experiment artifacts ===
434 
435 - within storage service, gather selected components (measurement and/or experiment artifacts) into bundle to form an object to be archived, and move this object into long-term archive service
436 
437 - identify object to be archived with a peristent identifier (handle)
438 
439 - include metadata to identify owner, and facilitate searching
440 
441 - allow retrieval by owner for further analysis and visualization, or for use as a reference
442 
443 - include policy for sharing with others or not (e.g., share with all others, or share with no one)
444 
445 - when object can be shared with others, allow searching from the Internet, and retrieval using the handle identifier
446
447
448Experiment management tools used:
449
450 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
451
452
453Persistent services required: 
454
455 - GENI Storage Service (admin OOB, web, API)
456 
457 - GENI Archive Service (admin OOB, web, API)
458
459
460Artifacts involved:
461
462 - measurement data files, with metadata files
463 
464 - object containing selected artifacts, with metadata
465 
466Status at completion:
467
468 - object stored in long-term archive service
469
470 - object identified with a peristent identifier (handle)
471 
472 - metadata is included with object to identify owner, and facilitate searching
473 
474 - object may be retrieved by owner for further analysis and visualization, or for use as a reference
475 
476 - when object can be shared with others, it can be searched from the Internet, and retrieved using the handle identifier
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500