TestTutorialExperimentWorkflow: 012513_v0.9_TestTutorialExperiment_Workflow.txt

File 012513_v0.9_TestTutorialExperiment_Workflow.txt, 18.0 KB (added by hmussman@bbn.com, 11 years ago)
Line 
1[[PageOutline]]
2
3== Basic GENI Experiment Workflow ==
4
5Version 0.9, January 25, 2013  [[BR]]
6(after discussion with Vic and Jeanne on 010913)  [[BR]]
7(after discussion with Jeanne on 012513)  [[BR]]
8
9
10=== 0)  Overview of experiment ===
11
120.1)  Title and abstract
13
140.2)  Goals and strategy for results
15
160.3)  Configuration  (e.g, topology of slice resources for experiment)
17
180.4)  Process  (i.e., flow chart of experiment)
19
20
21
22=== 1)  Design experiment, and find or write necessary software, rspecs and/or scripts ===
23
241.1)  Establish experiment goals and strategy for results
25
261.2)  Design configuration of experiment to meet goals, including topology, desired resources, desired tools and desired experiment application services;  consider previous, or reference, experiments as guides;  capture in a high-level configuration drawing or list
27
281.3)  Map desired resources (i.e., computing and networking resources) to aggregates that are likely to be available
29
301.4)  Map desired tools (i.e., I&M tools) to tools that are available, and identify necessary persistent services.
31
321.5)  Map desired experiment application services to software that is likely to be available, and note gaps.
33
341.6)  Find, or write, software images/packages/applications, rspecs and/or scripts necessary to realize remaining experiment appication services, identifying repositories and/or gathering into storage service
35
361.7)  Outline process for the experiment, and capture in a high-level flowchart
37
381.8)  Optionally, prototype the experiment using "GENI-in-a-box"
39
40Experiment management tools utilized:
41
42 - Gap:  Workflow description creation tool;  consider on-line entry or script
43 
44 - Option:  "GENI-in-a-box"
45
46In-slice services utilized: 
47
48 
49Persistent services utilized: 
50
51 - Repositories for images/packages/software
52
53 - GENI Storage Service (admin OOB, web, API)
54
55Artifacts utilized:
56
57 - Gap:  Artifacts from previous or reference experiments, used as tmeplates for this experiment
58 
59 - Gap:  Workflow description;  consider text, DB entries and/or scripts;  consider structure in GENI Storage Service
60
61 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
62
63 - Images/packages/software for tools (i.e., GIMI and/or GEMINI I&M tools)
64 
65 - Images/packages/software for experiment application services
66 
67Status at completion:
68
69 - Experiment goals and strategy for results extablished
70 
71 - Experiment configuration identified, and mapped into avaialble resources, etc.
72 
73 - Found or designed and coded software, rspecs and scripts necessary for experiment application services
74
75 - Experiment process outlined
76
77 
78=== 2)  Establish experiment management environment ===
79
802.1)  Establish user workspace environment, as repository for experiment management tools and local copy of some artifacts
81
822.2)  Gather necessary keys, passwords, certificates and credentials
83
842.3)  Verify availability of necessary persistent services, and access to those services
85
862.4)  Verify availability of desired resources in aggregates, including nodes and networking resources, by querying aggregates and/or GMOC;  optionally, reserve scarce resources
87
88Experiment management tools utilized:
89
90 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
91
92 - Option:  use GENI CH Portal to obtain certificates, etc.
93 
94 - Gap:  Persistent services availability verification tool
95 
96 - Gap:  Aggregate resources availability verification tool(s)
97
98 
99In-slice services utilized: 
100 
101 
102Persistent services utilized: 
103
104 - GENI CA
105 
106 - GENI SA
107 
108 - GENI Storage Service (admin OOB, web, API)
109 
110 - If using GIMI I&M tools:  GENI Messaging Service  (API)
111 
112 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
113 
114 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
115 
116 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
117 
118
119Artifacts utilized:
120
121 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
122 
123 - Experimenter usernames/passwords, certificates, keys, credentials
124
125Status at completion:
126
127 - Established user workspace environment, as repository for experiment management tools and local copy of some artifacts
128 
129 - Gathered necessary keys, passwords, certificates and credentials for experimenter
130
131 - Verified availability of necessary persistent services, and access to those services
132
133 - Verified availability of desired resources in aggregates
134 
135 - Optionally, reserved scarce resources
136
137
138
139
140=== 3)  Obtain slice of GENI resources, and then install I&M and experiment software ===
141
1423.1)  Select target aggregates
143
1443.2)  Select desired software images/packages for I&M tools and experiment application services
145
1463.3)  Formulate slice topology for experiment, and build request rspec
147
1483.4)  Acquire slice of resources and load images/packages for I&M and experiment application services
149
1503.5)  Verify expected slice topology and installed images/packages/software for I&M and experiment services, on computing and on networking resources, based on manifest rspec returned from aggregates
151
152 
153 
154Experiment management tools utilized:
155
156 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
157 
158 - If using InstaGENI rack(s):  FLACK graphical tool
159 
160 - If using ExoGENI rack(s):  Flukes graphical
161 
162 - Gap:  one grpahical tool to use for both InstaGENI and ExoGENI racks
163 
164 - Option:  Use of GENI CH Portal, then link to graphical assignement tool
165
166 - For both InstaGENI and ExoGENI racks, OMNI script-driven tool
167 
168 - Gap:  Topology and installation verification tool(s);  could use features in FLACK;  could push manifest rspec to UNIS, and use a GUI on UNIS to visualize topology;  could verify correct software loaded using ssh queries or features currently in GUSH
169
170 
171In-slice services utilized: 
172
173 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
174 
175 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement sotrage and presentation service(s)
176
177 - Experiment application service(s)
178 
179 
180Persistent services utilized: 
181
182 - Repositories for images/packages/software
183 
184 - Unified Network Information Service (UNIS) (stores experiment topology, based on manifest rspec)
185
186 - Gap:  UNIS provides web interface to display topology
187
188
189Artifacts utilized:
190
191 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
192 
193 - Images/packages/software for I&M tools and experiment application services
194 
195 - Request rspec (build and then submit)
196 
197 - Manifest rspec (received)
198
199
200Status at completion:
201
202 - Obtained slice of GENI resources, with desired topology
203 
204 - Installed images/packages/software for I&M and experiment services on computing and on networking resources
205
206
207=== 4) Configure and initialize I&M and experiment services ===
208
2094.1)  Configure I&M services
210
211 - include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
212 
213 - bind I&M services together (e.g., between each MP and the MC)
214 
215 - initialize I&M services
216
217 - verify that I&M services are ready to run and collect measurement data
218
2194.2)  Configure experiment services
220
221 - bind experiment services together to realize desired topology and functionality
222 
223 - initialize experiment services
224
225 - verify that experiment services are ready to run and collect data, and can be observed by I&M services
226
227 
228Experiment management tools utilized:
229
230 - If using GEMINI I&M tools:  include configuration in request rspec (p/o Step 3), then run instrumentize script
231 
232 - If using GIMI I&M tools:  p/o configuration is done in post-boot scripts (p/o Step 3), then completed using OMF EC with ruby scripts
233 
234 - Gap:  Experiment application configuration tool/scripts;  consider using OMF EC and OEDL
235
236 
237In-slice services utilized: 
238
239 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
240 
241 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement sotrage and presentation service(s)
242
243 - Experiment application service(s)
244 
245 
246Persistent services utilized: 
247
248 - GENI Storage Service (admin OOB, web, API)
249 
250 - If using GIMI I&M tools:  GENI Messaging Service  (API)
251 
252 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
253 
254 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
255 
256 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
257
258 
259Artifacts utilized:
260
261 - Manifest rspec (received)
262
263 
264Status at completion:
265
266 - I&M services have been configured to include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
267 
268 - I&M services have been bound together and initialized, so that they are ready to run and collect measurement data
269 
270 - Experiment application services have been configured to realize desired topology and functionality
271 
272 - Experiment application services have been bound together and initialized, so that they are ready to run and collect measurement data
273 
274 
275=== 5)  Run and orchestrate I&M and experiment services, to complete run of experiment  ===
276
2775.1)  Inititate basic measurements
278
279 - start basic host measurements to verify functionality of all hosts
280 
281 - start a set of ping active network measurements to verify topology of slice
282
283 - observe measurements with a real-time presentation service
284 
285 - continue measurements throughout duration of the experiment, to verify functionality of hosts and topology of slice
286 
2875.2)  Verify topology and bandwidth/performance of networking resources using active networking measurements (do a "continuity test")
288
289 - configure and start active network measurements (i.e., iperf) in a configuration designed to verify ability of slice to carry traffic expected during experiment
290
291 - observe measurements with a real-time presentation service
292 
293 - once satisfactory measurements have been observed, stop active network measurments
294   
2955.3)  Instrument and run experiment
296
297 - begin to run and orchestrate I&M services
298 
299 - begin to run and orchestrate experiment services
300 
301 - observe measurements with a real-time presentation service, to verify expected operation of experiment
302 
303 - collect all measurements for duration of experiment, and save for further analysis
304 
305 - stop experiment services, when this run of the experiment has been completed
306 
307 - stop I&M services
308   
3095.4)  Save collected measurements and other artifacts
310
311 - save collected measurements and other artifacts from experiment in storage service, for further analysis and reference
312
313 - then, slice resources can then be released at any time, without loss of any measurements or artifacts
314
315
316Experiment management tools utilized:
317
318 - User workspace, including windows with scripts and browsers
319 
320 - Gap:  experiment metadata creation and annotation service
321 
322 - If using GEMINI I&M tools:  browser;  Gap: consider adding OMF EC with ruby script   
323 
324 - If using GIMI tools:  OMF EC with ruby script
325 
326 - For xxperiment application:  Gap: consider adding OMF EC with ruby script
327
328
329In-slice services utilized: 
330
331 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
332 
333 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement sotrage and presentation service(s)
334
335 - Experiment application service(s)
336 
337 
338Persistent services utilized: 
339
340 - GENI Storage Service (admin OOB, web, API)
341 
342 - If using GIMI I&M tools:  GENI Messaging Service  (API)
343 
344 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
345 
346 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
347 
348 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
349
350
351
352Artifacts utilized:
353
354 - Ruby scripts
355 
356 - Measurement data files, with associated metadata files
357 
358
359Status at completion:
360
361 - one run of experiment has been completed
362
363 - real-time look at measurements has verfied expected operation of experiment (or not)
364
365 - a full set of measurements has been collected, for later analysis 
366
367 - collected measurments have been saved to storage service, for further analysis
368
369 - other exepriment artifacts have been saved to storage service, for further reference
370 
371 - slice resources can now be released at any time, without loss of any measurements or artifacts
372
373 
374=== 6)  Analyze experiment results, and decide on next step(s)  ===
375
376 - if necessary, retrieve measurement results from storage service, in a standardized format
377 
378 - as desired, format to facilitate analysis
379 
380 - as desired, visualize using a presentation serivce to facilitate analysis
381 
382 - analyze measurement results to evaluate experiment results
383 
384 - decide whether experiment goals have been met or not
385 
386 - decide whether additional experiment runs are required or not, or whether other changes need to be made in experiment before additional runs can be made
387
388 - decide whether slice resources can now be released 
389 
390 - as appropriate, save analyzed results and/or visualization in storage service
391
392
393Experiment management tools utilized:
394
395 - User workspace, including windows with scripts and browsers
396 
397 - Option if using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
398 
399 - Option if using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
400
401 - Gap:  presentation tools;  consider r and gnuplot;  others?
402
403 - Gap:  analysis tools;  which?
404 
405 - Option:  a different workspace, better configured for analyzing results
406
407 - Gap:  metadata creation and editing service
408
409 
410Persistent services utilized: 
411
412 - GENI Storage Service (admin OOB, web, API)
413
414
415Artifacts utilized:
416
417 - Measurement data files, with associated metadata files
418
419Status at completion:
420
421 - Current set of experiment measurement results have been analyzed, formatted and/or visualized
422 
423 - given current set of experiment measurement results, have decided whether experiemnt goals have been met or not, whether additional experiment runs are required or not, and whether slice resources can now be released
424 
425 - as appropriate, analyzed results and/or visualizations have been saved in storage service
426
427 
428=== 7)  When no longer needed, release all experiment resources  ===
429
430 - release all experiment resources as soon as possible, to free up them for use by others, but after being certain that they are not needed for another run of the experiment
431 
432 - release slice of GENI resources
433 
434 - erase entries in operational services that are no longer needed
435
436 - update any software images/packages with changes identifed during experiment
437
438 - update software applications, rspecs and/or scripts with changes identifed during experiment
439
440 
441 
442Experiment management tools utilized:
443
444 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
445 
446 
447 - If using InstaGENI rack(s):  FLACK graphical tool
448 
449 - If using ExoGENI rack(s):  Flukes graphical
450 
451 - Gap:  one grpahical tool to use for both InstaGENI and ExoGENI racks
452 
453 - Option:  Use of GENI CH Portal, then link to graphical assignement tool
454
455 - For both InstaGENI and ExoGENI racks, OMNI script-driven tool
456 
457 - Gap:  Topology and installation verification tool(s);  could use features in FLACK;  could push manifest rspec to UNIS, and use a GUI on UNIS to visualize topology;  could verify correct softwar loaded using ssh queries or features currently in GUSH
458
459Persistent services utilized: 
460
461Artifacts utilized:
462
463
464Status at completion:
465
466 - all experiment resources have been released, so that they can be used by others
467 
468 - entries in operational services that are no longer needed, have been erased
469 
470 - software images/packages/appications, and rspecs, scritps, etc., have been updated with changes identified during experiment
471 
472 
473=== 8)  Optionally, archive selected measurement and/or experiment artifacts ===
474 
475 - within storage service, gather selected components (measurement and/or experiment artifacts) into bundle to form an object to be archived, and move this object into long-term archive service
476 
477 - identify object to be archived with a peristent identifier (handle)
478 
479 - include metadata to identify owner, and facilitate searching
480 
481 - allow retrieval by owner for further analysis and visualization, or for use as a reference
482 
483 - include policy for sharing with others or not (e.g., share with all others, or share with no one)
484 
485 - when object can be shared with others, allow searching from the Internet, and retrieval using the handle identifier
486
487
488Experiment management tools utilized:
489
490 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
491
492
493Persistent services utilized: 
494
495 - GENI Storage Service (admin OOB, web, API)
496 
497 - GENI Archive Service (admin OOB, web, API)
498
499
500Artifacts utilized:
501
502 - measurement data files, with metadata files
503 
504 - object containing selected artifacts, with metadata
505 
506Status at completion:
507
508 - object stored in long-term archive service
509
510 - object identified with a peristent identifier (handle)
511 
512 - metadata is included with object to identify owner, and facilitate searching
513 
514 - object may be retrieved by owner for further analysis and visualization, or for use as a reference
515 
516 - when object can be shared with others, it can be searched from the Internet, and retrieved using the handle identifier
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540