TestTutorialExperimentWorkflow: 020413_v1.0_TestTutorialExperiment_Workflow.txt

File 020413_v1.0_TestTutorialExperiment_Workflow.txt, 27.6 KB (added by hmussman@bbn.com, 7 years ago)
Line 
1[[PageOutline]]
2
3== Basic GENI Experiment Workflow ==
4
5Version 1.0, February 4, 2013  [[BR]]
6(after discussion with Vic and Jeanne on 010913)  [[BR]]
7(after discussion with Jeanne on 012513)  [[BR]]
8(after work on Sol r document on 012913)
9
10
11=== 0)  Overview of experiment ===
12
130.1)  Title and abstract
14
150.2)  Goals and strategy for results
16
170.3)  Configuration  (e.g, topology of slice resources for experiment)
18
190.4)  Process  (i.e., flow chart of experiment)
20
21
22
23=== 1)  Design experiment, and find or write necessary software, rspecs and/or scripts ===
24
251.1)  Establish experiment goals and strategy for results
26
271.2)  Design configuration of experiment to meet goals, including topology, desired resources, desired tools and desired experiment application services;  consider previous, or reference, experiments as guides;  capture in a high-level configuration drawing or list
28
291.3)  Map desired resources (i.e., computing and networking resources) to aggregates that are likely to be available
30
311.4)  Map desired tools (i.e., I&M tools) to tools that are available, and identify necessary persistent services.
32
331.5)  Map desired experiment application services to software that is likely to be available, and note gaps.
34
351.6)  Find, or write, software images/packages/applications, rspecs and/or scripts necessary to realize remaining experiment appication services, identifying repositories and/or gathering into storage service
36
371.7)  Outline process for the experiment, and capture in a high-level flowchart
38
391.8)  Optionally, prototype the experiment using "GENI-in-a-box"
40
41Experiment management tools utilized:
42
43 - Gap:  Experiment planning tool, that assists an experimenter in designing an experiment configuration, and putting together a list of desired resources, desired tools (i.e., I&M) and available software.  Part of this tool could query GENI resources to evaluate potential availability.
44 
45 - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
46 
47 - GENI Storage Service client, that allows the experimenter to push/pull experiment artifacts  to/from the GENI Storage service, including rspecs, scripts, measurement data and associated descriptors.
48 
49 - Browser, to search and curate GUI on the GENI Storage service, that the experimenter can access using a browser.
50 
51 - Gap:  Image creation service(s), for target environemnt(s)
52 
53 - Option:  "GENI-in-a-box", to allow experimenter to prototype an experiment.
54 
55
56
57In-slice services utilized: 
58
59 
60Persistent services utilized: 
61
62 - On-line repositories to retrieve/store images/packages/software
63
64 - GENI Storage Service (admin OOB, web, API)
65
66Artifacts utilized:
67
68 - Gap:  Artifacts from previous or reference experiments, used as templates for this experiment
69 
70 - Gap:  Experiment description;  consider text, DB entries and/or scripts;  consider structure in GENI Storage Service
71
72 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
73
74 - Images/packages/software for tools (i.e., GIMI and/or GEMINI I&M tools)
75 
76 - Images/packages/software for experiment application services
77 
78Status at completion:
79
80 - Experiment goals and strategy for results extablished
81 
82 - Experiment configuration identified, and mapped into avaialble resources, etc.
83 
84 - Found or designed and coded software, rspecs and scripts necessary for experiment application services
85
86 - Experiment process outlined
87
88 
89=== 2)  Establish experiment management environment ===
90
912.1)  Establish user workspace environment, as repository for experiment management tools and local copy of some artifacts
92
932.2)  Gather (or create) the necessary certificates, logins, keys and credentials.
94
952.3)  Verify availability of necessary persistent services, and access to those services
96
972.4)  Verify availability of desired resources in aggregates, including nodes and networking resources, by querying aggregates and/or GMOC;  optionally, reserve scarce resources
98
99Experiment management tools utilized:
100
101 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
102 
103 - Scripts to gather (or create) the necessary certificates, logins, keys and credentials.
104
105 - Option:  GENI Clearinghouse Portal, a tool with a graphical interface that allows an experimenter to gather (or create) the necessary certificates, logins, keys and credentials, keep them in a registry, and access them via API for use by other tools.
106 
107  - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
108 
109 - Gap:  Services availability tool, that verifies the availability of persistent services needed for this experiment (i.e., GENI Storage Service or Unified Network Information Service), and access to those services by this experimenter (i.e., they can still access their account on the GENI Storage Service).
110 
111 - Gap:  Resource availability tool, that verifies the availability of resources in aggregates, including nodes and networking resources, by querying aggregates and/or GMOC;  this then allows the experimenter to request resources that are likely to be available. 
112 
113 - Option:  Resource reservation tool, that would allow an experimenter to reserve scarce resources.
114
115 
116In-slice services utilized: 
117 
118 
119Persistent services utilized: 
120
121 - GENI CA
122 
123 - GENI SA
124 
125 - GENI Storage Service (admin OOB, web, API)
126 
127 - If using GIMI I&M tools:  GENI Messaging Service  (API)
128 
129 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
130 
131 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
132 
133 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
134 
135
136Artifacts utilized:
137
138 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
139 
140 - Experimenter usernames/passwords, certificates, keys, credentials
141
142Status at completion:
143
144 - Established user workspace environment, as repository for experiment management tools and local copy of some artifacts
145 
146 - Gathered necessary keys, passwords, certificates and credentials for experimenter
147
148 - Verified availability of necessary persistent services, and access to those services
149
150 - Verified availability of desired resources in aggregates
151 
152 - Optionally, reserved scarce resources
153
154
155
156
157=== 3)  Obtain slice of GENI resources, and then install I&M and experiment software ===
158
1593.1)  Select target aggregates
160
1613.2)  Select desired software images/packages for I&M tools and experiment application services
162
1633.3)  Formulate slice topology for experiment, and build request rspec
164
1653.4)  Acquire slice of resources and load images/packages for I&M and experiment application services
166
1673.5)  Verify expected slice topology and installed images/packages/software for I&M and experiment services, on computing and on networking resources, based on manifest rspec returned from aggregates
168
169 
170 
171Experiment management tools utilized:
172
173 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
174 
175 - Option:  GENI Clearinghouse Portal, a tool with a graphical interface that allows an experimenter to gather (or create) the necessary certificates, logins, keys and credentials, keep them in a registry, and access them via API for use by other tools.
176
177  - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
178 
179 - Browser, to access graphical tools.
180 
181 - FLACK and Flukes tools with graphical web interfaces have been built to allow an experimenter to define and request a topology in InstaGENI and ExoGENI racks, respectively.  With each tool, the desired topology is input using a graphical interface, a request is developed, and then submitted to selected AM(s).  After resources have been assigned, a manifest is returned, and it is used to drive a graphical display.  The FLACK tool can operate with the GENI AM API, but it is based on software technology that will soon be obsolete. The  Flukes tool uses only the FxoGENI SM API.  However, it can be used to setup a topology that spans multiple ExoGENI racks at multiple sites.
182 
183 - For both InstaGENI and ExoGENI racks, experimenter can run scripts in the user workspace, that use the OMNI tool to access GENI AM APIs, request resources using a request rspec, and receive back a manifest rspec.  Given a known-good request rspec, this is repeatable.  However, it is essentially impossible to create a non-trivial request rspec, and hard to make significant edits in a request rspec.  Thus, this approach has limited utility for an experimenter.
184 
185 - Gap:  Combined resource assignment tool, that allows an experimenter to define and request resources in both InstaGENI and/or ExoGENI racks, where the desired topology is input using a graphical interface, a request is developed, and then submitted to selected AM(s) using the GENI AM API.  After resources have been assigned, a manifest is returned, and it is used to drive a graphical display for verification.
186
187 - Gap:  Extensions to the combined assignment tool, that allow an experimenter to request advanced topologies, such as those that span multiple racks at multiple sites, tied together by extensive L2 and/or OF paths.  For example, this tool would automatically provide any necessary L2 or OF “stitching”.
188
189 - Gap:  Extensions to the combined assignment tool, that allow an experimenter to setup and manage opt-in traffic from the local site (e.g., a campus, via wired or wireless access arrangements).
190
191 - Gap:  Extensions to the combined assignment tool that allow an experimenter to modify a topology request, submit it, and then see a confirmation.  The request could be modified by changing the graphical input, or editing the request rspec.
192
193 - Gap:  Topology and installation verification tool, to verify expected slice topology and installed images/packages/software for I&M and experiment services, on computing and on networking resources, based on manifest rspec returned from aggregates and queries of the resources (i.e., ssh to each assigned host).
194
195 
196In-slice services utilized: 
197
198 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
199 
200 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement sotrage and presentation service(s)
201
202 - Experiment application service(s)
203 
204 
205Persistent services utilized: 
206
207 - Aggregate managers
208 
209 - Repositories for images/packages/software
210 
211 - Unified Network Information Service (UNIS) (stores experiment topology, based on manifest rspec)
212
213 - Gap:  UNIS provides web interface to display topology
214
215
216Artifacts utilized:
217
218 - Gap:  List of all desired experiment resources, including services, software, rspecs, scripts;  consider structure in GENI Storage Service
219 
220 - Images/packages/software for I&M tools and experiment application services
221 
222 - Request rspec (build and then submit)
223 
224 - Manifest rspec (received)
225
226
227Status at completion:
228
229 - Obtained slice of GENI resources, with desired topology
230 
231 - Installed images/packages/software for I&M and experiment services on computing and on networking resources
232
233
234=== 4) Configure and initialize I&M and experiment services ===
235
2364.1)  Configure I&M services
237
238 - include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
239 
240 - bind I&M services together (e.g., between each MP and the MC)
241 
242 - initialize I&M services
243
244 - verify that I&M services are ready to run and collect measurement data
245
2464.2)  Configure experiment services
247
248 - bind experiment services together to realize desired topology and functionality
249 
250 - initialize experiment services
251
252 - verify that experiment services are ready to run and collect data, and can be observed by I&M services
253
254 
255Experiment management tools utilized:
256
257 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
258 
259 - Option:  GENI Clearinghouse Portal, a tool with a graphical interface that allows an experimenter to gather (or create) the necessary certificates, logins, keys and credentials, keep them in a registry, and access them via API for use by other tools.
260 
261  - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
262 
263 - Browser, to access tools.
264 
265 - To configure the GEMINI I&M tools, the experimenter includes the desired configuration in the request rspec (p/o Step 3), and then runs an instrumentize script (written in Python) that configures and initializes the tools.
266 
267 - To configure the GIMI I&M tools, the experimenter includes part of the configuration in post-boot scripts (p/o Step 3), then completes the configuration and initialization  using the OMF EC with ruby scripts.
268 
269 - Gap:  Experiment application configuration tool/scripts;  consider using OMF EC and OEDL
270
271 
272In-slice services utilized: 
273
274 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
275 
276 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement storage and presentation service(s)
277
278 - Experiment application service(s)
279 
280 
281Persistent services utilized: 
282
283 - GENI Storage Service (admin OOB, web, API)
284 
285 - If using GIMI I&M tools:  GENI Messaging Service  (API)
286 
287 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
288 
289 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
290 
291 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
292
293 
294Artifacts utilized:
295
296 - Manifest rspec (received)
297
298 
299Status at completion:
300
301 - I&M services have been configured to include desired measurement points within hosts, network resources and/or experiment services to instrument experiment, in a configuration designed to allow experiment goals to be met
302 
303 - I&M services have been bound together and initialized, so that they are ready to run and collect measurement data
304 
305 - Experiment application services have been configured to realize desired topology and functionality
306 
307 - Experiment application services have been bound together and initialized, so that they are ready to run and collect measurement data
308 
309 
310=== 5)  Run and orchestrate I&M and experiment services, to complete run of experiment  ===
311
3125.1)  Inititate basic measurements
313
314 - start basic host measurements to verify functionality of all hosts
315 
316 - start a set of ping active network measurements to verify topology of slice
317
318 - observe measurements with a real-time presentation service
319 
320 - continue measurements throughout duration of the experiment, to verify functionality of hosts and topology of slice
321 
3225.2)  Verify topology and bandwidth/performance of networking resources using active networking measurements (do a "continuity test")
323
324 - configure and start active network measurements (i.e., iperf) in a configuration designed to verify ability of slice to carry traffic expected during experiment
325
326 - observe measurements with a real-time presentation service
327 
328 - once satisfactory measurements have been observed, stop active network measurments
329   
3305.3)  Instrument and run experiment
331
332 - begin to run and orchestrate I&M services
333 
334 - begin to run and orchestrate experiment services
335 
336 - observe measurements with a real-time presentation service, to verify expected operation of experiment
337 
338 - collect all measurements for duration of experiment, and save for further analysis
339 
340 - stop experiment services, when this run of the experiment has been completed
341 
342 - stop I&M services
343   
3445.4)  Save collected measurements and other artifacts
345
346 - save collected measurements and other artifacts from experiment in storage service, for further analysis and reference
347
348 - then, slice resources can then be released at any time, without loss of any measurements or artifacts
349
350
351Experiment management tools utilized:
352
353 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
354 
355 - Option:  GENI Clearinghouse Portal, a tool with a graphical interface that allows an experimenter to gather (or create) the necessary certificates, logins, keys and credentials, keep them in a registry, and access them via API for use by other tools.
356
357 - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
358 
359 - GENI Storage Service client, that allows the experimenter to push/pull experiment artifacts  to/from the GENI Storage service, including rspecs, scripts, measurement data and associated descriptors.
360 
361 - Browser, to access graphical tools
362
363 - To run and orchestrate the GEMINI I&M tools, the experimenter utilizes a browser to interface with the configuration process running in the GEMINI global node, as part of the experimenter’s slice.
364
365 - To run and orchestrate the GIMI I&M tools, the experimenter uses an OMF EC with ruby scripts to communicate with the OMF RCs installed within each of the experimenter’s nodes.
366
367 - Gap:  Experiment application run and orchestration tool/scripts.  Consider using an OMF EC with ruby scripts to communicate with the OMF RCs installed within each of the experimenter’s nodes.
368
369 - Gap:  Combined run and orchestration tool for all I&M and experiment services.  Consider using an OMF EC with ruby scripts to communicate with the OMF RCs installed within each of the experimenter’s nodes.
370
371 - Gap:  Extend combined run and orchestration tool to include advanced topologies, such as those that include extensive L2 and/or OF paths.
372
373 - Gap:  Extend combined run and orchestration tool to include advanced topologies to allow an experimenter to manage opt-in traffic from the local site (e.g., a campus, via wired or wireless access arrangements).
374
375 - To observe measurement results in real time, both the GEMINI and GIMI I&M tools include portals that allow the experimenter to format and  view results using a browser. 
376
377 - After an experiment run has been completed, both the GEMINI and GIMI I&M tools include portals that allow the experimenter to push the measurement data to the GENI Storage service, for later retrieval and analysis.
378
379In-slice services utilized: 
380
381 - If using GIMI I&M tools:  GIMI measurement point services;  optional GIMI measurement collection and presentation service(s)
382 
383 - If using GEMINI I&M tools:  GEMINI measurement point services;  GEMINI measurement sotrage and presentation service(s)
384
385 - Experiment application service(s)
386 
387 
388Persistent services utilized: 
389
390 - GENI Storage Service (admin OOB, web, API)
391 
392 - If using GIMI I&M tools:  GENI Messaging Service  (API)
393 
394 - If using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
395 
396 - If using GEMINI I&M tools:  Unified Network Information Service (UNIS) (web?, API)
397 
398 - If using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
399
400
401
402Artifacts utilized:
403
404 - Ruby scripts
405 
406 - Measurement data files, with associated metadata files
407 
408
409Status at completion:
410
411 - one run of experiment has been completed
412
413 - real-time look at measurements has verfied expected operation of experiment (or not)
414
415 - a full set of measurements has been collected, for later analysis 
416
417 - collected measurments have been saved to storage service, for further analysis
418
419 - other exepriment artifacts have been saved to storage service, for further reference
420 
421 - slice resources can now be released at any time, without loss of any measurements or artifacts
422
423 
424=== 6)  Analyze experiment results, and decide on next step(s)  ===
425
426 - if necessary, retrieve measurement results from storage service, in a standardized format
427 
428 - as desired, format to facilitate analysis
429 
430 - as desired, visualize using a presentation serivce to facilitate analysis
431 
432 - analyze measurement results to evaluate experiment results
433 
434 - decide whether experiment goals have been met or not
435 
436 - decide whether additional experiment runs are required or not, or whether other changes need to be made in experiment before additional runs can be made
437
438 - decide whether slice resources can now be released 
439 
440 - as appropriate, save analyzed results and/or visualization in storage service
441
442
443Experiment management tools utilized:
444
445 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
446 
447  - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
448 
449 - GENI Storage Service client, that allows the experimenter to push/pull experiment artifacts  to/from the GENI Storage service, including rspecs, scripts, measurement data and associated descriptors.
450 
451 - Browser, to access graphical tools
452 
453 - Both the GEMINI and GIMI I&M tools include limited presentation functions in their respective portals.
454
455 - Gap:  Set of analysis and presentation tools, to be run in the user’s workspace, to provide all necessary format conversion, analysis and presentation functions.  Since such a set is completely open-ended, it is desirable to find an established structure for gathering and interfacing a multitude of tools.
456
457 - Option:  a different workspace, better configured for analyzing results
458
459
460In-slice services utilized: 
461
462
463Persistent services utilized: 
464
465 - GENI Storage Service (admin OOB, web, API)
466
467 - Option if using GIMI I&M tools:  GIMI Portal Service (admin OOB, web, API)
468 
469 - Option if using GEMINI I&M tools:  GEMINI Portal Service (admin OOB, web, API)
470
471Artifacts utilized:
472
473 - Measurement data files, with associated metadata files
474
475Status at completion:
476
477 - Current set of experiment measurement results have been analyzed, formatted and/or visualized
478 
479 - given current set of experiment measurement results, have decided whether experiemnt goals have been met or not, whether additional experiment runs are required or not, and whether slice resources can now be released
480 
481 - as appropriate, analyzed results and/or visualizations have been saved in storage service
482
483 
484=== 7)  When no longer needed, release all experiment resources  ===
485
486 - release all experiment resources as soon as possible, to free up them for use by others, but after being certain that they are not needed for another run of the experiment
487 
488 - release slice of GENI resources
489 
490 - erase entries in operational services that are no longer needed
491
492 - update any software images/packages with changes identifed during experiment
493
494 - update software applications, rspecs and/or scripts with changes identifed during experiment
495
496 
497 
498Experiment management tools utilized:
499
500 - User Workspace environment, including Linux with multiple windows, with browsers and command lines
501 
502 
503 - If using InstaGENI rack(s):  FLACK graphical tool
504 
505 - If using ExoGENI rack(s):  Flukes graphical
506 
507 - Gap:  one grpahical tool to use for both InstaGENI and ExoGENI racks
508 
509 - Option:  Use of GENI CH Portal, then link to graphical assignement tool
510
511 - For both InstaGENI and ExoGENI racks, OMNI script-driven tool
512 
513 - Gap:  Topology and installation verification tool(s);  could use features in FLACK;  could push manifest rspec to UNIS, and use a GUI on UNIS to visualize topology;  could verify correct softwar loaded using ssh queries or features currently in GUSH
514
515 
516In-slice services utilized: 
517
518Persistent services utilized: 
519
520Artifacts utilized:
521
522
523Status at completion:
524
525 - all experiment resources have been released, so that they can be used by others
526 
527 - entries in operational services that are no longer needed, have been erased
528 
529 - software images/packages/appications, and rspecs, scritps, etc., have been updated with changes identified during experiment
530 
531 
532=== 8)  Optionally, archive selected measurement and/or experiment artifacts ===
533 
534 - within storage service, gather selected components (measurement and/or experiment artifacts) into bundle to form an object to be archived, and move this object into long-term archive service
535 
536 - identify object to be archived with a peristent identifier (handle)
537 
538 - include metadata to identify owner, and facilitate searching
539 
540 - allow retrieval by owner for further analysis and visualization, or for use as a reference
541 
542 - include policy for sharing with others or not (e.g., share with all others, or share with no one)
543 
544 - when object can be shared with others, allow searching from the Internet, and retrieval using the handle identifier
545
546
547Experiment management tools utilized:
548
549 - Personal Linux user workspace environment for experimenter, with multiple windows, with browsers and command lines
550 
551  - Gap:  Experiment description tool, that allows the experimenter to precisely describe the steps of their experiment, and maintain a record of what was done.  A simple tool might allow the experimenter to record notes in an “online lab notebook”, while a much more complex tool might script much of the experiment.
552 
553 - GENI Storage Service client, that allows the experimenter to push/pull experiment artifacts  to/from the GENI Storage service, including rspecs, scripts, measurement data and associated descriptors.
554 
555 - Browser, to access graphical tools
556
557 - Search and curate GUI on the GENI Storage service, that the experimenter can access using a browser to define an object, and push it to the GENI Archive service, with an assigned persistent identifier (handle).
558
559In-slice services utilized: 
560
561Persistent services utilized: 
562
563 - GENI Storage Service (admin OOB, web, API)
564 
565 - GENI Archive Service (admin OOB, web, API)
566
567
568Artifacts utilized:
569
570 - measurement data files, with metadata files
571 
572 - object containing selected artifacts, with metadata
573 
574Status at completion:
575
576 - object stored in long-term archive service
577
578 - object identified with a peristent identifier (handle)
579 
580 - metadata is included with object to identify owner, and facilitate searching
581 
582 - object may be retrieved by owner for further analysis and visualization, or for use as a reference
583 
584 - when object can be shared with others, it can be searched from the Internet, and retrieved using the handle identifier
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608