| | 1 | = User Manual for archiving !OnTimeMeasure measurement data along with meta-data into the GENI Measurement Data Archive = |
| | 2 | '''Prasad Calyam, Yingxiao Xu'''[[BR]] |
| | 3 | '''June 28, 2011''' |
| | 4 | |
| | 5 | ''Software License:'' [http://www.geni.net/wp-content/uploads/2009/02/geniprojlic.pdf ''GENI Project License (GPL)''] |
| | 6 | |
| | 7 | == 1. Overview == |
| | 8 | |
| | 9 | This document describes the use cases and features for archiving experiment slice measurement datasets along with meta-data collected by [http://groups.geni.net/geni/wiki/OnTimeMeasure OnTimeMeasure] into the GENI Measurement Data Archive being developed/hosted by [http://www.doregistry.org/ Digital Object Registry(DOR)]. |
| | 10 | |
| | 11 | == 2. Use Case == |
| | 12 | === 2.1 Use Case 1: Archive and Share Subsets of Experiment Results === |
| | 13 | 1. Researcher1 queries "measurement time series with forecasts" from experiment slice and clicks "Transfer" to send the measurement query results to the GENI Measurement Data Archive. |
| | 14 | 2. Anyone who Researcher1 has authorized can access http://mda.doregistry.org/ manually or using API provided by DOR to download the archived results. |
| | 15 | === 2.2 Use Case 2: Archive and Share Entire Experiment Slice Measurement Results === |
| | 16 | 1. Researcher1 clicks "Query Archive" button to instruct !OnTimeMeasure to create a measurement archive file (i.e., {{{mysqldump}}} file). Subsequent click of "Transfer" button sends archive file along with meta-data to GENI Measurement Data Archive. |
| | 17 | 2. Anyone who Researcher1 has authorized can access http://mda.doregistry.org/ manually or using API provided by DOR to download the archive file with entire experiment slice measurement results. |
| | 18 | === 2.3 Use Case 3: Backup and Restore Entire Experiment Slice Measurement Results === |
| | 19 | 1. Researcher1 clicks "Query Archive" button to instruct !OnTimeMeasure to create a measurement archive file (i.e., {{{mysqldump}}} export file). Subsequent click of "Transfer" button sends archive file along with meta-data to GENI Measurement Data Archive. |
| | 20 | 2. If Researcher1 or anyone who Researcher1 has authorized wants to restore state of the experiment results to previous backup state, they can access http://mda.doregistry.org/ manually or using API provided by DOR to download the archive file. Subsequent import of the measurement archive file can be done using {{{mysqldump}}} utility or phpMyAdmin tool. |
| | 21 | |
| | 22 | == 3. Operation with Researcher Web-portal (!OnTimeBeacon) == |
| | 23 | |
| | 24 | Video of the operation of the use cases can be viewed at - http://ontime.oar.net/demo/archive_to_dor.htm |
| | 25 | |
| | 26 | === 3.1 Archive Experiment Query Results to GENI Measurement Data Archive === |
| | 27 | 1. Visit http://ontime.oar.net/OTM_DOR/ |
| | 28 | |
| | 29 | Login with a "demo" username and corresponding password provided to you for testing purposes(Please contact Prasad Calyam (email: pcalyam@osc.edu) for the password). |
| | 30 | |
| | 31 | You can also test by creating your own slice, registering an account in http://ontime.oar.net/OTM_DOR/ and configuring your slice as described in http://groups.geni.net/geni/wiki/OnTime-Install |
| | 32 | |
| | 33 | 2. Click "MEASUREMENT QUERY" on the left pane, select "query options" and "results type", and then click "Submit Query" button. |
| | 34 | |
| | 35 | [[Image(Query.png, 80%)]] |
| | 36 | |
| | 37 | 3. The following UI will appear, you can either click "View Result" link to download the result or click the "Transfer" Button to send query result to GENI Measurement Data Archive. |
| | 38 | |
| | 39 | [[Image(QueryResult.png, 80%)]] |
| | 40 | |
| | 41 | 4. The following UI will appear to confirm the success of your operation. |
| | 42 | |
| | 43 | [[Image(QueryToDORConfirm.png, 80%)]] |
| | 44 | |
| | 45 | === 3.2 Archive Experiment Slice Data along with Meta-Data to GENI Measurement Data Archive === |
| | 46 | 1. Visit http://ontime.oar.net/OTM_DOR/ |
| | 47 | |
| | 48 | Login with a "demo" username and corresponding password provided to you for testing purposes(Please contact Prasad Calyam (email: pcalyam@osc.edu) for the password). |
| | 49 | |
| | 50 | You can also test by creating your own slice, registering an account in http://ontime.oar.net/OTM_DOR/ and configuring your slice as described in http://groups.geni.net/geni/wiki/OnTime-Install |
| | 51 | |
| | 52 | 2. Click "MEASUREMENT QUERY" on the left-pane, then click "Query Archive" button. |
| | 53 | |
| | 54 | [[Image(QueryArchive.png, 80%)]] |
| | 55 | |
| | 56 | 3. The following UI will appear, you can either click "Download" link to download the archive file or click the "Transfer" button to send experiment slice data along with meta-data to GENI Measurement Data Archive. |
| | 57 | [[Image(QueryArchiveResult.png, 80%)]] |
| | 58 | |
| | 59 | 4. The following UI will appear to confirm the success of your operation. |
| | 60 | |
| | 61 | [[Image(QueryArchiveToDORConfirm.png, 80%)]] |
| | 62 | |
| | 63 | === 3.3 Search and Download Archived Data from GENI Measurement Data Archive === |
| | 64 | |
| | 65 | Access http://mda.doregistry.org/ with your browser, input username (i.e., demo) as keyword, you can search and download data along with its meta-data. For more details on the DOR capabilities, please refer to http://www.doregistry.org/ |
| | 66 | [[Image(DORUI.png, 80%)]] |
| | 67 | |
| | 68 | == 4. Operation with Command-line Tool (!OnTimeControl) == |
| | 69 | |
| | 70 | === 4.1 Archive Experiment Query Results to GENI Measurement Data Archive === |
| | 71 | |
| | 72 | Use "query.py" to query the measurement results from experiment slice and automatically download the results into a local folder. In addition to the download, a user can transfer measurement query results to the GENI Measurement Data Archive by using the -k parameter as follows: |
| | 73 | {{{ |
| | 74 | -f from time "YYYY-MM-DD HH:MM:SS" in UTC |
| | 75 | -t to time "YYYY-MM-DD HH:MM:SS" in UTC |
| | 76 | -s source node name |
| | 77 | -d destination node name |
| | 78 | -m metric {RountTripDelay|Throughput|Loss|Jitter} |
| | 79 | -r result type {RawFiles|TimeSeries|TimeSeriesPlusAnomalies|TimeSeriesPlusForecasts} |
| | 80 | -k a key word to identify the measurement data object being archived |
| | 81 | }}} |
| | 82 | |
| | 83 | An example is as follows: |
| | 84 | {{{ |
| | 85 | python query.py -f "2011-07-22 09:00:00" -t "2012-08-22 17:50:00" -s node1 -d node2 -m RoundTripDelay -r TimeSeries -k dordemo |
| | 86 | }}} |
| | 87 | |
| | 88 | [[Image(OTCQuery.png, 60%)]] |
| | 89 | |
| | 90 | |
| | 91 | In this example, two result files 20110722212234_Ping_RoundTripDelay.txt and 20110722212234_Ping_RoundTripDelay.png were created in the folder C:\!OnTimeControl\results. The result files are also accessible from the URL [http://mda.doregistry.org/ "http://mda.doregistry.org/"] by typing in the key word as the search parameter. |
| | 92 | |
| | 93 | |
| | 94 | |
| | 95 | === 4.2 Archive Experiment Slice Data to GENI Measurement Data Archive === |
| | 96 | |
| | 97 | Use "request_measurement_archive.py" to generate the measurement archive of the experiment slice data and automatically download the results into a local folder. In addition to the download, a user can transfer the experiment slice archive to the GENI Measurement Data Archive by using the -k parameter |
| | 98 | |
| | 99 | An example is as follows: |
| | 100 | {{{ |
| | 101 | python request_measurement_archive.py -k dordemo |
| | 102 | }}} |
| | 103 | |
| | 104 | [[Image(OTCArchive.png, 60%)]] |
| | 105 | |
| | 106 | In this example, a file !MeasurementDump.gz was created in the folder C:\!OnTimeControl\results. The result files are also accessible from the URL [http://mda.doregistry.org/ "http://mda.doregistry.org/"] by typing in the key word as the search parameter. |
| | 107 | |
| | 108 | === 4.3 Search and Download Archived Data from GENI Measurement Data Archive === |
| | 109 | |
| | 110 | Access http://mda.doregistry.org/ with your browser, input the key word (i.e. "dordemo" in this example), you can search and download experiment query results or entire experiment slice measurement results. For more details on the DOR capabilities, please refer to http://www.doregistry.org/ |
| | 111 | |
| | 112 | [[Image(OTCDOR.png, 60%)]] |
