Changes between Version 8 and Version 9 of PlasticSlices/BaselineEvaluation


Ignore:
Timestamp:
05/18/11 15:02:03 (13 years ago)
Author:
Josh Smift
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • PlasticSlices/BaselineEvaluation

    v8 v9  
    77        * Environment: System configuration, software versions, and resources for compute, OF, VLAN, and  Monitoring.
    88        * Evaluation details: Detailed description of the Topology, Traffic Profiles and tools used in the evaluation.
    9         * Evaluation criteria: Detailed description of the Criteria used to determine success along with any assumption made in defining success. 
     9        * Evaluation criteria: Detailed description of the Criteria used to determine success along with any assumption made in defining success.
    1010        * Baseline Evaluation Report (BER): Report capturing all results and analysis for each Baseline and an overall cumulative final version.
    1111
    12 In addition to the above areas, the GPO will also actively conduct project level coordination that will include requirements tracking, experiment design, GMOC monitoring data reviews. GPO will also provide campuses support for !OpenFlow resource, MyPLC resources, and campus GMOC monitoring data feeds. Additionally, the GPO will provide and support for a GENI AM API compliant ProtoGENI aggregate with four hosts. 
    13 
    14 = Baseline Areas = 
     12In addition to the above areas, the GPO will also actively conduct project level coordination that will include requirements tracking, experiment design, GMOC monitoring data reviews. GPO will also provide campuses support for !OpenFlow resource, MyPLC resources, and campus GMOC monitoring data feeds. Additionally, the GPO will provide and support for a GENI AM API compliant ProtoGENI aggregate with four hosts.
     13
     14= Baseline Areas =
    1515
    1616This section captures the implementation details for all major areas in each of the baselines planned.
    … …  
    1818== Environment ==
    1919
    20 Capturing the definition of the environment used to validate the baselines scenarios  is a crucial part of this activity.  A complete definition will 
     20Capturing the definition of the environment used to validate the baselines scenarios  is a crucial part of this activity.  A complete definition will
    2121be captured to facilitate the repeatability of the baseline scenario in the Quality Assurance step to be performed by Georgia Tech. Environment captures
    22 will detail: 
     22will detail:
    2323
    2424 * Configuration for all compute resources:
    2525        - Prerequisite environment changes (ex. ports to be opened in FW).
    26         - software versions 
    27         - configuration settings (ports) 
     26        - software versions
     27        - configuration settings (ports)
    2828        - OS versions, hardware plaforms
    2929
    30  * Configuration for all OF and VLAN resources: 
     30 * Configuration for all OF and VLAN resources:
    3131        - Firmaware versions
    32         - Hardware platform, device models. 
    33         - !FlowVisor and Expedient Controllers (OF) 
     32        - Hardware platform, device models.
     33        - !FlowVisor and Expedient Controllers (OF)
    3434
    3535 * Configuration for all Management and Monitoring resources
    3636        - MyPLC (OS, SW version, Settings)
    37         - GMOC API version, Data monitoring Host, 
     37        - GMOC API version, Data monitoring Host,
    3838        - GMOC STOP procedure definition.
    3939        - NCSA security plan versions
    4040
    4141== Evaluation details ==
    42         - Definition of each entity in test and of its role. 
     42        - Definition of each entity in test and of its role.
    4343        - Detailed topology of each test.
    4444        - Tools used for the evaluation (traffic generation, site monitoring, test scripts?)
    4545        - Traffic Profiles (iperf?, traffic type, packet rates, packet sizes, duration, etc)
    46         - Result gathering (collect logs(OF,expedient,MyPLC), collect monitoring statistics, collect traffic generation results, other?) 
     46        - Result gathering (collect logs(OF,expedient,MyPLC), collect monitoring statistics, collect traffic generation results, other?)
    4747
    4848== Evaluation criteria ==
    … …  
    5656== Baseline Evaluation Report ==
    5757
    58 As each baseline is completed, a Baseline Evaluation Report is generated that captures the results of the baseline, an analysis of the results, and any impact 
    59 that the findings may have on the current Plastic Slices Plan. 
    60 
    61 Baseline evaluation reports will also capture progress for requirements being tracked as well as progress towards the overall goals. A final version of the BER is generated to captures a cumulative representation of all Plastic Slices baseline assessments. 
     58As each baseline is completed, a Baseline Evaluation Report is generated that captures the results of the baseline, an analysis of the results, and any impact
     59that the findings may have on the current Plastic Slices Plan.
     60
     61Baseline evaluation reports will also capture progress for requirements being tracked as well as progress towards the overall goals. A final version of the BER is generated to captures a cumulative representation of all Plastic Slices baseline assessments.
    6262
    6363Question:  Does it make sense to report by major areas? Ex. OF, Compute resources,Monitoring, etc?
    … …  
    6565= Baseline Detailed Plans =
    6666
    67 This section provides a detailed description for each baseline and their respective Evaluation details (topology, tools, traffic profiles, etc), Evaluation Criteria, 
     67This section provides a detailed description for each baseline and their respective Evaluation details (topology, tools, traffic profiles, etc), Evaluation Criteria,
    6868and reporting.
    6969
    … …  
    9292=== plastic-101 ===
    9393
    94 [PlasticSlices/Experiments#GigaPing GigaPing], using count=140000, and this table of client/server pairs:
     94[PlasticSlices/Experiments#GigaPing GigaPing], using count=120000, and this table of client/server pairs:
    9595
    9696|| '''client'''                   || '''server'''                   || '''server address''' ||
    … …  
    9898|| planetlab4.clemson.edu         || pl5.myplc.grnoc.iu.edu         || server=10.42.101.73  ||
    9999|| pl4.myplc.grnoc.iu.edu         || of-planet2.stanford.edu        || server=10.42.101.91  ||
    100 || of-planet1.stanford.edu        || wings-openflow-3.wail.wisc.edu || server=10.42.101.96  ||
     100|| of-planet1.stanford.edu        || pl02.cs.washington.edu         || server=10.42.101.81  ||
     101|| pl01.cs.washington.edu.edu     || wings-openflow-3.wail.wisc.edu || server=10.42.101.96  ||
    101102|| wings-openflow-2.wail.wisc.edu || gardil.gpolab.bbn.com          || server=10.42.101.52  ||
    102103
    … …  
    105106{{{
    106107server=<ipaddr>
    107 sudo ping -i .001 -s $((1500-8-20)) -c 140000 $server
     108sudo ping -i .001 -s $((1500-8-20)) -c 120000 $server
    108109}}}
    109110
    110111=== plastic-102 ===
    111112
    112 [PlasticSlices/Experiments#GigaPing GigaPing], using count=140000, and this table of client/server pairs:
     113[PlasticSlices/Experiments#GigaPing GigaPing], using count=120000, and this table of client/server pairs:
    113114
    114115|| '''client'''                   || '''server'''                   || '''server address''' ||
    … …  
    116117|| planetlab5.clemson.edu         || pl4.myplc.grnoc.iu.edu         || server=10.42.102.72  ||
    117118|| pl5.myplc.grnoc.iu.edu         || of-planet4.stanford.edu        || server=10.42.102.93  ||
    118 || of-planet3.stanford.edu        || wings-openflow-2.wail.wisc.edu || server=10.42.102.95  ||
     119|| of-planet3.stanford.edu        || pl01.cs.washington.edu         || server=10.42.102.80  ||
     120|| pl02.cs.washinton.edu          || wings-openflow-2.wail.wisc.edu || server=10.42.102.95  ||
    119121|| wings-openflow-3.wail.wisc.edu || bain.gpolab.bbn.com            || server=10.42.102.54  ||
    120122
    … …  
    123125{{{
    124126server=<ipaddr>
    125 sudo ping -i .001 -s $((1500-8-20)) -c 140000 $server
     127sudo ping -i .001 -s $((1500-8-20)) -c 120000 $server
    126128}}}
    127129
    128130=== plastic-103 ===
    129131
    130 [PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using port=5103, size=100000000, and this table of client/server pairs:
     132[PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using port=5103, size=350, and this table of client/server pairs:
    131133
    132134|| '''client'''                   || '''server'''                   || '''server address''' ||
    133135|| of-planet1.stanford.edu        || navis.gpolab.bbn.com           || server=10.42.103.55  ||
     136|| ganel.gpolab.bbn.com           || pl01.cs.washington.edu         || server=10.42.103.80  ||
     137|| pl02.cs.washinton.edu          || of-planet2.stanford.edu        || server=10.42.103.91  ||
    134138
    135139One-time prep to run on each server:
    … …  
    149153
    150154{{{
    151 nice -n 19 iperf -c $server -p 5103 -n 1000M
     155nice -n 19 iperf -c $server -p 5103 -n 350M
    152156}}}
    153157
    … …  
    157161
    158162|| '''client'''                   || '''server'''                   || '''server address''' ||
    159 || planetlab4.clemson.edu         || ganel.gpolab.bbn.com           || server=10.42.104.51 ||
     163|| planetlab4.clemson.edu         || gardil.gpolab.bbn.com          || server=10.42.104.52 ||
    160164
    161165One-time prep to run on each server:
    … …  
    181185=== plastic-105 ===
    182186
    183 [PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using port=5105, size=100000000, and this table of client/server pairs:
    184 
    185 || '''client'''                   || '''server'''                   || '''server address''' ||
    186 || wings-openflow-2.wail.wisc.edu || gardil.gpolab.bbn.com          || server=10.42.105.52  ||
     187[PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using port=5105, size=350, and this table of client/server pairs:
     188
     189|| '''client'''                   || '''server'''                   || '''server address''' ||
     190|| wings-openflow-2.wail.wisc.edu || planetlab5.clemson.edu         || server=10.42.105.105 ||
     191|| planetlab4.clemson.edu         || sardis.gpolab.bbn.com          || server=10.42.105.53  ||
     192|| bain.gpolab.bbn.com            || wings-openflow-3.wail.wisc.edu || server=10.42.105.96  ||
    187193
    188194One-time prep to run on each server:
    … …  
    202208
    203209{{{
    204 nice -n 19 iperf -c $server -p 5103 -n 1000M
     210nice -n 19 iperf -c $server -p 5103 -n 350M
    205211}}}
    206212
    … …  
    210216
    211217|| '''client'''                   || '''server'''                   || '''server address''' ||
    212 || sardis.gpolab.bbn.com          || wings-openflow-3.wail.wisc.edu || server=10.42.106.96  ||
     218|| planetlab5.clemson.edu         || wings-openflow-2.wail.wisc.edu || server=10.42.106.95  ||
    213219
    214220One-time prep to run on each server:
    … …  
    234240=== plastic-107 ===
    235241
    236 [PlasticSlices/Experiments#GigaWeb GigaWeb], using count=118, port=4107, file=substrate.doc, md5sum=d4fcf71833327fbfef98be09deef8bfb, and this table of client/server pairs:
    237 
    238 || '''client'''                   || '''server'''                   || '''server address''' ||
    239 || planetlab4.clemson.edu         || pl5.myplc.grnoc.iu.edu         || server=10.42.107.73  ||
     242[PlasticSlices/Experiments#GigaWeb GigaWeb], using count=40, port=4107, file=substrate.doc, md5sum=d4fcf71833327fbfef98be09deef8bfb, and this table of client/server pairs:
     243
     244|| '''client'''                   || '''server'''                   || '''server address''' ||
     245|| planetlab5.clemson.edu         || pl4.myplc.grnoc.iu.edu         || server=10.42.107.72  ||
     246|| pl5.myplc.grnoc.iu.edu         || pl02.cs.washington.edu         || server=10.42.107.81  ||
     247|| pl01.cs.washington.edu         || planetlab4.clemson.edu         || server=10.42.107.104 ||
    240248
    241249One-time prep to run on each server:
    … …  
    275283mkdir ~/gigaweb
    276284cd ~/gigaweb
    277 for i in {1..118} ; do wget --no-check-certificate https://$server:4107/substrate.doc -O substrate.doc.$i ; done
     285for i in {1..40} ; do wget --no-check-certificate https://$server:4107/substrate.doc -O substrate.doc.$i ; done
    278286}}}
    279287
    … …  
    294302=== plastic-108 ===
    295303
    296 [PlasticSlices/Experiments#GigaWeb GigaWeb], using count=118, port=4108, file=substrate.doc, md5sum=d4fcf71833327fbfef98be09deef8bfb, and this table of client/server pairs:
    297 
    298 || '''client'''                   || '''server'''                   || '''server address''' ||
    299 || wings-openflow-3.wail.wisc.edu || of-planet1.stanford.edu        || server=10.42.108.90  ||
     304[PlasticSlices/Experiments#GigaWeb GigaWeb], using count=40, port=4108, file=substrate.doc, md5sum=d4fcf71833327fbfef98be09deef8bfb, and this table of client/server pairs:
     305
     306|| '''client'''                   || '''server'''                   || '''server address''' ||
     307|| wings-openflow-3.wail.wisc.edu || of-planet3.stanford.edu        || server=10.42.108.92  ||
     308|| of-planet4.stanford.edu        || pl5.myplc.grnoc.iu.edu         || server=10.42.108.73  ||
     309|| pl4.myplc.grnoc.iu.edu         || wings-openflow-2.wail.wisc.edu || server=10.42.108.95  ||
    300310
    301311One-time prep to run on each server:
    … …  
    335345mkdir ~/gigaweb
    336346cd ~/gigaweb
    337 for i in {1..118} ; do wget --no-check-certificate https://$server:4108/substrate.doc -O substrate.doc.$i ; done
     347for i in {1..40} ; do wget --no-check-certificate https://$server:4108/substrate.doc -O substrate.doc.$i ; done
    338348}}}
    339349
    … …  
    354364=== plastic-109 ===
    355365
    356 [PlasticSlices/Experiments#GigaNetcat GigaNetcat], using count=118, port=6109, file=substrate.doc, and this table of client/server pairs:
    357 
    358 || '''client'''                   || '''server'''                   || '''server address''' ||
    359 || pl4.myplc.grnoc.iu.edu         || of-planet2.stanford.edu        || server=10.42.109.91  ||
     366[PlasticSlices/Experiments#GigaNetcat GigaNetcat], using count=20, port=6109, file=substrate.doc, and this table of client/server pairs:
     367
     368|| '''client'''                   || '''server'''                   || '''server address''' ||
     369|| navis.gpolab.bbn.com           || pl5.myplc.grnoc.iu.edu         || server=10.42.109.73  ||
     370|| pl4.myplc.grnoc.iu.edu         || pl02.cs.washington.edu         || server=10.42.109.81  ||
     371|| pl01.cs.washinton.edu          || planetlab5.clemson.edu         || server=10.42.109.105 ||
     372|| planetlab4.clemson.edu         || of-planet3.stanford.edu        || server=10.42.109.92  ||
     373|| of-planet4.stanford.edu        || wings-openflow-3.wail.wisc.edu || server=10.42.109.96  ||
     374|| wings-openflow-2.wail.wisc.edu || ganel.gpolab.bbn.com           || server=10.42.109.51  ||
    360375
    361376Prep to run on each server and client:
    … …  
    370385server=<ipaddr>
    371386cd ~/plastic-slices
    372 for i in {1..118} ; do nc -l $server 6109 < substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; done
     387for i in {1..20} ; do nc -l $server 6109 < substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; done
    373388}}}
    374389
    … …  
    380395mkdir ~/giganetcat
    381396cd ~/giganetcat
    382 for i in {1..118} ; do nc $server 6109 > substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; mv substrate.doc substrate.doc.$i ; done
     397for i in {1..20} ; do nc $server 6109 > substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; mv substrate.doc substrate.doc.$i ; done
    383398}}}
    384399
    … …  
    399414=== plastic-110 ===
    400415
    401 [PlasticSlices/Experiments#GigaNetcat GigaNetcat], using count=118, port=6110, file=substrate.doc, and this table of client/server pairs:
    402 
    403 || '''client'''                   || '''server'''                   || '''server address''' ||
    404 || bain.gpolab.bbn.com            || pl4.myplc.grnoc.iu.edu         || server=10.42.110.72  ||
     416[PlasticSlices/Experiments#GigaNetcat GigaNetcat], using count=30, port=6110, file=substrate.doc, and this table of client/server pairs:
     417
     418|| '''client'''                   || '''server'''                   || '''server address''' ||
     419|| gardil.gpolab.bbn.com          || pl01.cs.washington.edu         || server=10.42.110.80  ||
     420|| pl02.cs.washington.edu         || of-planet1.stanford.edu        || server=10.42.110.90  ||
     421|| of-planet2.stanford.edu        || pl4.myplc.grnoc.iu.edu         || server=10.42.110.72  ||
     422|| pl5.myplc.grnoc.iu.edu         || sardis.gpolab.bbn.com          || server=10.42.110.53  ||
    405423
    406424Prep to run on each server and client:
    … …  
    415433server=<ipaddr>
    416434cd ~/plastic-slices
    417 for i in {1..118} ; do nc -l $server 6110 < substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; done
     435for i in {1..30} ; do nc -l $server 6110 < substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; done
    418436}}}
    419437
    … …  
    425443mkdir ~/giganetcat
    426444cd ~/giganetcat
    427 for i in {1..118} ; do nc $server 6110 > substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; mv substrate.doc substrate.doc.$i ; done
     445for i in {1..30} ; do nc $server 6110 > substrate.doc ; echo "completed transfer #$i" ; md5sum substrate.doc ; mv substrate.doc substrate.doc.$i ; done
    428446}}}
    429447
    … …  
    469487|| Assumptions ||  ||
    470488
    471 Detailed descritpion: 
     489Detailed descritpion:
    472490
    473491Similar to the previous baseline, cause the experiment running in each slice to move at least 1 GB of data per day, but do so repeatedly for 144 hours.
    … …  
    526544
    527545|| Due || 2011-06-20 ||
    528 || Summary Descr || Perform an Emergency Stop test while running ten slices,each moving at least 10 Mb/s continuously, for 144 hours.|| 
     546|| Summary Descr || Perform an Emergency Stop test while running ten slices,each moving at least 10 Mb/s continuously, for 144 hours.||
    529547||Traffic Model || TBD ||
    530548|| Tools ||  TBD ||
    … …  
    546564The purpose of this baseline is to test Emergency Stop procedures.
    547565
    548 == Baseline 8 == 
     566== Baseline 8 ==
    549567
    550568|| Due ||2011-06-20||