Changes between Version 4 and Version 5 of PlasticSlices/BaselineEvaluation


Ignore:
Timestamp:
05/16/11 18:44:40 (9 years ago)
Author:
Josh Smift
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PlasticSlices/BaselineEvaluation

    v4 v5  
    55The GENI Plastic Slices baselines evaluations capture progress throughout the Plastic Slices project. Each planned baseline will capture of the following major areas:
    66
    7         * Environment: System configuration, software versions, and resources for compute, OF, VLAN, and  Monitoring.
    8         * Evaluation details: Detailed description of the Topology, Traffic Profiles and tools used in the evaluation.
     7        * Environment: System configuration, software versions, and resources for compute, OF, VLAN, and  Monitoring.
     8        * Evaluation details: Detailed description of the Topology, Traffic Profiles and tools used in the evaluation.
    99        * Evaluation criteria: Detailed description of the Criteria used to determine success along with any assumption made in defining success.
    10         * Baseline Evaluation Report (BER): Report capturing all results and analysis for each Baseline and an overall cumulative final version.
     10        * Baseline Evaluation Report (BER): Report capturing all results and analysis for each Baseline and an overall cumulative final version.
    1111
    1212In addition to the above areas, the GPO will also actively conduct project level coordination that will include requirements tracking, experiment design, GMOC monitoring data reviews. GPO will also provide campuses support for !OpenFlow resource, MyPLC resources, and campus GMOC monitoring data feeds. Additionally, the GPO will provide and support for a GENI AM API compliant ProtoGENI aggregate with four hosts.
     
    2323
    2424 * Configuration for all compute resources:
    25         - Prerequisite environment changes (ex. ports to be opened in FW).
    26         - software versions
    27         - configuration settings (ports)
    28         - OS versions, hardware plaforms
     25        - Prerequisite environment changes (ex. ports to be opened in FW).
     26        - software versions
     27        - configuration settings (ports)
     28        - OS versions, hardware plaforms
    2929
    3030 * Configuration for all OF and VLAN resources: 
    31         - Firmaware versions
    32         - Hardware platform, device models.
    33         - !FlowVisor and Expedient Controllers (OF)
     31        - Firmaware versions
     32        - Hardware platform, device models.
     33        - !FlowVisor and Expedient Controllers (OF)
    3434
    3535 * Configuration for all Management and Monitoring resources
    36         - MyPLC (OS, SW version, Settings)
    37         - GMOC API version, Data monitoring Host,
    38         - GMOC STOP procedure definition.
    39         - NCSA security plan versions
     36        - MyPLC (OS, SW version, Settings)
     37        - GMOC API version, Data monitoring Host,
     38        - GMOC STOP procedure definition.
     39        - NCSA security plan versions
    4040
    4141== Evaluation details ==
    42         - Definition of each entity in test and of its role.
    43         - Detailed topology of each test.
    44         - Tools used for the evaluation (traffic generation, site monitoring, test scripts?)
    45         - Traffic Profiles (iperf?, traffic type, packet rates, packet sizes, duration, etc)
    46         - Result gathering (collect logs(OF,expedient,MyPLC), collect monitoring statistics, collect traffic generation results, other?)
     42        - Definition of each entity in test and of its role.
     43        - Detailed topology of each test.
     44        - Tools used for the evaluation (traffic generation, site monitoring, test scripts?)
     45        - Traffic Profiles (iperf?, traffic type, packet rates, packet sizes, duration, etc)
     46        - Result gathering (collect logs(OF,expedient,MyPLC), collect monitoring statistics, collect traffic generation results, other?)
    4747
    4848== Evaluation criteria ==
    4949
    5050Each baseline run will be evaluated to determine success, the following will be considered:
    51         - Traffic sent was successfully received.
    52         - All nodes are up and available through out the baseline time frame.
    53         - Logs and statistics are in line with the expected results and no FATAL or CRITICAL failure are found.  Lesser priority issues, such as WARNINGS are acceptable as long as they can be shown no to impact ability to communicate between endpoints.
    54         - Runtime assumptions can be made as long as reviewed and deemed reasonable.  For example, assuming that ?? (need relevant example).
     51        - Traffic sent was successfully received.
     52        - All nodes are up and available through out the baseline time frame.
     53        - Logs and statistics are in line with the expected results and no FATAL or CRITICAL failure are found.  Lesser priority issues, such as WARNINGS are acceptable as long as they can be shown no to impact ability to communicate between endpoints.
     54        - Runtime assumptions can be made as long as reviewed and deemed reasonable.  For example, assuming that ?? (need relevant example).
    5555
    5656== Baseline Evaluation Report ==
     
    9494[PlasticSlices/Experiments#GigaPing GigaPing], using count=140000, and this table:
    9595
    96 || '''client'''                   || '''server address''' ||
    97 || ganel.gpolab.bbn.com           || server=10.42.101.105 ||
    98 || planetlab4.clemson.edu         || server=10.42.101.73  ||
    99 || pl4.myplc.grnoc.iu.edu         || server=10.42.101.91  ||
    100 || of-planet1.stanford.edu        || server=10.42.101.96  ||
    101 || wings-openflow-2.wail.wisc.edu || server=10.42.101.52  ||
     96|| '''client'''                   || '''server'''                   || '''server address''' ||
     97|| ganel.gpolab.bbn.com           || planetlab5.clemson.edu         || server=10.42.101.105 ||
     98|| planetlab4.clemson.edu         || pl5.myplc.grnoc.iu.edu         || server=10.42.101.73  ||
     99|| pl4.myplc.grnoc.iu.edu         || of-planet2.stanford.edu        || server=10.42.101.91  ||
     100|| of-planet1.stanford.edu        || wings-openflow-3.wail.wisc.edu || server=10.42.101.96  ||
     101|| wings-openflow-2.wail.wisc.edu || gardil.gpolab.bbn.com          || server=10.42.101.52  ||
    102102
    103103=== plastic-102 ===
     
    105105[PlasticSlices/Experiments#GigaPing GigaPing], using count=140000, and this table:
    106106
    107 || '''client'''                   || '''server address''' ||
    108 || sardis.gpolab.bbn.com          || server=10.42.102.104 ||
    109 || planetlab5.clemson.edu         || server=10.42.102.72  ||
    110 || pl5.myplc.grnoc.iu.edu         || server=10.42.102.93  ||
    111 || of-planet3.stanford.edu        || server=10.42.102.95  ||
    112 || wings-openflow-3.wail.wisc.edu || server=10.42.102.54  ||
     107|| '''client'''                   || '''server'''                   || '''server address''' ||
     108|| sardis.gpolab.bbn.com          || planetlab4.clemson.edu         || server=10.42.102.104 ||
     109|| planetlab5.clemson.edu         || pl4.myplc.grnoc.iu.edu         || server=10.42.102.72  ||
     110|| pl5.myplc.grnoc.iu.edu         || of-planet4.stanford.edu        || server=10.42.102.93  ||
     111|| of-planet3.stanford.edu        || wings-openflow-2.wail.wisc.edu || server=10.42.102.95  ||
     112|| wings-openflow-3.wail.wisc.edu || bain.gpolab.bbn.com            || server=10.42.102.54  ||
    113113
    114114=== plastic-103 ===
     
    116116[PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using this table:
    117117
    118 || '''client'''                   || '''server address''' ||
    119 || of-planet1.stanford.edu        || server=10.42.103.55  ||
     118|| '''client'''                   || '''server'''                   || '''server address''' ||
     119|| of-planet1.stanford.edu        || navis.gpolab.bbn.com           || server=10.42.103.55  ||
    120120
    121121=== plastic-104 ===
     
    123123[PlasticSlices/Experiments#GigaPerfUDP GigaPerf UDP], using this table:
    124124
    125 || '''client'''                   || '''server address''' ||
    126 || ganel.gpolab.bbn.com           || server=10.42.104.104 ||
     125|| '''client'''                   || '''server'''                   || '''server address''' ||
     126|| ganel.gpolab.bbn.com           || planetlab4.clemson.edu         || server=10.42.104.104 ||
    127127
    128128=== plastic-105 ===
     
    130130[PlasticSlices/Experiments#GigaPerfTCP GigaPerf TCP], using this table:
    131131
    132 || '''client'''                   || '''server address''' ||
    133 || wings-openflow-2.wail.wisc.edu || server=10.42.105.52  ||
     132|| '''client'''                   || '''server'''                   || '''server address''' ||
     133|| wings-openflow-2.wail.wisc.edu || gardil.gpolab.bbn.com          || server=10.42.105.52  ||
    134134
    135135=== plastic-106 ===
     
    137137[PlasticSlices/Experiments#GigaPerfUDP GigaPerf UDP], using this table:
    138138
    139 || '''client'''                   || '''server address''' ||
    140 || sardis.gpolab.bbn.com          || server=10.42.106.96  ||
     139|| '''client'''                   || '''server'''                   || '''server address''' ||
     140|| sardis.gpolab.bbn.com          || wings-openflow-3.wail.wisc.edu || server=10.42.106.96  ||
    141141
    142142=== plastic-107 ===
     
    144144[PlasticSlices/Experiments#GigaWeb GigaWeb], using this table:
    145145
    146 || '''client'''                   || '''server address''' ||
    147 || planetlab4.clemson.edu         || server=10.42.107.73  ||
     146|| '''client'''                   || '''server'''                   || '''server address''' ||
     147|| planetlab4.clemson.edu         || pl5.myplc.grnoc.iu.edu         || server=10.42.107.73  ||
    148148
    149149=== plastic-108 ===
     
    151151[PlasticSlices/Experiments#GigaWeb GigaWeb], using this table:
    152152
    153 || '''client'''                   || '''server address''' ||
    154 || wings-openflow-3.wail.wisc.edu || server=10.42.108.94  ||
     153|| '''client'''                   || '''server'''                   || '''server address''' ||
     154|| wings-openflow-3.wail.wisc.edu || of-planet1.stanford.edu        || server=10.42.108.90  ||
    155155
    156156=== plastic-109 ===
     
    158158[PlasticSlices/Experiments#GigaNetcat GigaNetcat], using this table:
    159159
    160 || '''client'''                   || '''server address''' ||
    161 || pl4.myplc.grnoc.iu.edu         || server=10.42.109.91  ||
     160|| '''client'''                   || '''server'''                   || '''server address''' ||
     161|| pl4.myplc.grnoc.iu.edu         || of-planet2.stanford.edu        || server=10.42.109.91  ||
    162162
    163163=== plastic-109 ===
     
    165165[PlasticSlices/Experiments#GigaNetcat GigaNetcat], using this table:
    166166
    167 || '''client'''                   || '''server address''' ||
    168 || bain.gpolab.bbn.com            || server=10.42.110.72  ||
     167|| '''client'''                   || '''server'''                   || '''server address''' ||
     168|| bain.gpolab.bbn.com            || pl4.myplc.grnoc.iu.edu         || server=10.42.110.72  ||
    169169
    170170== Baseline 2 ==