= Experiment Measurements = This page captures status and execution details for the Experiment Measurements. For overall status see the [http://groups.geni.net/geni/wiki/GeniNetworkStitchingTestStatus GENI Network Stitching Test Status] page and for test details see the [http://groups.geni.net/geni/wiki/GeniNetworkStitchingTestPlan GENI Network Stitching Test Plan] page. ''Last update: Captured in each section.'' || '''Measurements''' || '''State''' || ''' Tickets''' || '''Comments''' || ||GNS-T1 - Utah InstaGENI to GPO InstaGENI ||[[Color(yellow,Complete)]] || || || ||GNS-T2 - InstaGENI Utah to GPO ExoGENI ||[[Color(yellow,Complete)]] || || || ||GNS-T3 - InstaGENI Utah to GPO ExoGENI to LONI and MAX|| || || || ||GNS-T4 - InstaGENI Utah to GPO ExoGENI to CENIC || || || || ||GNS-T5 - InstaGENI Utah to GPO ExoGENI to NYSERNet || || || || ||GNS-T6 - Utah PG to GPO InstaGENI || || || || ||PG Utah to IG Utah (not originally planned) ||[[Color(yellow,Complete)]] || || || ||IG GPO to PG Utah (not originally planned) ||[[Color(yellow,Complete)]] || || || ||PG UTAH to PG UKY (Not originally planned) ||[[Color(yellow,Complete)]] || || || For a summary of sliver timings see [http://groups.geni.net/geni/wiki/GeniNetworkStitchingTestStatus/Measurements#Sliverset-upandtear-down Sliver set up and tear-down] table. [[BR]] || '''State Legend''' || '''Description''' || || [[Color(green,Pass)]] || Test completed and met all criteria || || [[Color(#98FB98,Pass: most criteria)]] || Test completed and met most criteria. Exceptions documented || || [[Color(red,Fail)]] || Test completed and failed to meet criteria. || || [[Color(yellow,Complete)]] || Test completed but will require re-execution due to expected changes || || [[Color(orange,Blocked)]] || Blocked by ticketed issue(s). || || [[Color(#63B8FF,In Progress)]] || Currently under test. || [[BR]] = Test Execution = '''Assumptions:''' - GPO PG (pgeni.gpolab.bbn.com) credentials are used for tests in this page. - Some end-points exist in multiple test cases, so their measurements will be not be duplicated. '''Notes:''' - Stitcher currently does not support AM API v3, is delivered to run AM API V2. - Capacity '''must''' be specified for scenarios where Utah IG is end-point. == Measurements Goals == The set of GENI Network Stitching (GNS) test topologies selected to validate stitching experiment scenarios are also used to benchmark measurements to characterize various areas including setup timing, tear down timing, timing for an experiment to change data paths from one stitched VLAN to another, end-host to end-host data flow set up timing and delay. Throughput will also be measured with iperf between endpoint hosts using the following methodology: - All results are captured for default bandwidth allocation, unless otherwise stated. - All TCP results are for a 60 second test runs are collected for: * '1 client' scenario using command: 'iperf -c dest_host -t 60 -P 1' * '5 clients' scenario using command: 'iperf -c dest_host -t 60 -P 5' * '10 clients' scenario command: 'iperf -c dest_host -t 60 -P 10' - All TCP measurements to PG Utah use an MTU of 1400, see ticket http://groups.geni.net/geni/ticket/1086 - UDP iperf measurements are run for 10 seconds increments may be used to determine bandwidth supported == Measurements Collected == This sections captures all measurements collected for the Network Stitching test effort. Note that a capacity of 100,000 was requested which is also the default value if not capacity is specified. === GNS-T1 - Topology 1 - Utah InstaGENI to GPO InstaGENI === ||''' Description''' ||'''collected 06/28/13'''||'''collected 10/02/13'''|| || VLAN stitching setup timing || 2 min 24 sec || 3 min 25 sec || || VLAN stitching tear down timing || 1 min 45 sec || 2 min 33 sec || || Data path change from stitched VLAN to another || || || || End-host to end-host data flow set up timing || || || || End-host to end-host delay || 90.6 ms || 81.9 ms || || Throughput between end-hosts (1 Client) || || 1.52 Mbits/sec || || Throughput between end-hosts (5 Clients) || || 5.63 Mbits/sec || || Throughput between end-hosts (10 Clients) || || 10.8 Mbits/sec || || Throughput between end-hosts (1 UDP Client) || || 1.05 Mbits/sec || === GNS-T2 - Topology 2 - InstaGENI Utah to GPO ExoGENI === ||''' Description''' || '''collected 10/02/13 ''' || || VLAN stitching setup timing || 2 min 57 sec || || VLAN stitching tear down timing || 1 min 37 sec || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || 199 ms || || End-host to end-host delay ||92.032/92.598/100.979/1.965 ms|| || Throughput between end-hosts (1 Client) || 90.0 Kbits/sec || || Throughput between end-hosts (5 Clients) || 96.4 Kbits/sec || || Throughput between end-hosts (10 Clients) || 74.6 Kbits/sec || || Throughput between end-hosts (1 UDP Client) || 11.8 Kbits/sec || === GNS-T3 - Topology 3 - InstaGENI Utah to GPO ExoGENI to LONI and MAX === ||''' Description''' || '''Results Collected ''' || || VLAN stitching setup timing || || || VLAN stitching tear down timing || || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || || || End-host to end-host delay || || || Throughput between end-hosts (1 Client) || || || Throughput between end-hosts (5 Clients) || || || Throughput between end-hosts (10 Clients) || || || Throughput between end-hosts (1 UDP Client) || || === GNS-T4 - Topology 4 - InstaGENI Utah to GPO ExoGENI to CENIC === ||''' Description''' || '''Results Collected ''' || || VLAN stitching setup timing || || || VLAN stitching tear down timing || || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || || || End-host to end-host delay || || || Throughput between end-hosts (1 Client) || || || Throughput between end-hosts (5 Clients) || || || Throughput between end-hosts (10 Clients) || || || Throughput between end-hosts (1 UDP Client) || || === GNS-T5 - Topology 5 - InstaGENI Utah to GPO ExoGENI to NYSERNet === ||''' Description''' || '''Results Collected ''' || || VLAN stitching setup timing || || || VLAN stitching tear down timing || || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || || || End-host to end-host delay || || || Throughput between end-hosts (1 Client) || || || Throughput between end-hosts (5 Clients) || || || Throughput between end-hosts (10 Clients) || || || Throughput between end-hosts (1 UDP Client) || || === GNS-T6 - Topology 6 - Utah PG to GPO InstaGENI === ||''' Description''' || '''Results Collected ''' || || VLAN stitching setup timing || || || VLAN stitching tear down timing || || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || || || End-host to end-host delay || || || Throughput between end-hosts (1 Client) || || || Throughput between end-hosts (5 Clients) || || || Throughput between end-hosts (10 Clients) || || || Throughput between end-hosts (1 UDP Client) || || === Utah PG to Utah InstaGENI (not originally planned) === ||''' Description''' ||'''Collected 06/11/13 ''' || || VLAN stitching setup timing || 1 minute 50 seconds || || VLAN stitching tear down timing || 2 minutes 9 seconds || || Data path change from stitched VLAN to another || TDB || || End-host to end-host data flow set up timing || 1.142 ms || || End-host to end-host delay ||0.180/0.256/0.319/0.036 ms (*)|| || TCP Throughput between end-hosts (1 client) || 95.8 Mbits/sec || || TCP Throughput between end-hosts (5 clients) || 96.0 Mbits/sec || || TCP Throughput between end-hosts (10 clients) || 96.0 Mbits/sec || || Throughput between end-hosts (1 UDP Client) || || (*) rtt min/avg/max/mdev === GPO IG to Utah PG (not originally planned) === ||''' Description''' || '''Collected 10/02/13''' || || VLAN stitching setup timing || 2 minutes 56 seconds || || VLAN stitching tear down timing || 2 minutes 17 seconds || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || 164 ms || || End-host to end-host delay ||81.562/81.949/82.337/0.378 ms || || TCP Throughput between end-hosts (1 client) || 1.53 Mbits/sec || || TCP Throughput between end-hosts (5 clients) || 2.66 Mbits/sec || || TCP Throughput between end-hosts (10 clients) || 7.53 Mbits/sec || || Throughput between end-hosts (1 UDP Client) || 839 Kbits/sec || === Utah PG to UKY PG (not originally planned) === ||''' Description''' || '''Collected 10/02/13''' || || VLAN stitching setup timing || 2 minutes 56 seconds || || VLAN stitching tear down timing || 2 minutes 25 seconds || || Data path change from stitched VLAN to another || || || End-host to end-host data flow set up timing || 147 ms || || End-host to end-host delay ||71.391/71.605/72.155/0.419 ms || || TCP Throughput between end-hosts (1 client) || 1.36 Mbits/sec || || TCP Throughput between end-hosts (5 clients) || 5.78 Mbits/sec || || TCP Throughput between end-hosts (10 clients) || 10.8 Mbits/sec || || Throughput between end-hosts (1 UDP Client) || 839 Kbits/sec || == Sliver set-up and tear-down == Collected: 2013 June ||'''End-point Aggregates ''' ||'''Create Sliver'''||'''Delete sliver'''|| || GPO IG <-2links-> Utah IG || 2 min 36 sec || 8 min 14 sec || || GPO IG <-> Utah IG || 2 min 45 sec || 2 min 32 sec || || GPO IG <-> Kentucky PG || 2 min 11 sec || 1 min 52 sec || || GPO IG <-> Utah PG || 2 min 26 sec || 2 min 11 sec || || MAX MyPLC <-> GPO IG || 4 min 00 sec || 1 min 02 sec || || MAX <-> Utah PG || 4 min 59 sec || 1 min 49 sec || || MAX <-> PG KY || 40 min 14 sec (1) || 1 min 07 sec || || MAX <-> PG KY || 5 min 52 sec (2) || 1 min 40 sec || || Utah IG <-> Utah PG <-> GPO IG || 2 min 38 sec || 3 min 10 sec || || Kentucky PG to Utah IG ||18 min 26 sec (3) || 2 min 59 sec || || Kentucky PG to Utah IG || 4 min 05 sec (4) || 2 min 58 sec || || Utah PG <-> Kentucky PG <-> GPO IG <-> Utah PG || 3 min 34 sec ||3 min 03 sec|| ||Utah IG <-> Utah PG <-> Kentucky PG <-> GPO IG <-> Utah IG || || || (1) No VLAN available due to SCS bug, fix was applied as test was running, which allowed completion.[[BR]] (2) Fix was applied and test was re-ran test to collect new timing. [[BR]] (3) No errors or warnings were reported in this run, 10 min to create sliver at emulab. [[BR]] (4) Re-ran test to verify measurements collected in (3). [[BR]]