wiki:GENIRacksHome/Performance

Version 90 (modified by lnevers@bbn.com, 6 years ago) (diff)

--

GENI Racks Performance

This page captures performance measurements for GENI Racks using OpenFlow and Stitching Scenarios. The goal is to have the following topologies with Raw-PC/Bare Metal dedicated hosts to collect performance data:

  • Site Confirmations test OpenFlow connections over Meso-scale:
    • New IG or EG Site to GPO IG, GPO EG, Utah PG
  • Site Confirmations test for OpenFlow connection over AL2S:
    • New IG or EG Site to GPO IG, GPO EG, Utah PG
  • Site Stitching Confirmation test via stitched VLAN:
    • New IG or EG Site to GPO IG
    • New IG or EG Site to GPO EG Bare Metal

Some additional measurements are collected with Xen VM for IG for comparison. Also ExoGENI OpenStack VMs are used because GENI Network Stitching to Bare Metal nodes is not supported. For details about the performance test criteria see the Test Methodology section.

1.0 Performance Results

All results (meso-scale, AL2S, and stitching) captured on this page are also available in the attached GENI Racks Performance Spreadsheet.

1.1 OpenFlow connections over Meso-scale Connection

Meso-scale Run2 Raw-PC Results

All IG uses raw-pc, and all EG uses OpenStack VM. Results were collected on 2014-01-22.

End points One Client(tcp)5 Clients(tcp)10 Clients(tcp)1 client(udp)Ping rtt min/avg/max/mdev
Clemson IG to GPO IG
Clemson IG to GPO EG
Clemson IG to PG Utah
Wisconsin IG to GPO IG
Wisconsin IG to GPO EG
Wisconsin IG to PG Utah
GATech IG to GPO IG
GATech IG to GPO EG
GATech IG to PG Utah
Northwestern IG to GPO IG
Northwestern IG to GPO EG
Northwestern IG to PG Utah
Utah IG to GPO IG
Utah IG to GPO EG
Utah IG to PG Utah
Stanford IG to GPO IG
Stanford IG to GPO EG
Stanford IG to PG Utah
NYU IG to GPO IG
NYU IG to GPO EG
NYU IG to PG Utah
FIU EG to GPO IG
FIU EG to GPO EG
FIU EG to PG Utah
MAX IG to GPO IG
MAX IG to GPO EG
MAX IG to PG Utah
Cornell IG to GPO IG
Cornell IG to GPO EG
Cornell IG to PG Utah
Missouri IG to GPO IG
Missouri IG to GPO EG
Missouri IG to PG Utah
UKY IG to GPO IG
UKY IG to GPO EG
UKY IG to PG Utah
Illinois IG to GPO IG
Illinois IG to GPO EG
Illinois IG to PG Utah
Kansas IG to GPO IG
Kansas IG to GPO EG
Kansas IG to PG Utah
Princeton IG to GPO IG
Princeton IG to GPO EG
Princeton IG to PG Utah
SOX IG to GPO IG
SOX IG to GPO EG
SOX IG to PG Utah
UtahDDC IG to GPO IG
UtahDDC IG to GPO EG
UtahDDC IG to PG Utah
NYSERNet IG to GPO IG
NYSERNet IG to GPO EG
NYSERNet IG to PG Utah
Kettering IG to GPO IG
Kettering IG to GPO EG
Kettering IG to PG Utah

Meso-scale Run1 Xen Results

All IG uses Xen, and all EG uses OpenStack VM. Results were collected on 2014-01-21.

End points One Client(tcp)5 Clients(tcp)10 Clients(tcp)1 client(udp)Ping rtt min/avg/max/mdev
Clemson IG to GPO IG 5.57 Mbits/sec 27.1 Mbits/sec 54.9 Mbits/sec 101 Mbits/sec 164.824/165.121/169.982/0.836 ms
Clemson IG to GPO EG 5.62 Mbits/sec 28.7 Mbits/sec 56.2 Mbits/sec 94.0 Mbits/sec 164.733/164.892/165.271/0.437 ms
Clemson IG to PG Utah 3.39 Mbits/sec 15.3 Mbits/sec 25.3 Mbits/sec 65.5 Mbits/sec 110.188/113.837/121.941/2.853 ms
Wisconsin IG to GPO IG 4.18 Mbits/sec 19.8 Mbits/sec 38.3 Mbits/sec 100 Mbits/sec 223.678/223.978/225.363/0.704 ms
Wisconsin IG to GPO EG 4.29 Mbits/sec 20.5 Mbits/sec 40.8 Mbits/sec 101 Mbits/sec 223.723/224.018/225.043/0.462 ms
Wisconsin IG to PG Utah 3.44 Mbits/sec 13.2 Mbits/sec 23.2 Mbits/sec 75.0 Mbits/sec 142.347/145.478/153.134/2.625 ms
GATech IG to GPO IG 5.61 Mbits/sec 29.1 Mbits/sec 54.6 Mbits/sec 97.8 Mbits/sec 162.198/162.373/162.841/0.507 ms
GATech IG to GPO EG 5.91 Mbits/sec 30.0 Mbits/sec 59.1 Mbits/sec 101 Mbits/sec 162.202/162.354/163.019/0.361 ms
GATech IG to PG Utah 3.21 Mbits/sec 14.5 Mbits/sec 22.2 Mbits/sec 75.3 Mbits/sec 107.553/111.764/120.555/3.371 ms
Northwestern IG to GPO IG 33.0 Mbits/sec 163 Mbits/sec 337 Mbits/sec 75.0 Mbits/sec 24.491/24.768/28.323/0.700 ms
Northwestern IG to GPO EG 38.8 Mbits/sec 171 Mbits/sec 366 Mbits/sec 75.1 Mbits/sec 24.456/24.572/24.861/0.145 ms
Northwestern IG to PG Utah 2.06 Mbits/sec 9.03 Mbits/sec 16.1 Mbits/sec 75.4 Mbits/sec 244.785/248.565/256.759/2.810 ms
Utah IG to GPO IG X 824 Kbits/sec 2.77 Mbits/sec 3.54 Mbits/sec 76.8 Mbits/sec 268.828/273.533/284.110/3.708
Utah IG to GPO EG X 800 Kbits/sec 4.84 Mbits/sec 4.95 Mbits/sec 76.9 Mbits/sec 268.780/343.239/1888.241/298.675 ms
Utah IG to PG Utah 903 Mbits/sec 907 Mbits/sec 905 Mbits/sec 75.3 Mbits/sec 0.635/0.899/1.175/0.092 ms
Stanford IG to GPO IG 9.82 Mbits/sec 48.2 Mbits/sec 94.1 Mbits/sec 101 Mbits/sec 97.585/97.936/110.542/1.666 ms
Stanford IG to GPO EG 9.78 Mbits/sec 48.2 Mbits/sec 92.4 Mbits/sec 101 Mbits/sec 97.496/97.669/98.273/0.400 ms
Stanford IG to PG Utah 2.84 Mbits/sec 10.3 Mbits/sec 19.3 Mbits/sec 61.6 Mbits/sec 173.427/177.312/185.144/2.801 ms
NYU IG to GPO IG 4.46 Mbits/sec 22.3 Mbits/sec 43.1 Mbits/sec 100 Mbits/sec 180.711/181.282/183.143/0.557 ms
NYU IG to GPO EG 4.72 Mbits/sec 20.8 Mbits/sec 45.3 Mbits/sec 100 Mbits/sec 180.709/181.535/193.131/1.740 ms
NYU IG to PG Utah 11.0 Mbits/sec 53.0 Mbits/sec 62.2 Mbits/sec 75.0 Mbits/sec 88.051/92.170/100.498/3.439 ms
FIU EG to GPO IG
FIU EG to GPO EG
FIU EG to PG Utah
MAX IG to GPO IG 4.57 Mbits/sec 20.6 Mbits/sec 39.8 Mbits/sec 100 Mbits/sec 176.594/177.108/183.511/0.996 ms
MAX IG to GPO EG 4.45 Mbits/sec 21.0 Mbits/sec 42.8 Mbits/sec 101 Mbits/sec 176.571/176.973/178.118/0.484 ms
MAX IG to PG Utah 3.97 Mbits/sec 10.5 Mbits/sec 22.6 Mbits/sec 73.5 Mbits/sec 95.362/98.645/104.097/2.537 ms
Cornell IG to GPO IG 4.67 Mbits/sec 22.0 Mbits/sec 43.2 Mbits/sec 100 Mbits/sec 188.277/188.680/193.724/0.732 ms
Cornell IG to GPO EG 5.18 Mbits/sec 18.5 Mbits/sec 43.2 Mbits/sec 100 Mbits/sec 188.166/188.704/189.952/0.647 ms
Cornell IG to PG Utah 4.78 Mbits/sec 16.8 Mbits/sec 24.2 Mbits/sec 94.3 Mbits/sec 95.588/96.017/98.703/0.461 ms
Missouri IG to GPO IG 4.74 Mbits/sec 23.2 Mbits/sec 45.1 Mbits/sec 101 Mbits/sec 193.801/193.922/194.139/0.332 ms
Missouri IG to GPO EG 4.78 Mbits/sec 21.8 Mbits/sec 45.6 Mbits/sec 101 Mbits/sec 193.780/194.072/203.789/1.270 ms
Missouri IG to PG Utah 4.21 Mbits/sec 13.9 Mbits/sec 19.2 Mbits/sec 72.8 Mbits/sec 138.979/142.715/150.631/3.011 ms
UKY IG to GPO IG 5.21 Mbits/sec 25.6 Mbits/sec 48.1 Mbits/sec 101 Mbits/sec 174.672/174.909/175.817/0.565 ms
UKY IG to GPO EG 5.58 Mbits/sec 21.6 Mbits/sec 49.5 Mbits/sec 97.0 Mbits/sec 174.670/174.922/175.661/0.312 ms
UKY IG to PG Utah 3.21 Mbits/sec 15.7 Mbits/sec 22.9 Mbits/sec 74.6 Mbits/sec 120.079/123.622/130.746/2.660 ms
Illinois IG to GPO IG 4.19 Mbits/sec 18.5 Mbits/sec 39.5 Mbits/sec 100 Mbits/sec 219.745/222.410/342.145/15.703 ms
Illinois IG to GPO EG 4.40 Mbits/sec 21.1 Mbits/sec 41.2 Mbits/sec 101 Mbits/sec 219.794/220.207/221.565/0.635 ms
Illinois IG to PG Utah 6.76 Mbits/sec 37.3 Mbits/sec 63.8 Mbits/sec 67.6 Mbits/sec 127.361/131.403/139.986/3.168 ms
Kansas IG to GPO IG 4.91 Mbits/sec 24.3 Mbits/sec 47.0 Mbits/sec 100 Mbits/sec 191.722/191.870/192.146/0.593 ms
Kansas IG to GPO EG 5.07 Mbits/sec 24.8 Mbits/sec 48.3 Mbits/sec 101 Mbits/sec 191.709/191.866/192.384/0.318 ms
Kansas IG to PG Utah 3.78 Mbits/sec 13.1 Mbits/sec 22.0 Mbits/sec 75.3 Mbits/sec 136.936/141.348/151.848/3.269 ms
Princeton IG to GPO IG 4.96 Mbits/sec 25.4 Mbits/sec 50.0 Mbits/sec 100 Mbits/sec 179.595/180.056/181.664/0.671 ms
Princeton IG to GPO EG 5.21 Mbits/sec 23.5 Mbits/sec 50.1 Mbits/sec 101 Mbits/sec 179.520/179.988/181.158/0.591 ms
Princeton IG to PG Utah 5.09 Mbits/sec 16.3 Mbits/sec 23.4 Mbits/sec 75.3 Mbits/sec 98.388/102.827/111.358/3.322 ms
SOX IG to GPO IG 5.73 Mbits/sec 29.0 Mbits/sec 56.4 Mbits/sec 101 Mbits/sec 162.084/162.432/166.913/0.851 ms
SOX IG to GPO EG 5.57 Mbits/sec 27.9 Mbits/sec 57.3 Mbits/sec 97.4 Mbits/sec 162.077/162.305/163.287/0.554 ms
SOX IG to PG Utah 3.75 Mbits/sec 16.5 Mbits/sec 25.0 Mbits/sec 75.5 Mbits/sec 107.538/110.624/115.835/2.262 ms
UtahDDC IG to GPO IG X 3.58 Mbits/sec 17.4 Mbits/sec 21.7 Mbits/sec 76.2 Mbits/sec 268.764/276.932/438.777/23.790 ms
UtahDDC IG to GPO EG X 2.67 Mbits/sec 10.6 Mbits/sec 20.3 Mbits/sec 75.5 Mbits/sec 268.715/286.235/1095.464/106.287 ms
UtahDDC IG to PG Utah 907 Mbits/sec 907 Mbits/sec 909 Mbits/sec 75.2 Mbits/sec 0.865/1.043/1.346/0.087 ms
NYSERNet IG to GPO IG 4.84 Mbits/sec 22.4 Mbits/sec 41.7 Mbits/sec 100 Mbits/sec 186.686/187.128/189.823/0.521 ms
NYSERNet IG to GPO EG 4.77 Mbits/sec 22.9 Mbits/sec 47.2 Mbits/sec 101 Mbits/sec 186.642/187.133/188.837/0.654 ms
NYSERNet IG to PG Utah 4.03 Mbits/sec 21.1 Mbits/sec 22.9 Mbits/sec 94.8 Mbits/sec 94.093/94.636/100.070/0.839 ms
Kettering IG to GPO IG 2.59 Mbits/sec 8.34 Mbits/sec 16.0 Mbits/sec 74.9 Mbits/sec 31.986/32.093/32.246/0.209 ms
Kettering IG to GPO EG 2.45 Mbits/sec 9.42 Mbits/sec 15.7 Mbits/sec 75.0 Mbits/sec 31.919/32.072/32.223/0.105 ms
Kettering IG to PG Utah 861 Kbits/sec 3.98 Mbits/sec 5.62 Mbits/sec 74.7 Mbits/sec 252.265/255.785/265.095/3.014 ms

1.2 OpenFlow connection over AL2S Connection

End points One Client(tcp)5 Clients(tcp)10 Clients(tcp)1 client(udp)Ping rtt min/avg/max/mdev
Clemson IG to GPO IG
Clemson IG to GPO EG
Clemson IG to PG Utah
Wisconsin IG to GPO IG
Wisconsin IG to GPO EG
Wisconsin IG to PG Utah
GATech IG to GPO IG
GATech IG to GPO EG
GATech IG to PG Utah
Northwestern IG to GPO IG
Northwestern IG to GPO EG
Northwestern IG to PG Utah
Utah IG to GPO IG
Utah IG to GPO EG
Utah IG to PG Utah
Stanford IG to GPO IG
Stanford IG to GPO EG
Stanford IG to PG Utah
NYU IG to GPO IG
NYU IG to GPO EG
NYU IG to PG Utah
FIU EG to GPO IG
FIU EG to GPO EG
FIU EG to PG Utah

1.3 GENI Network Stitching connections

Stitching Results Raw-PC Run1

All IG are raw-pcs and all EG are OpenStack VMs. Results collected 2014-01-10, 2014-01-15, 2014-01-16, 2014-01-22.

End points One Client(tcp)5 Clients(tcp)10 Clients(tcp)1 client(udp)Ping rtt min/avg/max/mdev
Wisconsin IG to GPO IG 96.3 Mbits/sec 99.5 Mbits/sec 101 Mbits/sec 97.9 Mbits/sec 49.455/49.525/50.067/0.253 ms
GPO IG to Wisconsin IG 72.9 Mbits/sec 100 Mbits/sec 100 Mbits/sec 97.7 Mbits/sec 49.405/49.460/49.487/0.260 ms
Wisconsin IG to GPO EG
GPO EG to Wisconsin IG
MAX IG to GPO IG 100 Mbits/sec 101 Mbits/sec 101 Mbits/sec 97.8 Mbits/sec 11.528/11.619/11.828/0.105 ms
GPO IG to MAX IG 100 Mbits/sec 101 Mbits/sec 99.1 Mbits/sec 97.8 Mbits/sec 11.443/11.517/11.812/0.113 ms
MAX IG to GPO EG 714 Kbits/sec(?) 264 Kbits/sec 203 Kbits/sec 26.3 Mbits/sec 11.561/11.661/11.946/0.161 ms
GPO EG to MAX IG
NYSERnet IG to GPO IG 99.6 Mbits/sec 101 Mbits/sec 101 Mbits/sec 96.5 Mbits/sec 10.997/11.078/11.154/0.063 ms
GPO IG to NYSERNet IG 99.7 Mbits/sec 101 Mbits/sec 100 Mbits/sec (99.2 Mbits/sec 11.152/12.469/84.158/9.334 ms
NYSERnet IG to GPO EG 97.4 Kbits/sec(?) 109 Kbits/sec time-out 11.288/11.374/11.814/0.156 ms
GPO EG to NYSERNet IG 688 Kbits/sec (?) 331 Kbits/sec 349 Kbits/sec 22.8 Mbits/sec 11.263/11.388/11.610/0.161 ms
Illinois IG to GPO IG
GPO IG to Illinois IG
Illinois IG to GPO EG 1.23 Mbits/sec (?) 756 Kbits/sec 583 Kbits/sec (no ack) 34.713/39.593/319.990/36.505 ms
GPO EG to Illinois IG 97.4 Kbits/sec(?) 100 Kbits/sec 15.5 Kbits/sec 95.2 Kbits/sec 34.682/34.827/35.224/0.117 ms
Kentucky PG to GPO IG 499 Kbits/sec 2.43 Mbits/sec 4.61 Mbits/sec 92.9 Mbits/sec 35.788/36.491/66.378/3.942 ms
GPO IG to Kentucky IG 571 Kbits/sec 3.06 Mbits/sec 3.06 Mbits/sec 95.1 Mbits/sec 35.793/35.849/36.262/0.201 ms
Kentucky PG to GPO EG
GPO EG to Kentucky PG
Missouri IG to GPO IG 97.7 Mbits/sec 100 Mbits/sec 101 Mbits/sec 98.0 Mbits/sec 45.576/45.657/45.948/0.200 ms
GPO IG to Missouri IG 86.4 Mbits/sec 100 Mbits/sec 100 Mbits/sec 97.8 Mbits/sec 45.575/45.623/45.797/0.076 ms
Missouri IG to GPO EG
GPO EG to Missouri EG
Utah IG to GPO IG 51.8 Mbits/sec 99.4 Mbits/sec 99.8 Mbits/sec (no ack) 92.216/92.482/92.937/0.407 ms
GPO IG to Utah IG 45.9 Mbits/sec 99.3 Mbits/sec 99.9 Mbits/sec (no ack) 92.214/92.525/92.891/0.351 ms
Utah IG to GPO EG 6.54 Mbits/sec (?) 2.91 Mbits/sec 878 Kbits/sec (no ack) 92.350/92.663/93.212/0.443 ms
GPO EG to Utah IG 93.3 Kbits/sec (?) 93.9 Kbits/sec 77.4 Kbits/sec (no ack) 92.388/95.532/262.060/21.682 ms
GPO IG to GPO EG (?)
GPO EG to GPO IG (?)

Notes: (no ack) UDP iperf failed with "did not receive ack" for 100M, 50M and 25M. (?)Poor performance on ExoGENI Nodes,looking at new OS for EG and need to re-run all.

Stitching Results Raw-PC Run0

All IG are raw-pcs and all EG are OpenStack VMs. These results were collected 2013-12-06:

End points One Client(tcp)5 Clients(tcp)10 Clients(tcp)1 client(udp)Ping rtt min/avg/max/mdev
GPO IG to Missouri IG 98.3 Mbits/sec 100 Mbits/sec 100 Mbits/sec 98.0 Mbits/sec 45.583/45.678/46.016/0.260 ms
Missouri IG to GPO IG 99.4 Mbits/sec 100 Mbits/sec 100 Mbits/sec 97.8 Mbits/sec 45.605/46.493/92.008/5.931 ms
GPO EG to Missouri IG 97.2 Kbits/sec 93.2 Kbits/sec 120 Kbits/sec 95.4 Kbits/sec 45.664/45.827/46.500/0.279 ms
Missouri IG to GPO EG 7.30 Mbits/sec 581 Kbits/sec 431 Kbits/sec 26.3 Mbits/sec 45.688/45.809/46.319/0.221 ms
GPO IG to NYSERNet IG 99.8 Mbits/sec 101 Mbits/sec 101 Mbits/sec 98.0 Mbits/sec 11.137/11.218/11.532/0.092 ms
NYSERNet IG to GPO IG 100 Mbits/sec 101 Mbits/sec 101 Mbits/sec 97.6 Mbits/sec 11.191/11.238/11.546/0.097 ms
GPO EG to NYSERNet IG 96.4 Kbits/sec 121 Kbits/sec 75.0 Kbits/sec 96.8 Kbits/sec 11.213/11.418/15.838/0.589 ms
NYSERNet IG to GPO EG 689 Kbits/sec 396 Kbits/sec 171 Kbits/sec 20.9 Mbits/sec 11.200/11.402/12.017/0.161 ms
GPO IG to Wisconsin IG99.3 Mbits/sec 100 Mbits/sec 102 Mbits/sec 98.0 Mbits/sec 36.616/36.733/36.942/0.070 ms
Wisconsin IG to GPO IG95.0 Mbits/sec 100 Mbits/sec 101 Mbits/sec 97.8 Mbits/sec 36.655/36.752/37.063/0.232 ms
GPO EG to Wisconsin IG 97.3 Kbits/sec 99.6 Kbits/sec 88.4 Kbits/sec 95.6 Kbits/sec 36.807/36.967/37.688/0.153 ms
Wisconsin IG to GPO EG 1.38 Mbits/sec 670 Kbits/sec 337 Kbits/sec 23.9 Mbits/sec 36.789/37.015/40.590/0.527 ms
GPO IG to MAX IG 100 Mbits/sec 100 Mbits/sec 100 Mbits/sec 97.9 Mbits/sec 11.558/11.609/11.863/0.056 ms
MAX IG to GPO IG 100 Mbits/sec 101 Mbits/sec 101 Mbits/sec 97.7 Mbits/sec 11.526/11.606/11.946/0.159 ms
GPO EG to MAX IG 97.0 Kbits/sec 118 Kbits/sec 105 Kbits/sec 96.6 Kbits/sec 11.560/11.795/16.349/0.603 ms
MAX IG to GPO EG 719 Kbits/sec 406 Kbits/sec 190 Kbits/sec 27.8 Mbits/sec 11.539/11.716/12.407/0.198 ms

2.0 Miscellaneous Information

2.1 Test Methodology

Test Characteristics:

  • All test hosts are dedicate hosts (bare metal/raw-pc) unless otherwise stated
  • All results are captured for default bandwidth allocation (100Mb/s shaped), unless otherwise stated.
  • All results collected between January 2014

Iperf TCP measurements captured in this page have the following assumptions:

  • All results are for a 60 second test run.
  • For OpenFlow scenarios, traffic is exchanged before each of the measurements below to establish data flows:
    • iperf '1 client' scenario command: 'iperf -c dest_host -t 60'
    • iperf '5 clients' scenario command: 'iperf -c dest_host -t 60 -P 5'
    • iperf '10 clients' scenario command: 'iperf -c dest_host -t 60 -P 10'

Iperf UDP measurements captured in this page have the following assumptions:

  • Each test run is 60 seconds and traffic is exchanged before each of the measurements to establish a data flow.
  • UDP measurements between racks requested a bandwidth of 100 Mbits/sec, if results showed that 100Mb/s was possible, then increased to 1Gb/s.

2.2 Historical Performance Data

Various effort have captured performance for GENI Racks, following is a list of earlier pages that have capture some aspect of performance with OpenVZ, Xen and dedicated hosts. These pages should be considered obsolete:

Attachments (1)

Download all attachments as: .zip