[[PageOutline]] = GENI Racks Performance = This page captures performance measurements for GENI Racks using !OpenFlow and Stitching Scenarios. The following topologies will be used with raw-pc/bare metal dedicated hosts to collect performance data: * Site Confirmations test !OpenFlow connections over Meso-scale: * New IG or EG Site to GPO IG, GPO EG, Utah PG * Site Confirmations test for !OpenFlow connection over AL2S: * New IG or EG Site to GPO IG, GPO EG, Utah PG * Site Stitching Confirmation test via stitched VLAN: * New IG or EG Site to GPO IG * New IG or EG Site to GPO EG Bare Metal For details about the performance test criteria see the [wiki:GENIRacksHome/Performance#TestMethodology Test Methodology] section. = 1.0 Performance Results = All results (meso-scale, AL2S, and stitching) captured on this page are also available in the attached [http://groups.geni.net/geni/attachment/wiki/GENIRacksHome/Performance/GENIRacks-Performance.xlsx GENI Racks Performance] Spreadsheet. == 1.1 !OpenFlow connections over Meso-scale Connection == === Meso-scale Run2 Raw-PC Results === All IG uses raw-pc, and all EG uses !OpenStack VM. Results to be collected. === Meso-scale Run1 Xen Results === All IG uses Xen, and all EG uses !OpenStack VM. Results were collected on 2014-01-21. ||'''End points''' ||'''One Client(tcp)'''||'''5 Clients(tcp)'''||'''10 Clients(tcp)'''||'''1 client(udp)'''||'''Ping rtt min/avg/max/mdev'''|| || Clemson IG to GPO IG || 5.57 Mbits/sec || 27.1 Mbits/sec ||54.9 Mbits/sec ||101 Mbits/sec ||164.824/165.121/169.982/0.836 ms|| || Clemson IG to GPO EG ||5.62 Mbits/sec ||28.7 Mbits/sec ||56.2 Mbits/sec ||94.0 Mbits/sec ||164.733/164.892/165.271/0.437 ms|| || Clemson IG to PG Utah ||3.39 Mbits/sec ||15.3 Mbits/sec ||25.3 Mbits/sec ||65.5 Mbits/sec ||110.188/113.837/121.941/2.853 ms|| || || || || || Wisconsin IG to GPO IG || 4.18 Mbits/sec ||19.8 Mbits/sec ||38.3 Mbits/sec ||100 Mbits/sec ||223.678/223.978/225.363/0.704 ms|| || Wisconsin IG to GPO EG || 4.29 Mbits/sec ||20.5 Mbits/sec ||40.8 Mbits/sec ||101 Mbits/sec ||223.723/224.018/225.043/0.462 ms|| || Wisconsin IG to PG Utah || 3.44 Mbits/sec ||13.2 Mbits/sec ||23.2 Mbits/sec ||75.0 Mbits/sec ||142.347/145.478/153.134/2.625 ms|| || || || GATech IG to GPO IG ||5.61 Mbits/sec ||29.1 Mbits/sec ||54.6 Mbits/sec ||97.8 Mbits/sec ||162.198/162.373/162.841/0.507 ms|| || GATech IG to GPO EG ||5.91 Mbits/sec ||30.0 Mbits/sec ||59.1 Mbits/sec ||101 Mbits/sec ||162.202/162.354/163.019/0.361 ms || || GATech IG to PG Utah ||3.21 Mbits/sec ||14.5 Mbits/sec ||22.2 Mbits/sec ||75.3 Mbits/sec ||107.553/111.764/120.555/3.371 ms|| || || || || || Northwestern IG to GPO IG || 33.0 Mbits/sec ||163 Mbits/sec ||337 Mbits/sec ||75.0 Mbits/sec ||24.491/24.768/28.323/0.700 ms || || Northwestern IG to GPO EG || 38.8 Mbits/sec ||171 Mbits/sec ||366 Mbits/sec ||75.1 Mbits/sec ||24.456/24.572/24.861/0.145 ms || || Northwestern IG to PG Utah|| 2.06 Mbits/sec ||9.03 Mbits/sec ||16.1 Mbits/sec ||75.4 Mbits/sec ||244.785/248.565/256.759/2.810 ms || || || || || || Utah IG to GPO IG X ||824 Kbits/sec ||2.77 Mbits/sec || 3.54 Mbits/sec || no ack ||268.828/273.533/284.110/3.708|| || Utah IG to GPO EG X ||800 Kbits/sec ||4.84 Mbits/sec || 4.95 Mbits/sec ||76.9 Mbits/sec ||268.780/343.239/1888.241/298.675 ms|| || Utah IG to PG Utah ||903 Mbits/sec ||907 Mbits/sec ||905 Mbits/sec || 75.3 Mbits/sec || 0.635/0.899/1.175/0.092 ms || || || || || || Stanford IG to GPO IG ||9.82 Mbits/sec ||48.2 Mbits/sec ||94.1 Mbits/sec || 101 Mbits/sec ||97.585/97.936/110.542/1.666 ms || || Stanford IG to GPO EG ||9.78 Mbits/sec ||48.2 Mbits/sec ||92.4 Mbits/sec ||101 Mbits/sec ||97.496/97.669/98.273/0.400 ms || || Stanford IG to PG Utah ||2.84 Mbits/sec ||10.3 Mbits/sec ||19.3 Mbits/sec || 61.6 Mbits/sec ||173.427/177.312/185.144/2.801 ms|| || || || || || NYU IG to GPO IG || 4.46 Mbits/sec ||22.3 Mbits/sec ||43.1 Mbits/sec ||100 Mbits/sec ||180.711/181.282/183.143/0.557 ms|| || NYU IG to GPO EG || 4.72 Mbits/sec ||20.8 Mbits/sec ||45.3 Mbits/sec ||100 Mbits/sec ||180.709/181.535/193.131/1.740 ms|| || NYU IG to PG Utah || 11.0 Mbits/sec ||53.0 Mbits/sec ||62.2 Mbits/sec ||75.0 Mbits/sec ||88.051/92.170/100.498/3.439 ms|| || || || || || FIU EG to GPO IG || || || || || || || FIU EG to GPO EG || || || || || || || FIU EG to PG Utah || || || || || || || || || || || MAX IG to GPO IG ||4.57 Mbits/sec ||20.6 Mbits/sec ||39.8 Mbits/sec ||100 Mbits/sec ||176.594/177.108/183.511/0.996 ms|| || MAX IG to GPO EG ||4.45 Mbits/sec ||21.0 Mbits/sec ||42.8 Mbits/sec ||101 Mbits/sec ||176.571/176.973/178.118/0.484 ms || || MAX IG to PG Utah ||3.97 Mbits/sec ||10.5 Mbits/sec ||22.6 Mbits/sec ||73.5 Mbits/sec ||95.362/98.645/104.097/2.537 ms|| || || || || || Cornell IG to GPO IG ||4.67 Mbits/sec ||22.0 Mbits/sec ||43.2 Mbits/sec ||100 Mbits/sec ||188.277/188.680/193.724/0.732 ms|| || Cornell IG to GPO EG ||5.18 Mbits/sec ||18.5 Mbits/sec ||43.2 Mbits/sec ||100 Mbits/sec ||188.166/188.704/189.952/0.647 ms|| || Cornell IG to PG Utah ||4.78 Mbits/sec ||16.8 Mbits/sec ||24.2 Mbits/sec ||94.3 Mbits/sec ||95.588/96.017/98.703/0.461 ms || || || || Missouri IG to GPO IG ||4.74 Mbits/sec ||23.2 Mbits/sec ||45.1 Mbits/sec ||101 Mbits/sec ||193.801/193.922/194.139/0.332 ms|| || Missouri IG to GPO EG ||4.78 Mbits/sec ||21.8 Mbits/sec ||45.6 Mbits/sec ||101 Mbits/sec ||193.780/194.072/203.789/1.270 ms|| || Missouri IG to PG Utah ||4.21 Mbits/sec ||13.9 Mbits/sec ||19.2 Mbits/sec ||72.8 Mbits/sec ||138.979/142.715/150.631/3.011 ms|| || || || || || UKY IG to GPO IG ||5.21 Mbits/sec ||25.6 Mbits/sec || 48.1 Mbits/sec ||101 Mbits/sec ||174.672/174.909/175.817/0.565 ms|| || UKY IG to GPO EG ||5.58 Mbits/sec ||21.6 Mbits/sec || 49.5 Mbits/sec ||97.0 Mbits/sec ||174.670/174.922/175.661/0.312 ms|| || UKY IG to PG Utah ||3.21 Mbits/sec ||15.7 Mbits/sec || 22.9 Mbits/sec ||74.6 Mbits/sec ||120.079/123.622/130.746/2.660 ms|| || || || || || Illinois IG to GPO IG ||4.19 Mbits/sec ||18.5 Mbits/sec ||39.5 Mbits/sec ||100 Mbits/sec ||219.745/222.410/342.145/15.703 ms|| || Illinois IG to GPO EG ||4.40 Mbits/sec ||21.1 Mbits/sec ||41.2 Mbits/sec ||101 Mbits/sec ||219.794/220.207/221.565/0.635 ms|| || Illinois IG to PG Utah ||6.76 Mbits/sec ||37.3 Mbits/sec ||63.8 Mbits/sec ||67.6 Mbits/sec ||127.361/131.403/139.986/3.168 ms|| || || || || || Kansas IG to GPO IG ||4.91 Mbits/sec ||24.3 Mbits/sec ||47.0 Mbits/sec ||100 Mbits/sec ||191.722/191.870/192.146/0.593 ms|| || Kansas IG to GPO EG ||5.07 Mbits/sec ||24.8 Mbits/sec ||48.3 Mbits/sec ||101 Mbits/sec ||191.709/191.866/192.384/0.318 ms|| || Kansas IG to PG Utah ||3.78 Mbits/sec ||13.1 Mbits/sec ||22.0 Mbits/sec ||75.3 Mbits/sec ||136.936/141.348/151.848/3.269 ms|| || || || || || Princeton IG to GPO IG ||4.96 Mbits/sec ||25.4 Mbits/sec ||50.0 Mbits/sec ||100 Mbits/sec ||179.595/180.056/181.664/0.671 ms|| || Princeton IG to GPO EG ||5.21 Mbits/sec ||23.5 Mbits/sec ||50.1 Mbits/sec ||101 Mbits/sec ||179.520/179.988/181.158/0.591 ms|| || Princeton IG to PG Utah ||5.09 Mbits/sec ||16.3 Mbits/sec ||23.4 Mbits/sec ||75.3 Mbits/sec ||98.388/102.827/111.358/3.322 ms || || || || || || SOX IG to GPO IG ||5.73 Mbits/sec ||29.0 Mbits/sec ||56.4 Mbits/sec ||101 Mbits/sec ||162.084/162.432/166.913/0.851 ms|| || SOX IG to GPO EG ||5.57 Mbits/sec ||27.9 Mbits/sec ||57.3 Mbits/sec ||97.4 Mbits/sec ||162.077/162.305/163.287/0.554 ms|| || SOX IG to PG Utah ||3.75 Mbits/sec ||16.5 Mbits/sec ||25.0 Mbits/sec ||75.5 Mbits/sec ||107.538/110.624/115.835/2.262 ms|| || || || UtahDDC IG to GPO IG || || || || || || || UtahDDC IG to GPO EG || || || || || || || UtahDDC IG to PG Utah || || || || || || || || || || || NYSERNet IG to GPO IG ||4.84 Mbits/sec ||22.4 Mbits/sec ||41.7 Mbits/sec ||100 Mbits/sec ||186.686/187.128/189.823/0.521 ms|| || NYSERNet IG to GPO EG ||4.77 Mbits/sec ||22.9 Mbits/sec ||47.2 Mbits/sec ||101 Mbits/sec ||186.642/187.133/188.837/0.654 ms|| || NYSERNet IG to PG Utah ||4.03 Mbits/sec ||21.1 Mbits/sec ||22.9 Mbits/sec ||94.8 Mbits/sec ||94.093/94.636/100.070/0.839 ms|| || || || || || Kettering IG to GPO IG ||2.59 Mbits/sec ||8.34 Mbits/sec ||16.0 Mbits/sec ||74.9 Mbits/sec ||31.986/32.093/32.246/0.209 ms || || Kettering IG to GPO EG ||2.45 Mbits/sec ||9.42 Mbits/sec ||15.7 Mbits/sec ||75.0 Mbits/sec ||31.919/32.072/32.223/0.105 ms || || Kettering IG to PG Utah ||861 Kbits/sec ||3.98 Mbits/sec ||5.62 Mbits/sec ||74.7 Mbits/sec ||252.265/255.785/265.095/3.014 ms || || || == 1.2 !OpenFlow connection over AL2S Connection == ||'''End points''' ||'''One Client(tcp)'''||'''5 Clients(tcp)'''||'''10 Clients(tcp)'''||'''1 client(udp)'''||'''Ping rtt min/avg/max/mdev''' || || Clemson IG to GPO IG || || || || || || || Clemson IG to GPO EG || || || || || || || Clemson IG to PG Utah || || || || || || || || || Wisconsin IG to GPO IG || || || || || || || Wisconsin IG to GPO EG || || || || || || || Wisconsin IG to PG Utah || || || || || || || || || GATech IG to GPO IG || || || || || || || GATech IG to GPO EG || || || || || || || GATech IG to PG Utah || || || || || || || || || Northwestern IG to GPO IG || || || || || || || Northwestern IG to GPO EG || || || || || || || Northwestern IG to PG Utah|| || || || || || || || || Utah IG to GPO IG || || || || || || || Utah IG to GPO EG || || || || || || || Utah IG to PG Utah || || || || || || || || || Stanford IG to GPO IG || || || || || || || Stanford IG to GPO EG || || || || || || || Stanford IG to PG Utah || || || || || || || || || NYU IG to GPO IG || || || || || || || NYU IG to GPO EG || || || || || || || NYU IG to PG Utah || || || || || || || || || FIU EG to GPO IG || || || || || || || FIU EG to GPO EG || || || || || || || FIU EG to PG Utah || || || || || || || || == 1.3 GENI Network Stitching connections == === Stitching Results Raw-pc Run1 === All IG are raw-pcs and all EG are !OpenStack VMs. Results collected 2014-01-10, 2014-01-15, 2014-01-16: ||'''End points''' ||'''One Client(tcp)'''||'''5 Clients(tcp)'''||'''10 Clients(tcp)'''||'''1 client(udp)'''||'''Ping rtt min/avg/max/mdev'''|| || Wisconsin IG to GPO IG ||96.3 Mbits/sec || 99.5 Mbits/sec ||101 Mbits/sec ||97.9 Mbits/sec ||49.455/49.525/50.067/0.253 ms || || GPO IG to Wisconsin IG || || || || || || || Wisconsin IG to GPO EG || || || || || || || GPO EG to Wisconsin IG || || || || || || || || || MAX IG to GPO IG ||100 Mbits/sec ||101 Mbits/sec || 101 Mbits/sec || 97.8 Mbits/sec ||11.528/11.619/11.828/0.105 ms || || GPO IG to MAX IG ||100 Mbits/sec ||101 Mbits/sec ||99.1 Mbits/sec || 97.8 Mbits/sec ||11.443/11.517/11.812/0.113 ms || || MAX IG to GPO EG ||714 Kbits/sec(?) ||264 Kbits/sec || 203 Kbits/sec || 26.3 Mbits/sec ||11.561/11.661/11.946/0.161 ms || || GPO EG to MAX IG || || || || || || || || || NYSERnet IG to GPO IG || 99.6 Mbits/sec ||101 Mbits/sec || 101 Mbits/sec || (no ack) ||36.045/248.109/269.347/41.979 ms || || GPO IG to NYSERNet IG || 99.7 Mbits/sec ||101 Mbits/sec || 100 Mbits/sec || (no ack) ||11.152/12.469/84.158/9.334 ms || || NYSERnet IG to GPO EG || 97.4 Kbits/sec ||109 Kbits/sec || time-out || ||11.288/11.374/11.814/0.156 ms || || GPO EG to NYSERNet IG || 688 Kbits/sec (?) ||331 Kbits/sec || 349 Kbits/sec || 22.8 Mbits/sec ||11.263/11.388/11.610/0.161 ms || || || || Illinois IG to GPO IG || || || || || || || GPO IG to Illinois IG || || || || || || || Illinois IG to GPO EG ||1.23 Mbits/sec ||756 Kbits/sec ||583 Kbits/sec || (no ack) ||34.713/39.593/319.990/36.505 ms|| || GPO EG to Illinois IG ||97.4 Kbits/sec(?) ||100 Kbits/sec ||15.5 Kbits/sec || 95.2 Kbits/sec ||34.682/34.827/35.224/0.117 ms|| || || || Kentucky PG to GPO IG || 499 Kbits/sec ||2.43 Mbits/sec || 4.61 Mbits/sec || 92.9 Mbits/sec ||35.788/36.491/66.378/3.942 ms || || GPO IG to Kentucky IG || || || || || || || Kentucky PG to GPO EG || || || || || || || GPO EG to Kentucky PG || || || || || || || || || Missouri IG to GPO IG || || || || || || || GPO IG to Missouri IG || || || || || || || Missouri IG to GPO EG || || || || || || || GPO EG to Missouri EG || || || || || || || || || Utah IG to GPO IG ||51.8 Mbits/sec || 99.4 Mbits/sec || 99.8 Mbits/sec || (no ack) ||92.216/92.482/92.937/0.407 ms || || GPO IG to Utah IG ||45.9 Mbits/sec || 99.3 Mbits/sec || 99.9 Mbits/sec || (no ack) ||92.214/92.525/92.891/0.351 ms || || Utah IG to GPO EG ||6.54 Mbits/sec || 2.91 Mbits/sec ||878 Kbits/sec || (no ack) ||92.350/92.663/93.212/0.443 ms || || GPO EG to Utah IG ||93.3 Kbits/sec || 93.9 Kbits/sec ||77.4 Kbits/sec || (no ack) ||92.388/95.532/262.060/21.682 ms|| || || || GPO IG to GPO EG || || || || || || || GPO EG to GPO IG || || || || || || || || Notes: (no ack) UDP iperf failed with "did not receive ack" for 100M, 50M and 25M. (?) related to poor performance on ExoGENI Nodes,looking at new OS. === Stitching Results Raw-pc Run0 === All IG are raw-pcs and all EG are !OpenStack VMs. These results were collected 2013-12-06: ||'''End points''' ||'''One Client(tcp)'''||'''5 Clients(tcp)'''||'''10 Clients(tcp)'''||'''1 client(udp)'''||'''Ping rtt min/avg/max/mdev'''|| ||GPO IG to Missouri IG ||98.3 Mbits/sec || 100 Mbits/sec || 100 Mbits/sec || 98.0 Mbits/sec ||45.583/45.678/46.016/0.260 ms || ||Missouri IG to GPO IG ||99.4 Mbits/sec || 100 Mbits/sec || 100 Mbits/sec || 97.8 Mbits/sec ||45.605/46.493/92.008/5.931 ms || ||GPO EG to Missouri IG || 97.2 Kbits/sec || 93.2 Kbits/sec || 120 Kbits/sec || 95.4 Kbits/sec ||45.664/45.827/46.500/0.279 ms || ||Missouri IG to GPO EG || 7.30 Mbits/sec || 581 Kbits/sec || 431 Kbits/sec || 26.3 Mbits/sec ||45.688/45.809/46.319/0.221 ms || || || ||GPO IG to NYSERNet IG ||99.8 Mbits/sec || 101 Mbits/sec || 101 Mbits/sec || 98.0 Mbits/sec ||11.137/11.218/11.532/0.092 ms || ||NYSERNet IG to GPO IG ||100 Mbits/sec || 101 Mbits/sec || 101 Mbits/sec || 97.6 Mbits/sec ||11.191/11.238/11.546/0.097 ms || ||GPO EG to NYSERNet IG || 96.4 Kbits/sec || 121 Kbits/sec || 75.0 Kbits/sec || 96.8 Kbits/sec ||11.213/11.418/15.838/0.589 ms || ||NYSERNet IG to GPO EG || 689 Kbits/sec || 396 Kbits/sec || 171 Kbits/sec || 20.9 Mbits/sec ||11.200/11.402/12.017/0.161 ms || || || ||GPO IG to Wisconsin IG||99.3 Mbits/sec || 100 Mbits/sec || 102 Mbits/sec || 98.0 Mbits/sec ||36.616/36.733/36.942/0.070 ms || ||Wisconsin IG to GPO IG||95.0 Mbits/sec || 100 Mbits/sec || 101 Mbits/sec || 97.8 Mbits/sec ||36.655/36.752/37.063/0.232 ms || ||GPO EG to Wisconsin IG|| 97.3 Kbits/sec || 99.6 Kbits/sec || 88.4 Kbits/sec || 95.6 Kbits/sec ||36.807/36.967/37.688/0.153 ms || ||Wisconsin IG to GPO EG|| 1.38 Mbits/sec || 670 Kbits/sec || 337 Kbits/sec || 23.9 Mbits/sec ||36.789/37.015/40.590/0.527 ms || || || ||GPO IG to MAX IG ||100 Mbits/sec || 100 Mbits/sec || 100 Mbits/sec || 97.9 Mbits/sec ||11.558/11.609/11.863/0.056 ms || ||MAX IG to GPO IG ||100 Mbits/sec || 101 Mbits/sec || 101 Mbits/sec || 97.7 Mbits/sec ||11.526/11.606/11.946/0.159 ms || ||GPO EG to MAX IG || 97.0 Kbits/sec || 118 Kbits/sec || 105 Kbits/sec || 96.6 Kbits/sec ||11.560/11.795/16.349/0.603 ms || ||MAX IG to GPO EG ||719 Kbits/sec || 406 Kbits/sec || 190 Kbits/sec || 27.8 Mbits/sec ||11.539/11.716/12.407/0.198 ms || = 2.0 Miscellaneous Information = == 2.1 Test Methodology == Test Characteristics: - All test hosts are dedicate hosts (bare metal/raw-pc) unless otherwise stated - All results are captured for default bandwidth allocation (100Mb/s shaped), unless otherwise stated. - All results collected between January 2014 Iperf TCP measurements captured in this page have the following assumptions: - All results are for a 60 second test run. - For !OpenFlow scenarios, traffic is exchanged before each of the measurements below to establish data flows: - iperf '1 client' scenario command: 'iperf -c dest_host -t 60' - iperf '5 clients' scenario command: 'iperf -c dest_host -t 60 -P 5' - iperf '10 clients' scenario command: 'iperf -c dest_host -t 60 -P 10' Iperf UDP measurements captured in this page have the following assumptions: - Each test run is 60 seconds and traffic is exchanged before each of the measurements to establish a data flow. - UDP measurements between racks requested a bandwidth of 100 Mbits/sec, if results showed that 100Mb/s was possible, then increased to 1Gb/s. == 2.2 Historical Performance Data == Various effort have captured performance for GENI Racks, following is a list of earlier pages that have capture some aspect of performance with OpenVZ, Xen and dedicated hosts. These pages should be considered obsolete: - [http://groups.geni.net/geni/wiki/GENIRacksHome/Performance-old Initial Performance] Initial version of this page which includes incorrect 1 TCP, 1 UDP and max/mdev results for ICMP. - [http://groups.geni.net/geni/wiki/GENIRacksHome/InstageniRacks/ConfirmationTestStatus/InstageniPerformance InstaGENI Site Confirmation Performance] - [http://groups.geni.net/geni/wiki/GENIRacksHome/InstageniRacks/ConfirmationTestStatus/InstageniPerformanceOF Local OpenFlow Performance (xen/OpenVZ] - [http://groups.geni.net/geni/wiki/GENIRacksHome/ExogeniRacks/ConfirmationTestStatus/ExogeniPerformance ExoGENI Site Confirmation Performance] - [http://groups.geni.net/geni/wiki/GENIOESSTopologiesPerformance OESS Cross-connects Performance] - [http://groups.geni.net/geni/wiki/GeniNetworkStitchingConfirmationTestStatus/Performance GENI Network Stitching Performance]