[[PageOutline]] = Continuation Round 6 = In this round, I ran plastic-105 and plastic-106 for about N hours, with only GT and Wisconsin, to try to track down some issues with performance in Round 3 that seemed related to GT. This was just a repeat of Round 4. Further testing (not shown here) suggests that the problem is specific to data leaving the GT plnodes' dataplane interfaces: We get good results when we use the control interfaces, and with data leaving a wapgnode at GT, and bad performance even between nodes purely at GT. The only times we see problems are when GT plnodes are sending data, and it doesn't seem to matter at all where they're sending it. GT is investigating. The raw logs are at http://www.gpolab.bbn.com/plastic-slices/continuation/round-6/. = plastic-105 = [ggw:PlasticSlices/Experiments#SteadyPerfTCP SteadyPerf TCP], using port=5105, time=518400, and this table of client/server pairs: || '''client''' || '''server''' || '''server address''' || || plnode1.cip.gatech.edu || wings-openflow-2.wail.wisc.edu || server=10.42.105.95 || || plnode2.cip.gatech.edu || wings-openflow-3.wail.wisc.edu || server=10.42.105.96 || == One-time prep commands run on each client and server == {{{ sudo yum -y install iperf }}} == Commands run on each server == {{{ server= nice -n 19 iperf -B $server -p 5105 -s -i 1 }}} == Commands run on each client == {{{ server= nice -n 19 iperf -c $server -p 5105 -t 518400 }}} == Results == Generated with {{{ subnet=105 for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 5 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done }}} and then edited slightly to remove artifacts (like control characters, my prompt, etc). plnode1.cip.gatech.edu: {{{ ------------------------------------------------------------ Client connecting to 10.42.105.95, TCP port 5105 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.42.105.100 port 54474 connected with 10.42.105.95 port 5105 [ 3] 0.0-17262.0 sec 1.52 GBytes 755 Kbits/sec }}} plnode2.cip.gatech.edu: {{{ ------------------------------------------------------------ Client connecting to 10.42.105.96, TCP port 5105 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.42.105.101 port 38278 connected with 10.42.105.96 port 5105 [ 3] 0.0-17264.6 sec 1.93 GBytes 958 Kbits/sec }}} == Analysis == These were somewhat higher than previously because there were periods when we weren't running plastic-106; we discovered that the low TCP bandwidth only happened while plastic-106 was running. The server logs show this more clearly. = plastic-106 = [ggw:PlasticSlices/Experiments#SteadyPerfUDP SteadyPerf UDP], using port=5106, time=518400, rate=30, and this table of client/server pairs: || '''client''' || '''server''' || '''server address''' || || plnode1.cip.gatech.edu || wings-openflow-3.wail.wisc.edu || server=10.42.105.96 || || plnode2.cip.gatech.edu || wings-openflow-2.wail.wisc.edu || server=10.42.105.95 || == One-time prep commands run on each client and server == {{{ sudo yum -y install iperf }}} == Commands run on each server == {{{ server= nice -n 19 iperf -u -B $server -p 5106 -s -i 1 }}} == Commands run on each client == {{{ server= nice -n 19 iperf -u -c $server -p 5106 -t 518400 -b 30M }}} == Results == Generated with {{{ subnet=106 for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 10 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done }}} and then edited slightly to remove artifacts (like control characters, my prompt, etc). plnode1.cip.gatech.edu: {{{ ------------------------------------------------------------ Client connecting to 10.42.105.96, UDP port 5106 Sending 1470 byte datagrams UDP buffer size: 122 KByte (default) ------------------------------------------------------------ [ 3] local 10.42.105.100 port 48341 connected with 10.42.105.96 port 5106 [ 3] 0.0-16304.4 sec 56.9 GBytes 30.0 Mbits/sec [ 3] Sent 41551881 datagrams [ 3] Server Report: [ 3] 0.0-16304.1 sec 52.3 GBytes 27.6 Mbits/sec 0.134 ms 3318803/41551881 (8%) [ 3] 0.0-16304.1 sec 240 datagrams received out-of-order }}} plnode2.cip.gatech.edu: {{{ ------------------------------------------------------------ Client connecting to 10.42.105.95, UDP port 5106 Sending 1470 byte datagrams UDP buffer size: 122 KByte (default) ------------------------------------------------------------ [ 3] local 10.42.105.101 port 40375 connected with 10.42.105.95 port 5106 [ 3] 0.0-16319.8 sec 57.0 GBytes 30.0 Mbits/sec [ 3] Sent 41600427 datagrams [ 3] Server Report: [ 3] 0.0-16319.3 sec 52.6 GBytes 27.7 Mbits/sec 0.084 ms 3160813/41600427 (7.6%) [ 3] 0.0-16319.3 sec 232 datagrams received out-of-order }}} == Analysis == Much the same packet loss as before.