wiki:PlasticSlices/Continuation/Round6

Version 2 (modified by Josh Smift, 7 years ago) (diff)

--

Continuation Round 6

In this round, I ran plastic-105 and plastic-106 for about N hours, with only GT and Wisconsin, to try to track down some issues with performance in Round 3 that seemed related to GT. This was just a repeat of Round 4.

Further testing (not shown here) suggests that the problem is specific to data leaving the GT plnodes' dataplane interfaces: We get good results when we use the control interfaces, and with data leaving a wapgnode at GT, and bad performance even between nodes purely at GT. The only times we see problems are when GT plnodes are sending data, and it doesn't seem to matter at all where they're sending it. GT is investigating.

The raw logs are at http://www.gpolab.bbn.com/plastic-slices/continuation/round-6/.

plastic-105

SteadyPerf TCP, using port=5105, time=518400, and this table of client/server pairs:

client server server address
plnode1.cip.gatech.edu wings-openflow-2.wail.wisc.edu server=10.42.105.95
plnode2.cip.gatech.edu wings-openflow-3.wail.wisc.edu server=10.42.105.96

One-time prep commands run on each client and server

sudo yum -y install iperf

Commands run on each server

server=<ipaddr>
nice -n 19 iperf -B $server -p 5105 -s -i 1

Commands run on each client

server=<ipaddr>
nice -n 19 iperf -c $server -p 5105 -t 518400

Results

Generated with

subnet=105
for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 5 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done

and then edited slightly to remove artifacts (like control characters, my prompt, etc).

plnode1.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.95, TCP port 5105
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.100 port 54474 connected with 10.42.105.95 port 5105
[  3]  0.0-17262.0 sec  1.52 GBytes    755 Kbits/sec

plnode2.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.96, TCP port 5105
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.101 port 38278 connected with 10.42.105.96 port 5105
[  3]  0.0-17264.6 sec  1.93 GBytes    958 Kbits/sec

Analysis

These were somewhat higher than previously because there were periods when we weren't running plastic-106; we discovered that the low TCP bandwidth only happened while plastic-106 was running. The server logs show this more clearly.

plastic-106

SteadyPerf UDP, using port=5106, time=518400, rate=30, and this table of client/server pairs:

client server server address
plnode1.cip.gatech.edu wings-openflow-3.wail.wisc.edu server=10.42.105.96
plnode2.cip.gatech.edu wings-openflow-2.wail.wisc.edu server=10.42.105.95

One-time prep commands run on each client and server

sudo yum -y install iperf

Commands run on each server

server=<ipaddr>
nice -n 19 iperf -u -B $server -p 5106 -s -i 1

Commands run on each client

server=<ipaddr>
nice -n 19 iperf -u -c $server -p 5106 -t 518400 -b 30M

Results

Generated with

subnet=106
for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 10 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done

and then edited slightly to remove artifacts (like control characters, my prompt, etc).

plnode1.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.96, UDP port 5106
Sending 1470 byte datagrams
UDP buffer size:   122 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.100 port 48341 connected with 10.42.105.96 port 5106
[  3]  0.0-16304.4 sec  56.9 GBytes  30.0 Mbits/sec
[  3] Sent 41551881 datagrams
[  3] Server Report:
[  3]  0.0-16304.1 sec  52.3 GBytes  27.6 Mbits/sec  0.134 ms 3318803/41551881 (8%)
[  3]  0.0-16304.1 sec  240 datagrams received out-of-order

plnode2.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.95, UDP port 5106
Sending 1470 byte datagrams
UDP buffer size:   122 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.101 port 40375 connected with 10.42.105.95 port 5106
[  3]  0.0-16319.8 sec  57.0 GBytes  30.0 Mbits/sec
[  3] Sent 41600427 datagrams
[  3] Server Report:
[  3]  0.0-16319.3 sec  52.6 GBytes  27.7 Mbits/sec  0.084 ms 3160813/41600427 (7.6%)
[  3]  0.0-16319.3 sec  232 datagrams received out-of-order

Analysis

Much the same packet loss as before.