wiki:PlasticSlices/Continuation/Round4

Version 1 (modified by Josh Smift, 7 years ago) (diff)

--

Continuation Round 4

In this round, we ran plastic-105 and plastic-106 for about three hours, with only GT and Wisconsin, to try to track down some issues with performance in Round 3 that seemed related to GT. In particular, we wanted to find out whether there was a specific problem with plnode1.

The raw logs are at http://www.gpolab.bbn.com/plastic-slices/continuation/round-4/.

plastic-105

SteadyPerf TCP, using port=5105, time=518400, and this table of client/server pairs:

client server server address
plnode1.cip.gatech.edu wings-openflow-2.wail.wisc.edu server=10.42.105.95
plnode2.cip.gatech.edu wings-openflow-3.wail.wisc.edu server=10.42.105.96

One-time prep commands run on each client and server

sudo yum -y install iperf

Commands run on each server

server=<ipaddr>
nice -n 19 iperf -B $server -p 5105 -s -i 1

Commands run on each client

server=<ipaddr>
nice -n 19 iperf -c $server -p 5105 -t 518400

Results

Generated with

subnet=105
for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 5 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done

and then edited slightly to remove artifacts (like control characters, my prompt, etc).

plnode1.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.95, TCP port 5105
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.100 port 59778 connected with 10.42.105.95 port 5105
[  3]  0.0-13761.8 sec    405 MBytes    247 Kbits/sec

plnode2.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.96, TCP port 5105
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.101 port 58242 connected with 10.42.105.96 port 5105
[  3]  0.0-13768.4 sec    482 MBytes    294 Kbits/sec

Analysis

Less than 300 Kbits/sec is very low.

Looking at the server logs shows times when the rate jumped up to one or two Mbits/sec, before dropping back down to double-digt Kbits/sec.

The results are similar for the two plnodes, so there probably isn't a problem with just one or the other.

plastic-106

SteadyPerf UDP, using port=5106, time=518400, rate=30, and this table of client/server pairs:

client server server address
plnode1.cip.gatech.edu wings-openflow-3.wail.wisc.edu server=10.42.105.96
plnode2.cip.gatech.edu wings-openflow-2.wail.wisc.edu server=10.42.105.95

One-time prep commands run on each client and server

sudo yum -y install iperf

Commands run on each server

server=<ipaddr>
nice -n 19 iperf -u -B $server -p 5106 -s -i 1

Commands run on each client

server=<ipaddr>
nice -n 19 iperf -u -c $server -p 5106 -t 518400 -b 30M

Results

Generated with

subnet=106
for host in $(awk 'NR%2==1' ~/plastic-slices/logins/logins-plastic-$subnet.txt | sed -r -e 's/.+@//') ; do echo -e "$host:\n\n{{{" ; grep -A 10 -B 1 "Client connecting" pgenigpolabbbncom_plastic$subnet\@$host.log ; echo -e "}}}\n" ; done

and then edited slightly to remove artifacts (like control characters, my prompt, etc).

plnode1.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.96, UDP port 5106
Sending 1470 byte datagrams
UDP buffer size:   122 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.100 port 53151 connected with 10.42.105.96 port 5106
[  3]  0.0-13755.2 sec  48.0 GBytes  30.0 Mbits/sec
[  3] Sent 35057619 datagrams
[  3] Server Report:
[  3]  0.0-13754.8 sec  44.2 GBytes  27.6 Mbits/sec  0.078 ms 2767576/35057619 (7.9%)
[  3]  0.0-13754.8 sec  252 datagrams received out-of-order

plnode2.cip.gatech.edu:

------------------------------------------------------------
Client connecting to 10.42.105.95, UDP port 5106
Sending 1470 byte datagrams
UDP buffer size:   122 KByte (default)
------------------------------------------------------------
[  3] local 10.42.105.101 port 39327 connected with 10.42.105.95 port 5106
[  3]  0.0-13756.4 sec  48.0 GBytes  30.0 Mbits/sec
[  3] Sent 35068319 datagrams
[  3] Server Report:
[  3]  0.0-13756.0 sec  44.4 GBytes  27.7 Mbits/sec  0.252 ms 2666045/35068319 (7.6%)
[  3]  0.0-13756.0 sec  319 datagrams received out-of-order

Analysis

Similar packet loss to Round 3, where we saw 7.5% loss.

The results are similar for the two plnodes, so there probably isn't a problem with just one or the other.