[[PageOutline]] This page describes the procedure for setting up, running, and cleaning up each of the experiments that we're running for the [wiki:PlasticSlices Plastic Slices] project. In each experiment, it assumes you've already created your slice and all of its slivers, and divided your compute resources into "clients" and "servers" (the meaning of which will vary depending on the particular experiment). There are two kinds of experiments described here: * "Giga" experiments, which use various methods to send a particular quantity of total data. (1 GB as described below, but it's easy to vary this.) * "Steady" experiments, which use various methods to send data at a particular rate, either for a set length of time, or indefinitely. (10 Mbits/sec as described below, but it's easy to vary this.) ''FIXME: It would be better if we included commands to log output and inspect it later; we just handwave past this for now.'' = !GigaPing = This experiment uses the 'ping' command to send 1 GB of data via ICMP. == Setup == Divide the hosts that you want to use into pairs. In each pair, the client is the one that you'll run the 'ping' command on, and the server is the one that you'll ping from the client. Identify the precise IP address on each server that you'll want to ping. Make a table to keep track of which server you'll want to ping from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. To send 1 GB of data, you'll need to send a total of 666,667 1.5 KB packets. Rounding up for simplicity, divide 700,000 by the number of host pairs you have (the number of rows in your table); call this number , and use it below when you run the 'ping' commands. == Execution == On each client, run {{{ server= sudo ping -i .001 -s $((1500-8-20)) -c $server }}} using the "server=" line from your table, and the value you calculated earlier. That should print many lines on your terminal that look like {{{ 1480 bytes from : icmp_seq=44 ttl=64 time=135 ms 1480 bytes from : icmp_seq=49 ttl=64 time=87.7 ms 1480 bytes from : icmp_seq=50 ttl=64 time=77.9 ms }}} and then finish with a section that looks like {{{ --- ping statistics --- packets transmitted, received, 0% packet loss, time 5229ms rtt min/avg/max/mdev = 59.376/70.448/522.011/57.578 ms, pipe 55 }}} Copy that final block for your results. == Results == The results of the experiment are the three-line statistics summary at the end of the output on the client. == Cleanup == No special cleanup is needed. = !GigaNetcat = This experiment uses the 'netcat' command to send 1 GB of data via unencrypted TCP. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run a netcat listener on the server, and make a netcat connection to it from the client. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the netcat server and client commands. Identify a file that you'd like to transfer from the servers to the clients; call it . Find out how many times you'll need to copy it to get more than 1 GB: {{{ echo $((10**9/$(du -b | awk '{print $1;}')+1)) }}} Divide that number by the number of host pairs you have (the number of rows in your table); call this number , and use it below when you run the netcat commands. Copy the file to each of the servers. (When you run the experiment, you'll download it repeatedly to each of the clients.) On each client and server, install netcat: {{{ sudo yum install nc }}} == Execution == On each server, run {{{ server= for i in {1..} ; do nc -l $server < ; echo "completed transfer #$i" ; md5sum ; done }}} using the "server=" line from your table, and the , , and values you identified earlier. You won't see any output immediately, but when you start running clients, as each transfer completes, you should see lines on your terminal that look like {{{ completed transfer #1 }}} and the md5sum checksum of the file. On each client, run {{{ server= rm -rf ~/giganetcat mkdir ~/giganetcat cd ~/giganetcat for i in {1..} ; do nc $server > ; echo "completed transfer #$i" ; md5sum ; mv .$i ; done }}} You won't see any output immediately, but as each transfer completes, you should see lines on your terminal that look like {{{ completed transfer #1 }}} and the md5sum checksum of the file (and the filename). == Results == In each client-server pair, compare the output on the client and the server. They should be identical: Each should list the same set of transfers completed, and all of the md5sum lines should be identical on both. On each client, check the total size of the data transfered: {{{ du -sb . }}} Add up the results for all the clients; they should add up to 1 GB (or more). == Cleanup == On each client, remove the giganetcat directory: {{{ cd rm -rf ~/giganetcat }}} = !GigaWeb = This experiment uses the HTTPS protocol to send 1 GB of data via encrypted TCP. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run an HTTPS server on the server, and run wget on the client to fetch files from it. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the netcat server and client commands. On each server, install an HTTPS server. One way is to install pyOpenSSL, the 'patch' program, and use the Python server at http://code.activestate.com/recipes/442473-simple-http-server-supporting-ssl-secure-communica/, with a patch to allow you to bind to an address and port specified on the command line: {{{ sudo yum install pyOpenSSL patch rm -rf ~/gigaweb mkdir -p ~/gigaweb/docroot cd ~/gigaweb wget http://code.activestate.com/recipes/442473-simple-http-server-supporting-ssl-secure-communica/download/1/ -O httpsd.py wget http://groups.geni.net/geni/attachment/wiki/PlasticSlices/Experiments/httpsd.py.patch?format=raw -O httpsd.py.patch patch httpsd.py httpsd.py.patch rm httpsd.py.patch }}} On each server, generate a self-signed SSL key: {{{ openssl genrsa -passout pass:localhost -des3 -rand /dev/urandom -out localhost.localdomain.key 1024 openssl req -subj /CN=localhost.localdomain -passin pass:localhost -new -key localhost.localdomain.key -out localhost.localdomain.csr openssl x509 -passin pass:localhost -req -days 3650 -in localhost.localdomain.csr -signkey localhost.localdomain.key -out localhost.localdomain.crt openssl rsa -passin pass:localhost -in localhost.localdomain.key -out decrypted.localhost.localdomain.key mv decrypted.localhost.localdomain.key localhost.localdomain.key cat localhost.localdomain.key localhost.localdomain.crt > localhost.localdomain.pem rm localhost.localdomain.key localhost.localdomain.crt localhost.localdomain.csr }}} Identify a file that you'd like to transfer from the servers to the clients; call it . Find out how many times you'll need to copy it to get more than 1 GB: {{{ echo $((10**9/$(du -b | awk '{print $1;}')+1)) }}} Divide that number by the number of host pairs you have (the number of rows in your table); call this number , and use it below when you run the wget command on the clients. Copy the file to each of the servers, and put it into ~/gigaweb/docroot. (When you run the experiment, you'll download it repeatedly to each of the clients.) == Execution == On each server, run {{{ server= cd ~/gigaweb/docroot python ../httpsd.py $server }}} using the "server=" line from your table, and the value you identified earlier. That should print a line on your terminal like {{{ Serving HTTPS on port ... }}} immediately, and then lines like {{{ - - [17/May/2011 19:53:21] "GET / HTTP/1.1" 200 - }}} for each client connection when clients connect. On each client, run {{{ server= rm -rf ~/gigaweb mkdir ~/gigaweb cd ~/gigaweb for i in {1..} ; do wget --no-check-certificate https://$server:/ -O .$i ; done }}} You'll see output typical of wget as each transfer runs. When the downloads are finished, on the server, hit ctrl-c to kill the httpsd.py process. == Results == On each client, check the total size of the data transfered: {{{ du -sb . }}} Add up the results for all the clients; they should add up to 1 GB (or more). On the server, calculate the md5sum checksum for the file: Do {{{ md5sum | awk '{print $1;}' }}} Call the result of this command . On each client, calculate the md5sum checksums for all of the copies of the file, and find any that differ from the server: Do {{{ md5sum * | grep -v || echo "All checksums match." }}} and expect "All checksums match" as the only output. == Cleanup == On each client, remove the gigaweb directory: {{{ cd rm -rf ~/gigaweb }}} = GigaPerfTCP = This experiment uses the iperf command to send 1 GB of data via TCP, measuring performance throughout the transfer. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run an iperf listener on the server, and make an iperf connection to it from the client. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the iperf server and client commands. Divide 1000 by the number of host pairs you have (the number of rows in your table); call this number , and use it below when you run the iperf client command. (This is how much data (in megabytes) each client will send to the server.) On each client and server, install iperf: {{{ sudo yum -y install iperf }}} == Execution == On each server, run {{{ server= nice -n 19 iperf -B $server -p -s -i 1 }}} using the "server=" line from your table, and the value you identified earlier. That should print a few lines on your terminal immediately, including ones like {{{ Server listening on TCP port Binding to local address }}} and then more lines once a client connects. On each client, run {{{ server= nice -n 19 iperf -c $server -p -n M }}} using the "server=" line from your table, and the value you identified earlier. That should print a few lines on your terminal immediately, including ones like {{{ Client connecting to , TCP port ------------------------------------------------------------ [ 3] local 10.42.17.51 port 40997 connected with port }}} The client IP address and port will be different, of course. The server will then start to print lines like {{{ [ 4] local port connected with 10.42.17.51 port 40997 [ 4] 0.0- 1.0 sec 359 KBytes 2.94 Mbits/sec [ 4] 1.0- 2.0 sec 498 KBytes 4.08 Mbits/sec }}} When the transfer finishes, the client will print lines like {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0- 2.2 sec 1.00 MBytes 3.79 Mbits/sec }}} Copy that final block for your results. When the transfer finishes, on the server, hit ctrl-c to kill the iperf server process. == Results == The results of the experiment are the three-line statistics summary at the end of the output on the client. == Cleanup == No special cleanup is needed. = GigaPerfUDP = This experiment uses the iperf command to send 1 GB of data via UDP, measuring performance throughout the transfer. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run an iperf listener on the server, and make an iperf connection to it from the client. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the iperf server and client commands. Divide 1000 by the number of host pairs you have (the number of rows in your table); call this number , and use it below when you run the iperf client command. (This is how much data (in megabytes) each client will send to the server.) Decide how fast you'd like to send data (in megabits per second); call this number , and use it below when you run the iperf client command. On each client and server, install iperf: {{{ sudo yum -y install iperf }}} == Execution == On each server, run {{{ server= nice -n 19 iperf -u -B $server -p -s -i 1 }}} using the "server=" line from your table, and the value you identified earlier. That should print a few lines on your terminal immediately, including ones like {{{ Server listening on UDP port Binding to local address }}} and then more lines once a client connects. On each client, run {{{ server= nice -n 19 iperf -u -c $server -p -n M -b M }}} using the "server=" line from your table, and the , , and values you identified earlier. That should print a few lines on your terminal immediately, including ones like {{{ Client connecting to , UDP port ------------------------------------------------------------ [ 3] local 10.42.17.51 port 40997 connected with port }}} The client IP address and port will be different, of course. The server will then start to print lines like {{{ [ 4] local port connected with 10.42.17.51 port 40997 [ 4] 0.0- 1.0 sec 359 KBytes 2.94 Mbits/sec [ 4] 1.0- 2.0 sec 498 KBytes 4.08 Mbits/sec }}} When the transfer finishes, the client will print lines like {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-79.8 sec 954 MBytes 100 Mbits/sec [ 3] Sent 680273 datagrams [ 3] Server Report: [ 3] 0.0-79.1 sec 881 MBytes 93.4 Mbits/sec 0.040 ms 52088/680272 (7.7%) [ 3] 0.0-79.1 sec 256 datagrams received out-of-order }}} Copy that final block for your results. When the transfer finishes, on the server, hit ctrl-c to kill the iperf server process. == Results == The results of the experiment are the three-line statistics summary at the end of the output on the client. == Cleanup == No special cleanup is needed. = !SteadyPing = This experiment uses the 'ping' command to send data at 10 Mbits/second via ICMP. == Setup == Divide the hosts that you want to use into pairs. In each pair, the client is the one that you'll run the 'ping' command on, and the server is the one that you'll ping from the client. Identify the precise IP address on each server that you'll want to ping. Make a table to keep track of which server you'll want to ping from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Calculate the interval between ping packets. The formula is: {{{ seconds/packet = 1500 bytes/packet * 8 bits/byte * 1/1024 Kbit/bit * 1/1024 Mbit/Kbit * * 1/10 seconds/Mbit }}} where is the number of host pairs you have (the number of rows in your table). Call this number , and use it below when you run the 'ping' commands. (It's 1/10 seconds per Mbit because you want to calculate an interval expressed in seconds per packet, not packets per second.) == Execution == On each client, run {{{ server= sudo ping -i -s $((1500-8-20)) $server }}} using the "server=" line from your table, and the value you calculated earlier. That should print many lines on your terminal that look like {{{ 1480 bytes from : icmp_seq=44 ttl=64 time=135 ms 1480 bytes from : icmp_seq=49 ttl=64 time=87.7 ms 1480 bytes from : icmp_seq=50 ttl=64 time=77.9 ms }}} and then finish with a section that looks like {{{ --- ping statistics --- packets transmitted, received, 0% packet loss, time 5229ms rtt min/avg/max/mdev = 59.376/70.448/522.011/57.578 ms, pipe 55 }}} Copy that final block for your results. == Results == The results of the experiment are the three-line statistics summary at the end of the output on the client. == Cleanup == No special cleanup is needed. = !SteadyNetcat = This experiment uses the 'netcat' command to send data via unencrypted TCP. Note that it doesn't allow you to directly control the rate at which data is sent. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run a netcat listener on the server, and make a netcat connection to it from the client. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the netcat server and client commands. Identify a file that you'd like to transfer from the servers to the clients; call it . Copy the file to each of the servers. (When you run the experiment, you'll download it repeatedly to each of the clients.) On each client and server, install netcat: {{{ sudo yum install nc }}} == Execution == On each server, run {{{ server= while true ; do nc -l $server < ; done }}} using the "server=" line from your table, and the , and values you identified earlier. You won't see any output in the server window. On each client, run {{{ server= rm -rf ~/steadynetcat mkdir ~/steadynetcat cd ~/steadynetcat while true ; do nc $server > ; echo -n "$(date "+%F %T") " ; md5sum ; rm ; done }}} You won't see any output immediately, but as each transfer completes, you should see lines on your terminal that look like {{{ 2011-06-29 11:20:46 d4fcf71833327fbfef98be09deef8bfb substrate.doc }}} and the md5sum checksum of the file and the filename. Leave it running for a while; when you're done, hit ctrl-c on the client, and then on the server, to kill the netcat processes. == Results == Compare the md5sum checkums; they should all be the same. == Cleanup == On each client, remove the steadynetcat directory: {{{ cd rm -rf ~/steadynetcat }}} = !SteadyWeb = This experiment uses the HTTPS protocol to send data via encrypted TCP. Note that it doesn't allow you to directly control the rate at which data is sent. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run an HTTPS server on the server, and run wget on the client to fetch files from it. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the netcat server and client commands. On each server, install an HTTPS server. One way is to install pyOpenSSL, the 'patch' program, and use the Python server at http://code.activestate.com/recipes/442473-simple-http-server-supporting-ssl-secure-communica/, with a patch to allow you to bind to an address and port specified on the command line: {{{ sudo yum install pyOpenSSL patch rm -rf ~/steadyweb mkdir -p ~/steadyweb/docroot cd ~/steadyweb wget http://code.activestate.com/recipes/442473-simple-http-server-supporting-ssl-secure-communica/download/1/ -O httpsd.py wget http://groups.geni.net/geni/attachment/wiki/PlasticSlices/Experiments/httpsd.py.patch?format=raw -O httpsd.py.patch patch httpsd.py httpsd.py.patch rm httpsd.py.patch }}} On each server, generate a self-signed SSL key: {{{ openssl genrsa -passout pass:localhost -des3 -rand /dev/urandom -out localhost.localdomain.key 1024 openssl req -subj /CN=localhost.localdomain -passin pass:localhost -new -key localhost.localdomain.key -out localhost.localdomain.csr openssl x509 -passin pass:localhost -req -days 3650 -in localhost.localdomain.csr -signkey localhost.localdomain.key -out localhost.localdomain.crt openssl rsa -passin pass:localhost -in localhost.localdomain.key -out decrypted.localhost.localdomain.key mv decrypted.localhost.localdomain.key localhost.localdomain.key cat localhost.localdomain.key localhost.localdomain.crt > localhost.localdomain.pem rm localhost.localdomain.key localhost.localdomain.crt localhost.localdomain.csr }}} Identify a file that you'd like to transfer from the servers to the clients; call it . Copy the file to each of the servers, and put it into ~/steadyweb/docroot. (When you run the experiment, you'll download it repeatedly to each of the clients.) == Execution == On each server, run {{{ server= cd ~/steadyweb/docroot python ../httpsd.py $server }}} using the "server=" line from your table, and value you identified earlier. That should print a line on your terminal like {{{ Serving HTTPS on port ... }}} immediately, and then lines like {{{ - - [17/May/2011 19:53:21] "GET / HTTP/1.1" 200 - }}} for each client connection when clients connect. On each client, run {{{ server= rm -rf ~/steadyweb mkdir ~/steadyweb cd ~/steadyweb while true ; do wget --no-check-certificate https://$server:/ -O ; echo -n "$(date "+%F %T") " ; md5sum ; rm ; done }}} You'll see output typical of wget as each transfer runs, and an md5sum checksum line. Leave it running for a while; when you're done, hit ctrl-c on the client, and then on the server, to kill the netcat processes. == Results == Compare the md5sum checkums; they should all be the same. == Cleanup == On each client, remove the steadyweb directory: {{{ cd rm -rf ~/steadyweb }}} = SteadyPerfTCP = This experiment uses the iperf command to send data via TCP, measuring performance throughout the transfer. Note that it doesn't allow you to directly control the rate at which data is sent. == Setup == Divide the hosts that you want to use into pairs. In each pair, you'll run an iperf listener on the server, and make an iperf connection to it from the client. Identify the precise IP address on each server that you'll want to connect to. Make a table to keep track of which server you'll want to connect to from each client, e.g. || '''client''' || '''server''' || '''server address''' || || ganel.gpolab.bbn.com || planetlab5.clemson.edu || server=10.42.17.105 || || planetlab4.clemson.edu || pl5.myplc.grnoc.iu.edu || server=10.42.17.73 || || of-planet1.stanford.edu || wings-openflow-3.wail.wisc.edu || server=10.42.17.96 || You'll use the "server=" parts when you run the experiment. Identify a port that you'd like to use for the connection; call it , and use it below when you run the iperf server and client commands. Decide how long (in seconds) you'd like to transfer data for; call that