[[PageOutline]] = Sample Experiment: Writing a Web Server = == Purpose == The goal of this assignment is to build a functional web server. This assignment will guide you through you the basics of distributed programming, client/server structures, and issues in building high performance servers. This assignment is based on a course [http://www.cs.williams.edu/~jeannie/cs339/assignment1/index.html assignment] used at the Distributed Systems Course offered by the [http://www.cs.williams.edu/ Computer Science Department of Williams College] and taught by [http://www.cs.williams.edu/~jeannie/ Professor Jeannie Albrecht]. == Prerequisites == Before beginning this experiment, you should be prepared with the following. * You have GENI credentials to obtain GENI resources. (If not, see SignMeUp). * You are able to use '''Flack''' to request GENI resources. (If not, see [https://www.protogeni.net/trac/protogeni/wiki/FlackTutorial the Flack tutorial]). * You are comfortable using '''ssh''' and executing basic commands using a UNIX shell. * You are comfortable with coding in '''C or C++''' == Setup == * Download the attached [attachment:websrv.rspec?format=raw rspec file] and save it on your machine. (Make sure to save in raw format.) * Start Flack, create a new slice, load rspec ''websrv.rspec'' and submit for sliver creation (also fine to use omni, if you prefer). Your sliver should look something like this: [[Image(WebsrvExampleSliver.png, 50%)]] In this setup, there is one host acting as a web server. To test that the webserver is up visit the web page of the Server host. To do this * either press on the (i) button in Flack and then press the Visit button * or open a web browser and go to the webpage !http://.emulab.net, in the above example this would be !http://pc484.emulab.net). If the installation is successful you should see a page that is similar to this: [[Image(WebsrvIndex.png, 40%)]] == Techniques == You will use the following techniques during this experiment. === Start and stop the web server === In the original setup of your sliver there a webserver already installed and running on the `Server` host. As you implement your own webserver you might need to stop or start the installed webserver. * To '''Stop''' the webserver run: {{{ sudo /sbin/service httpd stop }}} To verify that you have stopped the webserver, try to visit the above web page, you should get an error. * To '''Start''' the webserver run: {{{ sudo /sbin/service httpd start }}} === Command Line Web Transfers === Except from using a web browser you can also use command line tools for web transfers. To do this, follow these steps: * Log in to `Client1`. * You can download the web page using this command {{{ [inki@Client1 ~]$ wget http://pc484.emulab.net --2012-07-06 04:32:59-- http://pc484.emulab.net/ Resolving pc484.emulab.net... 155.98.38.84 Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 548 [text/html] Saving to: “index.html” 100%[======================================>] 548 --.-K/s in 0s 2012-07-06 04:32:59 (127 MB/s) - “index.html” saved [548/548] }}} * The above command only downloads the `index.html` file from the webserver. As we are going to see later a web page might consist of multiple web pages or other objects such as pictures, videos etc. In order to force wget to download all dependencies of a page use the following options : {{{ [inki@Client1 ~]$ wget -m -p http://pc484.emulab.net --2012-07-06 04:33:49-- http://pc484.emulab.net/ Resolving pc484.emulab.net... 155.98.38.84 Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 548 [text/html] Saving to: “pc484.emulab.net/index.html” 100%[======================================>] 548 --.-K/s in 0s 2012-07-06 04:33:49 (118 MB/s) - “pc484.emulab.net/index.html” saved [548/548] Loading robots.txt; please ignore errors. --2012-07-06 04:33:49-- http://pc484.emulab.net/robots.txt Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2012-07-06 04:33:49 ERROR 404: Not Found. --2012-07-06 04:33:49-- http://pc484.emulab.net/top.html Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 917 [text/html] Saving to: “pc484.emulab.net/top.html” 100%[======================================>] 917 --.-K/s in 0s 2012-07-06 04:33:49 (172 MB/s) - “pc484.emulab.net/top.html” saved [917/917] --2012-07-06 04:33:49-- http://pc484.emulab.net/home.html Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 822 [text/html] Saving to: “pc484.emulab.net/home.html” 100%[======================================>] 822 --.-K/s in 0s 2012-07-06 04:33:49 (215 MB/s) - “pc484.emulab.net/home.html” saved [822/822] --2012-07-06 04:33:49-- http://pc484.emulab.net/links.html Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 958 [text/html] Saving to: “pc484.emulab.net/links.html” 100%[======================================>] 958 --.-K/s in 0s 2012-07-06 04:33:49 (267 MB/s) - “pc484.emulab.net/links.html” saved [958/958] --2012-07-06 04:33:49-- http://pc484.emulab.net/media/GENILogo.png Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 22678 (22K) [image/png] Saving to: “pc484.emulab.net/media/GENILogo.png” 100%[======================================>] 22,678 --.-K/s in 0.001s 2012-07-06 04:33:49 (42.8 MB/s) - “pc484.emulab.net/media/GENILogo.png” saved [22678/22678] --2012-07-06 04:33:49-- http://pc484.emulab.net/media/topgeni.png Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 116224 (114K) [image/png] Saving to: “pc484.emulab.net/media/topgeni.png” 100%[======================================>] 116,224 --.-K/s in 0.002s 2012-07-06 04:33:49 (66.7 MB/s) - “pc484.emulab.net/media/topgeni.png” saved [116224/116224] --2012-07-06 04:33:49-- http://pc484.emulab.net/media/hello.png Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 11497 (11K) [image/png] Saving to: “pc484.emulab.net/media/hello.png” 100%[======================================>] 11,497 --.-K/s in 0s 2012-07-06 04:33:49 (45.8 MB/s) - “pc484.emulab.net/media/hello.png” saved [11497/11497] --2012-07-06 04:33:49-- http://pc484.emulab.net/media/experimenter.png Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 14020 (14K) [image/png] Saving to: “pc484.emulab.net/media/experimenter.png” 100%[======================================>] 14,020 --.-K/s in 0s 2012-07-06 04:33:49 (47.5 MB/s) - “pc484.emulab.net/media/experimenter.png” saved [14020/14020] --2012-07-06 04:33:49-- http://pc484.emulab.net/media/keyboard.png Connecting to pc484.emulab.net|155.98.38.84|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 9533 (9.3K) [image/png] Saving to: “pc484.emulab.net/media/keyboard.png” 100%[======================================>] 9,533 --.-K/s in 0s 2012-07-06 04:33:49 (73.4 MB/s) - “pc484.emulab.net/media/keyboard.png” saved [9533/9533] FINISHED --2012-07-06 04:33:49-- Downloaded: 9 files, 173K in 0.003s (59.7 MB/s) [inki@Client1 ~]$ }}} * Use the dataplane interface. Up to this point we have used the public interface of the webserver. As you are testing your environment, you should use the dataplane connections between the clients and the server. These are the private connections that are represented with a lines between the machines in Flack. To do that you first need to figure out the IP address of the server on each of these links. In the above example, the line that connects the Server with Client1 has a box that is called `lan0`. Press on the (i) button of that box and see the IP server that is assigned on the server. Use this IP to run the wget command : {{{ [inki@Client1 ~]$ wget -m -p http://10.10.1.1 }}} === Viewing and Adjusting link characteristics === In this experiment, you'll be changing the characteristics of the link and measuring how they affect UDT and TCP performance. * Log into your delay node as you do with any other node. Then, on your delay node, use this command: {{{ %sudo ipfw pipe show }}} You'll get something like this: {{{ 60111: 100.000 Mbit/s 1 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 207.167.175.72/0 195.123.216.8/6 7 1060 0 0 0 60121: 100.000 Mbit/s 1 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 207.167.176.224/0 195.124.8.8/6 8 1138 0 0 0 }}} This information shows the internal configuration of the "pipes" used to emulate network characteristics. (Your output may look different, depending on the version of ipfw installed on your delay node. In any case, the information you need is on the first line of output for each pipe.) You'll want to make note of the two pipe numbers, one for each direction of traffic along your link. In the example above, they are 60111 and 60121. There are three link characteristics we'll manipulate in this experiment: bandwidth, delay, and packet loss rate. You'll find their values listed in the ipfw output above. The link bandwidth appears on the first line immediately after the pipe number. It's 100Mbps in the example shown above. The next value shown is the delay, 1 ms in the example above. The packet loss rate (PLR) is omitted if it's zero, as shown above. If non-zero, you'll see something like '''plr 0.000100''' immediately after the "50 sl." on the first output line. It is possible to adjust the parameters of the two directions of your link separately, to emulate asymmetric links. In this experiment, however, we are looking at symmetric links, so we'll always change the settings on both pipes together. Here are the command sequences you'll need to change your link parameters. In each case, you'll need to provide the correct pipe numbers, if they're different from the example. * To change bandwidth (100M means 100Mbits/s): {{{ sudo ipfw pipe 60111 config bw 100M sudo ipfw pipe 60121 config bw 100M }}} * Request a bandwidth of zero to use the full capacity of the link (unlimited): {{{ sudo ipfw pipe 60111 config bw 0 sudo ipfw pipe 60121 config bw 0 }}} * To change link delay (delays are measured in ms): {{{ sudo ipfw pipe 60111 config delay 10 sudo ipfw pipe 60121 config delay 10 }}} * To change packet loss rate (rate is a probability, so 0.001 means 0.1% packet loss): {{{ sudo ipfw pipe 60111 config plr .0001 sudo ipfw pipe 60121 config plr .0001 }}} * You can combine settings for bandwidth, delay, and loss by specifying more than one in a single ipfw command. We'll use this form in the procedure below. == Experiment Procedure == * Set your link parameters to use maximum bandwidth, no delay, no packet loss: {{{ sudo ipfw pipe 60111 config bw 0 delay 0 plr 0 sudo ipfw pipe 60121 config bw 0 delay 0 plr 0 }}} * Verify with {{{ sudo ipfw pipe show 60111: unlimited 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 207.167.175.72/0 195.123.216.8/6 7 1060 0 0 0 60121: unlimited 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 207.167.176.224/0 195.124.8.8/6 8 1138 0 0 0 }}} Note that bandwidth is set to ''unlimited'', delay to ''0 ms'', and no PLR is shown. * Using this initial setting, try a few UDT transfers, including the larger files. Now try FTP transfers. Record the transfer sizes and rates. * Now change the link parameters to reduce the available bandwidth to 10Mbps: {{{ sudo ipfw pipe 60111 config bw 10M delay 0 plr 0 sudo ipfw pipe 60121 config bw 10M delay 0 plr 0 }}} * Repeat your file transfers with the new settings. As before, note the transfer sizes and rates, as well as the link settings. * Continue with additional trials, varying each of the three link parameters over a range sufficient to observe meaningful performance differences. Record your data. == What to hand in == 1. Your raw data and appropriate graphs illustrating changes in performance for the two transfer protocols with differing link parameters. 1. Your analysis. Here are some questions to consider. a. Does one protocol outperform the other? b. Under what conditions are performance differences most clearly seen? Why? c. What shortcomings in the experiment design may affect your results? How might you improve the experiment design? d. What interesting characteristics of the transfer protocols are not measured in this experiment? How might you design an experiment to investigate these?