wiki:GENIExperimenter/Tutorials/SystematicExprCaseStudy/SmallTopo/Execute

Version 12 (modified by sedwards@bbn.com, 9 years ago) (diff)

--

A Tutorial on Systematic Experimental Design

Step I: Single Node

Step II: Small Topology

Image Map

5. End-to-End Validation

We first validate end-to-end communication between client and server by running ping and traceroute at the client, and then run iperf on both client and server.

Open two terminals, and login to client and server, respectively.

a. End-to-End traceroute and ping

  • Run ping from client (192.168.10.11) to server (192.168.20.10).

xuanliu@client:~$ ping 192.168.20.10
PING 192.168.20.10 (192.168.20.10) 56(84) bytes of data.
64 bytes from 192.168.20.10: icmp_req=1 ttl=61 time=3.28 ms
64 bytes from 192.168.20.10: icmp_req=2 ttl=61 time=2.69 ms
64 bytes from 192.168.20.10: icmp_req=3 ttl=61 time=2.73 ms
^C
--- 192.168.20.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.696/2.905/3.282/0.273 ms
  • Run traceroute from client (192.168.10.11) to server (192.168.20.10).
xuanliu@client:~$ traceroute 192.168.20.10
traceroute to 192.168.20.10 (192.168.20.10), 30 hops max, 60 byte packets
 1  router-1-lan4 (192.168.10.10)  0.536 ms  0.596 ms  0.562 ms
 2  router-2-lan0 (192.168.1.2)  1.048 ms  0.994 ms  0.979 ms
 3  router-3-lan1 (192.168.2.2)  1.368 ms  1.328 ms  1.291 ms
 4  server-lan5 (192.168.20.10)  1.878 ms  1.790 ms  1.706 ms

b. iperf Test

  • iperf End-to-End TCP test

Server Side:

xuanliu@server:~$ iperf -s -t 10 -i 1
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.20.10 port 5001 connected with 192.168.10.11 port 54611
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0- 1.0 sec  12.0 MBytes   101 Mbits/sec
[  4]  1.0- 2.0 sec  11.9 MBytes  99.6 Mbits/sec
[  4]  2.0- 3.0 sec  11.9 MBytes   100 Mbits/sec
[  4]  3.0- 4.0 sec  11.9 MBytes   100 Mbits/sec
[  4]  4.0- 5.0 sec  12.0 MBytes   101 Mbits/sec
[  4]  5.0- 6.0 sec  11.9 MBytes   100 Mbits/sec
[  4]  6.0- 7.0 sec  11.9 MBytes   100 Mbits/sec
[  4]  7.0- 8.0 sec  12.0 MBytes   101 Mbits/sec
[  4]  8.0- 9.0 sec  11.9 MBytes   100 Mbits/sec
[  4]  9.0-10.0 sec  12.0 MBytes   101 Mbits/sec
[  4]  0.0-10.1 sec   120 MBytes   100 Mbits/sec

Client Side:

xuanliu@client:~$ iperf -c 192.168.20.10 -t 10 -i 1
------------------------------------------------------------
Client connecting to 192.168.20.10, TCP port 5001
TCP window size: 23.5 KByte (default)
------------------------------------------------------------
[  3] local 192.168.10.11 port 54611 connected with 192.168.20.10 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  12.5 MBytes   105 Mbits/sec
[  3]  1.0- 2.0 sec  11.8 MBytes  98.6 Mbits/sec
[  3]  2.0- 3.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  3.0- 4.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  4.0- 5.0 sec  11.9 MBytes  99.6 Mbits/sec
[  3]  5.0- 6.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  6.0- 7.0 sec  11.9 MBytes  99.6 Mbits/sec
[  3]  7.0- 8.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  8.0- 9.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  9.0-10.0 sec  12.0 MBytes   101 Mbits/sec
[  3]  0.0-10.0 sec   120 MBytes   101 Mbits/sec
  • iperf End-to-End UDP Test, where the bandwidth is set as 1Mbps by default.

Server (192.168.20.10) Side:

xuanliu@server:~$ iperf -s -t 10 -i 1 -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:  224 KByte (default)
------------------------------------------------------------
[  3] local 192.168.20.10 port 5001 connected with 192.168.10.11 port 33345
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0- 1.0 sec   128 KBytes  1.05 Mbits/sec   0.090 ms    0/   89 (0%)
[  3]  1.0- 2.0 sec   128 KBytes  1.05 Mbits/sec   0.219 ms    0/   89 (0%)
[  3]  2.0- 3.0 sec   128 KBytes  1.05 Mbits/sec   0.058 ms    0/   89 (0%)
[  3]  3.0- 4.0 sec   128 KBytes  1.05 Mbits/sec   0.050 ms    0/   89 (0%)
[  3]  4.0- 5.0 sec   128 KBytes  1.05 Mbits/sec   0.038 ms    0/   89 (0%)
[  3]  5.0- 6.0 sec   129 KBytes  1.06 Mbits/sec   0.058 ms    0/   90 (0%)
[  3]  6.0- 7.0 sec   128 KBytes  1.05 Mbits/sec   0.079 ms    0/   89 (0%)
[  3]  7.0- 8.0 sec   128 KBytes  1.05 Mbits/sec   0.030 ms    0/   89 (0%)
[  3]  8.0- 9.0 sec   128 KBytes  1.05 Mbits/sec   0.089 ms    0/   89 (0%)
[  3]  9.0-10.0 sec   128 KBytes  1.05 Mbits/sec   0.029 ms    0/   89 (0%)
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec   0.028 ms    0/  893 (0%)

Client (192.168.10.11) Side:

xuanliu@client:~$ iperf -c 192.168.20.10 -t 10 -i 1 -u
------------------------------------------------------------
Client connecting to 192.168.20.10, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  224 KByte (default)
------------------------------------------------------------
[  3] local 192.168.10.11 port 33345 connected with 192.168.20.10 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   129 KBytes  1.06 Mbits/sec
[  3]  1.0- 2.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  2.0- 3.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  3.0- 4.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  4.0- 5.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  5.0- 6.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  6.0- 7.0 sec   129 KBytes  1.06 Mbits/sec
[  3]  7.0- 8.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  8.0- 9.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  9.0-10.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec
[  3] Sent 893 datagrams
[  3] Server Report:
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec   0.028 ms    0/  893 (0%)

The ping and iperf results showed that the communication between the client and the server were good. Since the client and the server are not directly connected, they can communicate with each other means that the OSPF routing worked properly.

The traceroute showed the hops from the client to the server, where the intermediate hops are router-1, router-2 and router-3. This information also verified the routing path between the client and the server.

6. Tracking Routing Table Updates

From the traceroute result above, we know that from client to server, the path is client --> router-1 --> router-2 --> router-3 --> server

Now let's try some failure events and see how many updates occurs at a particular router, and we select router-1 to view the updates, and trigger the failures at router-2.

  • Login to router-1 and start the script (i.e., xorp_log.sh) that periodically tracking the routing table updates. If there is no change, you will see No Change from the terminal. Here we run the script for 60 seconds. The script captures the routing table every 1 second.

xuanliu@router-1:~$ cd /local/xorp_run
xuanliu@router-1:/local/xorp_run$ /bin/bash xorp_log.sh 60 1
/local/xorp_run/logs/rt-change.csv
07-01-2014-18-28-28
Create the folder for routing table logs
/local/xorp_run/logs/xorp-rtable_07-01-2014-18-28-28.txt
07-01-2014-18-28-28 1404239308

No Change

No Change

No Change

......
  • Bring down the interface at router-2

We can make the link between router-1 and router-2 fail by bringing down the corresponding interface on router-2. While running xorp_log.sh at router-1, we disable the virtual interface at router-2, which is associated with the virtual link (192.168.1.0/24) to router-1. Use /sbin/ifconfig to find the name of that interface.

xuanliu@router-2:~$ /sbin/ifconfig
eth0      Link encap:Ethernet  HWaddr 02:6c:4e:0a:be:dc  
          inet addr:172.17.1.19  Bcast:172.31.255.255  Mask:255.240.0.0
          inet6 addr: fe80::6c:4eff:fe0a:bedc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1298 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1309 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:153645 (153.6 KB)  TX bytes:116549 (116.5 KB)
          Interrupt:25 

<snip>

eth2      Link encap:Ethernet  HWaddr 02:4a:cc:17:60:44  
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::4a:ccff:fe17:6044/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2551 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:127175180 (127.1 MB)  TX bytes:177712 (177.7 KB)
          Interrupt:26 

<snip>

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8368 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8368 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:895016 (895.0 KB)  TX bytes:895016 (895.0 KB)

For example, in the above the appropriate interface is eth2. So the command to bring down the interface to router-1 is:

xuanliu@router-2:$ sudo ifconfig eth2 down

We can see one routing table update occurred at router-1:

......

No Change

No Change
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!

No Change

....

When we bring up eth2 at router-2 again, we notice another routing table update at router-1 immediately. The command to bring the interface back up is:

xuanliu@router-2:$ sudo ifconfig eth2 up
  • Make router-2 fail.

As we mentioned before, to emulate a node failure, we can kill the XORP processes to disable the routing functionality at a router. We can use the command

ps -ef | grep xorp_ | /usr/bin/awk '{ if ( $1 == "root" ) {print "sudo kill -9 " $2}}' | sh

Now start xorp_log.sh again at router-1, and kill the XORP processes at router-2, the output from router-1 is

......

No Change
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!

No Change

No Change
......

At this time, we can run traceroute from client to server again, and we see that the path is changed: client --> router-1 --> router-4 --> router-3 --> server.

xuanliu@client:~$ traceroute 192.168.20.10
traceroute to 192.168.20.10 (192.168.20.10), 30 hops max, 60 byte packets
 1  router-1-lan4 (192.168.10.10)  0.668 ms  0.617 ms  0.584 ms
 2  router-4-lan3 (192.168.4.1)  1.279 ms  1.252 ms  1.220 ms
 3  router-3-lan2 (192.168.3.2)  1.753 ms  1.733 ms  1.700 ms
 4  server-lan5 (192.168.20.10)  2.680 ms  2.656 ms  2.624 ms

Rerun xorp_log.sh again at router-1, and restart XORP on router-2 to enable OSPF routing at router-2.

Start XORP by specifying the routing protocol configuration file.

xuanliu@router-2:~$ cd /usr/local/xorp/sbin/
xuanliu@router-2:/usr/local/xorp/sbin$ sudo ./xorp_rtrmgr -b /etc/xorp/ospfd.conf -l /tmp/xorp_rtrmgr_log -d

Verify the XORP process is running

xuanliu@router-2:/usr/local/xorp/sbin$ ps -ef | grep xorp
root      1972     1  0 13:30 ?        00:00:02 xorp_fea
root      1973     1  0 13:30 ?        00:00:00 xorp_rib
root      1974     1  0 13:30 ?        00:00:00 xorp_policy
root      1975     1  0 13:30 ?        00:00:01 xorp_ospfv2
root      1976     1  0 13:30 ?        00:00:00 ./xorp_rtrmgr -b /etc/xorp/ospfd.conf -l /tmp/xorp_rtrmgr_log -d

We see the OSPF routing table updates at router-1 this time! This is because when the XORP processes were started again, it sent OSPF hello messages to other routers to notify that router-2 was up, and the OSPF routing table was updated at each router afterwards.

......
No Change

No Change
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!

No Change
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!

No Change
Files /local/xorp_run/logs/tmp.txt and /local/xorp_run/logs/last_table.txt differ
routing table changed!

No Change

No Change
......

  • CSV File at router-1

For research purposes, we would like to mark down the timestamp for every second when we capture the routing table. xorp_log.sh actually generates a csv log file that stores the timestamp (in UTC) and routing table update information. To simplify the log format, we use 1 to represent There is an Update! , and 0 for No Update! . These csv files live in /local/xorp_run/logs/rt-change_*.csv. A sample csv file can be like:

router-1,1404240360,0
router-1,1404240362,1
router-1,1404240363,0
router-1,1404240365,1
router-1,1404240366,1
router-1,1404240368,0
router-1,1404240369,1
router-1,1404240371,0

In this way, we can plot figures for routing table updates over time.