Version 43 (modified by 10 years ago) (diff) | ,
---|
A Tutorial on Systematic Experimental Design
1. Software Configuration
Once we have installed the software on the VM, we may need to configure the software, and automate the configuration process for every node used in an experiment. For this tutorial, since we are setting a virtual routing environment, we will need to configure the routing protocol (e.g., OSPF protocol) for XORP
. Again, let's start with the smallest reasonable topology, which is a four-router virtual topology as shown below. The reason that we chose the four-router topology is that we need at least one pair of routers that are not directly connected to each other and each router must have at least two virtual interfaces.
a. Create Virtual Network
As we have mentioned, a four-router topology is the smallest topology to configure and verify the OSPF
routing. In this part, we illustrate how to create a virtual network with four virtual routers.
i. Create VMs and Virtual Links
Create four VMs from the same InstaGENI aggregate and add the links. Modify each VM's name, set the Sliver Type to emulab-xen
, and load the custom image by following Load Custom Image.
The network configuration is given below:
router-1 ----- router-2: 192.168.1.0/24 router-2 ----- router-3: 192.168.2.0/24 router-3 ----- router-4: 192.168.3.0/24 router-4 ----- router-1: 192.168.4.0/24 router-2 ----- router-4: 192.168.5.0/24
ii. Configure Virtual Interfaces
Configure the virtual interfaces' IP addresses of each link by clicking at the link icon.
After configuring all the links, submit the request at the bottom of your canvas.
iii. Save the Current RSpec
For version control purpose, It is important to save (or export) the user request RSpec at each step. Thus, when all VMs are ready for login, we save a copy of the current RSpec as 4node-v1.rspec
.
We can do so by clicking the View and choosing Preview requested document(s) from the pull-down list. You will see the RSpec generated by Flack looks like the following:
<rspec type="request" generated_by="Flack" generated="2014-08-07T22:23:33Z" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/request.xsd " xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.geni.net/resources/rspec/3"> <node client_id="router-2" component_manager_id="urn:publicid:IDN+instageni.stanford.edu+authority+cm" exclusive="false" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"> <rs:vnode name="pcvm3-13" xmlns="http://www.protogeni.net/resources/rspec/ext/emulab/1"/> <sliver_type name="emulab-xen"> <disk_image url="https://www.instageni.clemson.edu/image_metadata.php?uuid=5417bcef-224e-11e4-aa5a-000000000000"/> </sliver_type> <interface client_id="xen:if0"> <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"/> <flack:interface_info addressBound="false" bound="false"/> </interface> <interface client_id="xen:if1"> <ip address="192.168.2.1" netmask="255.255.255.0" type="ipv4"/> <flack:interface_info addressBound="false" bound="false"/> </interface> <interface client_id="xen:if2"> <ip address="192.168.5.1" netmask="255.255.255.0" type="ipv4"/> <flack:interface_info addressBound="false" bound="false"/> </interface> <flack:node_info x="422" y="170" unbound="true"/> </node>
Choose Save to file to export the RSpec. In this tutorial, we assume the file is saved to 4node-v1.rspec
, but you can choose any name as you like. (You can download a sample RSpec for 4node-v1.rspec
at here)
b. Manually Configure OSPF for XORP
In this tutorial, we configure each VM as an virtual router running OSPF
routing protocol. Examples of XORP
configurations can be found at XORPExamples. For each VM running as a software router, the OSPF
configuration file will be named ospfd.conf
and stored in /etc/xorp/
.
We first configure OSPF
on router-1
. The sample configuration file for router-1
can be downloaded from here, where you may find the interface information in the sample file may be different from router-1
's interfaces.
It's highly recommended that make sure you double check whether you have the interface information correct in the ospfd.conf
on router-1
before you start xorp
.
To do that create the /etc/xorp
directory:
sudo mkdir /etc/xorp
Then create and edit the file using your favorite editor:
sudo vi /etc/xorp/ospfd.conf
There are several key information we observed by manually creating the OSPF
configuration file for router-1
.
- Interface Information
interface eth1 { description: "virtual interface-eth1" disable: false discard: false vif eth1 { disable: false address 192.168.4.2 { prefix-length: 24 disable: false } } } interface eth2 { ...... }
OSPF
Metricsprotocols { ospf4 { router-id: 192.168.1.1 area 0.0.0.0 { area-type:"normal" interface eth1 { vif eth1 { address 192.168.4.2 { interface-cost 10 hello-interval 5 router-dead-interval 10 disable: false } } } interface eth2 { ...... } } } }
To simplify the configuration process, we assume the interface-cost
is 10 and the hello-interval
for every virtual interface. So the main information we need from each node is the virtual interface's address
and its prefix-length
, as well as the router's router-id
that is required for the protocol
section in the configuration file.
Similarly, we manually create ospfd.conf
under /etc/xorp/
for the other three virtual routers (see sample ospfd.conf
files for `router-2`, `router-3`, and `router-4`).
c. Start XORP
and Verify the Routing
i. Start XORP
Process
First, we need to add xorp
to the group. Login to router-1
and run the following command:
sudo groupadd xorp
Start XORP
by specifying the routing protocol configuration file.
xuanliu@router-1:~$ cd /usr/local/xorp/sbin/ xuanliu@router-1:/usr/local/xorp/sbin$ xuanliu@router-1:/usr/local/xorp/sbin$ sudo ./xorp_rtrmgr -b /etc/xorp/ospfd.conf -l /tmp/xorp_rtrmgr_log -d
Verify the XORP
process is running
xuanliu@router-1:/usr/local/xorp/sbin$ ps -ef | grep xorp root 1972 1 0 13:30 ? 00:00:02 xorp_fea root 1973 1 0 13:30 ? 00:00:00 xorp_rib root 1974 1 0 13:30 ? 00:00:00 xorp_policy root 1975 1 0 13:30 ? 00:00:01 xorp_ospfv2 root 1976 1 0 13:30 ? 00:00:00 ./xorp_rtrmgr -b /etc/xorp/ospfd.conf -l /tmp/xorp_rtrmgr_log -d
Then Start XORP
for router-2
, router-3
and router-4
in the same way.
If you need to stop XORP
for some reason, there is no one-key method to stop XORP
process, so we have to kill every XORP
-related process. We can use the command like the one below to kill all XORP
process at one time.
ps -ef | grep xorp_ | /usr/bin/awk '{ if ( $1 == "root" ) {print "sudo kill -9 " $2}}' | sh
ii. Verify the Routing
Login to router-1
and view the OSPF
routing table:
xuanliu@router-1:/usr/local/xorp/sbin$ sudo ./xorpsh Welcome to XORP v1.8.5 on router-1.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu Version tag: 00000000 Build Date: 2014-06-05 17:11 64-bit root@router-1.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu> show route table ipv4 unicast ospf 192.168.2.0/24 [ospf(110)/20] > to 192.168.1.2 via eth2/eth2 192.168.3.0/24 [ospf(110)/20] > to 192.168.4.1 via eth1/eth1 192.168.5.0/24 [ospf(110)/20] > to 192.168.1.2 via eth2/eth2
Login to router-3
and view the OSPF
routing table:
xuanliu@router-3:/usr/local/xorp/sbin$ sudo ./xorpsh Welcome to XORP v1.8.5 on router-3.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu Version tag: 00000000 Build Date: 2014-06-05 17:11 64-bit root@router-3.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu> show route table ipv4 unicast ospf 192.168.1.0/24 [ospf(110)/20] > to 192.168.2.1 via eth2/eth2 192.168.4.0/24 [ospf(110)/20] > to 192.168.3.1 via eth1/eth1 192.168.5.0/24 [ospf(110)/20] > to 192.168.2.1 via eth2/eth2 root@router-3.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu>
Since router-1
and router-3
are not directly connected, by OSPF
routing, we should be able to run ping
and traceroute
between them.
xuanliu@router-3:/usr/local/xorp/sbin$ ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. 64 bytes from 192.168.1.1: icmp_req=1 ttl=63 time=2.15 ms 64 bytes from 192.168.1.1: icmp_req=2 ttl=63 time=1.45 ms 64 bytes from 192.168.1.1: icmp_req=3 ttl=63 time=1.41 ms ^C --- 192.168.1.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.411/1.673/2.152/0.342 ms xuanliu@router-3:/usr/local/xorp/sbin$ traceroute 192.168.1.1 traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 60 byte packets 1 router-2-lan1 (192.168.2.1) 0.652 ms 0.595 ms 0.563 ms 2 router-1-lan0 (192.168.1.1) 1.146 ms 1.106 ms 1.076 ms xuanliu@router-3:/usr/local/xorp/sbin$
d. Automate OSPF
Configuration
By understanding the syntax of the OSPF
configuration file for XORP
, we are able to automate the OSPF
configuration for every virtual router in a topology, assuming these virtual routers are identical. You can download the automation scripts from XORPAutoStart.
i. Test the Automation
To test it, you will need to login to each router (e.g., router-3
) and run the start-xorp.sh
. You will see the output similar to the following:
dhcp89-69-127:~ xliu$ ssh -p 30781 -i /Users/xliu/.ssh/id_rsa xuanliu@pc3.instageni.stanford.edu Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-56-generic x86_64) * Documentation: https://help.ubuntu.com/ Last login: Mon Jun 30 14:56:51 2014 from 128.89.69.127 xuanliu@router-3:~$ cd /local xuanliu@router-3:/local$ sudo wget http://geni-myvini.umkc.gpeni.net/resources/experiments/xorp_autostart.tar.gz xuanliu@router-3:/local$ sudo tar -zxvf xorp_autostart.tar.gz xuanliu@router-3:/local/xorp_autostart$ /bin/bash start-xorp.sh router-3.xuan-wiki-prep.ch-geni-net.instageni.stanford.edu 2014-07-01 06:56:28 AM XORP dir exist xorp is already added to the group xorp is running, stop it first XORP is starting ...... Not doing pidfile...
The key procedure is to obtain the virtual interface's address
and its prefix
of the subnet information.
In the install script start-xorp.sh
, the Following components are the key parts to generate ospfd.conf
.
- Get Hostname and virtual interface information, and store it into a file (
vm_info.txt
)hostname | sudo tee vm_info.txt > /dev/null /sbin/ifconfig | egrep 'eth|inet addr' | sudo tee -a vm_info.txt > /dev/null
A samplevm_info.txt
is available at SampleVMInfo
- Create
ospfd.conf
by running anawk
script, available from heresudo /usr/bin/awk -f $ABS_PATH/ospfd-conf-gen.awk $vm_info_file "$timestamp" 24 | sudo tee $ABS_PATH/ospfd.conf > /dev/null
A sampleospfd.conf
is available at SampleOSPFConf
- Start
XORP
Process$sudo ./xorp_rtrmgr -b /etc/xorp/ospfd.conf -l /tmp/xorp_rtrmgr_log -d
The complete shell script to wrap up the above procedures is available at StartXORP
Please go to HOWTOWriteInstallScript for details on how to write an install script.
|
Next we are going to install the automate script as an install script at the postboot stage.
If you plan to use the same slice to do the next step, please delete your existing resources in your slice at this time, otherwise you can create a new slice to continue. |
e. Install Script
Launch Flack for your slice, and import the RSpec 4node-v1.rspec
we just saved.
Click Import and select Import from file from the pull-down list, then select 4node-v1.rspec
from the location where you have just saved to, and confirm it. You will see the four-node topology on your canvas, and each node has been configured to have the custom image.
Now, we are ready to add an install script by editing the node property. For each node, click to see the node information. By clicking , we are able to add a service that usually refers to an install script. In this tutorial, we will add an install script called xorp_autostart.tar.gz
, by entering the link http://geni-myvini.umkc.gpeni.net/resources/experiments/xorp_autostart.tar.gz, and the install path is set as /local. By clicking , we can add the commend that execute the install script. In the execute box, we enter /bin/bash /local/xorp_autostart/start-xorp.sh
, and the shell type is sh
.
Once we added the service and the execute service in Flack, apply the changes and submit the VM reservation request, the OSPF configuration file will be automatically created and the XORP will be started at router-1
once it is initialized.
Repeat the same procedures for the other three routers (i.e., router-2
, router-3
, router-4
), and then submit the request. When all resources are ready, Save the RSpec as 4node-v2.rspec
at this stage, we will load this RSpec in the next step to get the same setup. (You can download a sample RSpec for 4node-v2.rspec
at here)
Setting up the install script will automate installation or configuration process at every node's postboot stage, and we do not need to login to every node to run start-xorp.sh
manually.
2. Impact to Request RSpec
Whenever we make a change to node information or link information in Flack, the user request RSpec is modified accordingly. In other words, we can rely on Flack to easily modify the request RSpec.
Now let's take a look at how the steps Load a Custom Image and Adding Install Scripts will modify the request RSpec. For instance, before adding the custom image and install scripts, the RSpec describing router-1
was like:
<node client_id="router-1" component_manager_id="urn:publicid:IDN+instageni.stanford.edu+authority+cm" component_id="urn:publicid:IDN+instageni.stanford.edu+node+pc3" component_name="pc3" exclusive="false"> <vnode name="pcvm3-3" xmlns="http://www.protogeni.net/resources/rspec/ext/emulab/1"/> <location latitude="37.43" longitude="-122.17" country="US"/> <interface client_id="xen-0:if0"> <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"/> <flack:interface_info addressBound="true" bound="false"/> </interface> <interface client_id="xen-0:if1"> <ip address="192.168.4.2" netmask="255.255.255.0" type="ipv4"/> <flack:interface_info addressBound="true" bound="false"/> </interface> <flack:node_info x="150" y="407" unbound="true"/> </node>
First, when we added the custom image (see Load a Custom Image), a new tag <sliver>
will be added to router-1
's RSpec, which is
<sliver_type name="emulab-xen"> <disk_image url="https://www.instageni.clemson.edu/image_metadata.php?uuid=5417bcef-224e-11e4-aa5a-000000000000"/> </sliver_type>
Secondly, when we added the install scripts (see Adding Install Scripts), a new tag <service>
will be added to router-1
's RSpec, which is
<services> <execute command="/bin/bash /local/xorp_autostart/start-xorp.sh" shell="sh"/> <install install_path="/local" url="http://geni-myvini.umkc.gpeni.net/resources/experiments/xorp_autostart.tar.gz"/> </services>
This indicates that we can add custom image and postboot scripts by directly modifying the request RSpec, this is helpful when we have a larger topology.
3. Script to Capture Routing Table Periodically
In this tutorial, one metric we would like to evaluate is how often the routing table will be updated when there is a link/node failure. To emulate a link failure, we can disable the virtual interfaces, and to emulate a virtual router failure, we can stop the XORP process at the virtual router. A short script (downloaded from here is available to be loaded as an install script.
Delete any existing resources or create a new slice, LOAD the RSpec 4node-v2.rspec
, and add a new install service to every node in Flack. Now, the service portion for each node becomes to
<services> <execute command="/bin/bash /local/xorp_autostart/start-xorp.sh" shell="sh"/> <install install_path="/local" url="http://geni-myvini.umkc.gpeni.net/resources/experiments/xorp_autostart.tar.gz"/> <install install_path="/local" url="http://geni-myvini.umkc.gpeni.net/resources/experiments/xorp_run.tar.gz"/> </services>
After adding the new install script to every node, send the resource reservation request. When all nodes are ready to login, we save the new modified RSpec as 4node-v3.rspec
. (You can download a sample RSpec for 4node-v3.rspec
at here)
In the next step, we will add two end hosts to current four-node topology, so now we can delete existing resources from the current slice.
4. Add End Hosts and Install iperf
Once we have verified the routing functions for the virtual router, we add end hosts to validate the end-to-end communication. As the figure shows below, we create the same four-node topology by loading 4node-v3.rspec
in Flack, and create two more VMs as a client and a server, and add two virtual links. For client and server, we select regular Ubuntu 12.04
OS in the node information, so we need to install iperf
separately. In this example, there are only two hosts need iperf installed, so one option is to login to each host, and manually install it by running
sudo apt-get install iperf
Or, we can also write a script to install iperf
to the client and server, load it as an install script by following the same steps as we just did for XORP. The install scripts for client and server can be downloaded from here. An sample RSpec with iperf install script and execute command can be downloaded at `4node-v4.rspec`.
SAVE the new RSpec to 4node-v4.rspec
.
Attachments (27)
- 4node-config.png (23.4 KB) - added by 10 years ago.
- 4node-topo.png (54.8 KB) - added by 10 years ago.
- info.jpg (3.1 KB) - added by 10 years ago.
- add-service-btn.png (5.4 KB) - added by 10 years ago.
- add-exe-serv-btn.png (6.0 KB) - added by 10 years ago.
- node-install-service.png (99.1 KB) - added by 10 years ago.
- r1-ospf.conf (1.3 KB) - added by 10 years ago.
- r2-ospf.conf (1.7 KB) - added by 10 years ago.
- r3-ospf.conf (1.3 KB) - added by 10 years ago.
- r4-ospf.conf (1.7 KB) - added by 10 years ago.
- ospfd-conf-gen.awk (2.2 KB) - added by 10 years ago.
- vm_info.txt.example (490 bytes) - added by 10 years ago.
- ospfd.conf.example (1.3 KB) - added by 10 years ago.
- start-xorp.sh (2.4 KB) - added by 10 years ago.
- initial_install.sh (388 bytes) - added by 10 years ago.
- export-to-file.jpg (63.6 KB) - added by 10 years ago.
- import-rspec-from-file.png (28.5 KB) - added by 10 years ago.
- 4node-add-endhosts.png (45.1 KB) - added by 10 years ago.
- lan-config.png (46.5 KB) - added by 10 years ago.
- 4node-v1.rspec (40.2 KB) - added by 10 years ago.
- 4node-v2.rspec (41.6 KB) - added by 10 years ago.
- 4node-v4.rspec (51.9 KB) - added by 10 years ago.
- 4node-v3.rspec (43.2 KB) - added by 10 years ago.
- SmallTopo.jpg (65.8 KB) - added by 9 years ago.
- ConfigIF.jpg (91.8 KB) - added by 9 years ago.
- SaveRSpec.jpg (63.7 KB) - added by 9 years ago.
- ReserveResources.jpg (17.5 KB) - added by 9 years ago.
Download all attachments as: .zip