Version 17 (modified by, 9 years ago) (diff)


Custom Experiment on InstaGENI with GEMINI

1. Goals

  • Configure the "User Workspace" VM
  • Start an InstaGENI slice
  • Modify the request RSpec to include experiment-specific functions
  • Use Omni to create slivers at the chosen aggregates
  • Use Flack to view and modify the experiment properties as well as view slice status
  • Instrument the slice with GEMINI and configure some measurements using the GENI Desktop
  • Configure nodes using the command line to setup the experiment
  • Run an experiment using scripts to control the node interface properties
  • View results and explore the GEMINI services

2. Experiment Overview

This custom experiment will make use of a WAN accelerator service called Phoebus. Phoebus can help transfer performance of TCP flows over long-distance, or "challeneged", network environments through path segmentation, protocol adaptation, and automatic protocol tuning. Phoebus Gateways (PGs) are typically installed on nodes bordering WAN link(s), segmenting the WAN path from the edge networks (e.g., campus or data centers). This experiment will create a simple 4-node topology over which to test the operation of Phoebus when used with a common network benchmark. To simulate WAN latencies and edge loss, we will use the netem (Network Emulator) module in Linux to modify how packates are handled across the experimental topology. GEMINI will be used to verify connectivity and to collect some performance data.

The image above shows the topology we will create on InstaGENI. We will make some assumptions about addresses assigned to each interface to help with the consistency of this tutorial.

client0 :

gateway0 : and

gateway1 : and

client0 :

This IP assignment will give us 3 distinct subnets over which to send traffic. (edge 0), (WAN), and (edge 1) Host routing will be necessary to allow IP fowarding across each subnet.

3. Setting up your user environment

Follow these instructions to configure Omni

Now we want to decrypt our private key so Omni won't have to repeatedly prompt for the key password.

4. Create the experiment slice

OUr experiment topology will be generated in Flack on InstaGENI racks at Kentucky and Utah.

Use the "Flack GENI Client" link in the bookmarks par to load the Flack interface.

Select UKY and Utah instageni AMs.

Create your topology as shown using VM nodes from either UKY or Utah.

Click on the "i" on each node to configure additional properties, such as the node name and the desired IP addresses.

To allow for GEMINI instrumentation, enable the GEMINI extensions in the right-most tab as shown in the screenshot. Selecting this extension will automatically create an additional node in your experiment for measurement configuration and collection.

To modify the request RSpec in the following steps, you need to save the RSpec to your local filesystem. Click on View->Request RSpec to display the XML representation of your topology and save to file as shown.

Now we do some editing of the request RSpec. Either download or save the following script in your working directory.



       <execute command='wget -q -P /tmp;sudo bash /tmp/' shell='sh'/>\n\

sed "/\/node\>/ i\ ${TEXT_ADD}" ${FILE}
$ ./ myslice.xml > myslice_phoebus.xml

The script adds extra service elements to each node that will allow a custom script to be executed when the node boots. In our case, this script will download and install the Phoebus software packages and other supporting scripts needed for the experiment. Your own experiments may have many such scripts, each of which is completely customizeable.

At this point our request RSpec is ready to go. We now want to interact with the GENI control framework to instantiate our experiment. In this tutorial, we will use Omni.

First, renew the slice created in Flack via Omni.

You now want to invoke "createsliver" to create the slivers at each AM involved in your experiment topology. In our case, this is either ig-utah or ig-ky as configured in your Omni configuration file.

After the request is submitted, Omni can be used to monitor the slice status. More conveniently, you can reload the slice in Flack and watch the progress until your slice display turns green.

5. Instrument the slice

Now that the slice is created, you may instrument the slice with GEMINI. The GENI Desktop provides this same functionality via a browser interface, but here we will use the "instrumentize" scripts available on the UW VM.

Note that you specify your slice name and the location of your SSL certificate when executing these scripts.

geni@GEC16-Tutorials:~/src/GEMINI/geminiv2$ ./ -n ezraslice2 -f ~/.ssl/geni_cert.pem
geni@GEC16-Tutorials:~/src/GEMINI/geminiv2$ ./ -n ezraslice2 -f ~/.ssl/geni_cert.pem

Once these scripts return, your slice has been instrumentized and will begin collecting default, passive measurements. We will visit the GENI Desktop in later sections, but while the slice is being instrumented, we can use this time time to login and configure our nodes for our experiment.

6. Login and configure experiment nodes

ssh to each of the nodes in the experiment. Become root with 'sudo su'.

We will now add host routes so the client nodes in the experiment can talk to each other across the GW systems.

On client0:

$ route add -net gw

On gateway0:

$ route add -net gw

On gateway1:

$ route add -net gw

On client1:

$ route ad -net gw

From client0, you should now be able to ping and traceroute to client1:

$ ping
PING ( 56(84) bytes of data.
64 bytes from icmp_req=1 ttl=62 time=0.054 ms
64 bytes from icmp_req=2 ttl=62 time=0.062 ms
64 bytes from icmp_req=3 ttl=62 time=0.067 ms
--- ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.054/0.061/0.067/0.005 ms
$ traceroute
traceroute to (, 30 hops max, 60 byte packets
 1  gateway0-lan0 (  0.049 ms  0.013 ms  0.011 ms
 2  gateway1-lan1 (  0.031 ms  0.017 ms  0.016 ms
 3 (  0.028 ms  0.022 ms  0.021 ms

7. Configure measurements

Login to the GENI Desktop using the bookmark in Firefox. link

Note that this step requires that you first login to your Slice Authority Emulab site.

Explore the desktop.

For our experiment, we'd also like to measure the latency across all of our nodes. We can use the active measurement configuration page to do so.

You may add new ping tests under "Schedul BLiPP Test".

8. Run experiments

9. View results

Attachments (17)

Download all attachments as: .zip