wiki:GeniTmixTutorial

Version 24 (modified by Ben Newton, 6 years ago) (diff)

--

Using Tmix on GENI

This page describes how to run experiments with Tmix on GENI nodes. This tutorial assumes that you are already familiar with basic experimentation with GENI, and that you have access to some version of Linux/Unix running on a PC or virtual machine. We also assume basic familiarity with Linux/Unix.

1 Install Omni

This tutorial will use Omni, which is a command line interface tool for reserving GENI resources. If you do not already have Omni installed and set up, please follow Step 1 (Configuring Omni with your GENI Account) here. Please note, it may take 30-45 minutes to get Omni installed and configured. Also note when installing Omni, the ./omni-configure.py script may need to be run from your gcf/src directory, and you may need to create the .ssl subdirectory in your home directory.

2 Extra Omni Configuration

2.1 Add gcf/src to path (optional but recommended)

Although optional, it will be helpful to add the gcf/src directory to your path and python path. To do this, replace "/path/to/gcf" in the lines below with the path to your copy of gcf, and add the lines to the end of your .profile file or equivalent.

PATH="/path/to/gcf/src:$PATH"
export PYTHONPATH="/path/to/gcf/src:$PATH"

Once you have added the lines and saved the file, run

source .profile

to setup the path environment variables for this session. This will be done automatically from now on.

If you don't make this change, you may need to add python in front of all the omni commands, and call them from within the gcf/src directory.

2.2 Clear Passphrases (optional but recommended)

We recommend that you clear the passphrase from your SSL certificate and/or your SSH key that you will use to login to your nodes. This step is also optional, however, if you don't clear your passphrases some commands will require you to enter your it several times, which can become burdensome. To clear the passphrases from your certificate and key run this script:

clear-passphrases.py 

3 A Simple Tmix Experiment

3.1 Reserve Resources

The following is the request rspec for a simple two node network. It requests two raw nodes named left and right, each with a single interface named left:if0 and right:if0 respectively. A link named left-right-lan connects these interfaces.

rspec here

Think of a unique name for your slice. We recommend the slice name begin with your GENI username. Create a new slice by issuing the following command with your unique slice name in place of <sliceName>.

omni.py createslice <sliceName>

Next, create a sliver with the resources requested in the rspec by issuing the following command where <sliceName> is again replaced with the unique slice name you have chosen.

omni.py -a pg-utah <sliceName> two-nodes-tmix.rspec

Note: this and future steps assume your ~/.gcf/omni_config includes a mapping from nickname pg-utah to https://www.emulab.net:12369/protogeni/xmlrpc/am. If pg-utah causes issues, you can use the full url in the -a option.

If the resources were successfully reserved the Result Summary from Omni should say: Got Reserved resources ...

Look back further in the output and you should see two lines which include "hostname=pc???.emulab.net". These are the hostnames of the machines reserved for your experiment. Make note of these hostnames for later use.

3.2 Log in to nodes

  1. Now, let's find out the resources advertised by this aggregate using omni.
    $  omni.py -a http://geni-in-a-box.net:8001 listresources
    INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    INFO:omni:Using control framework gib_ch
    INFO:omni:Listed resources on 1 out of 1 possible aggregates.
    INFO:omni:<?xml version="1.0" encoding="UTF-8"?>
    INFO:omni:  <!-- Resources at AM:
    	URN: unspecified_AM_URN
    	URL: http://geni-in-a-box.net:8001
     -->
    INFO:omni:  
    <!-- Resources at AM:
    	URN: urn:publicid:IDN+geni-in-a-box.net+authority+cm
    	URL: http://geni-in-a-box.net:8001
     -->
    <rspec expires="2013-12-20T22:26:14Z" type="advertisement" xmlns="http://www.geni.net/resources/rspec/3" 
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/ad.xsd" >
      <node component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc101" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" component_name="pc101" exclusive="true">    
          <hardware_type name="openVZ container">      
          </hardware_type>    
          <sliver_type name="virtual-pc">      
          <disk_image description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>      
          <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>      
          </sliver_type>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth0" role="control">      
              <emulab:interface name="eth0"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" role="experimental">      
              <emulab:interface name="eth2"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth2" role="experimental">      
              <emulab:interface name="eth3"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth3" role="experimental">      
              <emulab:interface name="eth1"/>      
          </interface>
          <location country="Molvania" />    
      </node>        
          ... 
      <node component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc106" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" component_name="pc106" exclusive="true">    
          <hardware_type name="openVZ container">      
          </hardware_type>    
          <sliver_type name="virtual-pc">      
          <disk_image description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>      
          <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>      
          </sliver_type>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth0" role="control">      
              <emulab:interface name="eth0"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth1" role="experimental">      
              <emulab:interface name="eth2"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth2" role="experimental">      
              <emulab:interface name="eth3"/>      
          </interface>    
          <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth3" role="experimental">      
              <emulab:interface name="eth1"/>      
          </interface>
          <location country="Molvania" />    
      </node> 
    </rspec>
    
    INFO:omni: ------------------------------------------------------------
    INFO:omni: Completed listresources:
    
      Options as run:
    		aggregate: ['http://geni-in-a-box.net:8001']
    		framework: gib_ch
    		getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    
      Args: listresources
    
      Result Summary: Queried resources from 1 of 1 aggregate(s).
     
    INFO:omni: ============================================================
    $ 
    

The output of the command is the advertisement rspec from the GENI-in-a-Box aggregate. You'll see the aggregate advertises six compute resources. Only two are shown here for readability reasons but you will find the complete advertisement rspec here. Look at the description of the first resource in the rspec. This compute resource is an openVZ container running Fedora 15 image (or an Ubuntu 10 image if you are using the Ubuntu flavor of GENI-in-a-Box). The resource has four network interfaces: eth0 through eth3. eth0 is a control interface and is not available to the experimenter. All six compute resources advertised by this aggregate are identical. You can therefore use this aggregate to create experiment topologies with up to six nodes and up to three links per node.

Notice the -a option to Omni that gave it the URL of the aggregate manager on which we wish to invoke list resources. To save us from having to type this URL every time we use an Omni command that acts on this aggregate manager, we have defined a nickname gib for this aggregate. This nickname is defined in the file ~/.gcf/omni_config. From now on we will use this nickname instead of the aggregate manager URL with Omni commands that require an aggregate URL.

2.2 Acquire and Configure Resources

Before we run an experiment we need to acquire the resources needed the experiment. Our introductory experiment will consist of two hosts connected to one another by a single link. When the hosts come up they execute a script provided by us that:

  • Downloads and installs iperf on each host.
  • Runs iperf in server mode on one host and and in client mode on the other host. The outputs of the iperf commands are written to /var/www/html/iperf-server.log and /var/www/html/iperf-client.log on the respective hosts.

As a part of this experiment we will ssh into these hosts to view these logs.

The request rspec for this experiment (and other examples of request rspecs) are in the directory ~/geni-in-a-box.

$ cd geni-in-a-box
$ ls
example-scripts    install-example.rspec  README.txt             two-nodes-iperf.rspec
four-nodes.rspec   islands.rspec          three-nodes-lan.rspec
$

We'll use the rspec file called two-nodes-iperf.rspec. You can view it here or inside the VM. This rspec requests two hosts called left and right. Host left has one interface called left:if0 and host right has one interface called right:if0. The two interfaces are connected by a link called left-right-lan. The following figure illustrates the resources requested by this rspec.

No image "ResourcesRequested.png" attached to GeniTmixTutorial

The <install> element in the rspec asks the GENI-in-a-Box aggregate manager to download a script from the specified URL, uncompress and untar it, and save it in the /local directory. The <execute> element asks the aggregate to execute the specified script when the hosts come up. In this case the script to be executed is the one that was installed in /local.

The following steps will walk us through acquiring the resources for our experiment:

  1. First we need to create a slice using the command omni.py createslice myslice where myslice is the name or our slice (of course you can give your slice some other name).
    $ omni.py createslice myslice
    INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    INFO:omni:Using control framework gib_ch
    INFO:omni:Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
    INFO:omni: ------------------------------------------------------------
    INFO:omni: Completed createslice:
    
      Options as run:
    		framework: gib_ch
    		getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    
      Args: createslice myslice
    
      Result Summary: Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
     
    INFO:omni: ============================================================
    $
    
  1. We now send our request rspec to the GENI-in-a-Box aggregate manager using the command omni.py -a http://geni-in-a-box.net:8001 createsliver myslice two-nodes-iperf.rspec. If our request is successful, the aggregate manager will allocate the requested resources to our slice. In GENI parlance the resources allocated to our slice are called slivers. The createsliver request returns a manifest spec that provides details of the resources allocated to us.

This command will take many minutes to complete because it configures and boots up the requested virtual machines; does some additional configuration; and download, installs, and runs iperf.

The following is a snippet of the manifest your will get back from createsliver:

$ omni.py -a gib createsliver myslice two-nodes-iperf.rspec
INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
INFO:omni:Using control framework gib_ch
WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file two-nodes-iperf.rspec for slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice
INFO:omni:Got return from CreateSliver for slice myslice at http://geni-in-a-box.net:8001:
INFO:omni:<?xml version="1.0" ?>
INFO:omni:  <!-- Reserved resources for:
	Slice: myslice
	at AM:
	URN: unspecified_AM_URN
	URL: http://geni-in-a-box.net:8001
 -->
INFO:omni:  <rspec type="manifest" xmlns="http://www.geni.net/resources/rspec/3" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.geni.net/resources/rspec/3   http://www.geni.net/resources/rspec/3/manifest.xsd">  
    <node client_id="left" component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc101" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" exclusive="false" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+101">    
        <sliver_type name="virtual-pc">      
            <disk_image name="urn:publicid:geni-in-a-box.net+image+//FEDORA15-STD" os="Linux" version="10"/>      
        </sliver_type>    
        <services>      
            <execute command="/local/iperf-script.sh" shell="sh"/>      
            <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/geni-in-a-box/iperf-script.tar.gz"/>      
            <login authentication="ssh-keys" hostname="pc101.geni-in-a-box.net" port="22" username="gibuser"/>      
        </services>    
        <interface client_id="left:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" mac_address="00:0C:29:B4:DF:A7" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+1011">      
            <ip address="10.0.3.101"/>      
        </interface>    
        <rs:vnode name="pc101" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>    
        <host name="pc101.geni-in-a-box.net"/>    
    </node>  
    ...  
    <link client_id="left-right-lan">    
        <component_manager name="urn:publicid:IDN+geni-in-a-box.net+authority+cm"/>    
        <interface_ref client_id="left:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" sliver_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1"/>    
        <interface_ref client_id="right:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc102:eth1" sliver_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc102:eth1"/>    
        <property dest_id="right:if0" source_id="left:if0"/>    
        <property dest_id="left:if0" source_id="right:if0"/>    
        <link_type name="lan"/>    
    </link>  
</rspec>
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:

  Options as run:
		aggregate: ['gib']
		framework: gib_ch
		getversionCacheName: /home/gibuser/.gcf/get_version_cache.json

  Args: createsliver myslice two-nodes-iperf.rspec

  Result Summary: Got Reserved resources RSpec from geni-in-a-box-net-8001 
INFO:omni: ============================================================
]0;gibuser@GiB:~/geni-in-a-box[gibuser@GiB geni-in-a-box]$

Notice the manifest rspec is essentially your request rspec annotated with information about the resources that got allocated to you. For example, the <node> element of the manifest includes the MAC and IP addresses assigned to each network interface on the node (host) and information you will need to ssh into the nodes (e.g. the fully qualified domain name of the host ). The following figure illustrates the resource allocation made by the GENI-in-a-Box aggregate, as described in the manifest spec.

No image "ResourcesAllocated.png" attached to GeniTmixTutorial

2.3 Using your Resources

You can now ssh into the hosts allocated to you and do what you need to do to get your experiment working. For example, you can install and run software. If needed, you can use the sudo command to do things that require root privileges.

In the case of our experiment, the software needed (iperf) was installed by our startup script. Our startup script put the output of iperf in a file accessible from a web browser. Since our iperf server ran on node pc101 (the node labeled 'left' in the request spec), we can use a browser to view the output of this client. Start up Firefox in a terminal on the GENI-in-a-Box virtual machine and enter the url http://pc101.geni-in-a-box.net/iperf-server.log. The iperf server output should appear in your browser window. You can view the iperf client output at http://pc102.geni-in-a-box.net/iperf-client.log.

You can also ssh into the node pc101 or pc102 to view these log files. Your username on GENI-in-a-Box (gibuser) is given an account on the nodes allocated to your experiment and the public key for this user (~/.gcf/gibuser.pub is copied into gibuser's .ssh directory in each of these nodes. The private key for gibuser is in ~/.ssh/gibuser and must be provided to the ssh command.

$ ssh -i ~/.ssh/gibuser pc101
The authenticity of host 'pc101 (10.0.1.101)' can't be established.
RSA key fingerprint is e3:7c:8a:21:16:05:93:fd:dd:56:1c:f5:5c:5b:f3:c7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'pc101,10.0.1.101' (RSA) to the list of known hosts.

[gibuser@left ~]$ cd /var/www/html
[gibuser@left html]$ ls
iperf-server.log
[gibuser@left html]$ cat iperf-server.log 
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 10.0.3.101 port 5001 connected with 10.0.3.102 port 60542
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec  61.8 MBytes  51.5 Mbits/sec
[gibuser@left html]$

You can have the GENI-in-a-Box aggregate create additional user accounts on your experiment nodes; see this page for instructions.

2.4. Freeing up Resources

When you are done with your experiment you can return the resources allocated to you using omni's deletesliver command:

$ omni.py -a gib deletesliver myslice
INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
INFO:omni:Using control framework gib_ch
WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
INFO:omni:Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at http://geni-in-a-box.net:8001
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed deletesliver:

  Options as run:
		aggregate: ['gib']
		framework: gib_ch
		getversionCacheName: /home/gibuser/.gcf/get_version_cache.json

  Args: deletesliver myslice

  Result Summary: Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at http://geni-in-a-box.net:8001 
INFO:omni: ============================================================
$

Congratulations! You have successfully run an experiment using GENI-in-a-Box resources!

3 GENI-in-a-Box Tips

See this page for tips that you might find helpful while using GENI-in-a-Box.

4 What's Next?

You are now ready to run your own experiments inside GENI-in-a-Box. You might want to start with one of the sample rspecs in the ~/geni-in-a-box directory and edit it as needed by your experiment. See the README file in this directory for a description of the rspecs in this directory.

You can apply for a GENI account and use real GENI aggregates and resources for your experiments. See GiB-to-ProtoGENI for tips on moving your experiment from GENI-in-a-Box to other aggregates (and vice versa).