wiki:GiB-to-ProtoGENI

Version 16 (modified by lnevers@bbn.com, 12 years ago) (diff)

--

Moving Experiments from GENI-in-a-Box to ProtoGENI Aggregates

The GENI-in-a-Box aggregate is most similar to the GENI ProtoGENI aggregates. This page describes some of the similarities and differences between these aggregates.

Similarities

  • All experimenter nodes have 3 NICs available to the experimenter. These is, in addition, a fourth NIC that is connected to by the aggregate control network. This network should not be used for experiment traffic.
  • Experimenters get root access to their virtual machines.
  • Virtual machines from both aggregates are implemented as OpenVZ containers running Fedora 15. The Ubuntu flavor of GENI-in-a-Box provides OpenVZ containers running Ubuntu.
  • The manifest rspec is available in all the experimenter nodes at /proj/geni-in-a-box.net/exp/<slicename>/tbdata/geni_manifest and the fully qualified name of the node is available in /var/emulab/boot/nickname. An example of a fully qualified node name is left.myslice.geni-in-a-box.net where myslice is the name of the slice.

Differences

  • Mapping of device name to experimenter specified interface name. The mapping of the experimenter specified interface name to the operating system device name is available in the manifest rspec. For example, the <interface> element in the snippet of a manifest rspec shown below tells us that the interface with the experimenter specified name left:if0 is mapped to eth1 by the operating system (see the last part of the component_id for this interface). In the ProtoGENI aggregate this mapping is not guaranteed to be correct i.e. it might actually be mapped to a different operating system interface such as eth2 or eth3. With GENI-in-a-Box the mapping will be correct.
    <node client_id="left" component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc101" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" exclusive="false" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+101">    
            <sliver_type name="virtual-pc">      
                <disk_image name="urn:publicid:geni-in-a-box.net+image//FEDORA15-STD"/>      
            </sliver_type>    
            <services>      
                <install file_type="tar.gz" install_path="/local" url="http://groups.geni.net/geni/attachment/wiki/GiB-gettingStarted/iperf-execute-script.tar.gz?format=raw"/>      
                <execute command="/local/iperf-execute-script.sh" shell="sh"/>      
                <login authentication="ssh-keys" hostname="pc101.geni-in-a-box.net" port="22" username="alice"/>      
            </services>    
            <interface client_id="left:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" mac_address="00:0C:29:B4:DF:A7" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+1011">      
                <ip address="10.0.3.101"/>      
            </interface>    
            <rs:vnode name="pc101" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>    
            <host name="pc101.geni-in-a-box.net"/>    
    </node>  
    
  • Aggregate URN in rspec. As you might expect, the aggregate URN in a GENI-in-a-Box rspec will be different from that in a ProtoGENI rspec. However, you can use your ProtoGENI rspec, with the URN for a ProtoGENI aggregate, in your GENI-in-a-Box rspec. The GENI-in-a-Box aggregate manager does not check the aggregate URN in the rspec. This is a feature and not a bug. It is so you can use your ProtoGENI rspec with GENI-in-a-Box with very little or no changes.
  • Running commands as root in startup scripts. The startup script specified by the experimenter as an <execute> element of the request rspec is executed with root privileges in the GENI-in-a-Box aggregate. You therefore do not require to sudo to run commands that require root privileges. In fact, such scripts with sudo in them will fail as the VMs are set up to disallow sudo'ing without a tty.

Moving your experiment to ProtoGENI

This section highlights the process of moving an experiment from the GENI-in-a-Box aggregate running on a virtual machine to the ProtoGENI aggregate using your own system to run the experiment.

Get GENI Credentials

GENI Experimenters can get credentials by signing up for existing projects or by requesting new projects. These options are highlighted in the GENI Experimenter Sign Me Up page.

Install and use Omni

Omni is the tool used to manage GENI resources and it is part of the gcf package.

  1. The gcf package has dependencies that can be installed as detailed on the GCF Software Dependencies page.
  1. Get the latest GCF package release at the GPO Software Release page, or the latest development version of the GCF package using git as detailed in the Getting GCF page.
  1. Unpack the gcf tgz file and configure as detailed in the How to configure Omni page.
  1. You should add the following to your PATH and PYTHONPATH:
     gcf-version=gcf-2.1
     export PATH="$HOME/$gcf-version/src:$PATH"
     export PYTHONPATH=~/$gcf-version/src/
    

  1. You should now be able to use Omni to request information from the ProtoGENI aggregate. Note that the omni configuration includes a nickname for the ProtoGENI aggregate at Utah, pg-utah:
    $ omni.py -a pg-utah listresources -o 
    INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
    INFO:omni:Using control framework pg
    INFO:omni:Saving output to a file.
    INFO:omni:Substituting AM nickname pg-utah with URL https://www.emulab.net/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
    INFO:omni:Listed resources on 1 out of 1 possible aggregates.
    INFO:omni:Writing to 'rspec-www-emulab-net-protogeniv2.xml'
    INFO:omni: ------------------------------------------------------------
    INFO:omni: Completed listresources:
    
      Options as run:
    		aggregate: ['pg-utah']
    		framework: pg
    		getversionCacheName: /home/lnevers/.gcf/get_version_cache.json
    		output: True
    
      Args: listresources
    
      Result Summary: Queried resources from 1 of 1 aggregate(s).
    Wrote rspecs from 1 aggregate(s) to 1 file(s)
    Saved listresources RSpec at 'unspecified_AM_URN' to file rspec-www-emulab-net-protogeniv2.xml;  
    INFO:omni: ============================================================
    

Note: The command runs for several minutes, and the content of the listresources output file rspec-www-emulab-net-protogeniv2.xml is used later in this procedure to define the ProtoGENI RSpec.

Modify your GiB RSpec for use at ProtoGENI

In this section we will modify the GENI-in-a-Box RSpec named "three-nodes-lan.rspec" to be used at ProtoGENI. Edit the file "three-nodes-lan.rspec" to introduce the modification needed:

  1. Replace all instances of pgeni.gpolab.bbn.com with emulab.net
  2. Replace all instances of the image tag with an image available at ProtoGENI. The following line: <disk_image name="urn:publicid:IDN+pgeni.gpolab.bbn.com+image+emulab-ops:UBUNTU1004-STD" os="Linux" version="Ubuntu10.04"/> is replaced by

<disk_image description="Ubuntu 10 32-bit" name="urn:publicid:IDN+emulab.net+image+emulab-ops:UBUNTU10-STD" os="Linux" version="10"/> .

Note: You may chose a different image, simply run look for "disk_images" in the rspec-www-emulab-net-protogeniv2.xml generated above.

Run the experiment at ProtoGENI

  1. You may now create a slice and a sliver at the ProtoGENI aggregate using the modified RSpec file.
    $ omni.py createslice myslice
    $ omni.py createsliver -a pg-utah myslice three-nodes-lan-mod.rspec
    
  1. You get the information to login to the nodes with one of the gcf scripts:
$ gcf/examples/readyToLogin.py -a pg-utah myslice

 .....
================================================================================
LOGIN INFO for AM: https://www.emulab.net/protogeni/xmlrpc/am/2.0
================================================================================

middle's geni_status is: ready (am_status:ready) 
User lnevers logins to middle using:
	xterm -e ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc154.emulab.net &


right's geni_status is: ready (am_status:ready) 
User lnevers logins to right using:
	xterm -e ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc99.emulab.net &


left's geni_status is: ready (am_status:ready) 
User lnevers logins to left using:
	xterm -e ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc116.emulab.net &

You can now log in to the nodes and run your experiment!