Changes between Version 1 and Version 2 of GeniTmixTutorial


Ignore:
Timestamp:
02/27/13 10:37:49 (11 years ago)
Author:
Ben Newton
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GeniTmixTutorial

    v1 v2  
    1 Placeholder for GeniTmixTutorial
     1[[PageOutline(2-3, Table of Contents)]]
     2
     3= Using Tmix on GENI =
     4
     5This page describes how to run experiments with [wiki:GeniTmix] on GENI nodes.
     6
     7
     8== 1 Install GENI-in-a-Box ==
     91. Start up !VirtualBox (Version 4.1.23 or later) and from the !VirtualBox "File" menu, select "Import Appliance...".
     10
     112. The !VirtualBox Appliance Wizard will start up and guide you through the import of the GENI-in-a-Box virtual machine image you downloaded.  Please accept the default VM settings during the import.
     12
     133. When the import is done, and entry for GENI-in-a-Box will appear in the Oracle VM !VirtualBox Manager window.  Select this entry and click the green arrow labeled Start at the top of this window.  A new window will open up and you will see Linux boot up in this window.  You will automatically be logged into this Linux virtual machine when it is booted.
     14
     15''Windows Users:''  If your GENI-in-a-Box virtual machine fails to start up and you get an error message that reads something like "This kernel requires the following features not present on the CPU: pae", you will need to:
     16   * Power off the virtual machine
     17   * Select Machine -> Settings -> System -> Processor
     18   * Check the Enable PAE/NX box
     19   * Click OK
     20   * Start the virtual machine
     21
     22== 2 Get Started with a Simple Experiment ==
     23
     24=== 2.1 Discover Resources ===
     25
     26We start by finding out the aggregates available to us inside GENI-in-a-Box and learning about the resources advertised by these aggregates:
     27
     281. After you are logged into the GENI-in-a-Box virtual machine, open up a terminal window by clicking on the icon for a terminal on top of the screen.
     29
     302. In the terminal window, use Omni to list the aggregates available to you using the command {{{omni.py listaggregates}}}.   You should see something like this in your terminal window:
     31{{{
     32$ omni.py listaggregates
     33INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
     34INFO:omni:Using control framework gib_ch
     35INFO:omni:Listing 1 aggregates...
     36INFO:omni:  Aggregate 1:
     37urn:publicid:IDN+geni:gpo:gcf:am1+authority+am
     38http://geni-in-a-box.net:8001
     39INFO:omni: ------------------------------------------------------------
     40INFO:omni: Completed listaggregates:
     41
     42    Options as run:
     43                framework: gib_ch
     44                getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
     45
     46    Args: listaggregates
     47
     48    Result Summary: Found 1 aggregate. URN: urn:publicid:IDN+geni:gpo:gcf:am1+authority+am; URL: http://geni-in-a-box.net:8001
     49INFO:omni: ============================================================
     50$
     51}}}
     52
     53Look at the {{{Result Summary}}} from Omni.  You'll see that Omni found one aggregate available to you at URL {{{http://geni-in-a-box.net:8001}}}.  This is the aggregate you will use to run your experiments.
     54
     553. Now, let's find out the resources advertised by this aggregate using omni.
     56{{{
     57$  omni.py -a http://geni-in-a-box.net:8001 listresources
     58INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
     59INFO:omni:Using control framework gib_ch
     60INFO:omni:Listed resources on 1 out of 1 possible aggregates.
     61INFO:omni:<?xml version="1.0" encoding="UTF-8"?>
     62INFO:omni:  <!-- Resources at AM:
     63        URN: unspecified_AM_URN
     64        URL: http://geni-in-a-box.net:8001
     65 -->
     66INFO:omni: 
     67<!-- Resources at AM:
     68        URN: urn:publicid:IDN+geni-in-a-box.net+authority+cm
     69        URL: http://geni-in-a-box.net:8001
     70 -->
     71<rspec expires="2013-12-20T22:26:14Z" type="advertisement" xmlns="http://www.geni.net/resources/rspec/3"
     72       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     73       xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/ad.xsd" >
     74  <node component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc101" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" component_name="pc101" exclusive="true">   
     75      <hardware_type name="openVZ container">     
     76      </hardware_type>   
     77      <sliver_type name="virtual-pc">     
     78      <disk_image description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>     
     79      <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>     
     80      </sliver_type>   
     81      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth0" role="control">     
     82          <emulab:interface name="eth0"/>     
     83      </interface>   
     84      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" role="experimental">     
     85          <emulab:interface name="eth2"/>     
     86      </interface>   
     87      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth2" role="experimental">     
     88          <emulab:interface name="eth3"/>     
     89      </interface>   
     90      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth3" role="experimental">     
     91          <emulab:interface name="eth1"/>     
     92      </interface>
     93      <location country="Molvania" />   
     94  </node>       
     95      ...
     96  <node component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc106" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" component_name="pc106" exclusive="true">   
     97      <hardware_type name="openVZ container">     
     98      </hardware_type>   
     99      <sliver_type name="virtual-pc">     
     100      <disk_image description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>     
     101      <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="urn:publicid:IDN+geni-in-a-box.net+image:FEDORA15-STD" os="Fedora" version="15"/>     
     102      </sliver_type>   
     103      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth0" role="control">     
     104          <emulab:interface name="eth0"/>     
     105      </interface>   
     106      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth1" role="experimental">     
     107          <emulab:interface name="eth2"/>     
     108      </interface>   
     109      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth2" role="experimental">     
     110          <emulab:interface name="eth3"/>     
     111      </interface>   
     112      <interface component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc106:eth3" role="experimental">     
     113          <emulab:interface name="eth1"/>     
     114      </interface>
     115      <location country="Molvania" />   
     116  </node>
     117</rspec>
     118
     119INFO:omni: ------------------------------------------------------------
     120INFO:omni: Completed listresources:
     121
     122  Options as run:
     123                aggregate: ['http://geni-in-a-box.net:8001']
     124                framework: gib_ch
     125                getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
     126
     127  Args: listresources
     128
     129  Result Summary: Queried resources from 1 of 1 aggregate(s).
     130 
     131INFO:omni: ============================================================
     132$
     133}}}
     134The output of the command is the ''advertisement rspec'' from the GENI-in-a-Box aggregate.  You'll see the aggregate advertises six compute resources.   Only two are shown here for readability reasons but you will find the complete advertisement rspec [attachment:gib-advert.rspec here].  Look at the description of the first resource in the rspec.   This compute resource is an openVZ container running Fedora 15 image (or an Ubuntu 10 image if you are using the Ubuntu flavor of GENI-in-a-Box).  The  resource has four network interfaces: {{{eth0}}} through {{{eth3}}}.  {{{eth0}}} is a control interface and is not available to the experimenter.  All six compute resources advertised by this aggregate are identical.  You can therefore use this aggregate to create experiment topologies with up to six nodes and up to three links per node.
     135
     136''Notice the {{{-a}}} option to Omni that gave it the URL of the aggregate manager on which we wish to invoke {{{list resources}}}.  To save us from having to type this URL every time we use an Omni command that acts on this aggregate manager, we have defined a ''nickname'' {{{gib}}} for this aggregate.  This nickname is defined in the file {{{~/.gcf/omni_config}}}.  From now on we will use this nickname instead of the aggregate manager URL with Omni commands that require an aggregate URL.''
     137
     138=== 2.2 Acquire and Configure Resources ===
     139Before we run an experiment we need to acquire the resources needed the experiment.   Our introductory experiment will consist of two hosts connected to one another by a single link.   When the hosts come up they execute a script provided by us that:
     140    * Downloads and installs iperf on each host.
     141    * Runs iperf in server mode on one host and and in client mode on the other host.  The outputs of the iperf commands are written to /var/www/html/iperf-server.log and /var/www/html/iperf-client.log on the respective hosts.
     142As a part of this experiment we will ssh into these hosts to view these logs.
     143
     144The request rspec for this experiment (and other examples of request rspecs) are in the directory {{{~/geni-in-a-box}}}.
     145{{{
     146$ cd geni-in-a-box
     147$ ls
     148example-scripts    install-example.rspec  README.txt             two-nodes-iperf.rspec
     149four-nodes.rspec   islands.rspec          three-nodes-lan.rspec
     150$
     151}}}
     152We'll use the rspec file called two-nodes-iperf.rspec.  You can view it [attachment:two-nodes-iperf.rspec here] or inside the VM.  This rspec requests two hosts called {{{left}}} and {{{right}}}.  Host {{{left}}} has one interface called {{{left:if0}}} and host {{{right}}} has one interface called {{{right:if0}}}.  The two interfaces are connected by a link called {{{left-right-lan}}}.  The following figure illustrates the resources requested by this rspec.
     153
     154[[Image(ResourcesRequested.png, 60%)]]
     155
     156The {{{<install>}}} element in the rspec asks the GENI-in-a-Box aggregate manager to download a [attachment:iperf-script.sh script] from the specified URL, uncompress and untar it,  and save it in the {{{/local}}} directory.  The {{{<execute>}}} element asks the aggregate to execute the specified script when the hosts come up.  In this case the script to be executed is the one that was installed in {{{/local}}}.
     157
     158The following steps will walk us through acquiring the resources for our experiment:
     159
     1601. First we need to create a slice using the command {{{omni.py createslice myslice}}} where {{{myslice}}} is the name or our slice (of course you can give your slice some other name).
     161{{{
     162$ omni.py createslice myslice
     163INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
     164INFO:omni:Using control framework gib_ch
     165INFO:omni:Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
     166INFO:omni: ------------------------------------------------------------
     167INFO:omni: Completed createslice:
     168
     169  Options as run:
     170                framework: gib_ch
     171                getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
     172
     173  Args: createslice myslice
     174
     175  Result Summary: Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
     176 
     177INFO:omni: ============================================================
     178$
     179}}}
     180
     1812. We now send our [attachment:two-nodes-iperf.rspec request rspec] to the GENI-in-a-Box aggregate manager using the command {{{omni.py -a http://geni-in-a-box.net:8001 createsliver myslice two-nodes-iperf.rspec}}}.   If our request is successful, the aggregate manager will allocate the requested resources to our slice.  In GENI parlance the resources allocated to our slice are called slivers.  The {{{createsliver}}} request returns a manifest spec that provides details of the resources allocated to us. 
     182
     183''This command will take many minutes to complete because it configures and boots up the requested virtual machines; does some additional configuration; and download, installs, and runs iperf.''
     184
     185The following is a snippet of the manifest your will get back from {{{createsliver}}}:
     186{{{
     187$ omni.py -a gib createsliver myslice two-nodes-iperf.rspec
     188INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
     189INFO:omni:Using control framework gib_ch
     190WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
     191INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
     192INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
     193INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
     194INFO:omni:Creating sliver(s) from rspec file two-nodes-iperf.rspec for slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice
     195INFO:omni:Got return from CreateSliver for slice myslice at http://geni-in-a-box.net:8001:
     196INFO:omni:<?xml version="1.0" ?>
     197INFO:omni:  <!-- Reserved resources for:
     198        Slice: myslice
     199        at AM:
     200        URN: unspecified_AM_URN
     201        URL: http://geni-in-a-box.net:8001
     202 -->
     203INFO:omni:  <rspec type="manifest" xmlns="http://www.geni.net/resources/rspec/3" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.geni.net/resources/rspec/3   http://www.geni.net/resources/rspec/3/manifest.xsd"> 
     204    <node client_id="left" component_id="urn:publicid:IDN+geni-in-a-box.net+node+pc101" component_manager_id="urn:publicid:IDN+geni-in-a-box.net+authority+cm" exclusive="false" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+101">   
     205        <sliver_type name="virtual-pc">     
     206            <disk_image name="urn:publicid:geni-in-a-box.net+image+//FEDORA15-STD" os="Linux" version="10"/>     
     207        </sliver_type>   
     208        <services>     
     209            <execute command="/local/iperf-script.sh" shell="sh"/>     
     210            <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/geni-in-a-box/iperf-script.tar.gz"/>     
     211            <login authentication="ssh-keys" hostname="pc101.geni-in-a-box.net" port="22" username="gibuser"/>     
     212        </services>   
     213        <interface client_id="left:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" mac_address="00:0C:29:B4:DF:A7" sliver_id="urn:publicid:IDN+geni-in-a-box.net+sliver+1011">     
     214            <ip address="10.0.3.101"/>     
     215        </interface>   
     216        <rs:vnode name="pc101" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>   
     217        <host name="pc101.geni-in-a-box.net"/>   
     218    </node> 
     219    ... 
     220    <link client_id="left-right-lan">   
     221        <component_manager name="urn:publicid:IDN+geni-in-a-box.net+authority+cm"/>   
     222        <interface_ref client_id="left:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1" sliver_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc101:eth1"/>   
     223        <interface_ref client_id="right:if0" component_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc102:eth1" sliver_id="urn:publicid:IDN+geni-in-a-box.net+interface+pc102:eth1"/>   
     224        <property dest_id="right:if0" source_id="left:if0"/>   
     225        <property dest_id="left:if0" source_id="right:if0"/>   
     226        <link_type name="lan"/>   
     227    </link> 
     228</rspec>
     229INFO:omni: ------------------------------------------------------------
     230INFO:omni: Completed createsliver:
     231
     232  Options as run:
     233                aggregate: ['gib']
     234                framework: gib_ch
     235                getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
     236
     237  Args: createsliver myslice two-nodes-iperf.rspec
     238
     239  Result Summary: Got Reserved resources RSpec from geni-in-a-box-net-8001
     240INFO:omni: ============================================================
     241]0;gibuser@GiB:~/geni-in-a-box[gibuser@GiB geni-in-a-box]$
     242}}}
     243Notice the manifest rspec is essentially your request rspec annotated with information about the resources that got allocated to you.  For example, the {{{<node>}}} element of the manifest includes the MAC and IP addresses assigned to each network interface on the node (host) and information you will need to ssh into the nodes (e.g. the fully qualified domain name of the host ).  The following figure illustrates the resource allocation made by the GENI-in-a-Box aggregate, as described in the manifest spec.
     244
     245[[Image(ResourcesAllocated.png, 60%)]]
     246
     247=== 2.3 Using your Resources ===
     248You can now ssh into the hosts allocated to you and do what you need to do to get your experiment working.  For example, you can install and run software.  If needed, you can use the {{{sudo}}} command to do things that require root privileges. 
     249
     250In the case of our experiment, the software needed (iperf) was installed by our startup script.  Our startup script put the output of iperf in a file accessible from a web browser.   Since our iperf server ran on node {{{pc101}}} (the node labeled 'left' in the request spec), we can use a browser to view the output of this client.  Start up Firefox in a terminal on the GENI-in-a-Box virtual machine and enter the url {{{http://pc101.geni-in-a-box.net/iperf-server.log}}}.  The iperf server output should appear in your browser window.  You can view the iperf client output at {{{http://pc102.geni-in-a-box.net/iperf-client.log}}}.   
     251
     252You can also ssh into the node pc101 or pc102 to view these log files.  Your username on GENI-in-a-Box (gibuser) is given an account on the nodes allocated to your experiment and the public key for this user ({{{~/.gcf/gibuser.pub}}} is copied into gibuser's {{{.ssh}}} directory in each of these nodes.  The private key for gibuser is in {{{~/.ssh/gibuser}}} and must be provided to the {{{ssh}}} command. 
     253{{{
     254$ ssh -i ~/.ssh/gibuser pc101
     255The authenticity of host 'pc101 (10.0.1.101)' can't be established.
     256RSA key fingerprint is e3:7c:8a:21:16:05:93:fd:dd:56:1c:f5:5c:5b:f3:c7.
     257Are you sure you want to continue connecting (yes/no)? yes
     258Warning: Permanently added 'pc101,10.0.1.101' (RSA) to the list of known hosts.
     259
     260[gibuser@left ~]$ cd /var/www/html
     261[gibuser@left html]$ ls
     262iperf-server.log
     263[gibuser@left html]$ cat iperf-server.log
     264------------------------------------------------------------
     265Server listening on TCP port 5001
     266TCP window size: 85.3 KByte (default)
     267------------------------------------------------------------
     268[  4] local 10.0.3.101 port 5001 connected with 10.0.3.102 port 60542
     269[ ID] Interval       Transfer     Bandwidth
     270[  4]  0.0-10.1 sec  61.8 MBytes  51.5 Mbits/sec
     271[gibuser@left html]$
     272}}}
     273
     274You can have the GENI-in-a-Box aggregate create additional user accounts on your experiment nodes; see [wiki:GENI-in-a-Box/GiB-tips this page] for instructions.
     275
     276=== 2.4. Freeing up Resources ===
     277When you are done with your experiment you can return the resources allocated to you using omni's {{{deletesliver}}} command:
     278{{{
     279$ omni.py -a gib deletesliver myslice
     280INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
     281INFO:omni:Using control framework gib_ch
     282WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
     283INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
     284INFO:omni:Substituting AM nickname gib with URL http://geni-in-a-box.net:8001, URN unspecified_AM_URN
     285INFO:omni:Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at http://geni-in-a-box.net:8001
     286INFO:omni: ------------------------------------------------------------
     287INFO:omni: Completed deletesliver:
     288
     289  Options as run:
     290                aggregate: ['gib']
     291                framework: gib_ch
     292                getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
     293
     294  Args: deletesliver myslice
     295
     296  Result Summary: Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at http://geni-in-a-box.net:8001
     297INFO:omni: ============================================================
     298$
     299}}}
     300
     301Congratulations!  You have successfully run an experiment using GENI-in-a-Box resources!
     302
     303== 3 GENI-in-a-Box Tips ==
     304See [wiki:GENI-in-a-Box/GiB-tips this page] for tips that you might find helpful while using GENI-in-a-Box.
     305
     306
     307== 4 What's Next? ==
     308You are now ready to run your own experiments inside GENI-in-a-Box.  You might want to start with one of the sample rspecs in the {{{~/geni-in-a-box}}} directory and edit it as needed by your experiment.  See the {{{README}}} file in this directory for a description of the rspecs in this directory.
     309
     310You can [wiki:SignMeUp apply for a GENI account] and use real GENI aggregates and resources for your experiments.  See [wiki:GENI-in-a-Box/GiB-to-ProtoGENI GiB-to-ProtoGENI] for tips on moving your experiment from GENI-in-a-Box to other aggregates (and vice versa).