Changes between Version 27 and Version 28 of GeniTmixTutorial

02/27/13 13:00:28 (11 years ago)
Ben Newton



  • GeniTmixTutorial

    v27 v28  
    8686where HH:MM:SS is the chosen start time in hours minutes and seconds. 
    88 3. Now, let's find out the resources advertised by this aggregate using omni.
    89 {{{
    90 $ -a listresources
    91 INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    92 INFO:omni:Using control framework gib_ch
    93 INFO:omni:Listed resources on 1 out of 1 possible aggregates.
    94 INFO:omni:<?xml version="1.0" encoding="UTF-8"?>
    95 INFO:omni:  <!-- Resources at AM:
    96         URN: unspecified_AM_URN
    97         URL:
    98  -->
    99 INFO:omni: 
    100 <!-- Resources at AM:
    101         URN:
    102         URL:
    103  -->
    104 <rspec expires="2013-12-20T22:26:14Z" type="advertisement" xmlns=""
    105        xmlns:xsi=""
    106        xsi:schemaLocation="" >
    107   <node component_id="" component_manager_id="" component_name="pc101" exclusive="true">   
    108       <hardware_type name="openVZ container">     
    109       </hardware_type>   
    110       <sliver_type name="virtual-pc">     
    111       <disk_image description="Standard 32-bit Fedora 15 image" name="" os="Fedora" version="15"/>     
    112       <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="" os="Fedora" version="15"/>     
    113       </sliver_type>   
    114       <interface component_id="" role="control">     
    115           <emulab:interface name="eth0"/>     
    116       </interface>   
    117       <interface component_id="" role="experimental">     
    118           <emulab:interface name="eth2"/>     
    119       </interface>   
    120       <interface component_id="" role="experimental">     
    121           <emulab:interface name="eth3"/>     
    122       </interface>   
    123       <interface component_id="" role="experimental">     
    124           <emulab:interface name="eth1"/>     
    125       </interface>
    126       <location country="Molvania" />   
    127   </node>       
    128       ...
    129   <node component_id="" component_manager_id="" component_name="pc106" exclusive="true">   
    130       <hardware_type name="openVZ container">     
    131       </hardware_type>   
    132       <sliver_type name="virtual-pc">     
    133       <disk_image description="Standard 32-bit Fedora 15 image" name="" os="Fedora" version="15"/>     
    134       <disk_image default="true" description="Standard 32-bit Fedora 15 image" name="" os="Fedora" version="15"/>     
    135       </sliver_type>   
    136       <interface component_id="" role="control">     
    137           <emulab:interface name="eth0"/>     
    138       </interface>   
    139       <interface component_id="" role="experimental">     
    140           <emulab:interface name="eth2"/>     
    141       </interface>   
    142       <interface component_id="" role="experimental">     
    143           <emulab:interface name="eth3"/>     
    144       </interface>   
    145       <interface component_id="" role="experimental">     
    146           <emulab:interface name="eth1"/>     
    147       </interface>
    148       <location country="Molvania" />   
    149   </node>
    150 </rspec>
    152 INFO:omni: ------------------------------------------------------------
    153 INFO:omni: Completed listresources:
    155   Options as run:
    156                 aggregate: ['']
    157                 framework: gib_ch
    158                 getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    160   Args: listresources
    162   Result Summary: Queried resources from 1 of 1 aggregate(s).
    164 INFO:omni: ============================================================
    165 $
    166 }}}
    167 The output of the command is the ''advertisement rspec'' from the GENI-in-a-Box aggregate.  You'll see the aggregate advertises six compute resources.   Only two are shown here for readability reasons but you will find the complete advertisement rspec [attachment:gib-advert.rspec here].  Look at the description of the first resource in the rspec.   This compute resource is an openVZ container running Fedora 15 image (or an Ubuntu 10 image if you are using the Ubuntu flavor of GENI-in-a-Box).  The  resource has four network interfaces: {{{eth0}}} through {{{eth3}}}.  {{{eth0}}} is a control interface and is not available to the experimenter.  All six compute resources advertised by this aggregate are identical.  You can therefore use this aggregate to create experiment topologies with up to six nodes and up to three links per node.
    169 ''Notice the {{{-a}}} option to Omni that gave it the URL of the aggregate manager on which we wish to invoke {{{list resources}}}.  To save us from having to type this URL every time we use an Omni command that acts on this aggregate manager, we have defined a ''nickname'' {{{gib}}} for this aggregate.  This nickname is defined in the file {{{~/.gcf/omni_config}}}.  From now on we will use this nickname instead of the aggregate manager URL with Omni commands that require an aggregate URL.''
    171 === 2.2 Acquire and Configure Resources ===
    172 Before we run an experiment we need to acquire the resources needed the experiment.   Our introductory experiment will consist of two hosts connected to one another by a single link.   When the hosts come up they execute a script provided by us that:
    173     * Downloads and installs iperf on each host.
    174     * Runs iperf in server mode on one host and and in client mode on the other host.  The outputs of the iperf commands are written to /var/www/html/iperf-server.log and /var/www/html/iperf-client.log on the respective hosts.
    175 As a part of this experiment we will ssh into these hosts to view these logs.
    177 The request rspec for this experiment (and other examples of request rspecs) are in the directory {{{~/geni-in-a-box}}}.
    178 {{{
    179 $ cd geni-in-a-box
    180 $ ls
    181 example-scripts    install-example.rspec  README.txt             two-nodes-iperf.rspec
    182 four-nodes.rspec   islands.rspec          three-nodes-lan.rspec
    183 $
    184 }}}
    185 We'll use the rspec file called two-nodes-iperf.rspec.  You can view it [attachment:two-nodes-iperf.rspec here] or inside the VM.  This rspec requests two hosts called {{{left}}} and {{{right}}}.  Host {{{left}}} has one interface called {{{left:if0}}} and host {{{right}}} has one interface called {{{right:if0}}}.  The two interfaces are connected by a link called {{{left-right-lan}}}.  The following figure illustrates the resources requested by this rspec.
    187 [[Image(ResourcesRequested.png, 60%)]]
    189 The {{{<install>}}} element in the rspec asks the GENI-in-a-Box aggregate manager to download a [ script] from the specified URL, uncompress and untar it,  and save it in the {{{/local}}} directory.  The {{{<execute>}}} element asks the aggregate to execute the specified script when the hosts come up.  In this case the script to be executed is the one that was installed in {{{/local}}}.
    191 The following steps will walk us through acquiring the resources for our experiment:
    193 1. First we need to create a slice using the command {{{ createslice myslice}}} where {{{myslice}}} is the name or our slice (of course you can give your slice some other name).
    194 {{{
    195 $ createslice myslice
    196 INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    197 INFO:omni:Using control framework gib_ch
    198 INFO:omni:Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
    199 INFO:omni: ------------------------------------------------------------
    200 INFO:omni: Completed createslice:
    202   Options as run:
    203                 framework: gib_ch
    204                 getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    206   Args: createslice myslice
    208   Result Summary: Created slice with Name myslice, URN urn:publicid:IDN+geni:gpo:gcf+slice+myslice, Expiration 2012-09-28 23:49:43
    210 INFO:omni: ============================================================
    211 $
    212 }}}
    214 2. We now send our [attachment:two-nodes-iperf.rspec request rspec] to the GENI-in-a-Box aggregate manager using the command {{{ -a createsliver myslice two-nodes-iperf.rspec}}}.   If our request is successful, the aggregate manager will allocate the requested resources to our slice.  In GENI parlance the resources allocated to our slice are called slivers.  The {{{createsliver}}} request returns a manifest spec that provides details of the resources allocated to us. 
    216 ''This command will take many minutes to complete because it configures and boots up the requested virtual machines; does some additional configuration; and download, installs, and runs iperf.''
    218 The following is a snippet of the manifest your will get back from {{{createsliver}}}:
    219 {{{
    220 $ -a gib createsliver myslice two-nodes-iperf.rspec
    221 INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    222 INFO:omni:Using control framework gib_ch
    223 WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
    224 INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
    225 INFO:omni:Substituting AM nickname gib with URL, URN unspecified_AM_URN
    226 INFO:omni:Substituting AM nickname gib with URL, URN unspecified_AM_URN
    227 INFO:omni:Creating sliver(s) from rspec file two-nodes-iperf.rspec for slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice
    228 INFO:omni:Got return from CreateSliver for slice myslice at
    229 INFO:omni:<?xml version="1.0" ?>
    230 INFO:omni:  <!-- Reserved resources for:
    231         Slice: myslice
    232         at AM:
    233         URN: unspecified_AM_URN
    234         URL:
    235  -->
    236 INFO:omni:  <rspec type="manifest" xmlns="" xmlns:flack="" xmlns:planetlab="" xmlns:xsi="" xsi:schemaLocation=""> 
    237     <node client_id="left" component_id="" component_manager_id="" exclusive="false" sliver_id="">   
    238         <sliver_type name="virtual-pc">     
    239             <disk_image name="" os="Linux" version="10"/>     
    240         </sliver_type>   
    241         <services>     
    242             <execute command="/local/" shell="sh"/>     
    243             <install install_path="/local" url=""/>     
    244             <login authentication="ssh-keys" hostname="" port="22" username="gibuser"/>     
    245         </services>   
    246         <interface client_id="left:if0" component_id="" mac_address="00:0C:29:B4:DF:A7" sliver_id="">     
    247             <ip address=""/>     
    248         </interface>   
    249         <rs:vnode name="pc101" xmlns:rs=""/>   
    250         <host name=""/>   
    251     </node> 
    252     ... 
    253     <link client_id="left-right-lan">   
    254         <component_manager name=""/>   
    255         <interface_ref client_id="left:if0" component_id="" sliver_id=""/>   
    256         <interface_ref client_id="right:if0" component_id="" sliver_id=""/>   
    257         <property dest_id="right:if0" source_id="left:if0"/>   
    258         <property dest_id="left:if0" source_id="right:if0"/>   
    259         <link_type name="lan"/>   
    260     </link> 
    261 </rspec>
    262 INFO:omni: ------------------------------------------------------------
    263 INFO:omni: Completed createsliver:
    265   Options as run:
    266                 aggregate: ['gib']
    267                 framework: gib_ch
    268                 getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    270   Args: createsliver myslice two-nodes-iperf.rspec
    272   Result Summary: Got Reserved resources RSpec from geni-in-a-box-net-8001
    273 INFO:omni: ============================================================
    274 ]0;gibuser@GiB:~/geni-in-a-box[gibuser@GiB geni-in-a-box]$
    275 }}}
    276 Notice the manifest rspec is essentially your request rspec annotated with information about the resources that got allocated to you.  For example, the {{{<node>}}} element of the manifest includes the MAC and IP addresses assigned to each network interface on the node (host) and information you will need to ssh into the nodes (e.g. the fully qualified domain name of the host ).  The following figure illustrates the resource allocation made by the GENI-in-a-Box aggregate, as described in the manifest spec.
    278 [[Image(ResourcesAllocated.png, 60%)]]
    280 === 2.3 Using your Resources ===
    281 You can now ssh into the hosts allocated to you and do what you need to do to get your experiment working.  For example, you can install and run software.  If needed, you can use the {{{sudo}}} command to do things that require root privileges. 
    283 In the case of our experiment, the software needed (iperf) was installed by our startup script.  Our startup script put the output of iperf in a file accessible from a web browser.   Since our iperf server ran on node {{{pc101}}} (the node labeled 'left' in the request spec), we can use a browser to view the output of this client.  Start up Firefox in a terminal on the GENI-in-a-Box virtual machine and enter the url {{{}}}.  The iperf server output should appear in your browser window.  You can view the iperf client output at {{{}}}.   
    285 You can also ssh into the node pc101 or pc102 to view these log files.  Your username on GENI-in-a-Box (gibuser) is given an account on the nodes allocated to your experiment and the public key for this user ({{{~/.gcf/}}} is copied into gibuser's {{{.ssh}}} directory in each of these nodes.  The private key for gibuser is in {{{~/.ssh/gibuser}}} and must be provided to the {{{ssh}}} command. 
    286 {{{
    287 $ ssh -i ~/.ssh/gibuser pc101
    288 The authenticity of host 'pc101 (' can't be established.
    289 RSA key fingerprint is e3:7c:8a:21:16:05:93:fd:dd:56:1c:f5:5c:5b:f3:c7.
    290 Are you sure you want to continue connecting (yes/no)? yes
    291 Warning: Permanently added 'pc101,' (RSA) to the list of known hosts.
    293 [gibuser@left ~]$ cd /var/www/html
    294 [gibuser@left html]$ ls
    295 iperf-server.log
    296 [gibuser@left html]$ cat iperf-server.log
    297 ------------------------------------------------------------
    298 Server listening on TCP port 5001
    299 TCP window size: 85.3 KByte (default)
    300 ------------------------------------------------------------
    301 [  4] local port 5001 connected with port 60542
    302 [ ID] Interval       Transfer     Bandwidth
    303 [  4]  0.0-10.1 sec  61.8 MBytes  51.5 Mbits/sec
    304 [gibuser@left html]$
    305 }}}
    307 You can have the GENI-in-a-Box aggregate create additional user accounts on your experiment nodes; see [wiki:GENI-in-a-Box/GiB-tips this page] for instructions.
    309 === 2.4. Freeing up Resources ===
    310 When you are done with your experiment you can return the resources allocated to you using omni's {{{deletesliver}}} command:
    311 {{{
    312 $ -a gib deletesliver myslice
    313 INFO:omni:Loading config file /home/gibuser/.gcf/omni_config
    314 INFO:omni:Using control framework gib_ch
    315 WARNING:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires in <= 3 hours
    316 INFO:omni:Slice urn:publicid:IDN+geni:gpo:gcf+slice+myslice expires on 2012-10-16 14:51:59 UTC
    317 INFO:omni:Substituting AM nickname gib with URL, URN unspecified_AM_URN
    318 INFO:omni:Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at
    319 INFO:omni: ------------------------------------------------------------
    320 INFO:omni: Completed deletesliver:
    322   Options as run:
    323                 aggregate: ['gib']
    324                 framework: gib_ch
    325                 getversionCacheName: /home/gibuser/.gcf/get_version_cache.json
    327   Args: deletesliver myslice
    329   Result Summary: Deleted sliver urn:publicid:IDN+geni:gpo:gcf+slice+myslice on unspecified_AM_URN at
    330 INFO:omni: ============================================================
    331 $
    332 }}}
    334 Congratulations!  You have successfully run an experiment using GENI-in-a-Box resources!
    336 == 3 GENI-in-a-Box Tips ==
    337 See [wiki:GENI-in-a-Box/GiB-tips this page] for tips that you might find helpful while using GENI-in-a-Box.
    340 == 4 What's Next? ==
    341 You are now ready to run your own experiments inside GENI-in-a-Box.  You might want to start with one of the sample rspecs in the {{{~/geni-in-a-box}}} directory and edit it as needed by your experiment.  See the {{{README}}} file in this directory for a description of the rspecs in this directory.
    343 You can [wiki:SignMeUp apply for a GENI account] and use real GENI aggregates and resources for your experiments.  See [wiki:GENI-in-a-Box/GiB-to-ProtoGENI GiB-to-ProtoGENI] for tips on moving your experiment from GENI-in-a-Box to other aggregates (and vice versa).
     88n moving your experiment from GENI-in-a-Box to other aggregates (and vice versa).