Changes between Version 1 and Version 2 of GENIExperimenter/Tutorials/NSDI13/IntroToOmni

03/12/13 03:21:29 (7 years ago)



  • GENIExperimenter/Tutorials/NSDI13/IntroToOmni

    v1 v2  
     2= Simple OpenFlow Tutorial with OVS Using GENI Portal =
    3 = Prework =
    4 In this tutorial we are going to run a simple experiment in GENI, using the
    5 [ Omni command line tool]. We are going to take advantage of the Layer 2
    6 links between nodes and run a non-IP experiment.
     4== Experiment Description ==
     5In this tutorial we are going to use a simple topology using [ OVS] as an OpenFlow switch connected to three hosts.
     6OVS is a software switch running on a compute resource. The other three hosts can only communicate through the OVS switch. Although the controller for the switch can run anywhere in the internet, we are going to run it on the same host as the OVS switch to minimize the required resources. We are going to use this setup to write simple controllers and experiment with the capabilities of OpenFlow. For this tutorial we are going to use the [ PoX controller], this is just an example of [wiki:OpenFlow/Controllers many controllers].
     7The OVS switch that we use runs in userspace which although not optimized for performance, it is enough for getting acquainted with OpenFlow.
    8 This is the topology of our experiment:
    10 Before we start with the Omni tutorial, please follow these steps:
    11   i. Login to the VM
    12   i. Run Firefox
    13   i. Open all bookmarks under the Intro to Omni bookmark folder
    14   i. Open a terminal window
     11== Tutorial Instructions ==
    16 = 1. Configure Omni =
    17 Omni is a tool that will help you reserve resources in GENI, using your GENI
    18 account. The first step is to load into Omni your account information.
    19 Omni uses a configuration file, and there is a script that will automatically generate the file with
    20 some default options and place it in the default place of `~/.gcf/`. Follow
    21 these steps:
    22  i. In the web browser login to `` using the information
    23  in the paper slip
    24  i. On the menu on the left, press `Download your cert`
    25   [[Image(Tutorials/Graphics:download_cert.png)]]
    26  i. Select the option about downloading as a pem file.
    27  i. Press '''`Ctrl-s`''' to save this page. Type `~/.ssl/geni_cert.pem`, if file already exists replace it.
    28   [[Image(Tutorials/Graphics:save_cert.png, 40%)]]
    29  i. In the terminal type:
    30   {{{
    32 }}}
    33   This should configure your Omni, with your downloaded certificate and create an ssh key pair for logging in to nodes.
    35   '''Tip:''' When prompted for a passphrase, type the passphrase in your slip. You might be asked for a passphrase multiple times. '''Note:''' If you have taken other tutorials, you might have configured omni with other accounts, select to overwrite existing setting if asked.
    37   The output should look like:
    38   {{{
    39 geni@geni-vm:~$
    40 {'framework': 'pg', 'configfile': '~/.gcf/omni_config', 'plkey': '~/.ssh/geni_pl_key', 'cert': '~/.ssl/geni_cert', 'verbose': False}
    41 INFO:omniconfig:Using configfile: /home/geni/.gcf/omni_config
    42 INFO:omniconfig:Using certfile /home/geni/.ssl/geni_cert.pem
    43 INFO:omniconfig:
    46 INFO:omniconfig:Private key stored at: /home/geni/.ssh/geni_key
    47 Enter passphrase:
    48 INFO:omniconfig:Public key stored at: /home/geni/.ssh/
    49 INFO:omniconfig:Added to /home/geni/.ssh/config this line:
    50         'IdentityFile /home/geni/.ssh/geni_key
    51 '
    52 INFO:omniconfig:Wrote omni configuration file at: /home/geni/.gcf/omni_config
     16<table id="Table_01" height="409" border="0" cellpadding="0" cellspacing="0">
     17        <tr>
     18                <td>
     19                        <a href=""><img src="" width="152" height="152" alt="Design/Setup"></a>
     20               </td>
     21               <td>
     23             <li><a href="">Part I: Design/Setup</a></li>
     24             <ul>
     25                 <li><a href="">Step 1: Establish Management Environment: Setup your GENI account and join a GENI Project</a></li>
     26                 <li><a href="">Step 2: Obtain Resources: Create a slice and reserve resources</a></li>
     27             </ul></ul>
     28               </td>
     29        </tr>
     30        <tr>
     31                <td>
     32                        <a href=""><img src="" width="152" height="143" alt="Execute"></a>
     33                </td>
     34                <td>
     35             <ul>
     36<li><a href="">Part II: Execute</a></li>
     37             <ul>
     38                 <li><a href="">Configure and Initialize Services: Configure your OpenFlow switch</a> </li>
     39                 <li><a href="">Execute Experiment: Login to nodes and execute a simple layer 2 experiment</a> </li>
     40             </ul></ul>
     41                </td>
     42        </tr>
     43        <tr>
     44                <td>
     45                        <a href=""><img src="" width="152" height="114" alt="Finish"></a>
     46                </td>
     48                <td>
     49             <ul>
     50             <li><a href="">Part III: FInish</a></li>
     51             <ul>
     52                 <li><a href="">Teardown Experiment: Delete Resources</a></li>
     53             </ul></ul>
     54                </td>
     56        </tr>
    55 == 1a. Look around the omni_config file -- Optional ==
    56 Open the file `~/.gcf/omni_config` using either vim or emacs. Close to the top of the file you will see two parameters called `default_cf` and `users`. Your username should be at least listed in the user section. Look for the sections in the file that are named `[pg]` and  `[<username>]`.
    58 In the `[<username>]` section, the information need for logging-in to reserved compute resources are provided. It includes your unique user URN and a public key that would be uploaded to the hosts that you reserve.
    60 In the `[pg]` section you configure Omni to use your personal information. The cert and the key attribute point to files that we have manually downloaded from This is equivalent to the Download action of Flack.
    62 Another interesting section to look at is the `[aggregate-nicknames]` sections. Flack already knows the URL for all the AMs and present you a list of AMs to choose from using a short, descriptive name. In Omni a user is required to pass the URL for each call to the GENI AM API. In this section the user gets a chance to provide short descriptive names to the URLs that are easier to memorize and use.
    64 = 2. Clear the passphrase from your cert =
    65 When Omni talks to different authorities in GENI, it will need to identify you
    66 as a valid GENI user, so it will ask you for your passphrase multiple times.
    67 To save time during the tutorial, we are going to remove the passphrase from your cert and your ssh key. There is a script to clear the passphrases.
    68   i. In the terminal run
    69   {{{
    71 }}}
    72   Use the passphrase given to you in the paper slip, you might be prompted for a passphrase multiple times make sure you always use the same passphrase.
    73   The output should look like
    74   {{{
    75 geni@geni-vm:~$
    76 Do you want to remove the passphrase from your cert (/home/geni/.ssl/geni_cert.pem) [Y,n]?y
    77 INFO:clearcert:
    80 INFO:clearcert:The encoded certificate file is backed up at /home/geni/.ssl/geni_cert_enc.pem
    81 INFO:clearcert:Removing passphrase from cert...
    82 Enter pass phrase for /home/geni/.ssl/geni_cert.pem:
    83 writing RSA key
    84 INFO:clearcert:Change permissions of /home/geni/.ssl/geni_cert.pem to 0600
    85 Do you want to remove the passphrase from you ssh-key (/home/geni/.ssh/geni_key, key used to login to compute resources) [Y,n]?y
    86 INFO:clearcert:
    88 Enter PEM pass phrase:
    89 Enter passphrase:
    90 geni@geni-vm:~$
    91 }}}
    92   i. Add the key to the ssh agent
    93   {{{
    94   ssh-add ~/.ssh/geni_key
    95   }}}
    96   i. Verify that you have the necessary credential and key files
    97      {{{
    98      ls ~/.ssh ~/.ssl
    99      }}}
    100      The output looks like :
    101      {{{
    102 geni@geni-vm:~$ ls ~/.ssh ~/.ssl
    103 /home/geni/.ssh:
    104 config  geni_key
    106 /home/geni/.ssl:
    107 geni_cert_enc.pem  geni_cert.pem
    108      }}}
    110    || geni_cert.pem || Cleartext certificate, i.e. does not require any passphrase ||
    111    || geni_cert_enc.pem || Encrypted certificate  ||
    112    || geni_key || The private key that you will use to login to the nodes ||
    113    || || The corresponding public key that will be uploaded to the nodes ||
    115   '''Note''': You might see more files than the ones listed above, which are
    116   probably backup files from configuring omni, or files from other tutorials.
    118 == 3. Test Omni setup ==
    119 In order to test that our configuration is correct, you can issue a `getversion`
    120 command. For this step we have used !PlanetLab's AM, but you can use any
    121 Aggregate Manager(AM).
    122   i. In the terminal type:
    123   {{{
    124 getversion -a plc
    125 }}}
    126   In GENI each AM can be contacted using a specific URL. In order to make it
    127   easier to talk to AMs, Omni uses nicknames; i.e. it maps each URL into an easy
    128   to remember nickname. For example the PlanetLab AM listens for requests at
    129   ``, however you can use the nickname `plc` to
    130   refer to that URL. Nicknames are defined in each user's omni_config file.
    132   For the rest of the tutorial we are going to use aggregate nicknames.
    134 = 4. Create an experiment using Omni =
    135 In this step we are going to setup a sample GENI experiment, in order to
    136 get familiarized with most of the Omni commands.
    138 == 4a. Create and renew a slice ==
    139 The first thing to do when preparing to run a GENI experiment is to create a
    140 slice.
    141   i. To create a slice type, use the `<slicename>` given in your paper slip:
    142   {{{
    143 createslice <slicename>
    144   }}}
    145   ii. To verify that your slice has been created and in general to keep track of
    146   your slices, use the `listmyslices` command. use the <username> given in your
    147   paper slip. Type:
    148   {{{
    149 listmyslices <username>
    150   }}}
    151   iii. Renew your slice. To extend the lifetime of your slice type:
    152   {{{
    153 renewslice <slicename> 20121025
    154   }}}
    156 == 4b. Reserve resources, a.k.a create a sliver ==
    157 The next step in a GENI experiment is to actually reserve resources. For this
    158 experiment we are going to use the Aggregate manager of ProtoGENI in Utah.
    159   i. In Omni in order to see what each AM offers you can use the
    160   `listresources` command. Type:
    161    {{{
    162 listresources -a pg-utah -o
    163 }}}
    164   The `-o` option will save the output to a file. The filename is chosen by Omni and printed as part of the output. The output will look like :
    165   {{{
    166 geni@geni-VirtualBox:~$ listresources -a pg-utah -o
    167 INFO:omni:Loading config file /home/geni/.gcf/omni_config
    168 INFO:omni:Using control framework pg
    169 INFO:omni:Saving output to a file.
    170 INFO:omni:Substituting AM nickname pg-utah with URL, URN unspecified_AM_URN
    171 INFO:omni:Listed resources on 1 out of 1 possible aggregates.
    172 INFO:omni:Writing to 'rspec-www-emulab-net-protogeniv2.xml'
    173 INFO:omni: ------------------------------------------------------------
    174 INFO:omni: Completed listresources:
    176   Options as run:
    177                 aggregate: ['pg-utah']
    178                 framework: pg
    179                 output: True
    181   Args: listresources
    183   Result Summary: Queried resources from 1 of 1 aggregate(s).
    184 Wrote rspecs from 1 aggregate(s) to 1 file(s)
    185 Saved listresources RSpec at 'unspecified_AM_URN' (url '') to file rspec-www-emulab-net-protogeniv2.xml; 
    186 INFO:omni: ============================================================
    187 }}}
    188   In the last line of the output Omni will tell you the name of the file that output is saved at. In the example above this would be `rspec-www-emulab-net-protogeniv2.xml`. Open the file that Omni saved and just take a look to see how an advertisement RSpec looks like.
    189   In order to see only available resources type
    190   {{{
    191 listresources -a pg-utah --available -o
    192   }}}
    193   i. To be able to reserve resources you will need to craft a request rspec.
    194   For this example we have created the rspecs for you, use the rspec URL given
    195   in your paper slip. Type :
    196   {{{
    197 createsliver -a pg-utah <slicename> <rspec_url>
    198   }}}
    199   For example
    200   {{{
    201 createsliver -a pg-utah iomni30
    202   }}}
    203   '''Tip:''' If you are copying the above line make sure to substitute the slicename and the actual rspec name with the ones that are specified in your paper slip.
    204   i. Look at your reserved resources. Type:
    205   {{{
    206 listresources -a pg-utah <slicename>
    207   }}}
    208   i. Extend the lifetime of your sliver:
    209   {{{
    210 renewsliver -a pg-utah <slicename> 20121025
    211   }}}
    212   i. Check the status of your resources. Type:
    213   {{{
    214 sliverstatus -a pg-utah <slicename>
    215   }}}
    216   The `sliverstatus` command reports the status of your overall GENI slice. When
    217   the status is ready we are ready to continue to the next step.
    219 = 5. Run your experiment =
    220 Now that resources are ready we can start running our experiment. We will first
    221 start by logging in to our reserved nodes.
    223 == 5a. Logging Into the nodes -- Omni Scripting ==
    224 Depending on the Aggregate the login information are either part of the
    225 `sliverstatus` call, or as part of the manifest rspec, i.e. returned from the
    226 `listresources` call. Omni comes with a script that makes it easy to gather all
    227 this information.
    228   i. In the terminal run:
    229   {{{
    230 -a pg-utah <slicename>
    231 }}}
    232   The script will return the actual command that you would need to use for logging in.
    233   ii. Login to both nodes by copy-paste the commands from the output of the
    234   script.
    236   [[Image(Tutorials/Graphics:readytologin_output.png, 40%)]]
    238 == 5b. Send IP traffic ==
    239 The first simple experiment that we will run is to verify the IP connectivity
    240 between our hosts.
    241   i. Check the interfaces of your nodes. In the terminal type:
    242   {{{
    243    /sbin/ifconfig
    244   }}}
    245   You should see at least two interfaces:
    246     * The '''control interface'''. This is the interface from where you access
    247     the node, e.g. ssh into your host. The control interface is mainly used for
    248     control traffic, i.e. traffic for controlling the node and the experiment.
    249     The control interface usually has a publicly routable IP.
    250     * The '''data interface'''. This is the interface that is used for sending
    251     experimental traffic. This is the interface that connects to the other hosts
    252     of your experiment through GENI. The links between these interfaces are the
    253     ones that allow you to run non-IP experiments.
    254   i. Note the name and the MAC address of the control and of the data interface for each node. The data interface is the one that has an IP that starts with 10.
    255   i. From the client ping the server. From the xterm that is logged in to the
    256   client type :
    257   {{{
    258     ping server -c 5
    259   }}}
    260   '''Note''': You can use the name that is assigned to the host to directly ping
    261   the host, you can also ping using the IP of the data interface of the node.
    262   An example output should look like :
    263   {{{
    264 [[inki@r2 ~]$ ping server -c 5
    265 PING ( 56(84) bytes of data.
    266 64 bytes from icmp_seq=1 ttl=64 time=1.95 ms
    267 64 bytes from icmp_seq=2 ttl=64 time=2.27 ms
    268 64 bytes from icmp_seq=3 ttl=64 time=2.04 ms
    269 64 bytes from icmp_seq=4 ttl=64 time=1.95 ms
    270 64 bytes from icmp_seq=5 ttl=64 time=1.72 ms
    272 --- ping statistics ---
    273 5 packets transmitted, 5 received, 0% packet loss, time 4005ms
    274 rtt min/avg/max/mdev = 1.728/1.992/2.278/0.179 ms
    275 }}}
    277 == 5c. Send non-IP traffic ==
    278 GENI provides the capability of running non-IP experiments, since you can
    279 connect your hosts at Layer 2. For the purpose of this tutorial we have
    280 installed in all the hosts a very simple Layer 2 ping program that sends packets
    281 using a custom ethernet type.
    282   i. Disable the IP in your nodes. First of all we are going to completely
    283   disable IP on our nodes. In each of the terminals type:
    284   {{{
    285   sudo /sbin/ifconfig <data i/f name>
    286   }}}
    287   '''Note''': Be extra careful to bring down the IP on the data interface,
    288   bringing down the IP on the control interface means that you will lose
    289   connectivity to your host.
    290   ii. Try again to ping from the client to the server. In the xterm of the
    291   client type:
    292   {{{
    293   ping server -c 5
    294   }}}
    295   This time the ping should timeout.
    296   iii. Start the Layer 2 ping server: In the server xterm, type:
    297   {{{
    298   sudo /usr/local/bin/pingPlusListener 12345
    299   }}}
    300   iii. From the client try to ping the server at layer 2. You will need the mac
    301   address of the data interface of the server and the name of the data interface
    302   of the client. In the xterm of the client, type:
    303   {{{
    304   sudo /usr/local/bin/pingPlus <server data mac addr> <client data i/f name> 12345
    305   }}}
    307   The result should look like :
    308   {{{
    309 [uncusr20@client ~]$ sudo /usr/local/bin/pingPlus 02:00:9B:48:3E:20 mv1.1 12345
    310 RQ:'5582+2067' to 2:0:9b:48:3e:20.
    311 RQ:5582+2067 from 2:0:3e:15:6c:ab.
    312 }}}
    314 Congratulations you have run a Layer 2 experiment in GENI!
    316 = 6. Cleanup resources =
    317 Although all your reservations, have expiration times, you should always release
    318 your resources once you have completed your experiment to make them available
    319 to other experimenters.
    320   i. In the terminal, where you have been running your omni commands do:
    321   {{{
    322 deletesliver -a  pg-utah <slicename>
    323 }}}
    325 = 7. Wrap up =
    326 Congratulations, you have finished the tutorial! Now you are ready to design and
    327 run your own experiments. Don't hesitate to [ email us] with
    328 any questions you might have.
    330 == 7a. Fill out the tutorial survey ==
    331 We love to hear what you think so that we can improve the tutorials in the future.
    332 Please complete the [ online survey] and get a temporary GENI tatoo!
    334 == 7b. Get Your Own Account ==
    335 The accounts, that you used in the above steps will be deactivated after the
    336 tutorial. If you do not already have an account at GPO's ProtoGENI cluster, you
    337 should can follow [wiki:GEC15Agenda/ExperimenterDropIn/GetAGENIAccount these instructions]
    338 to get an account and start using GENI.