Changes between Initial Version and Version 1 of GENIExperimenter/Tutorials/HelloOmni

Timestamp:

11/07/12 17:39:44 (11 years ago)

Author:

nriga@bbn.com

Comment:

--

GENIExperimenter/Tutorials/HelloOmni

@@ +1 @@
+[[PageOutline]]
+This page will guide you through a simple GENI experiment using Omni. The only thing you will need a GENI account. If you don't already have one, [wiki:SignMeUp sign up!]
+= Prework = 
+In this tutorial we are going to run a simple experiment in GENI, using the
+[http://trac.gpolab.bbn.com/gcf/wiki/Omni Omni command line tool]. We are going to take advantage of the Layer 2
+links between nodes and run a non-IP experiment. 
+
+This is the topology of our experiment:
+
+Before we start with the Omni tutorial, please follow these steps:
+  i. Login to the VM
+  i. Run Firefox
+  i. Open all bookmarks under the Intro to Omni bookmark folder
+  i. Open a terminal window
+
+= 1. Configure Omni =
+Omni is a tool that will help you reserve resources in GENI, using your GENI
+account. The first step is to load into Omni your account information. 
+Omni uses a configuration file, and there is a script that will automatically generate the file with 
+some default options and place it in the default place of `~/.gcf/`. Follow
+these steps:
+ i. In the web browser login to `www.pgeni.gpolab.bbn.com` using the information
+ in the paper slip
+ i. On the menu on the left, press `Download your cert`
+  [[Image(Tutorials/Graphics:download_cert.png)]]
+ i. Select the option about downloading as a pem file. 
+ i. Press '''`Ctrl-s`''' to save this page. Type `~/.ssl/geni_cert.pem`, if file already exists replace it. 
+  [[Image(Tutorials/Graphics:save_cert.png, 40%)]]
+ i. In the terminal type:
+  {{{
+omni-configure.py
+}}} 
+  This should configure your Omni, with your downloaded certificate and create an ssh key pair for logging in to nodes. 
+  
+  '''Tip:''' When prompted for a passphrase, type the passphrase in your slip. You might be asked for a passphrase multiple times. '''Note:''' If you have taken other tutorials, you might have configured omni with other accounts, select to overwrite existing setting if asked.
+  
+  The output should look like:
+  {{{
+geni@geni-vm:~$ omni-configure.py 
+{'framework': 'pg', 'configfile': '~/.gcf/omni_config', 'plkey': '~/.ssh/geni_pl_key', 'cert': '~/.ssl/geni_cert', 'verbose': False}
+INFO:omniconfig:Using configfile: /home/geni/.gcf/omni_config
+INFO:omniconfig:Using certfile /home/geni/.ssl/geni_cert.pem
+INFO:omniconfig:
+
+        CREATING SSH KEYPAIR
+INFO:omniconfig:Private key stored at: /home/geni/.ssh/geni_key
+Enter passphrase: 
+INFO:omniconfig:Public key stored at: /home/geni/.ssh/geni_key.pub
+INFO:omniconfig:Added to /home/geni/.ssh/config this line:
+        'IdentityFile /home/geni/.ssh/geni_key
+'
+INFO:omniconfig:Wrote omni configuration file at: /home/geni/.gcf/omni_config
+}}}
+
+== 1a. Look around the omni_config file -- Optional ==
+Open the file `~/.gcf/omni_config` using either vim or emacs. Close to the top of the file you will see two parameters called `default_cf` and `users`. Your username should be at least listed in the user section. Look for the sections in the file that are named `[pg]` and  `[<username>]`.
+
+In the `[<username>]` section, the information need for logging-in to reserved compute resources are provided. It includes your unique user URN and a public key that would be uploaded to the hosts that you reserve.
+
+In the `[pg]` section you configure Omni to use your personal information. The cert and the key attribute point to files that we have manually downloaded from pgeni.gpolab.bbn.com. This is equivalent to the Download action of Flack. 
+
+Another interesting section to look at is the `[aggregate-nicknames]` sections. Flack already knows the URL for all the AMs and present you a list of AMs to choose from using a short, descriptive name. In Omni a user is required to pass the URL for each call to the GENI AM API. In this section the user gets a chance to provide short descriptive names to the URLs that are easier to memorize and use. 
+
+= 2. Clear the passphrase from your cert =
+When Omni talks to different authorities in GENI, it will need to identify you
+as a valid GENI user, so it will ask you for your passphrase multiple times. 
+To save time during the tutorial, we are going to remove the passphrase from your cert and your ssh key. There is a script to clear the passphrases. 
+  i. In the terminal run 
+  {{{
+clear-passphrases.py 
+}}}
+  Use the passphrase given to you in the paper slip, you might be prompted for a passphrase multiple times make sure you always use the same passphrase. 
+  The output should look like 
+  {{{
+geni@geni-vm:~$ clear-passphrases.py 
+Do you want to remove the passphrase from your cert (/home/geni/.ssl/geni_cert.pem) [Y,n]?y
+INFO:clearcert:
+        THIS SCRIPT WILL REPLACE /home/geni/.ssl/geni_cert.pem WITH AN UNENCREPTED CERT. A BACKUP OF THE ORIGINAL CERT WILL BE CREATED
+
+INFO:clearcert:The encoded certificate file is backed up at /home/geni/.ssl/geni_cert_enc.pem
+INFO:clearcert:Removing passphrase from cert...
+Enter pass phrase for /home/geni/.ssl/geni_cert.pem:
+writing RSA key
+INFO:clearcert:Change permissions of /home/geni/.ssl/geni_cert.pem to 0600
+Do you want to remove the passphrase from you ssh-key (/home/geni/.ssh/geni_key, key used to login to compute resources) [Y,n]?y
+INFO:clearcert:
+        THIS SCRIPT WILL REMOVE THE PASSPHRASE FROM YOUR SSH KEY. NO COPY OF THE ORIGINAL PRIVATE KEY WILL BE KEPT
+Enter PEM pass phrase:
+Enter passphrase:
+geni@geni-vm:~$ 
+}}}
+  i. Add the key to the ssh agent
+  {{{
+  ssh-add ~/.ssh/geni_key
+  }}}
+  i. Verify that you have the necessary credential and key files
+     {{{
+     ls ~/.ssh ~/.ssl
+     }}}
+     The output looks like :
+     {{{
+geni@geni-vm:~$ ls ~/.ssh ~/.ssl
+/home/geni/.ssh:
+config  geni_key  geni_key.pub
+
+/home/geni/.ssl:
+geni_cert_enc.pem  geni_cert.pem
+     }}}
+
+   || geni_cert.pem || Cleartext certificate, i.e. does not require any passphrase ||
+   || geni_cert_enc.pem || Encrypted certificate  ||
+   || geni_key || The private key that you will use to login to the nodes ||
+   || geni_key.pub || The corresponding public key that will be uploaded to the nodes ||
+
+  '''Note''': You might see more files than the ones listed above, which are
+  probably backup files from configuring omni, or files from other tutorials. 
+
+== 3. Test Omni setup ==
+In order to test that our configuration is correct, you can issue a `getversion`
+command. For this step we have used !PlanetLab's AM, but you can use any
+Aggregate Manager(AM). 
+  i. In the terminal type:
+  {{{
+omni.py getversion -a plc
+}}}
+  In GENI each AM can be contacted using a specific URL. In order to make it
+  easier to talk to AMs, Omni uses nicknames; i.e. it maps each URL into an easy
+  to remember nickname. For example the PlanetLab AM listens for requests at
+  `http://www.planet-lab.org:12346`, however you can use the nickname `plc` to
+  refer to that URL. Nicknames are defined in each user's omni_config file. 
+
+  For the rest of the tutorial we are going to use aggregate nicknames. 
+
+= 4. Create an experiment using Omni = 
+In this step we are going to setup a sample GENI experiment, in order to
+get familiarized with most of the Omni commands. 
+
+== 4a. Create and renew a slice ==
+The first thing to do when preparing to run a GENI experiment is to create a
+slice. 
+  i. To create a slice type, use the `<slicename>` given in your paper slip:
+  {{{
+  omni.py createslice <slicename>
+  }}}
+  ii. To verify that your slice has been created and in general to keep track of
+  your slices, use the `listmyslices` command. use the <username> given in your
+  paper slip. Type:
+  {{{
+  omni.py listmyslices <username>
+  }}}
+  iii. Renew your slice. To extend the lifetime of your slice type:
+  {{{
+  omni.py renewslice <slicename> 20121025
+  }}}
+
+== 4b. Reserve resources, a.k.a create a sliver ==
+The next step in a GENI experiment is to actually reserve resources. For this
+experiment we are going to use the Aggregate manager of ProtoGENI in Utah. 
+  i. In Omni in order to see what each AM offers you can use the
+  `listresources` command. Type:
+   {{{
+ omni.py listresources -a pg-utah -o
+}}}
+  The `-o` option will save the output to a file. The filename is chosen by Omni and printed as part of the output. The output will look like : 
+  {{{
+geni@geni-VirtualBox:~$ omni.py listresources -a pg-utah -o
+INFO:omni:Loading config file /home/geni/.gcf/omni_config
+INFO:omni:Using control framework pg
+INFO:omni:Saving output to a file.
+INFO:omni:Substituting AM nickname pg-utah with URL https://www.emulab.net/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
+INFO:omni:Listed resources on 1 out of 1 possible aggregates.
+INFO:omni:Writing to 'rspec-www-emulab-net-protogeniv2.xml'
+INFO:omni: ------------------------------------------------------------
+INFO:omni: Completed listresources:
+
+  Options as run:
+                aggregate: ['pg-utah']
+                framework: pg
+                output: True
+
+  Args: listresources
+
+  Result Summary: Queried resources from 1 of 1 aggregate(s).
+Wrote rspecs from 1 aggregate(s) to 1 file(s)
+Saved listresources RSpec at 'unspecified_AM_URN' (url 'https://www.emulab.net/protogeni/xmlrpc/am/2.0') to file rspec-www-emulab-net-protogeniv2.xml;  
+INFO:omni: ============================================================
+}}}
+  In the last line of the output Omni will tell you the name of the file that output is saved at. In the example above this would be `rspec-www-emulab-net-protogeniv2.xml`. Open the file that Omni saved and just take a look to see how an advertisement RSpec looks like. 
+  In order to see only available resources type 
+  {{{
+  omni.py listresources -a pg-utah --available -o
+  }}}
+  i. To be able to reserve resources you will need to craft a request rspec.
+  For this example we have created the rspecs for you, use the rspec URL given
+  in your paper slip. Type :
+  {{{
+  omni.py createsliver -a pg-utah <slicename> <rspec_url>
+  }}}
+  For example 
+  {{{
+  omni.py createsliver -a pg-utah iomni30 http://www.gpolab.bbn.com/experiment-support/gec15/omni-intro/rspecs/iomni-30.rspec
+  }}}
+  '''Tip:''' If you are copying the above line make sure to substitute the slicename and the actual rspec name with the ones that are specified in your paper slip. 
+  i. Look at your reserved resources. Type:
+  {{{
+  omni.py listresources -a pg-utah <slicename>
+  }}}
+  i. Extend the lifetime of your sliver:
+  {{{
+omni.py renewsliver -a pg-utah <slicename> 20121025
+  }}}
+  i. Check the status of your resources. Type:
+  {{{
+  omni.py sliverstatus -a pg-utah <slicename>
+  }}}
+  The `sliverstatus` command reports the status of your overall GENI slice. When
+  the status is ready we are ready to continue to the next step. 
+
+= 5. Run your experiment =
+Now that resources are ready we can start running our experiment. We will first
+start by logging in to our reserved nodes. 
+
+== 5a. Logging Into the nodes -- Omni Scripting ==
+Depending on the Aggregate the login information are either part of the
+`sliverstatus` call, or as part of the manifest rspec, i.e. returned from the
+`listresources` call. Omni comes with a script that makes it easy to gather all
+this information. 
+  i. In the terminal run:
+  {{{
+readyToLogin.py -a pg-utah <slicename>
+}}}
+  The script will return the actual command that you would need to use for logging in.
+  ii. Login to both nodes by copy-paste the commands from the output of the
+  script. 
+
+  [[Image(Tutorials/Graphics:readytologin_output.png, 40%)]]
+
+== 5b. Send IP traffic == 
+The first simple experiment that we will run is to verify the IP connectivity
+between our hosts. 
+  i. Check the interfaces of your nodes. In the terminal type:
+  {{{
+   /sbin/ifconfig
+  }}}
+  You should see at least two interfaces:
+    * The '''control interface'''. This is the interface from where you access
+    the node, e.g. ssh into your host. The control interface is mainly used for
+    control traffic, i.e. traffic for controlling the node and the experiment.
+    The control interface usually has a publicly routable IP. 
+    * The '''data interface'''. This is the interface that is used for sending
+    experimental traffic. This is the interface that connects to the other hosts
+    of your experiment through GENI. The links between these interfaces are the
+    ones that allow you to run non-IP experiments. 
+  i. Note the name and the MAC address of the control and of the data interface for each node. The data interface is the one that has an IP that starts with 10.
+  i. From the client ping the server. From the xterm that is logged in to the
+  client type :
+  {{{
+    ping server -c 5
+  }}}
+  '''Note''': You can use the name that is assigned to the host to directly ping
+  the host, you can also ping using the IP of the data interface of the node. 
+  An example output should look like :
+  {{{
+[[inki@r2 ~]$ ping server -c 5
+PING 10.10.2.1 (10.10.2.1) 56(84) bytes of data.
+64 bytes from 10.10.2.1: icmp_seq=1 ttl=64 time=1.95 ms
+64 bytes from 10.10.2.1: icmp_seq=2 ttl=64 time=2.27 ms
+64 bytes from 10.10.2.1: icmp_seq=3 ttl=64 time=2.04 ms
+64 bytes from 10.10.2.1: icmp_seq=4 ttl=64 time=1.95 ms
+64 bytes from 10.10.2.1: icmp_seq=5 ttl=64 time=1.72 ms
+
+--- 10.10.2.1 ping statistics ---
+5 packets transmitted, 5 received, 0% packet loss, time 4005ms
+rtt min/avg/max/mdev = 1.728/1.992/2.278/0.179 ms
+}}}
+
+== 5c. Send non-IP traffic ==
+GENI provides the capability of running non-IP experiments, since you can
+connect your hosts at Layer 2. For the purpose of this tutorial we have
+installed in all the hosts a very simple Layer 2 ping program that sends packets
+using a custom ethernet type. 
+  i. Disable the IP in your nodes. First of all we are going to completely
+  disable IP on our nodes. In each of the terminals type:
+  {{{
+  sudo /sbin/ifconfig <data i/f name> 0.0.0.0
+  }}}
+  '''Note''': Be extra careful to bring down the IP on the data interface,
+  bringing down the IP on the control interface means that you will lose
+  connectivity to your host. 
+  ii. Try again to ping from the client to the server. In the xterm of the
+  client type:
+  {{{
+  ping server -c 5
+  }}}
+  This time the ping should timeout.
+  iii. Start the Layer 2 ping server: In the server xterm, type:
+  {{{
+  sudo /usr/local/bin/pingPlusListener 12345
+  }}}
+  iii. From the client try to ping the server at layer 2. You will need the mac
+  address of the data interface of the server and the name of the data interface
+  of the client. In the xterm of the client, type:
+  {{{
+  sudo /usr/local/bin/pingPlus <server data mac addr> <client data i/f name> 12345
+  }}}
+
+  The result should look like :
+  {{{
+[uncusr20@client ~]$ sudo /usr/local/bin/pingPlus 02:00:9B:48:3E:20 mv1.1 12345
+RQ:'5582+2067' to 2:0:9b:48:3e:20.
+RQ:5582+2067 from 2:0:3e:15:6c:ab.
+}}}
+
+Congratulations you have run a Layer 2 experiment in GENI!
+
+= 6. Cleanup resources = 
+Although all your reservations, have expiration times, you should always release
+your resources once you have completed your experiment to make them available
+to other experimenters. 
+  i. In the terminal, where you have been running your omni commands do:
+  {{{
+omni.py deletesliver -a  pg-utah <slicename>
+}}}
+
+= 7. Wrap up =
+Congratulations, you have finished the tutorial! Now you are ready to design and
+run your own experiments. Don't hesitate to [mailto:help@geni.net email us] with
+any questions you might have. 
+
+== 7a. Fill out the tutorial survey ==
+We love to hear what you think so that we can improve the tutorials in the future. 
+Please complete the [http://tinyurl.com/8kxm6ys online survey] and get a temporary GENI tatoo!
+
+== 7b. Get Your Own Account ==
+The accounts, that you used in the above steps will be deactivated after the
+tutorial. If you do not already have an account at GPO's ProtoGENI cluster, you
+should can follow [wiki:GEC15Agenda/ExperimenterDropIn/GetAGENIAccount these instructions]
+to get an account and start using GENI.