Changes between Initial Version and Version 1 of GENIExperimenter/Tutorials/GENI-SAVI/DesignSetup


Ignore:
Timestamp:
05/22/15 15:15:51 (9 years ago)
Author:
acb@cs.princeton.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GENIExperimenter/Tutorials/GENI-SAVI/DesignSetup

    v1 v1  
     1= [wiki:GENIExperimenter/Tutorials/GENIExperimentEngine Get to Know the GENI Experiment Engine] =
     2{{{
     3#!html
     4<table border="0">
     5   
     6      <tr>
     7         <td>
     8         <a href="http://groups.geni.net/geni/wiki/GENIExperimenter/Tutorials/GENIExperimentEngine/DesignSetup"> <img border="0" src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/design_on.2.png?format=raw" alt="Hello GENI index"  height="90"  />  </a>
     9       </td>
     10       <td>
     11         <a href="http://groups.geni.net/geni/wiki/GENIExperimenter/Tutorials/GENIExperimentEngine/Execute"> <img border="0" src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/execute.2.png?format=raw" alt="Hello GENI index"  height="90" />  </a>
     12       </td>
     13       <td>
     14         <a href="http://groups.geni.net/geni/wiki/GENIExperimenter/Tutorials/GENIExperimentEngine/IntroToOmni/Finish"> <img border="0" src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/finish.2.png?format=raw" alt="Hello GENI index"  height="90" />  </a>
     15       </td>
     16     </tr>
     17 </table>
     18}}}
     19
     20----
     21
     22= STEPS FOR SETTING UP =
     23
     24----
     25
     26== 1. Get a GEE Slicelet and download GEE helper files ==
     27
     28Visit http://gee-project.org and login.  Once logged in, click on ''Get a slicelet''.  Reload the dashboard every few seconds until the slice is in state ''Running'' -- this should take about 30 seconds.  Then click on ''Download Slicelet File'' to download the GEE helper files.  Unpack the tarball and take a look at '''README.txt'''.
     29
     30----
     31
     32== 2. Learn about GEE networking ==
     33
     34Pick a node from your '''ssh-config''' file and log in to it with SSH.  For example:
     35
     36{{{
     37$ ssh -i id_rsa -F ssh-config slice338.pcvm3-1.instageni.metrodatacenter.com
     38}}}
     39
     40Type '''ifconfig eth0'''.  You should see an eth0 interface with an IP address in the 10.0.0.0/8 range, a ''private'' IP address. 
     41
     42{{{
     43root@slice347:~# ifconfig eth0
     44eth0      Link encap:Ethernet  HWaddr a6:24:c1:df:c7:09
     45          inet addr:10.20.0.243  Bcast:0.0.0.0  Mask:255.255.0.0
     46          inet6 addr: fe80::a424:c1ff:fedf:c709/64 Scope:Link
     47          UP BROADCAST RUNNING  MTU:1500  Metric:1
     48          RX packets:56555 errors:0 dropped:2 overruns:0 frame:0
     49          TX packets:49525 errors:0 dropped:0 overruns:0 carrier:0
     50          collisions:0 txqueuelen:1000
     51          RX bytes:93890170 (93.8 MB)  TX bytes:8052865 (8.0 MB)
     52}}}
     53
     54This IP address is behind a NAT and not directly reachable from outside (i.e., from the Internet).  However all nodes in your slicelet can contact each other using the private addresses.
     55
     56The nodes in your '''ssh-config''' file are Docker containers running on (virtual) hosts.  These hosts have ''public'' IP addresses that are routable from the Internet, that's why you can login to the Docker containers of your slicelet.  A port on the public IP address (e.g., 49155) is forwarded to the SSH port (22) on the container's private IP address.  The '''ssh-config''' file contains the names of the virtual hosts and the port forwarding information.   
     57
     58When you run a server on a node in your slicelet, it will listen on the private address, because that is the only one it can see.  By default it will only be visible to other nodes in your slicelet.  Docker provides facilities for exposing ports to the public Internet, but this topic is outside the scope of this tutorial. 
     59
     60''Note that later on you will need to find both the ''public'' and ''private'' IP addresses for each node in the slicelet.''
     61
     62----
     63
     64== 3. Configure the Ansible controller for your slicelet ==
     65
     66If you already have Ansible installed on your laptop, then you can use it as the controller for this experiment.  Otherwise, you will use one of the nodes in your slicelet as the Ansible controller.  Pick one, log into it, and install ansible using apt-get:
     67
     68{{{
     69$ apt-get update
     70$ apt-get install ansible
     71}}}
     72
     73Then upload your slicelet helper files to that node.  {{{scp}}} can be used for this; in the directory where you unzipped the files, run a command like the following:
     74
     75{{{
     76$ scp -F ssh-config * ansible-hosts slice323.pcvm3-1.instageni.metrodatacenter.com:
     77}}}
     78
     79|| [[Image(wiki:GENIExperimenter/Tutorials/Graphics:tip.png, nolink, 50px, bottom)]] || '''Pro Tip:''' remove your controller node from the '''ansible_hosts''' file after you’ve uploaded. ||
     80
     81----
     82
     83== 4. Learn some basic concepts of Ansible ==
     84
     85Ansible (http://docs.ansible.com) is a free, open-source, intuitive IT automation tool that is well-suited to the tasks in this tutorial.  Ansible commands can be run from the command line or put in a YAML file called a ''playbook''.  We will be creating an Ansible playbook to run the parameterized HTTP query described earlier.
     86
     87Two basic concepts in Ansible are ''inventories'' and ''modules''.  An inventory is a list of hosts to be managed by Ansible, organized into groups.  When you run Ansible commands, either from the command-line or in a playbook, you specify the host group that the command should operate on.  In this way Ansible commands can operate on many hosts in parallel.  Take a look at the Ansible inventory in your '''ansible-hosts''' file.  This is basically the equivalent of the '''ssh-config''' except its specialized for Ansible.
     88
     89A ''task'' in Ansible consists of a module and some arguments for the module.  A module provides a declarative abstraction on top of standard shell commands.  So for example, in the shell on an Ubuntu machine you might install package “foo” like this:
     90
     91{{{
     92$ sudo apt-get update
     93$ sudo apt-get install foo
     94}}}
     95
     96An equivalent Ansible task in a playbook would look like:
     97
     98{{{
     99- apt: name=foo state=latest update_cache=yes
     100}}}
     101
     102Or the same Ansible task could be invoked directly on the command line like this:
     103
     104{{{
     105$ ansible remote-machine -m apt -a "name=foo state=latest update_cache=yes"
     106}}}
     107
     108The task uses the '''apt''' module, and tells Ansible: “Make sure the latest version of package foo is installed”.  There are many other modules which are well-documented at http://docs.ansible.com.
     109
     110Here are a few Ansible tasks to run, to get some experience with the command-line interface. 
     111
     112=== (a) The ping module ===
     113
     114The '''ping''' module simply tries to do a SSH login to a node and reports success or failure.  Run the following command on your controller:
     115
     116{{{
     117$ ansible nodes -i ansible-hosts -m ping
     118}}}
     119
     120If you don’t see success everywhere then there is something wrong with your setup.  Ask one of the tutorial leaders for help. 
     121
     122=== (b) The shell module ===
     123
     124The '''shell''' module lets you run arbitrary SSH commands in parallel across a set of hosts.  It’s useful for poking around, or if there is no Ansible module with the functionality you need.  Try it out:
     125
     126{{{
     127$ ansible nodes -i ansible-hosts -m shell -a "hostname"
     128}}}
     129
     130You can replace ''hostname'' above with any other Linux command.
     131
     132=== (c) The setup module ===
     133
     134The '''setup''' module gathers a bunch of information about each node and saves it in variables that you can reference in your Ansible playbooks.  This will be really useful to do the tutorial!   Try it out on a node to see what it collects (replace `<your-slicelet>` with your slicelet’s name):
     135
     136{{{
     137$ ansible <your-slicelet>.pcvm1-1.instageni.wisc.edu -i ansible-hosts -m setup
     138}}}
     139
     140=== (d) A simple playbook ===
     141
     142Next, we will look at a simple Ansible playbook.  An Ansible playbook is a YAML file containing a list of Ansible tasks.  Copy the playbook below into a file called test.yaml:
     143
     144{{{
     145#!python
     146---
     147- hosts: nodes
     148  remote_user: root
     149  tasks:
     150  - name: An example of a debug statement
     151    debug: var=ansible_hostname
     152}}}
     153
     154Run the playbook as:
     155
     156{{{
     157$ ansible-playbook -i ansible-hosts test.yaml
     158}}}
     159
     160The '''setup''' module is run automatically at the beginning of a playbook to populate variables for each node.  The above playbook will dump the value of each node’s ''ansible_hostname'' variable.  To run the playbook on a single node, replace ''nodes'' with the name of one of your slice nodes (e.g., slice338.pcvm3-7.instageni.nps.edu).
     161
     162----
     163
     164== 5. Create and run an Ansible playbook to install the software you'll need ==
     165
     166Now you should have enough knowledge about Ansible to write a playbook to install the software you'll need on all the nodes.  Below is a [attachment:software-install.yaml skeleton Ansible playbook] that you can use.  You will need to fill in the bits marked `# INSERT ARGUMENTS HERE` to perform the actions described in the ''name'' lines.  If you get stuck at any point, you can take a look at this [attachment:software-install-solution.yaml solution].
     167
     168{{{
     169#!python
     170---
     171- hosts: nodes
     172  remote_user: root
     173  tasks:
     174  - name: Update apt cache
     175    apt: # INSERT ARGUMENTS HERE
     176
     177  - name: Install dnsutils (for dig)
     178    apt: # INSERT ARGUMENTS HERE
     179
     180  - name: Install geoip-bin (for geoiplookup)
     181    apt: # INSERT ARGUMENTS HERE
     182}}}
     183
     184Run this playbook on your Ansible control machine against all the nodes in your slice. 
     185
     186|| [[Image(wiki:GENIExperimenter/Tutorials/Graphics:tip.png, nolink, 50px, bottom)]] || '''Pro Tip:''' use the `-f` argument to `ansible-playbook` to speed things up -- it lets you control the number of nodes to operate on in parallel, and the default is 5.  Specifying `-f 20` will run the playbook's tasks against all your slicelet nodes in parallel.   ||
     187
     188
     189----
     190
     191= [wiki:GENIExperimenter/Tutorials/GENIExperimentEngine/Execute Next: Run Experiment] =