Version 52 (modified by 10 years ago) (diff) | ,
---|
Understanding the AM API using a simple Hadoop experiment
1. Design the Experiment
|
2. Establish the Environment
2.1 Pre-work: Ensure SSH keys are setup
Verify that you have at least one public key associated with your account. To do that, after you login to the portal check under your Profile, under the SSH keys
tab. If you do not have SSH keys associated yet, please follow the instructions on that tab of the Portal.
2.2 Configure Omni
|
Figure 2-1 Click on the Configure omni tab under Profile. |
- Click on the `Download your omni data` button under step 2.
Figure 2-2 Download your omni data under step 2.
- If this is the first time you try to access your GENI certificate you will have to generate one. Click on the `generate a certificate` link.
Figure 2-3 Click on Generate a certificate.
- Unless you really understand how SSL certificates work, choose the simple option. Click on the `Generate Combined Certificate and Key File` button and then click on `Close`. You will be taken back to the download page with the warning. Reload the page to enable the download button.
Figure 2-4 Click on Generate Combined Certificate and Key File.
- If you are a member of more than one project, select which project you would like to be the default one for running experiments in GENI. You can always change the project that is used by the `-r` command line option of Omni. Then click on `Download your omni data`.
Figure 2-5 Click on Download your omni data.
|
3. Obtain Resources
3.1 Create a slice
Create a slice using omni
and the slice name of your choice. From now on that slice name will be referred to as SLICENAME
.
$ omni createslice SLICENAME
3.2. Load a simple topology in Flack
For this exercise, we will edit an existing RSpec file. Start by loading this predefined topology into Flack.
|
Figure 3-2 Import an RSpec into Flack. |
|
3.3. Modify the RSpec to automatically install and execute network test software
For this experiment, we'd like to run a simple experiment with Hadoop using one Hadoop master and two Hadoop worker nodes.
|
Figure 3-3 Edit the nodes |
||||
|
|
Field | Value |
---|---|
Name | hadoop-master |
Sliver Type | XOLarge |
Disk Image Name | http://geni-images.renci.org/images/GENIWinterCamp/images/gwc-hadoop.v0.4a.xml |
Disk Image Version | 16ff128df4cf10f2472a8d20796146bcd5a5ddc3 |
Install Archive URL | http://geni-images.renci.org/images/GENIWinterCamp/master.sh |
Install Location | /tmp |
Execute Service | chmod +x /tmp/master.sh; /tmp/master.sh |
Execute Service Shell | sh |
Table 3-1 The details for the hadoop-master node.
Figure 3-5 Configure the details on the hadoop-master node.
Field | Value |
---|---|
Name | hadoop-worker-0 |
Sliver Type | XOMedium |
Disk Image Name | http://geni-images.renci.org/images/GENIWinterCamp/images/gwc-hadoop.v0.4a.xml |
Disk Image Version | 16ff128df4cf10f2472a8d20796146bcd5a5ddc3 |
Install Archive URL | http://geni-images.renci.org/images/GENIWinterCamp/worker.sh |
Install Location | /tmp |
Execute Service | chmod +x /tmp/worker.sh; /tmp/worker.sh $hadoop-master.Name() $hadoop-master.IP("link0") $hadoop-worker-0.Name() $hadoop-worker-0.IP("link0") |
Execute Service Shell | sh |
Table 3-2 The details for the hadoop-worker-0 node.
Field | Value |
---|---|
Name | hadoop-worker-1 |
Sliver Type | XOMedium |
Disk Image Name | http://geni-images.renci.org/images/GENIWinterCamp/images/gwc-hadoop.v0.4a.xml |
Disk Image Version | 16ff128df4cf10f2472a8d20796146bcd5a5ddc3 |
Install Archive URL | http://geni-images.renci.org/images/GENIWinterCamp/worker.sh |
Install Location | /tmp |
Execute Service | chmod +x /tmp/worker.sh; /tmp/worker.sh $hadoop-master.Name() $hadoop-master.IP("link0") $hadoop-worker-1.Name() $hadoop-worker-1.IP("link0") |
Execute Service Shell | sh |
Table 3-3 The details for the hadoop-worker-1 node.
Figure 3-6 Configure the details on the hadoop-worker-0 node.
|
Figure 3-7 Configure the link bandwidth |
Table 3-4 The details for link0. |
Figure 3-8 Configure link0.
Optional:Download and view the worker.sh
and master.sh
files yourself. This is a good example of how you can ask for GENI components to automatically perform tasks for you; very useful for large experiments!
3.4. Export the modified request RSpec
Let's look at the RSpec that Flack has prepared for us behind the scenes. Nodes and links have a corresponding element in the RSpec, and the details of the component configuration (such as the install and execute services we requested above) are specified with attributes, or sometimes child elements, within those portions of the document.
|
Figure 3-9 View and save the final request RSpec |
3.5. Instantiate the new experiment using Omni
For this step, we are going to use the Omni client that you have configured during setup.
From a terminal, please enter the command:
$ omni -a AM_NICKNAME createsliver SLICENAME RSPEC_FILE
where AM_NICKNAME
is the nickname for your assigned aggregate manager and SLICENAME
is the name of the slice you created earlier (both of these are given on your worksheet; if you don't have a worksheet and are using the RENCI ExoGENI rack then use an AM_NICKNAME
of renci-eg
). RSPEC_FILE
should be replaced with the filename of the RSpec you saved in step 4.
If all is well, Omni should give you a number of informational messages, such as:
INFO:omni:Loading config file /home/geni/.gcf/omni_config
It should quickly proceed to the point where it makes the request to the remote manager:
INFO:omni:Creating sliver(s) from rspec file /home/geni/Downloads/experiments.rspec for slice ...
This step can sometimes be time-consuming, so please be patient. If it succeeds, within a couple of minutes Omni should report:
INFO:omni: Completed createsliver:
and your resource reservation is complete!