Version 30 (modified by 10 years ago) (diff) | ,
---|
Understanding the AM API using a simple Hadoop experiment
1. Design the Experiment
|
2. Establish the Environment
2.1 Pre-work: Ensure SSH keys are setup
Verify that you have at least one public key associated with your account. To do that, after you login to the portal check under your Profile, under the SSH keys
tab. If you do not have SSH keys associated yet, please follow the instructions on that tab of the Portal.
2.2 Configure Omni
|
Figure 2-1 Click on the Configure omni tab under Profile. |
- Click on the `Download your omni data` button under step 2.
Figure 2-2 Download your omni data under step 2.
- If this is the first time you try to access your GENI certificate you will have to generate one. Click on the `generate a certificate` link.
Figure 2-3 Click on Generate a certificate.
- Unless you really understand how SSL certificates work, choose the simple option. Click on the `Generate Combined Certificate and Key File` button and then click on `Close`. You will be taken back to the download page with the warning. Reload the page to enable the download button.
Figure 2-4 Click on Generate Combined Certificate and Key File.
- If you are a member of more than one project, select which project you would like to be the default one for running experiments in GENI. You can always change the project that is used by the `-r` command line option of Omni. Then click on `Download your omni data`.
Figure 2-5 Click on Download your omni data.
- The bundle will be saved at
~/Downloads/omni-bundle.zip
- Open a terminal window and type:
omni-configure
The cert and key files you need will be installed in the appropriate folders.
3. Obtain Resources
3.1 Create a slice
Create a slice using omni
and the slice name of your choice. From now on that slice name will be referred to as SLICENAME
.
$ omni createslice SLICENAME
3.2. Load a simple topology in Flack
For this exercise, we will edit an existing RSpec file. Start by loading this predefined topology into Flack.
|
Figure 3-2 Import an RSpec into Flack. |
|
|
3.3. Modify the RSpec to automatically install and execute network test software
For this experiment, we'd like to run a simple experiment with Hadoop using one Hadoop master and two Hadoop worker nodes.
|
Figure 3-3 Edit the nodes |
||||
|
|
- For the link, go to the Interfaces tab and set the IP addresses and masks as specified in the table below.
- Switch to the properties tab and set the link’s bandwidth 100Mb/s Set capacity: 100000000 (note: Flack will report that this is 100Gb/s but it is actually 100Mb/s)
Figure 3-5 Configure the link bandwidth
Table 3-1 The details for the hadoop-master node. |
Figure 3-6 Configure the details on the hadoop-master node.
Table 3-2 The details for the hadoop-worker-0 node.
Table 3-3 The details for the hadoop-worker-1 node. |
Figure 3-7 Configure the details on the hadoop-worker-0 node.
Table 3-4 The details for link0. |
Figure 3-8 Configure link0.
If you have time, it would be instructive to download the worker.sh
and master.sh
files yourself, and inspect the contents. This is a good example of how you can ask for GENI components to perform tasks for you without any manual intervention. This is a very useful facility for large experiments!
3.4. Export the modified request RSpec
Now we will pull back some of the covers and inspect exactly what Flack has been doing for us when preparing the RSpecs for the experiments we design. Each node and link has a corresponding element in the RSpec, and the details of the component configuration (such as the install and execute services we requested above) are specified with attributes, or sometimes child elements, within those portions of the document.
|
Figure 3-6 View and save the final request RSpec |
3.5. Instantiate the new experiment using Omni
For this step, we'll change the approach a bit and switch to a new client tool, the command line Omni client.
From a terminal, please enter the command:
$ omni -a AM_NICKNAME createsliver SLICENAME RSPEC_FILE
where AM_NICKNAME
is the nickname for your assigned aggregate manager and SLICENAME
is the name of the slice you created earlier (both of these are given on your worksheet). RSPEC_FILE
should be replaced with the filename of the RSpec you saved in step 4.
If all is well, Omni should give you a number of informational messages, such as:
INFO:omni:Loading config file /home/geni/.gcf/omni_config
It should quickly proceed to the point where it makes the request to the remote manager:
INFO:omni:Creating sliver(s) from rspec file /home/geni/Downloads/experiments.rspec for slice ...
This step can sometimes be time-consuming, so please be patient. If it succeeds, within a couple of minutes Omni should report:
INFO:omni: Completed createsliver:
and your resource reservation is complete!