= [.. Hadoop in a Slice] =
'''We are actively updating this tutorial. If you need help with this tutorial please contact: help@geni.net'''
== Part I: Obtain Resources: create a slice and reserve resources ==
{{{
#!html
}}}
= 1. Establish the Environment =
== 1.1 Pre-work: Ensure SSH keys are setup ==
Verify that you have at least one public key associated with your account. To do that, after you login to the portal check under your Profile, under the `SSH keys` tab. If you do not have SSH keys associated yet, please follow the instructions on that tab of the Portal.
== 1.2 Configure Omni ==
If you have not installed and configured omni yet, please follow [http://trac.gpolab.bbn.com/gcf/wiki/Omni these instructions].
= 2. Obtain Resources =
== 2.1 Create a slice Slice ==
Create a slice using `omni` and the slice name of your choice. From now on that slice name will be referred to as `SLICENAME`.
{{{
$ omni createslice SLICENAME
}}}
== 2.2 Create your RSpec ==
We are going to use the graphical interface of the portal (Jacks) to create the RSpec file for this tutorial but we are going to use Omni for reserving the resources.
=== 2.2.1 Load a simple topology in Jacks ===
{{{
#!html
- In the Portal, open the Slice page for the slice you just created. Notice that you created the slice with omni and it is available via the Portal.
- Press the Add Resources button to launch Jacks for this slice.
- From the Choose RSpec menu (see figure), select the URL button.
- Enter the URL for the RSpec:
http://geni-images.renci.org/images/tutorials/GENI-hadoop/hadoop_dynamic_request_rspec.xml
then click Select.
- After you click Select, a couple of nodes will appear.
|
}}}
=======
{{{
#!html
- Click "Site 0" in the Jacks window
- Select an ExoGENI site (If you are participating in an organized tutorial, please bind the VMs to the rack assigned to you)
|
|
}}}
== 3. Create the Hadoop Master ==
{{{
#!html
- Add an ExoGENI VM to your slice.
|
- Select the VM to set its properties:
- Name: hadoop-master
- Node Type: ExoGENI Medium
- Custom Disk Image Name: http://geni-images.renci.org/images/standard/hadoop/hadoop_debian6.v1.0/hadoop_debian6.v1.0.xml
- Disk Image Version: 54ddc5ed7d4a22331651d7b78dc5d39138f8a165
- Add an Install Script:
URL: http://geni-images.renci.org/images/tutorials/GENI-hadoop/master.sh Path: /tmp
- Add an execute service to execute the script at boot time:
chmod +x /tmp/master.sh; /tmp/master.sh
|
}}}
== 3. Create the Hadoop Workers ==
{{{
#!html
- Add a 2 more VMs to the same rack as the first VM.
|
|
- Edit each worker’s attributes
- Names: hadoop-worker-0 and hadoop-worker-1
- Node Type: ExoGENI Medium
- Disk Image Name:
http://geni-images.renci.org/images/standard/hadoop/hadoop_debian6.v1.0/hadoop_debian6.v1.0.xml
- Disk Image Version: 54ddc5ed7d4a22331651d7b78dc5d39138f8a165
- Add an Install Script:
URL: http://geni-images.renci.org/images/tutorials/GENI-hadoop/worker.sh Path: /tmp
- Add an execute service to execute the script at boot time. For each VM, substitute
the VM’s name for where the following uses “hadoop-worker-0”. (Note: the following
should be placed on one line)
chmod +x /tmp/worker.sh; /tmp/worker.sh $hadoop-master.Name() $hadoop-master.IP("link-0") $hadoop-worker-0.Name() $hadoop-worker-0.IP("link-0")
- Repeat for worker 1
|
}}}
== 4. Create the Network ==
{{{
#!html
- Link all three VMs with a broadcast network
- Set the master's IP to 172.16.1.1
- Set the master's netmask to 255.255.255.0
- Set the worker0's IP to 172.16.1.10
- Set the worker0's netmask to 255.255.255.0
- Set the worker1's IP to172.16.1.11
- Set the worker1's netmask to 255.255.255.0
|
}}}
== 5. Instantiate the Slice ==
You can use any tool to reserve this topology, today we are going to use Omni. To do that you will need to:
{{{
#!html
- Download the rspec and save it to your local disk by clicking the download button
- Submit the request by running this omni command in a command line window:
omni createsliver slicename rspec_filename -a AM_NICKNAME
- Wait until the slice is up
- Log into the master node
|
}}}
----
= [.. Introduction] =
= [../ExecuteExperiment Next: Execute the Hadoop Experiment] =