wiki:GENIExperimenter/Tutorials/jacks/HadoopInASlice/ObtainResources

Version 14 (modified by pruth@renci.org, 9 years ago) (diff)

--

Hadoop in a Slice

Part I: Obtain Resources: create a slice and reserve resources

Image Map

Instructions

1. Create a slice

  1. Login to https://portal.geni.net
  2. Create a new slice in one of your projects
  3. Launch Jacks (use the beta version available in the GENI portal )
  4. Click "Add Resources"

2. Bind the Slice

  1. Click the "Site 0" in the Jacks window
  2. Select an ExoGENI site (If you are participating in an organized tutorial, please bind the VMs to the rack assigned to you)
Bind the Slice

3. Create the Hadoop Master

  • Custom Disk Image Name: http://geni-images.renci.org/images/GENIWinterCamp/images/gwc-hadoop.v0.4a.xml
  • Disk Image Version: 16ff128df4cf10f2472a8d20796146bcd5a5ddc3
  • Set the Master Image
  • Add an Install Script:
    URL: http://geni-images.renci.org/images/GENIWinterCamp/master.sh
    Path: /tmp
  • Add an execute service to execute the script at boot time:
    chmod +x /tmp/master.sh; /tmp/master.sh
  • Set the Master Execute Scripts
    1. Add an ExoGENI VM to your slice.
    Add the Hadoop Master
    1. Select the VM to set its properties:
      1. Name: hadoop-master
      2. Node Type: ExoGENI Medium
    Set the Master Name
    Set the Master Install Script

    3. Create the Hadoop Workers

    1. Add a 2 more VMs to the same rack as the first VM.
    Set the Master Execute Scripts
    1. Edit each worker’s attributes
      1. Names: hadoop-worker-0 and hadoop-worker-1
      2. Sliver Type: XOMedium
      3. Set the Master Execute Scripts
      4. Disk Image Name: http://geni-images.renci.org/images/GENIWinterCamp/images/gwc-hadoop.v0.4a.xml
      5. Disk Image Version: 16ff128df4cf10f2472a8d20796146bcd5a5ddc3
      6. Set the Master Execute Scripts
      7. Add an Install Script:
        URL: http://geni-images.renci.org/images/GENIWinterCamp/worker.sh
        Path: /tmp
      8. Set the Master Execute Scripts
      9. Add an execute service to execute the script at boot time. For each VM, substitute the VM’s name for where the following uses “hadoop-worker-0”. (Note: the following should be placed on one line)
        chmod +x /tmp/worker.sh; /tmp/worker.sh $hadoop-master.Name() $hadoop-master.IP("link-0") $hadoop-worker-0.Name() $hadoop-worker-0.IP("link-0")
      10. Set the Master Execute Scripts
      11. Repeat for worker 1
      12. Set the Master Execute Scripts

    4. Create the Network

    1. Link all three VMs with a broadcast network
    Set the Master Execute Scripts
    1. Set the master's IP to172.16.1.1
    2. Set the master's netmask to 255.255.255.0
      1. Set the Master Execute Scripts
      2. Set the worker0's IP to172.16.1.10
      3. Set the worker0's netmask to 255.255.255.0
        1. Set the Master Execute Scripts
        2. Set the worker0's IP to172.16.1.11
        3. Set the worker0's netmask to 255.255.255.0
          1. Set the Master Execute Scripts

    5. Instantiate the Slice

    1. Submit the request
    2. Wait until the slice is up
    Add the Hadoop Master

    Introduction

    Next: Execute the Hadoop Experiment

    Attachments (24)