Changes between Version 10 and Version 11 of GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment


Ignore:
Timestamp:
01/08/14 08:53:49 (10 years ago)
Author:
pruth@renci.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment

    v10 v11  
    2020= Instructions =
    2121
    22 Now that you have reserved your resources, you are ready to...
     22Now that you have reserved your resources, you are ready to login to the slice and run some Hadoop examples.
    2323
    2424== 1. Login to Hadoop Master ==
     
    3333       <td >
    3434         <ol type="A">
    35            <li>Login (ssh) to the hadoop-master using the key you associated with the
     35           <li>Login (ssh) to the hadoop-master using the credentials associated with the
    3636GENI Portal and the IP address displayed by Flack. The ssh application you use will
    37 depend on the configuration of laptop/desktop that you are using.</li>
     37depend on the configuration of your laptop/desktop.</li>
    3838         </op>
    3939</td></tr>
     
    220220=== A. Create a small test file ===
    221221{{{
    222 # hadoop fs -put hello.txt hello.txt
     222# echo Hello GENI World > hello.txt
    223223}}}
    224224
     
    243243== 4.   Run the Hadoop Sort Testcase ==
    244244
    245  Test the true power of the Hadoop filesystem by creating and sorting a large random dataset.   It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test.
     245 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset.   It may be useful/interesting to login to the master and/or worker VMs and use tools like top, iotop, and iftop to observe the resource utilization on each of the VMs during the sort test.  Note: on these VMs iotop and iftop must be run as root.
    246246
    247247===  A. Create a 1 GB random data set.   ===
     
    271271}}}
    272272
    273 === B. Sort the datasets. === 
    274 
    275 On your own, you can use Hadoop's cat and/or get  functionally to look at the random and sorted files to confirm their size and that the sort actually worked.
     273=== B. Sort the dataset. === 
     274
     275Note: you can use Hadoop's cat and/or get  functionally to look at the random and sorted files to confirm their size and that the sort actually worked.
    276276
    277277{{{
     
    339339== 5.   Advanced Example ==
    340340
    341  Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker  instance types.  Warning:  Be courteous to  other users and do not take all the resources.
    342 
    343 === A. Time the performance of runs with different resources  ===
    344 === B. Observe largest size file you can create with different settings. ===
     341 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker  instance types.  Warning:  be courteous to  other users and do not use too many of the resources.
     342
     343=== A. Time the performance of runs with different resources.  ===
     344=== B. Observe largest size file you can create with different resources. ===
    345345
    346346