Changes between Version 10 and Version 11 of GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment
- Timestamp:
- 01/08/14 08:53:49 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment
v10 v11 20 20 = Instructions = 21 21 22 Now that you have reserved your resources, you are ready to ...22 Now that you have reserved your resources, you are ready to login to the slice and run some Hadoop examples. 23 23 24 24 == 1. Login to Hadoop Master == … … 33 33 <td > 34 34 <ol type="A"> 35 <li>Login (ssh) to the hadoop-master using the key youassociated with the35 <li>Login (ssh) to the hadoop-master using the credentials associated with the 36 36 GENI Portal and the IP address displayed by Flack. The ssh application you use will 37 depend on the configuration of laptop/desktop that you are using.</li>37 depend on the configuration of your laptop/desktop.</li> 38 38 </op> 39 39 </td></tr> … … 220 220 === A. Create a small test file === 221 221 {{{ 222 # hadoop fs -put hello.txthello.txt222 # echo Hello GENI World > hello.txt 223 223 }}} 224 224 … … 243 243 == 4. Run the Hadoop Sort Testcase == 244 244 245 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset. It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test.245 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset. It may be useful/interesting to login to the master and/or worker VMs and use tools like top, iotop, and iftop to observe the resource utilization on each of the VMs during the sort test. Note: on these VMs iotop and iftop must be run as root. 246 246 247 247 === A. Create a 1 GB random data set. === … … 271 271 }}} 272 272 273 === B. Sort the dataset s. ===274 275 On your own,you can use Hadoop's cat and/or get functionally to look at the random and sorted files to confirm their size and that the sort actually worked.273 === B. Sort the dataset. === 274 275 Note: you can use Hadoop's cat and/or get functionally to look at the random and sorted files to confirm their size and that the sort actually worked. 276 276 277 277 {{{ … … 339 339 == 5. Advanced Example == 340 340 341 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker instance types. Warning: Be courteous to other users and do not take allthe resources.342 343 === A. Time the performance of runs with different resources ===344 === B. Observe largest size file you can create with different settings. ===341 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker instance types. Warning: be courteous to other users and do not use too many of the resources. 342 343 === A. Time the performance of runs with different resources. === 344 === B. Observe largest size file you can create with different resources. === 345 345 346 346