Changes between Version 9 and Version 10 of GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment
- Timestamp:
- 01/07/14 21:30:48 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment
v9 v10 77 77 78 78 79 === Observe the contents of the NEuca user data file. This file includes a script that will install and execute the script that you configured for the VM. ===79 === B. Observe the contents of the NEuca user data file. This file includes a script that will install and execute the script that you configured for the VM. === 80 80 {{{ 81 81 # neuca-user-data … … 110 110 111 111 112 === Observe the contents of the of the script that was installed and executed on the VM. ===112 === C. Observe the contents of the of the script that was installed and executed on the VM. === 113 113 {{{ 114 114 # cat /tmp/master.sh … … 148 148 149 149 150 === Test for connectivity between the VMs. ===150 === D. Test for connectivity between the VMs. === 151 151 152 152 {{{ … … 173 173 == 3. Check the status of the Hadoop filesystem. == 174 174 175 === Query for the status of the filesystem and its associated workers. ===175 === A. Query for the status of the filesystem and its associated workers. === 176 176 177 177 {{{ … … 215 215 216 216 217 == Test the filesystem with a small file ==218 219 220 === Create a small test file ===217 == 4. Test the filesystem with a small file == 218 219 220 === A. Create a small test file === 221 221 {{{ 222 222 # hadoop fs -put hello.txt hello.txt 223 223 }}} 224 224 225 === Push the file into the Hadoop filesystem ===225 === B. Push the file into the Hadoop filesystem === 226 226 {{{ 227 227 # hadoop fs -put hello.txt hello.txt 228 228 }}} 229 229 230 === C heck for the file's existence ===230 === C. Check for the file's existence === 231 231 {{{ 232 232 # hadoop fs -ls … … 235 235 }}} 236 236 237 === Check the contents of the file ===237 === D. Check the contents of the file === 238 238 {{{ 239 239 # hadoop fs -cat hello.txt … … 245 245 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset. It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test. 246 246 247 == Create a 1 GB random data set. After the data is created, use the \verb$ls$ functionally to confirm the data exists. Note that the data is composed of several files in a directory. == 247 === A. Create a 1 GB random data set. === 248 249 After the data is created, use the \verb$ls$ functionally to confirm the data exists. Note that the data is composed of several files in a directory. 250 248 251 {{{ 249 252 # hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar teragen 10000000 random.data.1G … … 268 271 }}} 269 272 270 == Sort the datasets. On your own, you can use the \verb$cat$ and/or \verb$get$ functionally to look at the random and sorted files to confirm their size and that the sort actually worked. == 273 === B. Sort the datasets. === 274 275 On your own, you can use Hadoop's cat and/or get functionally to look at the random and sorted files to confirm their size and that the sort actually worked. 271 276 272 277 {{{ … … 336 341 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker instance types. Warning: Be courteous to other users and do not take all the resources. 337 342 338 A. Time the performance of runs with different resources 339 B. Observe largest size file you can create with different settings. 343 === A. Time the performance of runs with different resources === 344 === B. Observe largest size file you can create with different settings. === 340 345 341 346
