Changes between Version 9 and Version 10 of GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment


Ignore:
Timestamp:
01/07/14 21:30:48 (6 years ago)
Author:
pruth@renci.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment

    v9 v10  
    7777 
    7878
    79 ===  Observe the contents of the NEuca user data file.  This file includes a script that will install and execute the script that you configured for the VM. ===
     79===  B. Observe the contents of the NEuca user data file.  This file includes a script that will install and execute the script that you configured for the VM. ===
    8080{{{
    8181# neuca-user-data
     
    110110
    111111
    112 === Observe the contents of the of the script that was installed and executed on the VM. ===
     112=== C. Observe the contents of the of the script that was installed and executed on the VM. ===
    113113{{{
    114114# cat /tmp/master.sh
     
    148148
    149149
    150 === Test for connectivity between the VMs. ===
     150=== D. Test for connectivity between the VMs. ===
    151151
    152152{{{
     
    173173== 3. Check the status of the Hadoop filesystem. ==
    174174
    175 === Query for the status of the filesystem and its associated workers. ===
     175===  A. Query for the status of the filesystem and its associated workers. ===
    176176
    177177{{{
     
    215215
    216216
    217 == Test the filesystem with a small file ==
    218 
    219 
    220 === Create a small test file ===
     217== 4. Test the filesystem with a small file ==
     218
     219
     220=== A. Create a small test file ===
    221221{{{
    222222# hadoop fs -put hello.txt hello.txt
    223223}}}
    224224
    225 === Push the file into the Hadoop filesystem ===
     225=== B. Push the file into the Hadoop filesystem ===
    226226{{{
    227227# hadoop fs -put hello.txt hello.txt
    228228}}}
    229229
    230 === Check for the file's existence ===
     230=== C. Check for the file's existence ===
    231231{{{
    232232# hadoop fs -ls
     
    235235}}}
    236236
    237 ===  Check the contents of the file ===
     237===  D. Check the contents of the file ===
    238238{{{
    239239# hadoop fs -cat hello.txt
     
    245245 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset.   It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test.
    246246
    247 ==  Create a 1 GB random data set.  After the data is created, use the \verb$ls$ functionally to confirm the data exists.  Note that the data is composed of several files in a directory. ==
     247===  A. Create a 1 GB random data set.   ===
     248
     249After the data is created, use the \verb$ls$ functionally to confirm the data exists.  Note that the data is composed of several files in a directory.
     250
    248251{{{
    249252#  hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar teragen 10000000 random.data.1G
     
    268271}}}
    269272
    270 == Sort the datasets.  On your own, you can use the \verb$cat$ and/or \verb$get$ functionally to look at the random and sorted files to confirm their size and that the sort actually worked. ==
     273=== B. Sort the datasets. === 
     274
     275On your own, you can use Hadoop's cat and/or get  functionally to look at the random and sorted files to confirm their size and that the sort actually worked.
    271276
    272277{{{
     
    336341 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker  instance types.  Warning:  Be courteous to  other users and do not take all the resources.
    337342
    338 A. Time the performance of runs with different resources
    339 B. Observe largest size file you can create with different settings.
     343=== A. Time the performance of runs with different resources  ===
     344=== B. Observe largest size file you can create with different settings. ===
    340345
    341346