Changes between Version 15 and Version 16 of GENIExperimenter/Tutorials/GettingStarted_PartII_Hadoop/Procedure/Execute


Ignore:
Timestamp:
03/09/14 13:41:20 (10 years ago)
Author:
sedwards@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIExperimenter/Tutorials/GettingStarted_PartII_Hadoop/Procedure/Execute

    v15 v16  
    6161
    6262  </li>
    63   <li>Now that your nodes are ready, go ahead and log into <tt>hadoop-master</tt>.</li>
     63  <li>Now that your nodes are ready, go ahead and log into <tt>hadoop-master</tt> as yourself (not as root).</li>
    6464</ol>
    6565}}}
     
    7474
    7575{{{
    76 # /sbin/ifconfig
     76$ /sbin/ifconfig
    7777eth0      Link encap:Ethernet  HWaddr fa:16:3e:72:ad:a6 
    7878          inet addr:10.103.0.20  Bcast:10.103.0.255  Mask:255.255.255.0
     
    107107====  5.1.2 Observe the contents of the NEuca user data file.  This file includes a script that will install and execute the script that you configured for the VM. ====
    108108{{{
    109 # neuca-user-data
     109$ neuca-user-data
    110110[global]
    111111actor_id=67C4EFB4-7CBF-48C9-8195-934FF81434DC
     
    142142
    143143{{{
    144 # cat /tmp/master.sh
    145 #!/bin/bash
     144$ cat /tmp/master.sh
     145$!/bin/bash
    146146
    147147 echo "Hello from neuca script" > /home/hadoop/log
     
    181181
    182182{{{
    183 # ping hadoop-worker-0
     183$ ping hadoop-worker-0
    184184PING hadoop-worker-0 (172.16.1.10) 56(84) bytes of data.
    18518564 bytes from hadoop-worker-0 (172.16.1.10): icmp_req=1 ttl=64 time=0.747 ms
     
    206206
    207207{{{
    208 # hadoop dfsadmin -report
     208$ hadoop dfsadmin -report
    209209Configured Capacity: 54958481408 (51.18 GB)
    210210Present Capacity: 48681934878 (45.34 GB)
     
    251251==== 5.3.1 Create a small test file ====
    252252{{{
    253 # echo Hello GENI World > hello.txt
    254 }}}
    255 
    256 ==== 5.3.2 Push the file into the Hadoop filesystem ===
    257 {{{
    258 # hadoop fs -put hello.txt hello.txt
    259 }}}
    260 
    261 ==== 5.3.3 Check for the file's existence ===
    262 {{{
    263 # hadoop fs -ls
     253$ echo Hello GENI World > hello.txt
     254}}}
     255
     256==== 5.3.2 Push the file into the Hadoop filesystem ====
     257{{{
     258$ hadoop fs -put hello.txt hello.txt
     259}}}
     260
     261==== 5.3.3 Check for the file's existence ====
     262{{{
     263$ hadoop fs -ls
    264264Found 1 items
    265265-rw-r--r--   3 root supergroup         12 2014-01-04 21:59 /user/root/hello.txt
    266266}}}
    267267
    268 ==== 5.3.4 Check the contents of the file ===
    269 {{{
    270 # hadoop fs -cat hello.txt
     268==== 5.3.4 Check the contents of the file ====
     269{{{
     270$ hadoop fs -cat hello.txt
    271271Hello GENI World
    272272}}}
     
    281281
    282282{{{
    283 #  hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar teragen 10000000 random.data.1G
     283$  hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar teragen 10000000 random.data.1G
    284284Generating 10000000 using 2 maps with step of 5000000
    28528514/01/05 18:47:58 INFO mapred.JobClient: Running job: job_201401051828_0003
     
    307307
    308308{{{
    309 # hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar terasort random.data.1G sorted.data.1G
     309$ hadoop jar /usr/local/hadoop-0.20.2/hadoop-0.20.2-examples.jar terasort random.data.1G sorted.data.1G
    31031014/01/05 18:50:49 INFO terasort.TeraSort: starting
    31131114/01/05 18:50:49 INFO mapred.FileInputFormat: Total input paths to process : 2