Changes between Version 7 and Version 8 of GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment


Ignore:
Timestamp:
01/07/14 21:17:52 (10 years ago)
Author:
pruth@renci.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GENIExperimenter/Tutorials/HadoopInASlice/ExecuteExperiment

    v7 v8  
    2222Now that you have reserved your resources, you are ready to...
    2323
    24 == 9. Login to nodes ==
     24== 1. Login to Hadoop Master ==
    2525{{{
    2626#!html
    … …  
    3131
    3232       <td >
    33          <ol>
    34            <li>Login (ssh) to the hadoop-master using a yourself using the key you associated with the
     33         <ol type="A">
     34           <li>Login (ssh) to the hadoop-master using the key you associated with the
    3535GENI Portal and the IP address displayed by Flack. The ssh application you use will
    3636depend on the configuration of laptop/desktop that you are using.</li>
    37            <li>Check the status/properties of the VMs.</li>
    38 
    39 <ol type="a">
     37         </op>
     38</td></tr>
     39</table>
     40}}}
     41
     42== 2. Check the status/properties of the VMs. ==
     43
     44{{{
     45#!html
     46
     47
     48<table border="0">
     49      <tr>
     50
     51       <td >
     52         <ol type="A">
     53           
    4054<li> Observe the properties of the network interfaces </li>
    4155
    … …  
    168182rtt min/avg/max/mdev = 0.468/0.607/0.852/0.174 ms
    169183</code></pre>
    170 
    171 
    172 <li> Check the status of the Hadoop filesystem</li>
     184</op>
     185
     186</td></tr>
     187</table>
     188}}}
     189
     190== 3. Check the status of the Hadoop filesystem. ==
     191
     192{{{
     193#!html
     194
     195
     196<table border="0">
     197      <tr>
     198
     199       <td >
     200         <ol type="A">
     201
     202
     203<li> Query for the status of the filesystem and its associated workers. </li>
    173204<pre><code>
    174205# hadoop dfsadmin -report
    … …  
    210241
    211242
     243
    212244<li> Test the filesystem with a small file </li>
    213245
    … …  
    237269
    238270</ol>
    239 
    240 
    241 <li> Test the true power of the Hadoop filesystem by creating and sorting a large random dataset.   It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test.  </li>
    242 
    243 
    244 <ol type="a">
     271</op>
     272
     273</td></tr>
     274</table>
     275}}}
     276
     277== 4.   Run the Hadoop Sort Testcase ==
     278
     279 Test the true power of the Hadoop filesystem by creating and sorting a large random dataset.   It may be useful/interesting to login to the master and/or worker VMs and use tools like \verb$top$, \verb$iotop$, and \verb$iftop$ to observe the resource utilization on each of the VMs during the sort test.
     280
     281{{{
     282#!html
     283
     284
     285<table border="0">
     286      <tr>
     287
     288       <td >
     289         <ol type="A">
    245290<li> Create a 1 GB random data set.  After the data is created, use the \verb$ls$ functionally to confirm the data exists.  Note that the data is composed of several files in a directory. </li>
    246291<pre><code>
    … …  
    331376</ol>
    332377
    333 <li>Re-do tutorial with a different number of workers, amount of bandwidth, and/or worker  instance types.  Warning:  Be courteous to  other users and do not take all the resources.  </li>
    334 
    335 <ol type="a">
     378</td></tr>
     379</table>
     380}}}
     381
     382== 5.   Advanced Example ==
     383
     384 Re-do the tutorial with a different number of workers, amount of bandwidth, and/or worker  instance types.  Warning:  Be courteous to  other users and do not take all the resources.
     385
     386{{{
     387#!html
     388
     389
     390<table border="0">
     391      <tr>
     392
     393       <td >
     394         <ol type="A">
     395
     396
    336397<li> Time the performance of runs with different resources </li>
    337398<li> Observe largest size file you can create with different settings. </li>