64 | | ======= |
| 64 | |
| 65 | The topology you loaded has two node-types: `master` and `worker`. Let's see what each node-type is comprised of: |
| 66 | |
| 67 | {{{ |
| 68 | #!html |
| 69 | <table> |
| 70 | <tr><td> |
| 71 | <ol type='a' start='6'> |
| 72 | <li>Inspect the properties of the hadoop master. Note that not only the usual attributes (e.g. node name, node type) are set, but there are also custom OS Image, install and execute attributes. These attributes is what customizes a generic GENI node a specialized one like a hadoop master. |
| 73 | </ol> |
| 74 | </td> |
| 75 | </tr> |
| 76 | </table> |
| 77 | }}} |
| 78 | |
| 79 | |
| 80 | === 2.2.2 Create the topology for the experiment === |
| 81 | |
| 82 | |
| 83 | Now we will create our Hadoop cluster. For the cluster we need: |
| 84 | * 1 hadoop master |
| 85 | * 2 or more workers |
| 86 | * all the nodes need to be on the same Layer 2 Domain and IP Subnet (172.16.1.0/24) |
| 87 | So let's go ahead and draw our topology: |
| 88 | |
| 89 | {{{ |
| 90 | #!html |
| 91 | <table> |
| 92 | <tr><td> |
| 93 | <ol type='a'> |
| 94 | <li>Select the worker node. </li> |
| 95 | <li>Press the "Duplicate Nodes only" button. </li> |
| 96 | <li> Repeat the above two steps so that you have at least three workers but no more than six (please be respectful to the cluster size you create, especially if you are doing this in the context of a tutorial). </li> |
| 97 | <li> Delete the original worker node (select it and on the attributes on the left click the Delete button).<br> |
| 98 | <b> DO NOT </b> press the delete button since this is equivalent to pressing the Back button on the browser. <br> |
| 99 | <it> We only delete the original worker to make the naming of our workers consistent since the install scripts are sensitive to the node names </it> |
| 100 | <li> Ensure that all your workers are named in the pattern `worker-i` with i=0,1.2,... (e.g. `worker-0, worker-1, worker-2`) <br> |
| 101 | <b> WARNING</b> This step is important, if the workers are not named based on the above convention your cluster <b> WILL NOT </b> be configured correctly |
| 102 | <li>On the <code> master </code> node make sure that on the Execute script the input number matches the number of workers, e.g. if you have 3 workers the entry should be <br> |
| 103 | <code> /home/hadoop/hadoop_config_dynamic.sh 3 </code></li> |
| 104 | <li> Connect all your nodes in one LAN |
| 105 | <ol type='i'> |
| 106 | <li> Drag a line between two of your nodes </li> |
| 107 | <li> From each unconnected node draw a line from that node to the middle of the link where there is a small square</li> |
| 108 | </ol> |
| 109 | </li> |
| 110 | <li> Assign IPs to your nodes: <br> |
| 111 | Click on the small square in the middle of your lan and set the IPs and netmask for each interface according to this list: |
| 112 | <ul> |
| 113 | <li> the netmask for all interfaces should be <code> 255.255.255.0 </code> |
| 114 | <li> master IP: 172.16.1.1 </li> |
| 115 | <li> worker-i IP: 172.16.1.<10+i>, e.g. worker-0: 172.16.1.10, worker-1: 172.16.1.11, worker-5: 172.16.1.15, etc |
| 116 | </ul> |
| 117 | |
| 118 | </ol> |
| 119 | </td> |
| 120 | </tr> |
| 121 | </table> |
| 122 | }}} |
| 123 | Now based on the loaded node types we will create our Hadoop cluster. For the cluster we need |
| 124 | * 1 hadoop master |
| 125 | * 2 or more workers |
| 126 | * all the nodes need to be on the same Layer 2 Domain |
| 127 | |
| 128 | Let's design our topology. |
| 129 | {{{ |
| 130 | #!html |
| 131 | <table> |
| 132 | <tr><td> |
| 133 | <ol type='a' start='6' > |
| 134 | <li>Select the worker node </li> |
| 135 | <li>Press the <b>Add Resources</b> button to launch Jacks for this slice.</li> |
| 136 | <li>From the <b>Choose RSpec</b> menu (see figure), select the <b>URL</b> button.</li> |
| 137 | <li>Enter the URL for the RSpec: <br> |
| 138 | <tt> http://geni-images.renci.org/images/tutorials/GENI-hadoop/hadoop_dynamic_request_rspec.xml </tt> <br> |
| 139 | then click <b>Select</b>.<br/> |
| 140 | </li> |
| 141 | <li>After you click <b>Select</b>, a couple of nodes will appear. </li> |
| 142 | </ol> |
| 143 | </td> |
| 144 | </tr> |
| 145 | </table> |
| 146 | |