Changes between Initial Version and Version 1 of GENIExperimenter/Tutorials/GettingStarted_PartII_Hadoop/Procedure/DesignSetup

Timestamp:

03/07/14 10:22:11 (10 years ago)

Author:

sedwards@bbn.com

Comment:

--

GENIExperimenter/Tutorials/GettingStarted_PartII_Hadoop/Procedure/DesignSetup

@@ +1 @@
+= [http://groups.geni.net/geni/wiki/GENIEducation/SampleAssignments/UnderstandAMAPI/Procedure Understanding the AM API] =
+{{{
+#!html
+
+
+<div style="text-align:center; width:495px; margin-left:auto; margin-right:auto;">
+<img id="Image-Maps_5201305222028436" src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/Setup.jpg?format=raw" usemap="#Image-Maps_5201305222028436" border="0" width="495" height="138" alt="" />
+<map id="_Image-Maps_5201305222028436" name="Image-Maps_5201305222028436">
+<area shape="rect" coords="18,18,135,110" href="./DesignSetup" alt="" title=""    />
+<area shape="rect" coords="180,18,297,111" href="./Execute" alt="" title=""    />
+<area shape="rect" coords="344,17,460,110" href="./Finish" alt="" title=""    />
+<area shape="rect" coords="493,136,495,138" href="http://www.image-maps.com/index.php?aff=mapped_users_5201305222028436" alt="Image Map" title="Image Map" />
+</map>
+<!-- Image map text links - End - -->
+
+</div>
+}}}
+
+
+
+== 1.  Design the Experiment ==
+  
+{{{
+#!html
+<table border="0">
+   <tr>
+     <td >
+        <ol type='a'><li>In today's experiment you will use resources at the aggregate listed on the worksheet. If you don't have a worksheet use <i>Clemson InstaGENI (aka instageni.clemson.edu)</i> </li>
+ </li>
+        </ol>
+     </td>
+   </tr>
+</table>
+}}}
+
+== 2.  Establish the Environment ==
+
+=== 2.1 Pre-work: Ensure SSH keys are setup ===
+
+Verify that you have at least one public key associated with your account. To do that, after you login to the portal check under your Profile, under the `SSH keys` tab. If you do not have SSH keys associated yet, please follow the instructions on that tab of the Portal.
+
+=== 2.2 Configure Omni ===
+
+
+{{{
+#!html
+<table>
+<tr><td>
+        <ol type='a'>
+          <li>Login to the <a href="http://portal.geni.net"> GENI Portal </a> </li>
+           <li> Click on the `Profile tab` link on the top of your
+           screen. Then click on the `Configure omni` tab under
+           `PROFILE`.</li>
+        </ol>
+</td><td>
+           <img
+           src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/OmniConfigure/Automatic/ConfigureOmniTab.png"
+           width="400" /> 
+     <br />
+     <b>Figure 2-1</b> Click on the <i>Configure omni</i> tab under
+     <i>Profile</i>.</i>
+</td></tr>
+<tr><td>
+        <ol type='a' start='3'>
+           <li> Click on the `Download your omni data` button under step 2. </li>
+</ol>
+</td><td>
+           <img
+           src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/OmniConfigure/Automatic/DownloadOmniDataButton.jpg"
+           width="400" />
+     <br />
+     <b>Figure 2-2</b> <i>Download your omni data</i> under step 2.
+</td></tr>
+<tr><td>
+        <ol type='a' start='4'>
+           <li> If this is the first time you try to access your GENI
+           certificate you will have to generate one. Click on the
+           `generate a certificate` link.</li>
+</ol>
+</td><td>
+           <img
+           src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/OmniConfigure/Automatic/GenerateCertLink.jpg"
+           width="400" />
+     <br />
+     <b>Figure 2-3</b> Click on <i>Generate a certificate</i>.</i>
+</td></tr>
+<tr><td>
+        <ol type='a' start='5'>
+           <li> Unless you really understand how SSL certificates
+           work, choose the simple option. Click on the `Generate
+           Combined Certificate and Key File` button and then click on
+           `Close`.  You will be taken back to the download page with
+           the warning.  Reload the page to enable the download
+           button.</li>
+</ol>
+</td><td>
+           <img
+           src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/OmniConfigure/Automatic/SimpleOption.jpg"
+           width="350"/> 
+     <br />
+     <b>Figure 2-4</b> Click on <i>Generate Combined Certificate and Key File</i>.
+</td></tr>
+<tr><td>
+        <ol type='a' start='6'>
+           <li> If you are a member of more than one project, select
+           which project you would like to be the default one for
+           running experiments in GENI. You can always change the
+           project that is used by the `-r` command line option of
+           Omni.  Then click on `Download your omni data`.</li> 
+</ol>
+</td><td>
+        <img
+        src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/OmniConfigure/Automatic/DownloadOmniData.jpg"
+        width="400"/>  
+     <br />
+     <b>Figure 2-5</b> Click on <i>Download your omni data</i>.
+</td></tr>
+<tr><td>
+        <ol type='a' start='7'>
+           <li> The bundle will be saved at <code>~/Downloads/omni-bundle.zip</code> </li>
+<li>Open a terminal window and type:<pre
+class="wiki">omni-configure</pre>The cert and key files you need will
+be installed in the appropriate folders.</li>
+</ol>
+</td></tr>
+</table>
+}}}
+
+== 3.  Obtain Resources ==
+
+
+=== 3.1 Create a slice ===
+
+Create a slice using `omni` and the slice name of your choice.  From now on that slice name will be referred to as `SLICENAME`.
+{{{
+$ omni createslice SLICENAME
+}}}
+
+=== 3.2. Load a simple topology in Flack ===
+For this exercise, we will edit an existing RSpec file.  Start by loading this predefined topology into Flack.
+
+{{{
+#!html
+<table>
+<tr><td>
+  <ol type='a'>
+    <li>In the Portal, launch Flack for this slice.</li>
+    <li>From the <b>Import</b> menu (see figure), select the <b>Import from the web</b> item.</li>
+  </ol>
+  </td>
+  <td> 
+     <img border="0"
+        src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/PortalOmniExample/ObtainResources/flack-import-small.png?format=raw"
+        alt="Import an RSpec into Flack"  title="Import an RSpec into Flack" /> 
+     <br />
+     <b>Figure 3-2</b> <i>Import an RSpec into Flack.</i>
+  </td>
+</tr>
+<tr><td>
+  <ol type='a' start='3'>
+    <li>Enter the URL corresponding to the rack type you are using
+    then click <b>Ok</b>.<br/>
+      <table border='1'>
+      <tr>
+      <th><b>Aggregate Type</b></th>
+      <th><b>RSpec URL</b></th>
+      </tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/instageni-logo.png?format=raw" alt='InstaGENI' width="75" height="75" />
+      </td><td>
+      <tt class="wiki">http://www.gpolab.bbn.com/exp/UnderstandAMAPI/ig/understandAMAPI_ig.rspec</tt>
+      </td></tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/exogeni-logo.png?format=raw" alt='ExoGENI'  width="75" height="75" />
+      </td><td>
+      <tt class="wiki">http://www.gpolab.bbn.com/exp/UnderstandAMAPI/eg/understandAMAPI_eg.rspec</tt>
+      </tr>
+      </table>
+    </li>
+    <li>When Flack pops up a message asking you to pick a manager to use click <b>OK</b> and then select the manager listed on your worksheet from the
+    list.  After you click <b>Accept</b>, and a network topology should appear
+    on the canvas.
+
+
+<table id="Table_01" border="0" cellpadding="5" cellspacing="0">
+        <tr>
+                <td>
+                        <img src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/Graphics/4NotesIcon_512x512.png" width="50" height="50" alt="Note">
+               </td>
+<td>
+Because this RSpec file contains a generic description of resources and does not specify any particular aggregate manager, Flack will pop up a message asking you to specify an aggregate to use:
+<br/>
+<i>
+There were resources detected without a manager selected, please select which manager you would like to use.
+</i>
+</td>
+        </tr>
+</table>
+
+    </li>
+  </ol>
+</td>
+
+
+</tr></table>
+}}}
+
+=== 3.3. Modify the RSpec to automatically install and execute network test software ===
+
+For this experiment, we'd like some additional software
+(`Apache`, `iperf`, and a couple of custom scripts) loaded
+onto the VMs to perform traffic measurements.  
+
+{{{
+#!html
+
+<table><tr>
+<td>
+  <ol type='a'>
+     <li>Click on the the <i>i</i> button on the <b>client node</b> (see figure) to manipulate details of the node configuration.
+
+
+<table id="Table_01" border="0" cellpadding="5" cellspacing="0">
+        <tr>
+                <td>
+                        <img src="http://trac.gpolab.bbn.com/gcf/raw-attachment/wiki/Graphics/4NotesIcon_512x512.png" width="50" height="50" alt="Note">
+               </td>
+<td>
+When you open the node information, you should see options like those
+in the figure.  The controls available to you will vary depending on the
+types of the aggregate manager and the component, but many features are
+widely available throughout GENI.
+</td>
+        </tr>
+</table>
+
+
+     </li>
+  </ol>
+</td>
+<td>
+  <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/PortalOmniExample/ObtainResources/flack-node-info-small.png?format=raw" width='400'/>
+     <br />
+     <b>Figure 3-3</b> <i>Edit the nodes</i>
+</td>
+</tr>
+
+<tr><td>
+  <ol type='a' start='2'>
+
+<li>The way we will request installation of the proper software and execution
+of our experiment is to add <i>install</i> and <i>execute</i> services.
+</li>
+<li>
+First, select "<tt>+ Add Install Service</tt>" as shown.  Complete the "Archive URL and a location" using the appropriate information from the following table.
+
+      <table border='1'>
+      <tr>
+      <th><b>Aggregate Type</b></th>
+      <th><b>Archive URL</b></th>
+      <th><b>Installation Location</b></th>
+      </tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/instageni-logo.png?format=raw" alt='InstaGENI' width="75" height="75" />
+      </td><td>
+      <tt class="wiki">http://www.gpolab.bbn.com/exp/UnderstandAMAPI/ig/underAMAPI_ig.tar.gz</tt></td>
+      <td><tt class='wiki'>/local</tt>
+      </td></tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/exogeni-logo.png?format=raw" alt='ExoGENI'  width="75" height="75" />
+      </td><td>
+      <tt class="wiki">http://www.gpolab.bbn.com/exp/UnderstandAMAPI/eg/underAMAPI_eg.tar.gz</tt></td>
+      <td><tt class='wiki'>/local</tt>
+      </td></tr>
+      </table>
+  </li>
+  <li>Similarly, please use the "<tt>+ Add Execute Service</tt>" button to request
+that the appropriate command be executed at boot time.
+      <table border='1'>
+      <tr>
+      <th><b>Aggregate Type</b></th>
+      <th><b>Client Install Script</b></th>
+      <th><b>Server Install Script</b></th>
+      <th><b>Shell</b></th>
+      </tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/instageni-logo.png?format=raw" alt='InstaGENI' width="75" height="75" />
+      </td>
+      <td><tt class="wiki">sudo /local/install-script.sh</tt></td>
+      <td><tt class="wiki">sudo /local/install-script.sh</tt></td>
+      <td><tt class='wiki'>sh</tt></td>
+      </tr>
+      <tr><td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/exogeni-logo.png?format=raw" alt='ExoGENI'  width="75" height="75" />
+      </td>
+      <td><tt class="wiki">sudo /local/install-client.sh</tt></td>
+      <td><tt class="wiki">sudo /local/install-server.sh</tt></td>
+      <td><tt class='wiki'>sh</tt></td>
+      </tr>
+      </table>
+
+<table border="0" cellpadding="0" cellspacing="0"><tr><td><img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/Graphics/warning-icon-hi.png?format=raw" width='50' height='50'/></td><td>Be very careful when entering this information -- these commands will not
+be executed yet, so it will be some time before you will see any relevant
+error messages if there is a mistake here.</td></tr></table>
+  </li>
+</ol>
+</td>
+<td>
+   <table>
+   <td><img width='400' src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/PortalOmniExample/ObtainResources/flack-services-small.png?format=raw" />
+     <br />
+     <b>Figure 3-4</b> <i>Specify the tarball and install script</i>
+   </td> 
+   </tr>
+   </table>
+</td></tr>
+
+<tr><td>
+  <ol type='a' start='6'>
+  <li>Once both the <i>install</i> and <i>execute</i> services are specified, please
+choose <b>Apply</b>.  
+  </li>
+  <li><b>Repeat the previous five steps on the server node</b> to add the appropriate <i>install</i> and
+<i>execute</i> services.
+  </li>
+  <li> In addtion, on the server node, click on the <i>link</i> tab
+      and select <b>Routable Control IP</b> as shown in the figure. 
+  </li>
+</ol>
+</td>
+        <td> 
+        <img
+      src="http://groups.geni.net/geni/attachment/wiki/GENIEducation/SampleAssignments/UnderstandAMAPI/Graphics/publicIP_labeled_v1.png?format=raw"
+      alt="Select the 'Routable Control IP' button"  width="400"
+      title="Select the 'Routable Control IP' button" /> </a>
+         <br/>
+         <b>Figure 3-5</b> On the <i>Link</i> tab, select the
+      "Routable Control IP" button.
+       </td>
+
+</tr>
+</table>
+}}}
+
+If you have time, it would be instructive to download the `underAMAPI_*.tar.gz` file yourself, and inspect the contents.  You will see the `install-script.sh` file referred to in the execute service, and this is a good example of how you can ask for GENI components to perform tasks for you without any manual intervention.  This is a very useful facility for large experiments!
+
+
+
+=== 3.4. Export the modified request RSpec ===
+Now we will pull back some of the covers and inspect exactly what Flack has been doing for us when preparing the RSpecs for the experiments we design.  Each node and link has a corresponding element in the RSpec, and the details of the component configuration (such as the install and execute services we requested above) are specified with attributes, or sometimes child elements, within those portions of the document.
+
+{{{
+#!html
+<table><tr>
+<td>
+   <ol type='a'>
+   <li>From the <i>View</i> menu (see figure), select the <b>Preview
+request document(s)</b> item.  This will bring up a window showing the
+current RSpec -- please take a moment to inspect it.  The XML elements
+under the "<tt>flack</tt>" namespace were added for Flack's internal
+use (containing information about the canvas layout, editing history, etc.),
+and are ignored by aggregate managers.  The `&lt;node&gt;` and `&lt;link&gt;` elements
+contain the specification for the components we will request, and if you
+look closely, you will be able to see the install and execute service
+elements you added earlier.
+   </li>
+   <li>Use the <b>Save to file</b> button (in the upper left) to
+a local copy of your RSpec with the name <tt>lab1_rspec.xml</tt>.  We'll use this in the next step to
+demonstrate how other client tools also use RSpec files to communicate
+requests to aggregate managers.</li>
+   </ol>
+   </td>
+   <td>
+      <img src="http://groups.geni.net/geni/attachment/wiki/GENIExperimenter/Tutorials/PortalOmniExample/ObtainResources/flack-view-small.png?format=raw" width='400'/>
+         <br/>
+         <b>Figure 3-6</b> View and save the final request RSpec
+   </td>
+</tr></table>
+}}}
+
+=== 3.5. Instantiate the new experiment using Omni ===
+
+For this step, we'll change the approach a bit and switch to a new client tool, the command line Omni client.  
+
+From a terminal, please enter the command:
+{{{
+$ omni -a AM_NICKNAME createsliver SLICENAME RSPEC_FILE
+}}}
+where {{{AM_NICKNAME}}} is the nickname for your assigned aggregate manager and {{{SLICENAME}}} is the name of the slice you created earlier (both of these are given on your worksheet).  {{{RSPEC_FILE}}} should be replaced with the filename of the RSpec you saved in step 4.
+
+
+If all is well, Omni should give you a number of informational messages, such as:
+{{{
+INFO:omni:Loading config file /home/geni/.gcf/omni_config
+}}}
+
+It should quickly proceed to the point where it makes the request to the remote manager:
+{{{
+INFO:omni:Creating sliver(s) from rspec file /home/geni/Downloads/experiments.rspec for slice ...
+}}}
+
+This step can sometimes be time-consuming, so please be patient.  If it succeeds, within a couple of minutes Omni should report:
+{{{
+INFO:omni: Completed createsliver:
+}}}
+
+and your resource reservation is complete!
+
+
+----
+= [wiki:GENIEducation/SampleAssignments/UnderstandAMAPI/Procedure Introduction] =
+= [wiki:GENIEducation/SampleAssignments/UnderstandAMAPI/Procedure/Execute Next:  Execute] =