[[PageOutline]] = INSTOOLS Evaluation = The INSTOOLS software is available in two ways: as part of the ProtoGENI Flack Client and as a standalone software package. Both are evaluated as part of the GIR 3.2 effort. The ProtoGENI FLACK Client has been integrated the INSTOOLS, so that a user may simply click a button to ''instrumentize'' nodes in an experiment. The ProtoGENI Flack Client is available [http://protogeni.net/flack.html here]. Flack Client version vGEC11.9 was used for this evaluation. Two version of the INSTOOLS software were evaluated both named [http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz V3.3]. Time Frame: Evaluation started September 14th, 2011. = INSTOOLS Findings = The ProtoGENI Flack Client Instrumentation was reviewed on September 15 and 16, 2011, version vGEC11.9 was used for this evaluation. No issues were found! The first version of the standalone software package was [http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz INSTOOLS V3.3] was found from the [ http://groups.geni.net/geni/wiki/Instrumentize Instrumentize ] wiki page. After running into problem, found out that this version should not be used due to issue that are to be addressed in the next version. A second version was made available named [http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz INSTOOLS V3.3] using the same version number and location. This version was successfully used to create an experiment and instrumentize it. The following issues were found: * archive_using_mda.py script fails due to Utah firewall preventing samba mounts from outside the Utah Network. CNRI uses samba mounts as a user workspace to which INSTOOLS dumps its data. * various rspec are used in the docs/README which are not part of the rspec delivered as part of the package. * creating a UKY and Utah tunnel experiment with the file rspecs/tunnel-rspec causes an exception in the Flack client. = INSTOOLS How-to = Both Flack Client and package version of the INSTOOLS software were evaluated. Detailed of usage are captured in this section. == INSTOOLS FLACK Client == Evaluation used the [http://www.netlab.uky.edu/p/instools/ Instrumentation Tools for a GENI Prototype] GEC11 Tutorial portion as a reference. Prerequisites include: * User account on one of the Emulab aggregates * Flash ready browser Once the prerequisites are met, the user can login to the [http://protogeni.net/flack.html ProtoGENI Flack Client]: [[Image(flack-login-1.jpeg)]] User is then presented with the certificate to be used for the login, enter your pass phrase and click "OK". [[Image(cert-login-2.jpeg)]] If you need to import the certificate select the ''Open from file'' button [[Image(open-from-file-3.jpeg)]]. Once logged in the following is shown: [[Image(flack-main-4.jpeg)]] You may now create a slice and add resources or you can simply select the ''Show only mine'' option, which will show any of your pre-existing slices. This scenario creates a three emulab nodes slice using the flack client: [[Image(create-slice-resources-5.jpeg)]] When creating the slice, modify each of the Node to use a Fedora 8 or greater disk image, this is done by selecting the ''Node Information'' button: [[Image(node-information-6.jpeg)]] You may now create the slivers. Once the resources are ready you may select the ''Instrumentize'' button: [[Image(slice-ready-7.jpeg)]] This will kicks the INSTOOLS installation on the nodes that are part of the experiment. Note this is a slow process, be patient. Once the installation is complete you may click on the ''Go to portal'' button to get access to Instools Portal where you may visualize the slice data. [[Image(instrumentation-complete-8.jpeg)]] The slice portal looks as follows: [[Image(slice-portal-9.jpeg)]] In the portal you may choose which statistics to monitor simply by clicking on a node an selecting the graphing options that are available: [[Image(graphing-10.jpeg)]] You may choose to go directly to the node by clicking on the node and then clicking on the ''All Graphs'' button: [[Image(node-graphs-11.jpeg)]] Access to each of the nodes is available by clicking on a node an selecting the ''VNC'' button: [[Image(vnc-12.jpeg)]] == INSTOOLS V3.3 package (candidate 2) == The second [http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz INSTOOLS V3.3] package was made available on 9/20/2011, using naming and version number from the previous candidate. The following steps were executed: {{{ $ wget http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz $ tar xzf INSTOOLSv3_3.tgz $ cd INSTOOLS/v3.3 }}} The file '' docs/INSTOOLS-SETUP'' defines the following software as prerequisites for the user's client system: a. Openssl [[BR]] b. ssh [[BR]] c. ssh-agent [[BR]] d. Python v2.5 or higher [[BR]] e. Python crypto libraries [[BR]] Other prerequisites include: 1. Get an Emulab account, as described [http://www.protogeni.net/trac/protogeni/wiki/Tutorial#GetanEmulabAccount here] 2. Create an SSL certificate as described [http://www.protogeni.net/trac/protogeni/wiki/Tutorial#SSLCertificate here] and saved to $HOME/.ssl/encrypted.pem 3. Create a .protogeni-config as described [http://www.protogeni.net/trac/protogeni/wiki/Tutorial#CertificateProblems here] 4. Create a file $HOME/.ssl/password containing the clear text password for the SSL certificate. 5. Upload your SSH keys as described [http://www.protogeni.net/trac/protogeni/wiki/Tutorial#UploadingSSHKeys here] 6 Start an ssh-agent to manage keys. Overall, it is expected that the user can execute an experiment and can get node access via SSH. === Create an experiment === Using a simple two node rspec created an experiment: {{{ $ ./createExp.py -n nevers -m https://www.uky.emulab.net/protogeni/xmlrpc/cm uky.rspec CMURI = https://www.uky.emulab.net/protogeni/xmlrpc Got my SA credential No such slice registered here:Creating new slice called nevers New slice created Asking for a ticket from the local CM Got a ticket from the CM. Redeeming the ticket ... Created the sliver Using manifest Version = 2 Hostname to login : pc47.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni1 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30900 Will boot it in a min.... Hostname to login : pc38.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni2 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30901 Will boot it in a min.... Booting your Slivers Please be patient... Got the sliver credential, calling StartSliver on the sliver Sliver state is changing. Please wait for additional 30 sec... Sliver state is changing. Please wait for additional 30 sec... Sliver is ready. Your Experiment is now complete. You may now try to login into your nodes. }}} === Instrumentize the slice === Using the slice and sliver created by the createExp.py command above, next the node can be instrumentized: {{{ $ ./instrumentize.py -m https://www.emulab.net/protogeni/xmlrpc/cm -n nevers Got my SA credential Asking for slice credential for nevers This Slice spans these CMs ['urn:publicid:IDN+uky.emulab.net+authority+cm'] Got the slice credential Found user record at the SA Your user account details obtained are: Username : lnevers Email ID : lnevers@bbn.com Certificate Issuer : utahemulab Talking to CM HRN = ukgeni.cm / Resolving the slice to get the sliver_urn.... Node : "geni1" passed pre-check test Node : "geni2" passed pre-check test ALL Nodes on this CM capable of being instrumentized. Proceeding to instrumentize. Updating the Sliver ... Updated the sliver, got a new ticket from CM Obtaining updated sliver Credentials Redeeming updated Ticket Created the sliver Hostname to login : pc47.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni1 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30900 Will boot it in a min.... Hostname to login : pc38.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni2 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30901 Will boot it in a min.... Hostname to login : pc64.uky.emulab.net SSHD Port number to use : 22 Virtual ID : MCukgeni CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30936 pc64.uky.emulab.net is your Measurement Controller Booting your Slivers Please be patient... Node "MCukgeni" state is changing Please wait for additional 10 sec... Node: "geni1" is booted and ready Node: "geni2" is booted and ready Node "MCukgeni" state is changing Please wait for additional 10 sec... Sliver is ready. Please wait a few more minutes while your Measurement Controller is being installed and configured MC is not yet ready. Will check again after 10 sec MC is not yet ready. Will check again after 10 sec MC is not yet ready. Will check again after 10 sec MC is not yet ready. Will check again after 10 sec MC is not yet ready. Will check again after 10 sec MC is ready. Continuing with configuration process locking MC Running Node Configuration Scripts on Node: "geni2" Running Node Configuration Scripts on Node: "geni1" Node Configuration Scripts on Node: "geni1" completed. Node Configuration Scripts on Node: "geni2" completed. Checking if Netflow data collection started for this Topology Netflow Data collection not started yet.. Will check again after 30 seconds Checking if Netflow data collection started for this Topology Netflow Data collection not started yet.. Will check again after 30 seconds Checking if Netflow data collection started for this Topology Netflow Data collection has started .. Continuing Intrumentize.... Updating the drupal Admin account info Sending POLL_DATA file to MC Sending this MC's info to INSTOOLS portal site Initializing Drupal Menu creation for this Topology unlocking MC ukgeni.cm instrumentization complete Visit http://portal.uky.emulab.net and fill out the requested information for a complete View of your Network Topology. Your Portal login details are Username : lnevers Password : 23e95baf15a711cea44ddecf9f6bf0dfd7ed52c1 Email Address : lnevers@bbn.com Certificate Issuer : utahemulab Slicename : nevers Your Experiment setup is now complete.You may login to your nodes and begin your experimentation. Logs for your setup are available in logs/instrumentize-nevers_20110920T10:01:08.log *********Other INSTOOLS Details*********** Measurement Controller URL for the ukgeni.cm Aggregate is http://pc64.uky.emulab.net/ }}} === Access Measurements === Using the information provided in the results accessed the INSTOOLS Slice Map Logon: [[Image(map-logon-12.jpeg)]] Where the nodes can be managed: [[Image(map-content-13.jpeg)]] === Archive Measuraments === All measurament data can be archived to the [http://groups.geni.net/geni/wiki/DigitalObjectRegistry CNRI Digital Object Repository project] (DOR) with the archive_using_mda.py script which uploads the INSTOOLS data (tarball containing INSTOOLS data in RRD,HTML and PNG format ) with a METADATA.txt file with details about the tar file contents. To uniquely identify each archive, a time stamp is also included in the metadata.txt file. To save this data into the Digital archive space a user account for the [http://mda.doregistry.org/ MDA] must be requested pior to using the script by sending an email request to Giridhar Manepalli (gmanepalli@cnri.reston.va.us). Once you have an account, you can store your measuraments as follows: {{{ ./archive_using_mda.py -m https://www.emulab.net/protogeni/xmlrpc/cm -n slicename}}} }}} Currently a firewall rule prevents the samba mounts from loading results for Utah slices. === De-instumentize Experiment === The data collection can be stopped as follows: {{{ $ ./de-instrumentize.py -m https://www.emulab.net/protogeni/xmlrpc/cm -n nevers Got my SA credential Asking for slice credential for nevers This Slice spans these CMs ['urn:publicid:IDN+uky.emulab.net+authority+cm'] Got the slice credential Talking to CM HRN = ukgeni.cm Resolving the slice to get the sliver_urn.... Got the sliver URN. Resolving manifest... Removing your MC for ukgeni.cm Updating the Sliver ... Updated the sliver, got a new ticket from CM Hostname to login : pc47.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni1 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30900 Hostname to login : pc38.uky.emulab.net SSHD Port number to use : 22 Virtual ID : geni2 CM URN : urn:publicid:IDN+uky.emulab.net+authority+cm Sliver URN : urn:publicid:IDN+uky.emulab.net+sliver+30901 Sliver is ready. Please wait a few more minutes while INSTOOLS is being purged from your Nodes Running Node de-Configuration Scripts on Node: "geni2" Running Node de-Configuration Scripts on Node: "geni1" De-Configuring Node: "geni2" Complete De-Configuring Node: "geni1" Complete Your Experiment De-intrumentization at ukgeni.cm is now complete. De-Intrumentization Now Complete }}} == INSTOOLS V3.3 package (candidate 1) == The INSTOOLS v3.3 package software was found by reviewing the [http://groups.geni.net/geni/wiki/InstrumentationTools-August2011-status August2011 status report] . From the status page, users are redirected to the [http://groups.geni.net/geni/wiki/INSTOOLSSummary INSTOOLS summary] page, where the user can find that the package and documentation are found at [http://groups.geni.net/geni/wiki/Instrumentize Instrumentize] page. Downloaded the [http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz INSTOOLS V3.3] package, which contains both the INSTOOLS python scripts and it supporting documentation. After downloading and unpacking the package, reviewed the docs/README and the docs/INSTOOLSf-SETUP files for additional instructions. Set up the prerequisites which require: * placing the file SSL Certificate ''encrypted.pem'' and the ''password'' files in ''~/.ssl/'' directory. * creating a file ''.protogeni-config.py'' defining the ''XMLRPC_SERVER'' value. Create an experiment. Note: this step creates a slice and assigns resources to the slice: {{{ $ ./createExp.py -n neverslice2 -m https://www.uky.emulab.net/protogeni/xmlrpc/cm uky.rspec CMURI = https://www.uky.emulab.net/protogeni/xmlrpc Got my SA credential No such slice registered here:Creating new slice called neverslice2 New slice created Traceback (most recent call last): File "./createExp.py", line 149, in current_slice_expiration = datetime.datetime(*time.strptime(instools_util.findSliceExpiry(slicecred, debug,LOGFILE),"%Y-%m-%dT%H:%M:%S")[0:6]) File "/usr/lib/python2.6/_strptime.py", line 454, in _strptime_time return _strptime(data_string, format)[0] File "/usr/lib/python2.6/_strptime.py", line 328, in _strptime data_string[found.end():]) ValueError: unconverted data remains: Z }}} At this point was directed by INSTOOLS team to stop using this version and that a new version would be available shortly.