wiki:GIR3.2_INSTOOLS

Version 36 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

INSTOOLS Evaluation

The INSTOOLS software is available in two ways: as part of the ProtoGENI Flack Client and as a standalone software package. Both are evaluated as part of the GIR 3.2 effort.

The ProtoGENI FLACK Client has been integrated the INSTOOLS, so that a user may simply click a button to instrumentize nodes in an experiment. The ProtoGENI Flack Client is available here. Flack Client version vGEC11.9 was used for this evaluation.

Three versions of the INSTOOLS software were evaluated all named V3.3.

Time Frame: Evaluation started September 14th, 2011 through September 26, 2011.

INSTOOLS Findings

The ProtoGENI Flack Client Instrumentation was reviewed on September 15 and 16, 2011, version vGEC11.9 was used for this evaluation. No issues were found!

The first version of the standalone software package was INSTOOLS V3.3 was found from the Instrumentize wiki page. After running into problem, found out that this version should not be used due to issue that are to be addressed in the next version.

A second version was made available named INSTOOLS V3.3 using the same version number and location. This version was successfully used to create an experiment and instrumentize it. The following issues were found:

  • archive_using_mda.py script fails due to Utah firewall preventing samba mounts from outside the Utah Network. CNRI uses samba mounts as a user workspace to which INSTOOLS dumps its data.
  • various rspec are used in the docs/README which are not part of the rspec delivered as part of the package.
  • creating a UKY and Utah tunnel experiment with the file rspecs/tunnel-rspec causes an exception in the Flack client.

INSTOOLS How-to

Both Flack Client and package version of the INSTOOLS software were evaluated. Detailed of usage are captured in this section.

INSTOOLS V3.3 package (candidate 2)

The second INSTOOLS V3.3 package was made available on 9/20/2011, using naming and version number from the previous candidate. The following steps were executed:

$ wget http://www.uky.emulab.net/downloads/INSTOOLSv3_3.tgz       
$ tar xzf INSTOOLSv3_3.tgz 
$ cd INSTOOLS/v3.3

The file docs/INSTOOLS-SETUP defines the following software as prerequisites for the user's client system:

  1. Openssl
  2. ssh
  3. ssh-agent
  4. Python v2.5 or higher
  5. Python crypto libraries

Other prerequisites include:

  1. Get an Emulab account, as described here
  2. Create an SSL certificate as described here and saved to $HOME/.ssl/encrypted.pem
  3. Create a .protogeni-config as described here
  4. Create a file $HOME/.ssl/password containing the clear text password for the SSL certificate.
  5. Upload your SSH keys as described here
    6 Start an ssh-agent to manage keys.

Overall, it is expected that the user can execute an experiment and can get node access via SSH.

Create an experiment

Using a simple two node rspec created an experiment:

$ ./createExp.py -n nevers -m https://www.uky.emulab.net/protogeni/xmlrpc/cm uky.rspec 
CMURI = https://www.uky.emulab.net/protogeni/xmlrpc

Got my SA credential
No such slice registered here:Creating new slice called nevers
New slice created

Asking for a ticket from the local CM
Got a ticket from the CM. Redeeming the ticket ...

Created the sliver
Using manifest Version = 2

Hostname to login               : pc47.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni1
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30900

 Will boot it in a min....

Hostname to login               : pc38.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni2
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30901

 Will boot it in a min....

Booting your Slivers
Please be patient...

Got the sliver credential, calling StartSliver on the sliver
Sliver state is changing. Please wait for additional 30 sec...
Sliver state is changing. Please wait for additional 30 sec...
Sliver is ready.

Your Experiment is now complete.
You may now try to login into your nodes.

Instrumentize the slice

Using the slice and sliver created by the createExp.py command above, next the node can be instrumentized:

$ ./instrumentize.py -m https://www.emulab.net/protogeni/xmlrpc/cm  -n nevers
Got my SA credential

Asking for slice credential for nevers

This Slice spans these CMs ['urn:publicid:IDN+uky.emulab.net+authority+cm']

Got the slice credential

Found user record at the SA
Your user account details obtained are: 
Username                : lnevers
Email ID                : lnevers@bbn.com
Certificate Issuer      : utahemulab

Talking to CM HRN = ukgeni.cm

/

Resolving the slice to get the sliver_urn....

Node : "geni1" passed pre-check test
Node : "geni2" passed pre-check test

ALL Nodes on this CM capable of being instrumentized.
Proceeding to instrumentize.

Updating the Sliver ...
Updated the sliver, got a new ticket from CM
Obtaining updated sliver Credentials
Redeeming updated Ticket

Created the sliver
Hostname to login               : pc47.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni1
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30900

 Will boot it in a min....

Hostname to login               : pc38.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni2
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30901

 Will boot it in a min....

Hostname to login               : pc64.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : MCukgeni
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30936

pc64.uky.emulab.net is your Measurement Controller

Booting your Slivers 
Please be patient...

Node "MCukgeni" state is changing Please wait for additional 10 sec...

Node: "geni1" is booted and ready
Node: "geni2" is booted and ready
Node "MCukgeni" state is changing Please wait for additional 10 sec...

Sliver is ready.
 
Please wait a few more minutes while your Measurement Controller is being installed and configured

MC is not yet ready. Will check again after 10 sec
MC is not yet ready. Will check again after 10 sec
MC is not yet ready. Will check again after 10 sec
MC is not yet ready. Will check again after 10 sec
MC is not yet ready. Will check again after 10 sec

MC is ready. Continuing with configuration process
locking MC

Running Node Configuration Scripts on Node: "geni2"
Running Node Configuration Scripts on Node: "geni1"

Node Configuration Scripts on Node: "geni1" completed.
Node Configuration Scripts on Node: "geni2" completed.

Checking if Netflow data collection started for this Topology
Netflow Data collection not started yet.. Will check again after 30 seconds 

Checking if Netflow data collection started for this Topology
Netflow Data collection not started yet.. Will check again after 30 seconds 

Checking if Netflow data collection started for this Topology
Netflow Data collection has started .. Continuing Intrumentize.... 

Updating the drupal Admin account info
Sending POLL_DATA file to MC

Sending this MC's info to INSTOOLS portal site
Initializing Drupal Menu creation for this Topology
unlocking MC

ukgeni.cm instrumentization complete

Visit http://portal.uky.emulab.net and fill out the requested information for a complete View of your Network Topology. 
Your Portal login details are
Username : lnevers
Password : 23e95baf15a711cea44ddecf9f6bf0dfd7ed52c1 
Email Address : lnevers@bbn.com
Certificate Issuer      : utahemulab
Slicename : nevers

Your Experiment setup is now complete.You may login to your nodes and begin your experimentation.
Logs for your setup are available in logs/instrumentize-nevers_20110920T10:01:08.log

*********Other INSTOOLS Details***********
 Measurement Controller URL for the ukgeni.cm Aggregate is http://pc64.uky.emulab.net/

Access Measurements

Using the information provided in the results accessed the INSTOOLS Slice Map Logon:

Where the nodes can be managed:

Archive Measuraments

All measurament data can be archived to the CNRI Digital Object Repository project (DOR) with the archive_using_mda.py script which uploads the INSTOOLS data (tarball containing INSTOOLS data in RRD,HTML and PNG format ) with a METADATA.txt file with details about the tar file contents. To uniquely identify each archive, a time stamp is also included in the metadata.txt file.

To save this data into the Digital archive space a user account for the MDA must be requested pior to using the script by sending an email request to Giridhar Manepalli (gmanepalli@cnri.reston.va.us). Once you have an account, you can store your measuraments as follows:

./archive_using_mda.py -m https://www.emulab.net/protogeni/xmlrpc/cm -n slicename}}}

Currently a firewall rule prevents the samba mounts from loading results for Utah slices.

De-instumentize Experiment

The data collection can be stopped as follows:

$  ./de-instrumentize.py -m https://www.emulab.net/protogeni/xmlrpc/cm  -n nevers
Got my SA credential

Asking for slice credential for nevers
This Slice spans these CMs ['urn:publicid:IDN+uky.emulab.net+authority+cm']
Got the slice credential

Talking to CM HRN = ukgeni.cm

Resolving the slice to get the sliver_urn....
Got the sliver URN.  Resolving manifest...
Removing your MC for ukgeni.cm
Updating the Sliver ...
Updated the sliver, got a new ticket from CM

Hostname to login               : pc47.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni1
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30900

Hostname to login               : pc38.uky.emulab.net
SSHD Port number to use         : 22
Virtual ID                      : geni2
CM URN                          : urn:publicid:IDN+uky.emulab.net+authority+cm
Sliver URN                      : urn:publicid:IDN+uky.emulab.net+sliver+30901

Sliver is ready.
Please wait a few more minutes while INSTOOLS is being purged from your Nodes

Running Node de-Configuration Scripts on Node: "geni2"
Running Node de-Configuration Scripts on Node: "geni1"

De-Configuring Node: "geni2" Complete
De-Configuring Node: "geni1" Complete

Your Experiment De-intrumentization at ukgeni.cm is now complete.
De-Intrumentization Now Complete

INSTOOLS FLACK Client

Evaluation used the Instrumentation Tools for a GENI Prototype GEC11 Tutorial portion as a reference. Prerequisites include:

  • User account on one of the Emulab aggregates
  • Flash ready browser

Once the prerequisites are met, the user can login to the ProtoGENI Flack Client:

User is then presented with the certificate to be used for the login, enter your pass phrase and click "OK".

If you need to import the certificate select the Open from file button .

Once logged in the following is shown:

You may now create a slice and add resources or you can simply select the Show only mine option, which will show any of your pre-existing slices. This scenario creates a three emulab nodes slice using the flack client:

When creating the slice, modify each of the Node to use a Fedora 8 or greater disk image, this is done by selecting the Node Information button:

You may now create the slivers. Once the resources are ready you may select the Instrumentize button:

This will kicks the INSTOOLS installation on the nodes that are part of the experiment. Note this is a slow process, be patient. Once the installation is complete you may click on the Go to portal button to get access to Instools Portal where you may visualize the slice data.

The slice portal looks as follows:

In the portal you may choose which statistics to monitor simply by clicking on a node an selecting the graphing options that are available:

You may choose to go directly to the node by clicking on the node and then clicking on the All Graphs button:

Access to each of the nodes is available by clicking on a node an selecting the VNC button:

INSTOOLS V3.3 package (candidate 1)

The INSTOOLS v3.3 package software was found by reviewing the August2011 status report . From the status page, users are redirected to the INSTOOLS summary page, where the user can find that the package and documentation are found at Instrumentize page.

Downloaded the INSTOOLS V3.3 package, which contains both the INSTOOLS python scripts and it supporting documentation. After downloading and unpacking the package, reviewed the docs/README and the docs/INSTOOLSf-SETUP files for additional instructions.

Set up the prerequisites which require:

  • placing the file SSL Certificate encrypted.pem and the password files in ~/.ssl/ directory.
  • creating a file .protogeni-config.py defining the XMLRPC_SERVER value.

Create an experiment. Note: this step creates a slice and assigns resources to the slice:

$ ./createExp.py -n neverslice2 -m https://www.uky.emulab.net/protogeni/xmlrpc/cm uky.rspec 
CMURI = https://www.uky.emulab.net/protogeni/xmlrpc
Got my SA credential
No such slice registered here:Creating new slice called neverslice2
New slice created

Traceback (most recent call last):
  File "./createExp.py", line 149, in <module>
    current_slice_expiration = datetime.datetime(*time.strptime(instools_util.findSliceExpiry(slicecred, debug,LOGFILE),"%Y-%m-%dT%H:%M:%S")[0:6])
  File "/usr/lib/python2.6/_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib/python2.6/_strptime.py", line 328, in _strptime
    data_string[found.end():])
ValueError: unconverted data remains: Z

At this point was directed by INSTOOLS team to stop using this version and that a new version would be available shortly.


Email us with any questions and feedback on this page!

Attachments (14)

Download all attachments as: .zip