= LAMP Evaluation = The Leveraging and Abstracting Measurements with perfSONAR (LAMP) Evaluation was based on instruction found at the [http://groups.geni.net/geni/wiki/LAMP/Tutorial LAMP I&M System Tutorial] page. No versioning was available for either of the two scripts available for download, [http://groups.geni.net/geni/attachment/wiki/LAMP/Tutorial/lamp-getcertificate.py lamp-getcertificate.py] and [http://groups.geni.net/geni/attachment/wiki/LAMP/Tutorial/lamp-sendmanifest.py lamp-sendmanifest.py]. Evaluation Time Frame: November 16 and 22 2011 = LAMP GENI Findings = Several issues were found including: * There is no versioning in the tools attached on the tutorial page. * The [http://groups.geni.net/geni/attachment/wiki/LAMP/Tutorial/lamp-sendmanifest.py lamp-sendmanifest.py] script has problems uploading manifests for RSpec v0.2. * Error handling for failed manifest upload is not obvious and it is difficult to tell that a failure occurred. * Updating the bundle cert on the lamp server is not documented in tutorial, and should be documented with the perfSONAR process (psconfig) restart requirements. = LAMP How-to = Using instructions from the [http://groups.geni.net/geni/wiki/LAMP/Tutorial LAMP I&M System Tutorial] page, downloaded the 2 LAMP scripts and the required ProtoGENI Test script a these locations: * [http://groups.geni.net/geni/attachment/wiki/LAMP/Tutorial/lamp-getcertificate.py lamp-getcertificate.py] * [http://groups.geni.net/geni/attachment/wiki/LAMP/Tutorial/lamp-sendmanifest.py lamp-sendmanifest.py]. * [http://www.emulab.net/downloads/protogeni-tests.tar.gz ProtoGENI Test tools] Unpacked the protogeni-test tools and placed the two lamp scripts in the same directory as the ProtoGENI tests. '''1. Create Slice''' Set up a ProtoGENI slice to be used in this experiment. This example uses the Omni tool in the GCF package: {{{ $ cd ~/gcf-1.4 $ ./src/omni.py createslice lnevers-lamp-slice1 INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni:Created slice with Name lnevers-lamp-slice1, URN urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1, Expiration 2011-11-17 00:09:25+00:00 INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createslice: Options as run: framework: pg native: True Args: createslice lnevers-lamp-slice1 Result Summary: Created slice with Name lnevers-lamp-slice1, URN urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1, Expiration 2011-11-17 00:09:25+00:00 INFO:omni: ============================================================ }}} Modified the LAMP [http://groups.geni.net/geni/wiki/LAMP/Tutorial#a1.ModifytheRSpec example RSpec] to include my slice name and my user information: {{{ 100 0.05 }}} '''2. Add LAMP Resources ''' Using the above RSpec created a sliver that uses the required LAMP image named UBUNTU91-LAMP,: {{{ $ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am -n createsliver lnevers-lamp-slice1 lamp.rspec INFO:omni:Loading config file omni_config INFO:omni:Using control framework pg INFO:omni:Slice urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 expires within 1 day on 2011-11-17 00:09:25 UTC INFO:omni:Creating sliver(s) from rspec file lamp.rspec for slice urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 INFO:omni:Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result: INFO:omni: INFO:omni: INFO:omni: 100 0.05 INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab.net/protogeni/xmlrpc/am framework: pg native: True Args: createsliver lnevers-lamp-slice1 lamp.rspec Result Summary: Slice urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 expires within 1 day(s) on 2011-11-17 00:09:25 UTC Reserved resources on https://www.emulab.net/protogeni/xmlrpc/am. INFO:omni: ============================================================ }}} '''3. Upload Slice Manifest'''. Upload your topology to allow services configuration within the UNIS system. This step requires upload the slice manifest and your user credentials for the slich, which can be generated with the omni tools as follows: {{{ $ cd /path/to/gcf-1.4 $ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am -o listresources lnevers-lamp-slice1 $ ./src/omni.py -o -a https://www.emulab.net/protogeni/xmlrpc/am getslicecred lnevers-lamp-slice1 }}} The first omni command creates a manifest Rspec output file named lnevers-lamp-slice1-rspec-www-emulab-net-protogeni.xml, later renamed it lnevers-lamp-slice1-manifest.xml for conciseness. The second omni command generate a user credentials file for the slice which is named lnevers-lamp-slice1-cred.xml. {{{ $ cd /path/to/protogeni-tests/ $ cp path/to/gcf-1.4/lnevers-lamp-slice1-rspec-www-emulab-net-protogeni.xml lnevers-lamp-slice1-manifest.xml $ ./getslicecredential.py -n lnevers-lamp-slice1 >lnevers-lamp-slice1-cred.xml }}} To upload the manifest executed the following: {{{ $ cd /path/to/protogeni-tests/ $ cp /path/to/gcf-1.4/lnevers-lamp-slice1-rspec-www-emulab-net-protogeni.xml lnevers-lamp-slice1-manifest.xml $ cp /path/to/gcf-1.4/lnevers-lamp-slice1-cred.xml . }}} With the latest lamp-sendmanifest.py, you should use the syntax below, but due to a bug with support for RSpec V0.2, the command does not work: {{{ $ ./lamp-sendmanifest.py 0.2 lnevers-lamp-slice1-manifest.xml urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 lnevers-lamp-slice1-cred.xml }}} The failing lamp-sendmanifest.py is the version introduced to support RSpec v2.0, but it no longer works with V0.2 Rspecs. The earlier version of the [http://groups.geni.net/geni/attachment/wiki/GIR3.2_LAMP/lamp-sendmanifest.py lamp-sendmanifest.py] script was used to upload the V.0.2 RSpec, which worked using the following syntax: {{{ $ ./lamp-sendmanifest.py lnevers-lamp-slice1-manifest.xml urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 lnevers-lamp-slice1-cred.xml http://ggf.org/ns/nmwg/topology/20070809 privilege 179245 MIIDVDCCAr2gAwIBAgIDASPNMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJV UzENMAsGA1UECBMEVXRhaDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHTAbBgNV BAoTFFV0YWggTmV0d29yayBUZXN0YmVkMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBB dXRob3JpdHkxGDAWBgNVBAMTD2Jvc3MuZW11bGFiLm5ldDEoMCYGCSqGSIb3DQEJ ARYZdGVzdGJlZC1vcHNAZmx1eC51dGFoLmVkdTAeFw0xMTA5MDcxNTI4NTVaFw0x MjA5MDYxNTI4NTVaMIGqMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEdMBsG A1UEChMUVXRhaCBOZXR3b3JrIFRlc3RiZWQxGzAZBgNVBAsTEnV0YWhlbXVsYWIu bG5ldmVyczEtMCsGA1UEAxMkZWUzYjNiMWEtMWNkNC0xMWRmLTg2MzMtMDAxMTQz ZTQ1M2ZlMSEwHwYJKoZIhvcNAQkBFhJsbmV2ZXJzQGVtdWxhYi5uZXQwgZ8wDQYJ KoZIhvcNAQEBBQADgY0AMIGJAoGBALK88CkpB82ElRkqApjci+HfYFeLtfuJ8dIU 0DnhdbOdg1SB5K8RsGw3DGMmpWYDv5AUMbGZnFsw9ob2u47Y2polg8+7sG8NG//Z Ix47m2DjBCDvk95t2zb4K4M6Dn2eGXljv7LEg4Kx5AXEQLQdLHO/VuS10OqnLgKF 8a68wGkvAgMBAAGjeDB2MAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFBk2a7X6jRTj lVn3Pf5O6P2YPg+AMEcGA1UdEQRAMD6GKHVybjpwdWJsaWNpZDpJRE4rZW11bGFi Lm5ldCt1c2VyK2xuZXZlcnOBEmxuZXZlcnNAZW11bGFiLm5ldDANBgkqhkiG9w0B AQQFAAOBgQCET0lR/Oy4AIqxKEyy/nEj4ZTCMWMrEFkoAL3QcYuHrXHhv87CdL7S I7lelLyMbOEYuXmq9j0bjCw0iXRiqdNoDD+QC+0VSxgEuhWL3BIZvxmu/0q0WZ6m 6uM9m1hY2akZ31VTquNmgPPm6S2YXqlI3inV9j1pFRDr3otve03BJw== urn:publicid:IDN+emulab.net+user+lnevers MIIDVjCCAr+gAwIBAgIDAUwPMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJV UzENMAsGA1UECBMEVXRhaDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHTAbBgNV BAoTFFV0YWggTmV0d29yayBUZXN0YmVkMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBB dXRob3JpdHkxGDAWBgNVBAMTD2Jvc3MuZW11bGFiLm5ldDEoMCYGCSqGSIb3DQEJ ARYZdGVzdGJlZC1vcHNAZmx1eC51dGFoLmVkdTAeFw0xMTExMTYxNzA5MjFaFw0x NzA1MDgxODA5MjNaMIGzMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEdMBsG A1UEChMUVXRhaCBOZXR3b3JrIFRlc3RiZWQxJzAlBgNVBAsTHnV0YWhlbXVsYWIu bG5ldmVycy1sYW1wLXNsaWNlMTEtMCsGA1UEAxMkMWJhZGQ5NDQtMTA3ZS0xMWUx LWI0N2EtMDAxMTQzZTQ1M2ZlMR4wHAYJKoZIhvcNAQkBFg9sbmV2ZXJzQGJibi5j b20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMXJKsvqPHVjqhkrQ6JqvX/v ylVc8Nf2fbX83Nkd95TKbKDiYqMUGCdIOMjwn0rai9lLH2aEhVJYmzto+e6GQ34I clnahLiy8C8RKnK0xPuSDMlCyUjCO+ueo8ENj7T5dQjF0cX5C0Cfquf03RPXLtLe qtcMr6hcT60ZTLsJBw5lAgMBAAGjcTBvMB0GA1UdDgQWBBSw2YIgtHuxIa918qEz 5rmvtpfJvTBABgNVHREEOTA3hjV1cm46cHVibGljaWQ6SUROK2VtdWxhYi5uZXQr c2xpY2UrbG5ldmVycy1sYW1wLXNsaWNlMTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3 DQEBBAUAA4GBABnRRiHDxXkR5sgzPy3gmtYEErim6A23149jnV9fR6mSEL41sx44 dHnWIsxTQ2TIUkWwFNoXJqpBQalB4Q/YWd/qlu09kfxIVd9UU0E9gzG5Ir9R/i0f UFFT1tQR0LPOQ+XcacDrpDU6o4PSzPt++3vHt28pnbuwXhQ2OsDKK/nQ urn:publicid:IDN+emulab.net+slice+lnevers-lamp-slice1 652f6ae8-1519-11e1-9d24-001143e453fe 2011-11-28T00:00:00Z *1 mjy4Et+9cXTN/a2wASmIfAiixrQ= iAi25x4Uupnzo0is3Dp/e77mYBc+ymkGY3ShsE05BOM5lULUxOTKF3dkgY/Xirre J1PiwIuk5YR/zufph1xMKYqe06al+dnDQMdEfneGKychrwhxg6sXNApATsq0Jr+s K3E0NG6yC/sGUOqFCQKc9Vsj+YO13dzyVZP4quTY/VI= MIIDoTCCAwqgAwIBAgIDAS/uMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJV UzENMAsGA1UECBMEVXRhaDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHTAbBgNV BAoTFFV0YWggTmV0d29yayBUZXN0YmVkMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBB dXRob3JpdHkxGDAWBgNVBAMTD2Jvc3MuZW11bGFiLm5ldDEoMCYGCSqGSIb3DQEJ ARYZdGVzdGJlZC1vcHNAZmx1eC51dGFoLmVkdTAeFw0xMTEwMDUxOTUxMDZaFw0x NzAzMjcyMDUxMDZaMIGsMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEdMBsG A1UEChMUVXRhaCBOZXR3b3JrIFRlc3RiZWQxFjAUBgNVBAsTDXV0YWhlbXVsYWIu c2ExLTArBgNVBAMTJDJiNDM3ZmFhLWFhMDAtMTFkZC1hZDFmLTAwMTE0M2U0NTNm ZTEoMCYGCSqGSIb3DQEJARYZdGVzdGJlZC1vcHNAZmx1eC51dGFoLmVkdTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA1ayN3cGHH9hsmTgVWVjb2ZOqF8zFJ1Ew TFRpXVtI//wk05+Z7uunpxn/QL1F3NjdcIEToEupo1q2tRUfCc2hquLBgC5zNfut YD/b5ukEsF5COKHb+pYl2RZly9BVckt+ySFLnC23erKW7ILyO2fGBD/QzHZNPhdY /fs18iCh58cCAwEAAaOBwjCBvzAdBgNVHQ4EFgQUU2CjacFUMyUNL++CplFi++MF Sl0wMwYDVR0RBCwwKoYodXJuOnB1YmxpY2lkOklETitlbXVsYWIubmV0K2F1dGhv cml0eStzYTAPBgNVHRMBAf8EBTADAQH/MFgGCCsGAQUFBwEBBEwwSjBIBhRpg8yT gKiYzKjHvbGngICqrteKG4YwaHR0cHM6Ly93d3cuZW11bGFiLm5ldDoxMjM2OS9w cm90b2dlbmkveG1scnBjL3NhMA0GCSqGSIb3DQEBBAUAA4GBAIDXwcvEu3HJApFQ bQduTiHGXQ8Og/2ZIFLXHkqu4SW81RaYVbHwRFxnKHOktKm7js9wjEPo/F0tqIRT 21x7yE7uOce/8tWNW241fVuIRyO/o/DNd/FVFyFU5WNqP6f/rzEu92iuO6zIJPBg fmkqRvZqMOm5R//SSNBFl83lZzlu emailAddress=testbed-ops@flux.utah.edu,CN=2b437faa-aa00-11dd-ad1f-001143e453fe,OU=utahemulab.sa,O=Utah Network Testbed,ST=Utah,C=US emailAddress=testbed-ops@flux.utah.edu,CN=boss.emulab.net,OU=Certificate Authority,O=Utah Network Testbed,L=Salt Lake City,ST=Utah,C=US 77806 1ayN3cGHH9hsmTgVWVjb2ZOqF8zFJ1EwTFRpXVtI//wk05+Z7uunpxn/QL1F3Njd cIEToEupo1q2tRUfCc2hquLBgC5zNfutYD/b5ukEsF5COKHb+pYl2RZly9BVckt+ ySFLnC23erKW7ILyO2fGBD/QzHZNPhdY/fs18iCh58c= AQAB http://perfsonar.net/ns/protogeni/auth/credential/1
pc86.emulab.net
eth1
00034773a242
10.10.1.1
pc101.emulab.net
eth3
0002b33f7dc7
10.10.1.2
pc103.emulab.net
ethernet 100 0.05 urn:ogf:network:domain=emulab.net+slice+lnevers-lamp-slice1:node=node1:port=iface0 urn:ogf:network:domain=emulab.net+slice+lnevers-lamp-slice1:node=node2:port=iface0
Enter PEM pass phrase: Received: success.ma.replaceddata element(s) successfully replaced http://ggf.org/ns/nmwg/topology/20070809 }}} '''4. Get the LAMP certificate'''. The LAMP services require a special certificate for the user and the slice within the expiration time to access topology information withing UNISIS. {{{ $ ./lamp-getcertificate.py -n lnevers-lamp-slice1 Got my SA credential, looking up lnevers-lamp-slice1 Asking for slice credential for lnevers-lamp-slice1 Got the slice credential Asking for my lamp certificate Paste the following certificate *as is* into a file called lampcert.pem Upload the certificate to all LAMP enabled nodes at /usr/local/etc/protogeni/ssl/lampcert.pem -----BEGIN RSA PRIVATE KEY----- .. many lines of output not show here.... -----END CERTIFICATE----- }}} '''5. Upload LAMP Certificate''' Determined which node to which to upload the LAMP certificate, and then upload certificate to /usr/local/etc/protogeni/ssl/lampcert.pem on each node. {{{ $ grep "login" lnevers-lamp-slice1-manifest.xml $ for node in pc86.emulab.net pc101.emulab.net pc103.emulab.net; do scp lampcert.pem lnevers@$node:.; ssh lnevers@$node "sudo mv lampcert.pem /usr/local/etc/protogeni/ssl/lampcert.pem"; ssh lnevers@$node "sudo chown root.perfsonar /usr/local/etc/protogeni/ssl/lampcert.pem"; ssh lnevers@$node "sudo chmod 440 /usr/local/etc/protogeni/ssl/lampcert.pem"; ssh lnevers@$node "sudo /etc/init.d/psconfig restart"; done lampcert.pem 100% 4226 4.1KB/s 00:00 /etc/init.d/psconfig stop: perfSONAR pSConfig Service stopped waiting... /opt/perfsonar_ps/perfSONAR_PS-pSConfig/bin/daemon.pl --ssl-enable --config=/usr/local/etc/perfSONAR/pSConfig.conf --pidfile=psconfig.pid --piddir=/var/run --logger=/usr/local/etc/perfSONAR/pSConfig_logger.conf --user=perfsonar --group=perfsonar /etc/init.d/psconfig start: perfSONAR pSConfig Service started lampcert.pem 100% 4226 4.1KB/s 00:00 /etc/init.d/psconfig stop: perfSONAR pSConfig Service stopped waiting... /opt/perfsonar_ps/perfSONAR_PS-pSConfig/bin/daemon.pl --ssl-enable --config=/usr/local/etc/perfSONAR/pSConfig.conf --pidfile=psconfig.pid --piddir=/var/run --logger=/usr/local/etc/perfSONAR/pSConfig_logger.conf --user=perfsonar --group=perfsonar /etc/init.d/psconfig start: perfSONAR pSConfig Service started lampcert.pem 100% 4226 4.1KB/s 00:00 /etc/init.d/psconfig stop: perfSONAR pSConfig Service stopped waiting... /opt/perfsonar_ps/perfSONAR_PS-pSConfig/bin/daemon.pl --ssl-enable --config=/usr/local/etc/perfSONAR/pSConfig.conf --pidfile=psconfig.pid --piddir=/var/run --logger=/usr/local/etc/perfSONAR/pSConfig_logger.conf --user=perfsonar --group=perfsonar /etc/init.d/psconfig start: perfSONAR pSConfig Service started }}} After the services were started still could connect to the LAMP Portal, it was suggested that Web Server Certificate Bundle should be refreshed: {{{ lamp:~> sudo /usr/local/etc/protogeni/ssl/getcacerts Files /usr/local/etc/protogeni/ssl/genica.bundle and /tmp/genica.bundle.6774 differ Files /usr/local/etc/protogeni/ssl/genicrl.bundle and /tmp/genicrl.bundle.6774 differ /etc/init.d/psconfig stop: perfSONAR pSConfig Service stopped /opt/perfsonar_ps/perfSONAR_PS-pSConfig/bin/daemon.pl --ssl-enable --config=/usr/local/etc/perfSONAR/pSConfig.conf --pidfile=psconfig.pid --piddir=/var/run --logger=/usr/local/etc/perfSONAR/pSConfig_logger.conf --user=perfsonar --group=perfsonar /etc/init.d/psconfig start: perfSONAR pSConfig Service started * Starting MySQL database server mysqld [ OK ] * Checking for corrupt, not cleanly closed and upgrade needing tables. lamp:~> }}} After refreshing the Webserver Certificate bundle, the service should be restarted on the node by using the following: {{{ lamp:~> sudo /etc/init.d/psconfig restart }}} ''' 6. You can now connect to the LAMP Portal'''. Following is a capture of the main page or "Slice Overview -> Configuration Status" page [[Image(Lamp-portal-1.jpg)]] Selecting the ''Registered Services'' brings up the following page: [[Image(Lamp-portal-2.jpg)]] The "Configuration -> Enabled Services" shows the following: [[Image(Lamp-portal-3.jpg)]] The "Configuration -> Clock Synchronization" shows the following: [[Image(Lamp-portal-4.jpg)]] After selecting an NTP server, selected Bandwitdth and Latency services for node1 and node2: [[Image(Lamp-portal-5.jpg)]] ''' 7. You can now enable/disable services on each node'''. Following instructions in the tutorial enabled the following services on each node: * node1: Host Monitoring Daemon, Latency services, BWCTL, and the NTP server. * node2: Host Monitoring Daemon, Bandwidth services, OWAMP, and the NTP server. * lamp: LAMP Portal, Host Monitoring Collector, the Ganglia Measurement Archive, and NTP server. Once the services are enables switched to "Configuration Status" page to "Push Configuration to UNISIS" [[Image(Lamp-portal-6.jpg)]] ''' 8. Schedule Tests'''. In the "Scheduled Tests" page you can configure latency and bandwidth tests, following is the capture for enabling Throughput test node2: [[Image(Lamp-portal-7.jpg)]] After several minutes you may view results by going to the "Registered Services" and clicking on the "Pull Registered Services from UNISIS": [[Image(Lamp-portal-8.jpg)]] To view ganglia data on the lamp node just connect to the https:///ganglia/ (collector_node usually = lamp node): [[Image(Lamp-portal-9.jpg)]]