Changes between Initial Version and Version 1 of PlasticSlices/MonitoringRecommendations/FoamConfiguration

Timestamp:

04/30/12 10:27:39 (12 years ago)

Author:

chaos@bbn.com

Comment:

--

PlasticSlices/MonitoringRecommendations/FoamConfiguration

@@ +1 @@
+[[PageOutline]]
+
+= How to configure Plastic Slices monitoring on an OpenFlow FOAM AM =
+
+== Introduction ==
+
+This page explains how to configure an !OpenFlow FOAM server for [wiki:PlasticSlices/MonitoringRecommendations central GENI operational monitoring].  This page was written for
+{{{
+tango-monitor-foam_0.1-1
+}}}
+and may not work well with earlier versions.  We have tested the FOAM utilities on Ubuntu 10.04 only.  If you are running a different OS and it is not obvious how to translate the instructions, please contact GPO infra.
+
+== Upgrading from previous versions ==
+
+If you are currently running a development snapshot of `tango-monitor-foam_0.1`, you need only do the following steps of this update:
+ * [#II.Installthemonitoringsoftware II. Upgrade the monitoring software itself]
+ * [#IV.Testthescript IV. Test the new script and make sure it still works] 
+
+If you are installing `tango-monitor-foam` for the first time or upgrading from an earlier version, read all sections to see which steps you need to take.
+
+== Variables ==
+
+ * `<cronuser>`: The user as which to run the FOAM monitoring.  The scripts can run as root, or as a non-root user of your choice.  (At GPO, we use the `ganglia` user.)
+ * `<site>`: a one-word phrase describing your site, probably the name of your university or lab (e.g. `stanford`, `gpolab`).  Use the same value for all hosts your site is monitoring.
+ * `<fqdn>`: The fully-qualified domain name as which your FV should report data.  ''IF this is not the same as the output of `uname -n` on your node, pay attention to section IV below.''
+
+== Steps to configure FOAM monitoring ==
+
+=== I. Prepare to submit authenticated monitoring data to GMOC ===
+
+You need to do these steps if you are installing monitoring for the first time.
+
+ 1. Register site credentials: IF you have not yet registered your site to submit data to GMOC, follow the instructions at [wiki:GENIMetaOps/SiteCredentials].  All site resources use the same credential; you only need to do this step once for your entire site.  '''Note: the password you use for your site registration is a new password generated for the purpose of submitting monitoring data.  Do not reuse any existing passwords from your site.'''
+
+ 2. Store your site password in a file which FOAM monitoring can use:
+{{{
+sudo touch /usr/local/etc/monitoring_passwd
+sudo chown <cronuser> /usr/local/etc/monitoring_passwd
+sudo chmod 600 /usr/local/etc/monitoring_passwd
+sudo vi /usr/local/etc/monitoring_passwd
+}}}
+ Add exactly one line to this file, containing the site monitoring password you registered with GMOC.
+
+=== II. Install the monitoring software ===
+
+Do these steps every time you install or upgrade the monitoring software.
+
+ 1. Download the `tango-monitor-foam_0.1-1.deb` package file from [http://software.geni.net/local-sw/], and copy it onto your FOAM host, e.g. into `~/tango-monitor-foam_0.1-1.deb`.
+
+ 2. Use dpkg to install the local package.  This may fail due to dependencies and say it is leaving the package "unconfigured".  Therefore, invoke apt-get to fix any missing dependencies:
+{{{
+sudo dpkg -i ~/tango-monitor-foam_0.1-1.deb
+sudo apt-get -f install
+}}}
+
+=== III. Create required directories and files ===
+
+These steps only need to be done the first time you install the monitoring software.
+ 
+ 1. Create required directories:
+{{{
+sudo mkdir -p /var/cache/fvmetrics/monitoring
+sudo chown <cronuser> /var/cache/fvmetrics/monitoring
+
+sudo mkdir -p /var/spool/rrds/$(uname -n) 
+sudo mkdir -p /var/spool/rrds/REPORT
+sudo chown -R <cronuser> /var/spool/rrds
+}}}
+
+ 2. IF `uname -n` does not report your node's FQDN for whatever reason, create a symlink so that reporting and reading scripts can find your RRD files:
+{{{
+cd /var/spool/rrds
+sudo ln -s $(uname -n) <fqdn>
+}}}
+
+ 3. The `metric_foam` script requires a FOAM password file, which it expects to find in:
+{{{
+/opt/foam/etc/foampasswd
+}}}
+ This file should contain the admin password for FOAM.  (This is the password you use with `foamctl` to run FOAM commands.)
+
+=== IV. Test the script ===
+
+Run the monitoring and submission scripts by hand to make sure they work.
+
+ 1. Check the time on your FOAM host, and make sure it is accurate. The data reporting format relies on clock accuracy, so your nodes must be running ntpd (or ntpdate regularly out of cron) in order for monitoring to work.
+
+ 2. Run the metric gathering script by hand to make sure it works:
+{{{
+sudo -u <cronuser> /usr/bin/metric_foam
+}}}
+   * This should produce no output
+   * After this runs, a number of RRD files should be created in `/var/spool/rrds/$(uname -n)`
+
+ 3. Run the data reporting script by hand to make sure it works:
+{{{
+sudo -u <cronuser> /usr/bin/report_data_to_gmoc <site> <fqdn>
+}}}
+   * This should produce no output
+   * You should be able to browse to [http://gmoc-db.grnoc.iu.edu/api-demo/], view data by Location, select your site, and find an entry for `<fqdn>`.
+
+=== VI. Install the scripts to run from cron ===
+
+This only needs to be done the first time you install the monitoring software.
+
+Add the two scripts to cron: modify the `<cronuser>` crontab:
+{{{
+sudo -u <cronuser> crontab -e
+}}}
+and add the new lines:
+{{{
+*/1 * * * * /usr/bin/metric_foam
+*/5 * * * * /usr/bin/report_data_to_gmoc <site> <fqdn>
+}}}