wiki:GENIRacksHome/OpenGENIRacks/AdministrationTools

Version 8 (modified by tmitchel@bbn.com, 9 years ago) (diff)

--

HealthCheck

The healthcheck.py script must be run in a root shell as it requires admin privileges for several operations. To use this tool, run the following commands from the control node:

cd /home/gram/gram/src/gram/am/gram
sudo bash
source /etc/novarc
export PYTHONPATH=$PYTHONPATH:/home/gram/gram/src
python healthcheck.py

Healthcheck performs various checks including:

  • Checking the state of the GRAM services
  • Checking the state of the OpenStack services
  • Checking if the management namespace exists and is properly configured
  • Checking that the compute hosts are reachable
  • Checking that the AM in functioning by doing Allocate-Provision-Delete

If it detects errors, it prints out warnings and attempts to fix them.

Occasionally, after restart, there is an issue with OpenStack where the management namespace is not instantiated. In this case, healthcheck will attempt to recover the namespace by restarting the Quantum-L3 agent up to 10 times. If this does not work, as a last resort, the admin can tear down the recreate the management and public networks. Healthcheck will prompt to recreate the network.

  • Before recreating the networks, the old ones must be removed.

The simplest method is to use the Horizon browser interface to delete the routers and networks. Then use healthcheck to recreate these components.

Deleting Management Network

The Horizon interface can be accessed through a web browser by navigating to the following address: http://<ip of control node>/horizon/

The login credentials are the 'os_username' and 'os_password' in /etc/config.json on the control node.

To delete the external/management:

  1. Use omni/portal to delete all slivers on the AM as they will no longer be accessible
  2. Click on the admin tab at the top (Where there is "Project" and "Admin")
  3. Delete the router: Click on "Routers" in the left panel and click on the "Delete Router" button next to "externalRouter"
  4. Delete the networks: Click on "Networks" in the left panel, then click on "public" under "Network Names". This will bring up a page with the ports and subnets on that network. Delete all the ports, then delete all the subnets. Then, click on "Networks" again in the left tab and delete the "public" network.
  5. Repeat step (4) with "GRAM-mgmt-net"

Once this is complete, you can return to the health check terminal and proceed with recreating the management and public networks.

After creating the new management networks, you need to update the config.json file to reflect the new management namespace.

cd /home/gram/gram/src/gram/am/gram
sudo bash
source /etc/novarc
export PYTHONPATH=$PYTHONPATH:/home/gram/gram/src
python set_namespace.py