wiki:GENIRacksHome/OpenGENIRacks/PowerUp

Version 1 (modified by Jeanne Ohren, 5 years ago) (diff)

--

GRAM Rack Power Up Sequence

It is recommended that GRAM rack devices be powered in the following order:

  1. UPS (if included)
  2. Management Switch - Dell Powerconnect 7048
  3. OpenFlow Switch - Dell Force10
  4. Control Node
  5. Compute Nodes

Make sure that the above sequence is followed in starting the rack devices. If not followed, GRAM services may fail to start.

Checking GRAM Services

Once devices have powered up, the local administrator shoul check the services status and restore any suspended VMs. This section outlines the produre to be used:

  1. Login to the Control Node as an admin. For example, type "ssh -Y -i ~/.ssh/id_rsa admin_user@<control addr>"
  2. Run the GRAM tool healthcheck.py as directed in the Administration Tools? page.
  3. If processes have been started properly, you should expected output similar to the following, but specific to your site naming:
     gram-mon - running
     gram-ch - running
     checking OpenStack services...
     nova-api - running
     nova-cert - running
     nova-conductor - running
     nova-consoleauth  - running
     nova-novncproxy - running
     nova-scheduler - running
     quantum-dhcp-agent - running
     quantum-metadata-agent - running
     quantum-server - running
     quantum-l3-agent - running
     quantum-plugin-openvswitch-agent - running
     glance-registry - running
     glance-api - running
     keystone - running
    WARNING: Management namespace NOT found
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 0
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 1
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 2
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 3
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 4
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 5
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 6
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 7
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 8
    Restarting Quantum-L3 service to attempt to recover the namespace - attempt 9
    Found management namespace and it matches config
    Checking the status of the compute hosts: 
     
     Binary           Host                                 Zone             Status     State Updated_At
     nova-compute     boscompute2                          nova             enabled    :-)   2014-04-15 13:39:58
     nova-compute     boscompute1                          nova             enabled    :-)   2014-04-15 13:39:58
     
     Checking status of Openstack newtworking software modules: 
    
     +--------------------------------------+--------------------+-----------------------------------+-------+----------------+
     | id                                   | agent_type         | host                              | alive | admin_state_up |
     +--------------------------------------+--------------------+-----------------------------------+-------+----------------+
     | 97c56061-2dc7-4760-90c4-77fffea0e10b | Open vSwitch agent | boscompute1.gram.gpolab.bbn.com   | :-)   | True           |
     | a6b774d4-1e9d-4db3-b7f7-5ed39324598d | DHCP agent         | boscontroller.gram.gpolab.bbn.com | :-)   | True           |
     | c3cfb1ce-1cb0-4651-854c-65a1280a54d5 | L3 agent           | boscontroller.gram.gpolab.bbn.com | :-)   | True           |
     | cae16682-40db-488b-8b43-12812c8618a7 | Open vSwitch agent | boscompute2.gram.gpolab.bbn.com   | :-)   | True           |
     | d2387a09-1767-4798-a754-196bd2e842cc | Open vSwitch agent | boscontroller.gram.gpolab.bbn.com | :-)   | True           |
     +--------------------------------------+--------------------+-----------------------------------+-------+----------------+
    
     Keystone - pass
     Nova - pass
     Glance - pass
     Quantum - pass
     AM is up : Get-Version succeeded at AM
     Allocate - success
     Provision - success
     Delete - success
     AM is functioning
    
    Note: AM is functioning signifies that the rack is operational. If the AM is not functioning, the healthcheck script will print out out warnings and attempts to fix them.
  1. Resume VMs that were suspended on the powering down of the rack, they must be restored to their original state. For each ID suspended, you should issue a "nova resume ID". For example, "nova resume b840058e-4511-420c-aaf4-577562b2dce6". So, when typing "nova list --all-tenants", the state of this ID is back to ACTIVE.
+--------------------------------------+------+--------+--------------------------------------------+
| ID                                   | Name | Status | Networks                                   |
+--------------------------------------+------+--------+--------------------------------------------+
| b840058e-4511-420c-aaf4-577562b2dce6 | VM-1 | ACTIVE | GRAM-mgmt-net=192.168.10.3; lan0=10.0.37.1 |
| cfa1aa58-e68f-4176-beed-60e9a4257ab3 | VM-2 | ACTIVE | GRAM-mgmt-net=192.168.10.4; lan0=10.0.37.2 |
+--------------------------------------+------+--------+--------------------------------------------+