Changes between Initial Version and Version 1 of GENIRacksHome/OpenGENIRacks/AcceptanceTestStatusDec2013/OG-ADM-7


Ignore:
Timestamp:
05/23/14 11:34:32 (10 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIRacksHome/OpenGENIRacks/AcceptanceTestStatusDec2013/OG-ADM-7

    v1 v1  
     1= GR-ADM-7: Documentation Review Test =
     2
     3
     4This page captures status for the test case GR-ADM-7. For additional information see the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusDec2013 Acceptance Test Status - December 2013] page overall status, or the [wiki:GENIRacksHome/AcceptanceTests/GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan] for details about the planned evaluation.
     5
     6''Last Update: 2013/12/18"
     7
     8= Test Status =
     9
     10This section captures the status for each step in the acceptance test plan.
     11
     12|| '''Documentation'''                                                        ||'''State'''                                      || ''' Tickets ''' || '''Notes''' ||
     13||1. GRAM Rack Installation Documentation                                     ||[[span(Pass: most criteria, style=background-color: #98FB98)]]||#74                 ||             ||
     14||2. GRAM Rack Part List Documentation                                        ||[[span(Pass: most criteria, style=background-color: #98FB98)]]||#75                 ||             ||
     15||3. GRAM Software Components Identification Documentation                    ||[[span(Pass, style=background-color: green )]]                ||#76                 ||             ||
     16||4. GRAM and OS Upgrade Plans                                                ||[[span(In Progress, style=background-color: #63B8FF)]]        ||#77                 ||             ||
     17||5. GRAM Software Components Functional Description and Access to Source code||[[span(Pass, style=background-color: green )]]                ||#78             ||
     18||6. GRAM Rack Resources Usage and Policy Documentation                       ||[[span(Fail, style=background-color: red)]]                   ||#79                 ||            ||
     19||7. GRAM Rack Resources Administration                                       ||[[span(Pass: most criteria, style=background-color: #98FB98)]]||#80                 ||            ||
     20||8. GRAM Rack Administrative procedure for user accounts                     ||[[span(Pass: most criteria, style=background-color: #98FB98)]]||#81                 ||            ||
     21||9. GRAM Rack Administrative procedure for rack operations                   ||[[span(Fail, style=background-color: red)]]                   ||#82                 ||            ||
     22||10. GRAM Rack Administrative procedure for rack scheduled shutdown          ||[[span(In Progress, style=background-color: #63B8FF)]]        ||#83              ||            ||
     23||11. GRAM Rack Administrative procedure for Emergency Stop.                  ||[[span(In Progress, style=background-color: #63B8FF)]]        ||#84              ||            ||
     24||12. GRAM Rack Administrative procedure for health status                    ||[[span(Fail, style=background-color: red)]]                   ||#85              ||            ||
     25||13. GRAM Rack Site Operations documentation                                 ||[[span(Fail, style=background-color: red)]]                   ||#86              ||            ||
     26||14. Site Specific GRAM Deployment Data                                      ||[[span(Fail, style=background-color: red)]]                   ||#87              ||            ||
     27
     28
     29[[BR]]
     30|| '''State Legend'''                                           || '''Description'''                                                    ||
     31||[[span(Pass, style=background-color: green )]]                ||Documentation review completed and met all criteria                                  ||
     32||[[span(Pass: most criteria, style=background-color: #98FB98)]]||Documentation review completed and met most criteria. Exceptions documented  ||
     33||[[span(Fail, style=background-color: red)]]                   ||Documentation review completed and failed to meet criteria .                          ||
     34||[[span(Complete, style=background-color: yellow)]]            ||Documentation review completed but will require re-execution due to expected changes  ||
     35||[[span(Partial, style=background-color: orange)]]             ||Minimal Documentation was found                                          ||
     36||[[span(In Progress, style=background-color: #63B8FF)]]        ||Currently under review                                                 ||
     37
     38[[BR]]
     39
     40= GRAM Documentation Tests =
     41
     42The GRAM Documentation Tests focus on documentation review, and apply the following criteria for each document:
     43 * The document has been provided in a public location. This criteria failed to be met, all documentation is internal for initial evaluation.
     44 * The document contains the required information.
     45 * The documented information appears to be accurate.
     46
     47Rack documentation available and reviewed to find required information includes:
     48 * [https://superior.bbn.com/trac/bbn-rack/wiki/AdministrationGuide GRAM Administration Guide] - No changes were made to this page in since the last evaluation.
     49 * [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideGrizzly GRAM Installation Guide with Grizzly !OpenStack] Significant changes where made to this guide.
     50 * [https://superior.bbn.com/trac/bbn-rack/wiki/ArchitectureDescription GRAM Network and Software Architecture Description]  - Minimal changes were made to this document since last evaluation.
     51 * [https://superior.bbn.com/trac/bbn-rack/wiki/TestGuide GRAM Testing Guide] - No changes were made to this page in since the last evaluation.
     52 * [https://superior.bbn.com/trac/bbn-rack/wiki/UserGuide  GRAM Users Guide] - Changes have been made to this guide to capture new features.
     53 * [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails Details on BBN-CAM GRAM Rack Installation and Configuration] details pages.
     54
     55
     56== 1. GRAM Rack Installation Documentation ==   
     57
     58'''The goal of this evaluation:'''
     59
     60To find pre-installation document that lists specific minimum requirements for all site-provided services for potential rack sites (e.g. space, number and type of power plugs, number and type of power circuits, cooling load, public addresses, NLR or Internet2 layer2 connections, etc.).   This document should also list all standard expected rack interfaces (e.g. 10GBE links to at least one research network).
     61
     62'''Evaluation Results''': [[span(Pass: most criteria, style=background-color: #98FB98)]]
     63
     64The page [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails Details on BBN-CAM GRAM Rack Installation and Configuration] and found:
     65 - Rack space requirements
     66 - Power requirements 
     67 - Network and Hardware Configuration
     68 - Cooling requirements
     69
     70Not found:
     71 - Although public IP addresses are listed in the initial topology captured on this page, there is no indication of the minimum required number of public address for rack components and for experiment usage. 
     72 - Core backbone connectivity requirements (layer2).
     73
     74== 2. GRAM Rack Part List Documentation ==     
     75
     76
     77'''The goal of this evaluation:'''
     78
     79To find a summary GRAM Rack parts list, including vendor part numbers for "standard" equipment intended for all sites (e.g. a VM server) and per-site equipment options (e.g. transceivers, PDUs etc.), if any.  This document should also indicate approximately how much headroom, if any, remains in the standard rack PDUs' power budget to support other equipment that sites may add to the rack.
     80
     81'''Evaluation Results''': [[span(Pass: most criteria, style=background-color: #98FB98)]]
     82
     83A parts list was found in the page [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails Details on BBN-CAM GRAM Rack Installation and Configuration].
     84 
     85Not found:
     86 - Document that indicates approximately how much headroom, if any, remains in the standard rack PDUs' power budget to support additional equipment added by site to the rack.
     87
     88== 3. GRAM Software Components Identification Documentation  ==       
     89
     90'''The goal of this evaluation:'''
     91
     92Find a procedure for identifying the software versions and system file configurations running on a rack, and how to get information about recent changes to the rack software and configuration.
     93 
     94'''Evaluation Results:''' [[span(Pass, style=background-color: green )]]
     95
     96Found:
     97 - Clearly defined components in [https://superior.bbn.com/trac/bbn-rack/wiki/ArchitectureDescription GRAM Network and Software Architecture Description].
     98 - Information for system configurations used in GRAM rack in the  [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideRevised  GRAM Installation Guide - Revised].
     99 - Software Configuration versioning for components installed on the   [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails#SoftwareConfiguration BBN-CAM] rack.
     100 - [https://superior.bbn.com/trac/bbn-rack/wiki/SoftwareRepository Software Repository] page provides a pointer to gram-dev@bbn.com for software help and for access to git repository, where recent configuration and software changes are captured.
     101
     102== 4. GRAM and OS Upgrade Plans   ==                     
     103
     104'''The goal of this evaluation:'''
     105
     106Find documentation that explains how and when GRAM software and OS updates can be performed on a rack, including plans for notification and update if important security vulnerabilities in rack software are discovered.
     107
     108'''Evaluation Results:''' [[span(In Progress, style=background-color: #63B8FF)]]
     109
     110Found software update information in:
     111 - [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideGrizzly GRAM Installation Guide with OpenStack Grizzly] includes:
     112    * Configuration Overview
     113    * Network Configuration for !OpenFlow Switch for the Data Network and for Management Switch
     114    * GRAM and OpenStack Installation and Configuration
     115    * Testing GRAM installation
     116    * Debugging Notes
     117
     118 - [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideRevised  GRAM Installation Guide - Revised] includes instructions for installing !OpenStack and GRAM on the control and compute nodes.
     119
     120
     121In Progress
     122 - Updates being implemented for package update, only first time install available at this time
     123
     124Not Found:
     125 - Notification process for updates
     126 - Security updates plans
     127 
     128
     129== 5. GRAM Software Components Functional Description and Access to Source code ==   
     130   
     131'''The goal of this evaluation:'''
     132
     133Find documentation that provides a description of the GRAM software running on a standard rack, and explanation of how to get access to the source code of each piece of standard GENI software.
     134
     135'''Evaluation Results:''' [[span(Pass, style=background-color: green )]]
     136
     137Found:
     138 - Clearly defined components in [https://superior.bbn.com/trac/bbn-rack/wiki/ArchitectureDescription GRAM Network and Software Architecture Description].
     139 - [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideGrizzly Installation Instructions] for OpenStack Grizzly package and GRAM installation.
     140 - [https://superior.bbn.com/trac/bbn-rack/wiki/SoftwareRepository Software Repository] page with instructions for getting account.
     141
     142
     143 
     144== 6. GRAM Rack Resources Usage and Policy Documentation ==   
     145 
     146'''The goal of this evaluation:'''
     147
     148Find documentation that provides a description of all the GENI experimental resources within the rack, and what policy options exist for each, including: how to configure rack nodes as bare metal vs. VM server, what options exist for configuring automated approval of compute and network resource requests and how to set them, how to configure rack aggregates to trust additional GENI slice authorities, and whether it is possible to trust local users within the rack.
     149
     150'''Evaluation Results:'''   [[span(Fail, style=background-color: red)]]
     151
     152Did not find any Resources Usage and Policy documents.
     153
     154
     155== 7. GRAM Rack Resources Administration ==               
     156
     157'''The goal of this evaluation:'''
     158
     159Find documentation that provides a description of the expected state of all the GENI experimental resources in the rack, including how to determine the state of an experimental resource and what state is expected for an unallocated bare metal node.
     160
     161'''Evaluation Results:'''  [[span(Pass: most criteria, style=background-color: #98FB98)]]
     162
     163Found:
     164 - [https://superior.bbn.com/trac/bbn-rack/wiki/InstallationGuideGrizzly Installation Instructions] for OpenStack Grizzly package and GRAM installation.
     165 - [https://superior.bbn.com/trac/bbn-rack/wiki/UserGuide  GRAM Users Guide] give minimal insight into node roles in GRAM
     166 - [https://superior.bbn.com/trac/bbn-rack/wiki/BOSRackDetails BBN GRAM Rack Details] which defines the actual node settings for the GRAM Rack.
     167
     168Did not find:
     169 - documentation that would give an administrator a description of the expected state of all the GENI experimental resources in the rack.
     170
     171== 8. GRAM Rack Administrative procedure for user accounts ==         
     172
     173'''The goal of this evaluation:'''
     174
     175Find documentation that provides a procedure for creating new site administrator and operator accounts.
     176
     177'''Evaluation Results:''' [[span(Pass: most criteria, style=background-color: #98FB98)]]
     178
     179Found:
     180 - The [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails#AdministrationProcedures Administration Procedures] section in the [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails BBN-CAM rack details] page includes a link to the [https://superior.bbn.com/trac/bbn-rack/wiki/RackAdminAccounts Rack Admin Accounts] page.
     181- [https://superior.bbn.com/trac/bbn-rack/wiki/RackAdminAccounts Rack Admin Accounts] page includes instructions for adding and removing administrative users on the control and compute nodes.
     182
     183Not Found:
     184
     185- [https://superior.bbn.com/trac/bbn-rack/wiki/RackAdminAccounts Rack Admin Accounts] page  does not includes instructions for Administrative access to switches and to the dRAC console.
     186
     187
     188== 9.  GRAM Rack Administrative procedure for rack operations ==   
     189
     190'''The goal of this evaluation:'''
     191
     192Find documentation that provides a procedure for changing IP addresses for all rack components.
     193
     194'''Evaluation Results:''' [[span(Fail, style=background-color: red)]]
     195
     196No procedure was found for changing IP addresses for all rack components.
     197
     198== 10. GRAM Rack Administrative procedure for rack scheduled shutdown == 
     199
     200'''The goal of this evaluation:'''
     201
     202Find documentation that provides a procedure for cleanly shutting down an entire rack in case of a scheduled site outage.
     203
     204'''Evaluation Results:''' [[span(In Progress, style=background-color: #63B8FF)]]   
     205
     206Found:
     207 - Skeleton outline for [https://superior.bbn.com/trac/bbn-rack/wiki/PowerDown Rack Shutdown] and [https://superior.bbn.com/trac/bbn-rack/wiki/PowerUp Rack Power up]. 
     208
     209Was not able to verify, expect to execute [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatusDec2013/GR-ADM-3 GR-ADM-3: Full Rack Reboot Test] which will allow verification of the procedure.   
     210
     211     
     212== 11. GRAM Rack Administrative procedure for Emergency Stop. ==     
     213 
     214'''The goal of this evaluation:'''
     215
     216Find documentation that provides a procedure for performing a shutdown operation on any type of sliver on a rack, in support of an Emergency Stop request.
     217
     218'''Evaluation Results:''' [[span(In Progress, style=background-color: #63B8FF)]]       
     219
     220Found:
     221 - Skeleton outline for [https://superior.bbn.com/trac/bbn-rack/wiki/EmergencyStop Emergency Stop Procedure].
     222
     223Was unable to verify, expect to execute [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatusDec2013/GR-ADM-4 GR-ADM-4: Emergency Stop Test] in order to verify content of procedure.
     224
     225== 12. GRAM Rack Administrative procedure for health status ==   
     226     
     227'''The goal of this evaluation:'''
     228
     229Find documentation that provides a procedure for performing comprehensive health checks for a rack (or, if those health checks are being run automatically, how to view the current/recent results).
     230
     231'''Evaluation Results:''' [[span(Fail, style=background-color: red)]]
     232
     233No documentation exist that defines how to monitor GRAM component health.  Found healthcheck.py but there is no documentation for it.????
     234 
     235== 13. GRAM Rack Site Operations documentation ==     
     236 
     237'''The goal of this evaluation:'''
     238
     239Find documentation that provides a Technical plan for handing off primary rack operations to site operators at all sites.
     240
     241
     242'''Evaluation Results:''' [[span(Fail, style=background-color: red)]]
     243
     244No documentation found.
     245
     246== 14. Site Specific GRAM Deployment Data ==               
     247   
     248'''The goal of this evaluation:'''
     249
     250Find documentation that provides Per-site deployment details/documentation.  This documentation should be prepared before sites are installed and kept updated after installation to reflect any changes or upgrades after delivery.  Text, network diagrams, wiring diagrams and labeled photos are all acceptable for site documents.  Per-site documentation should include the following items for each site:
     251
     252   1.  Part numbers and quantities of PDUs, with NEMA input power connector types, and an inventory of which equipment connects to which PDU.
     253   2.  Physical network interfaces for each control and data plane port that connects to the site's existing network(s), including type, part numbers, maximum speed etc. (eg. 10-GB-SR fiber)
     254   3.  Public IP addresses allocated to the rack, including: number of distinct IP ranges and size of each range, hostname to IP mappings which should be placed in site DNS, whether the last-hop routers for public IP ranges subnets sit within the rack or elsewhere on the site, and what firewall configuration is desired for the control network.
     255   4.  Data plane network connectivity and procedures for each rack, including core backbone connectivity and documentation, switch configuration options to set for compatibility with the L2 core, and the site and rack procedures for connecting non-rack-controlled VLANs and resources to the rack data plane.  A network diagram is highly recommended (See existing !OpenFlow meso-scale network diagrams on the GENI wiki for examples.)
     256
     257
     258'''Evaluation Results:''' [[span(Fail, style=background-color: red)]]
     259
     260 1. Found a part list, but no part numbers. Power requirements are available [https://superior.bbn.com/trac/bbn-rack/wiki/DellRackDetails#PowerRequirements here],  there are no inventory of which equipment connects to which PDU.
     261 2. Items in item 2 not found.
     262 3. Items in item 3 not found
     263 4. Items in item 4 not found