Changes between Version 1 and Version 2 of GENIRacksHome/OpenGENIRacks/AcceptanceTestReportMay2013


Ignore:
Timestamp:
05/23/14 12:31:19 (9 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GENIRacksHome/OpenGENIRacks/AcceptanceTestReportMay2013

    v1 v2  
    1 = GRAM Acceptance Test Report - May 2013 =
     1= OpenGENI Acceptance Test Report - May 2013 =
    22
    3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan].
     3This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan OpenGENI Acceptance Test Plan].
    44
    55
    … …  
    1515 - ticket 68 Requesting host with two interfaces results in allocated host with one interface.
    1616 - ticket 69 renewsliver fails with slice_object not defined error   -- MAJOR ISSUE --
    17  - ticket 72 Unable to connect local campus resources nor remote meso-scale resources via GRAM !OpenFlow
    18  - ticket 49 There is no GMOC data collection or GRAM monitoring data reporting for GR-MON-4/5 
     17 - ticket 72 Unable to connect local campus resources nor remote meso-scale resources via OpenGENI !OpenFlow
     18 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting for OG-MON-4/5 
    1919
    2020
    2121== Experimenter Acceptance Tests ==
    2222
    23 === GR-EXP-1: Bare Metal Support Acceptance Test ===
     23=== OG-EXP-1: Bare Metal Support Acceptance Test ===
    2424
    25 This test case failed due to a several issues. There is no supported for bare metal nodes in GRAM.
     25This test case failed due to a several issues. There is no supported for bare metal nodes in OpenGENI.
    2626Modified the test case to replace all dedicated nodes (aka bare metal) with VMs to get test coverage.   
    27 !OpenStack does support MS Windows, but in order for GRAM to support this GENI required OS, there needs
     27!OpenStack does support MS Windows, but in order for OpenGENI to support this GENI required OS, there needs
    2828to be work done to add the set up of accounts and installation of SSH keys on the allocated VMs.
    2929Issues captured in tickets:
    3030
    31  - ticket 48 GPO PG account not recognized at by GRAM aggregate. (Used local Slice authority on controller node for testing).
    32  - ticket 51 Bare metal support feature is missing for GR-EXP-1
     31 - ticket 48 GPO PG account not recognized at by OpenGENI aggregate. (Used local Slice authority on controller node for testing).
     32 - ticket 51 Bare metal support feature is missing for OG-EXP-1
    3333 - ticket 52 Advertisement RSpec fails rspeclint
    3434 - ticket 54 performoperationalaction start fails with "code 13: Unsupported API call"
    35  - ticket 70 MS Windows support requires handling of account setup in GRAM
     35 - ticket 70 MS Windows support requires handling of account setup in OpenGENI
    3636
    37 === GR-EXP-2: GRAM Single Site Acceptance Test ===
     37=== OG-EXP-2: OpenGENI Single Site Acceptance Test ===
    3838
    3939This test was completed because most steps could be executed, but two major issue was discovered.
    … …  
    4848 - ticket 58 slivers with traffic are not deleted and remain available after sliver is deleted
    4949
    50 === GR-EXP-3: GRAM Single Site 100 VM Test ===
     50=== OG-EXP-3: OpenGENI Single Site 100 VM Test ===
    5151
    5252This test case failed due to a several issues. Configuration limitations such as 10 VLAN
    … …  
    5555Additional testing may have been run, but it was deemed not worth the time initial results.
    5656
    57  - ticket 62 compute node 4 is not available, even though it is configured as part of GRAM
     57 - ticket 62 compute node 4 is not available, even though it is configured as part of OpenGENI
    5858 - ticket 63 Requesting 5 VM on 1 lan only results in ssh access via port 0   
    5959 - ticket 64 Creating 20 slices with 2 VMs on 1 vlan per sliver fails due to configuration limitations (10 VLAN available only) 
    6060
    61 === GR-EXP-4: GRAM Multi-site Acceptance Test ===
     61=== OG-EXP-4: OpenGENI Multi-site Acceptance Test ===
    6262
    6363This test case failed due to a several issues. Was unable to request a host with two interfaces,
    … …  
    6767
    6868 - ticket 57 Allocated nodes do not have sudo access setup for user   
    69  - ticket 67 readyToLogin.py fails for GRAM
     69 - ticket 67 readyToLogin.py fails for OpenGENI
    7070 - ticket 68 Requesting host with two interfaces results in allocated host with one interface.
    7171
    72 === GR-EXP-5: GRAM Network Resources Acceptance Test ===
     72=== OG-EXP-5: OpenGENI Network Resources Acceptance Test ===
    7373
    74 This test case failed because it could not be executed. GRAM does not support local campus non-rack
     74This test case failed because it could not be executed. OpenGENI does not support local campus non-rack
    7575resource and remote meso-scale site connections via rack OF switch.  Ticket captures issue:
    7676
    77  - ticket 72 Unable to connect local campus resources nor remote meso-scale resources via GRAM !OpenFlow 
     77 - ticket 72 Unable to connect local campus resources nor remote meso-scale resources via OpenGENI !OpenFlow 
    7878
    79 === GR-EXP-6: GRAM and Meso-scale Multi-site !OpenFlow Acceptance Test ===
     79=== OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test ===
    8080
    8181This test case failed.  Test case was modified to run in in one rack, also there are no remote Meso-scale
    … …  
    8989 - ticket 71 Failed create sliver attempts keep failed resource information in place
    9090
    91 === GR-EXP-7: Click Router Experiment Acceptance Test ===
     91=== OG-EXP-7: Click Router Experiment Acceptance Test ===
    9292This test case failed and could not be completed because it requires sudo privileges and multiple interface.
    9393Issues are captured by the following tickets:
    … …  
    100100== Administration Acceptance Tests ==
    101101
    102 === GR-ADM-1: Rack Receipt and Inventory Test ===
     102=== OG-ADM-1: Rack Receipt and Inventory Test ===
    103103
    104104This test case walks through the delivery, initial connection and setup for a rack. Test case was
    105105not executed because no rack delivery, initial connection or set up occurred as part of the evaluation.
    106106
    107 === GR-ADM-2: Rack Administrator Access Test ===
     107=== OG-ADM-2: Rack Administrator Access Test ===
    108108
    109109This test case verified administrative access to all rack components. All components were accessible
    110110and console connections were available for all hosts and switches although no remote access was
    111111available for the consoles. An IPKVM was available in the rack, but was powered off and not configured
    112 for the current GRAM rack environment,this issue was captured in ticket:
     112for the current OpenGENI rack environment,this issue was captured in ticket:
    113113
    114114 - ticket 65 Remote console not available, IPKVM powered off and disconnected.
    115115
    116 === GR-ADM-3: Full Rack Reboot Test ===
     116=== OG-ADM-3: Full Rack Reboot Test ===
    117117
    118118This test case verifies the existence of a procedure that a site administrator may need to perform
    119 for site maintenance. The procedure should define the shutdown and power sequence for all GRAM
     119for site maintenance. The procedure should define the shutdown and power sequence for all OpenGENI
    120120components. No such document was found, this issue was captured in ticket:
    121121
    122  - ticket 83 GRAM Rack Administrative procedure for rack scheduled shutdown - Documentation requirement
     122 - ticket 83 OpenGENI Rack Administrative procedure for rack scheduled shutdown - Documentation requirement
    123123
    124 === GR-ADM-4: Emergency Stop Test ===
     124=== OG-ADM-4: Emergency Stop Test ===
    125125
    126126This test case cannot be executed because an emergency stop is not supported for the initial evaluation.
    127 There is no GMOC involvement for the GRAM rack.
     127There is no GMOC involvement for the OpenGENI rack.
    128128 
    129129
    130 === GR-ADM-5: Software Update Test ===
     130=== OG-ADM-5: Software Update Test ===
    131131
    132 This test passed. A list of dependencies was determined for GRAM, but initial list of package candidate
     132This test passed. A list of dependencies was determined for OpenGENI, but initial list of package candidate
    133133for updates were all up to date.  Chose the ''glance'' package for updating. Glance is the !OpenStack
    134134Image Registry and Delivery Service. Packages updated include Glance Daemons, API, Common, Registry,
    135135Python Library and Documentation. No issues were found.
    136136
    137 === GR-ADM-6: Control Network Disconnection Test ===
     137=== OG-ADM-6: Control Network Disconnection Test ===
    138138
    139139This test case cannot be executed because no GMOC monitoring is supported for the initial evaluation.
    140 There is no GMOC involvement for the GRAM rack.
     140There is no GMOC involvement for the OpenGENI rack.
    141141
    142 === GR-ADM-7: Documentation Review Test ===
     142=== OG-ADM-7: Documentation Review Test ===
    143143
    144144This documentation test case checks for the existence of 14 separate documents. Each are listed below
    145145with the overall status and associated ticket.
    146146
    147  1. GRAM Rack Installation Documentation - Partially met -  Ticket ticket 74   
    148  2. GRAM Rack Part List Documentation   - Fail - Ticket ticket 75
    149  3. GRAM Software Components Identification Documentation  - Pass: most criteria - Ticket ticket 76     
    150  4. GRAM and OS Upgrade Plans - Fail - Ticket ticket 77         
    151  5. GRAM Software Components Functional Description and Access to Source code   - Partially met - Ticket ticket 78
    152  6. GRAM Rack Resources Usage and Policy Documentation  - Fail - Ticket ticket 79       
    153  7. GRAM Rack Resources Administration  - Fail - Ticket ticket 80       
    154  8. GRAM Rack Administrative procedure for user accounts  -  Fail - Ticket ticket 81   
    155  9. GRAM Rack Administrative procedure for rack operations - Fail - Ticket ticket 82   
    156  10. GRAM Rack Administrative procedure for rack scheduled shutdown -   Fail - Ticket ticket 83         
    157  11. GRAM Rack Administrative procedure for Emergency Stop. -   Fail - Ticket ticket 84         
    158  12. GRAM Rack Administrative procedure for health status -     Fail - Ticket ticket 85         
    159  13. GRAM Rack Site Operations documentation    - Fail - Ticket ticket 86       
    160  14. Site Specific GRAM Deployment Data         - Fail - Ticket ticket 87       
     147 1. OpenGENI Rack Installation Documentation - Partially met -  Ticket ticket 74   
     148 2. OpenGENI Rack Part List Documentation   - Fail - Ticket ticket 75
     149 3. OpenGENI Software Components Identification Documentation  - Pass: most criteria - Ticket ticket 76     
     150 4. OpenGENI and OS Upgrade Plans - Fail - Ticket ticket 77         
     151 5. OpenGENI Software Components Functional Description and Access to Source code   - Partially met - Ticket ticket 78
     152 6. OpenGENI Rack Resources Usage and Policy Documentation  - Fail - Ticket ticket 79       
     153 7. OpenGENI Rack Resources Administration  - Fail - Ticket ticket 80       
     154 8. OpenGENI Rack Administrative procedure for user accounts  -  Fail - Ticket ticket 81   
     155 9. OpenGENI Rack Administrative procedure for rack operations - Fail - Ticket ticket 82   
     156 10. OpenGENI Rack Administrative procedure for rack scheduled shutdown -   Fail - Ticket ticket 83         
     157 11. OpenGENI Rack Administrative procedure for Emergency Stop. -   Fail - Ticket ticket 84         
     158 12. OpenGENI Rack Administrative procedure for health status -     Fail - Ticket ticket 85         
     159 13. OpenGENI Rack Site Operations documentation    - Fail - Ticket ticket 86       
     160 14. Site Specific OpenGENI Deployment Data         - Fail - Ticket ticket 87       
    161161
    162162
    163163== Monitoring Acceptance Tests - May 2013 ==
    164164
    165 === GR-MON-1: Control Network Software and VLAN Inspection Test ===
     165=== OG-MON-1: Control Network Software and VLAN Inspection Test ===
    166166This test case was marked as "Pass: met most criteria". There is a requirement in GENI for the aggregate software
    167 to be available externally, which is not the case for GRAM at this time.  This is reasonable for the stage of GRAM,
     167to be available externally, which is not the case for OpenGENI at this time.  This is reasonable for the stage of OpenGENI,
    168168so the test case passed because all function could be verified.  These are the issues encountered by this test:
    169169
    170  - ticket 62 compute node 4 is not available, even though it is configured as part of GRAM
    171  - ticket 66 GRAM git access is BBN internal only, external access will be needed after initial evaluation
     170 - ticket 62 compute node 4 is not available, even though it is configured as part of OpenGENI
     171 - ticket 66 OpenGENI git access is BBN internal only, external access will be needed after initial evaluation
    172172
    173173
    174 === GR-MON-2: GENI Software Configuration Inspection Test ===
     174=== OG-MON-2: GENI Software Configuration Inspection Test ===
    175175This test case passed.  As an administrator, it is possible to get access to administrative interfaces to determine
    176 the configuration of GRAM resources as well as determining resource allocated to experiments.  It is also possible
     176the configuration of OpenGENI resources as well as determining resource allocated to experiments.  It is also possible
    177177to determine network resources configured and in use. No issue were reported for this test case.
    178178
    179 === GR-MON-3: GENI Active Experiment Inspection Test ===
     179=== OG-MON-3: GENI Active Experiment Inspection Test ===
    180180
    181181This test case passed.  As an administrator, it possible to determine all required information information to map
    … …  
    185185
    186186
    187 === GR-MON-4: Infrastructure Device Performance Test ===
     187=== OG-MON-4: Infrastructure Device Performance Test ===
    188188
    189189This test could be executed because there is no GMOC monitoring available. Ticket tracking missing GMOC support:
    190190
    191  - ticket 49 There is no GMOC data collection or GRAM monitoring data reporting
     191 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting
    192192
    193 === GR-MON-5: GMOC Data Collection Test ===
     193=== OG-MON-5: GMOC Data Collection Test ===
    194194
    195 This test could be executed because there is no monitoring data collection at the GMOC for the GRAM rack and the
    196 GRAM site is not reporting to the GMOC. Ticket tracking missing GMOC support:
     195This test could be executed because there is no monitoring data collection at the GMOC for the OpenGENI rack and the
     196OpenGENI site is not reporting to the GMOC. Ticket tracking missing GMOC support:
    197197
    198  - ticket 49 There is no GMOC data collection or GRAM monitoring data reporting
     198 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting