Changes between Version 1 and Version 2 of GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014


Ignore:
Timestamp:
05/23/14 12:21:41 (10 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014

    v1 v2  
    1 = GRAM Intermediate Acceptance Test Report - April 2014 =
     1= OpenGENI Intermediate Acceptance Test Report - April 2014 =
    22
    3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan]. Results are grouped by major area including: Experimenter, Administrative and Monitoring Acceptance tests. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. Once again there has been significant improvement since the last evaluation results captured in the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Most tickets reported in earlier evaluation have been successfully addressed.
     3This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan GRAM Acceptance Test Plan]. Results are grouped by major area including: Experimenter, Administrative and Monitoring Acceptance tests. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. Once again there has been significant improvement since the last evaluation results captured in the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Most tickets reported in earlier evaluation have been successfully addressed.
    44
    55== Experimenter Acceptance Tests ==
    66
    7 === GR-EXP-1: Bare Metal Support Acceptance Test ===
     7=== OG-EXP-1: Bare Metal Support Acceptance Test ===
    88
    99Test Case Status: "Fail"
     
    1111Test case fails due to lack of supported for bare metal nodes and for Windows OS. Test Case Tickets:
    1212
    13  - 51 Bare metal support feature is missing for GR-EXP-1
    14  - 70 MS Windows support requires handling of account setup in GRAM
     13 - 51 Bare metal support feature is missing for OG-EXP-1
     14 - 70 MS Windows support requires handling of account setup in OpenGENI
    1515
    16 === GR-EXP-2: GRAM Single Site Acceptance Test ===
     16=== OG-EXP-2: OpenGENI Single Site Acceptance Test ===
    1717
    1818Test Case Status: "Pass: most criteria"
    1919
    20 This test uses resources in one GRAM resources and it was successfully executed. Test Case Tickets:
     20This test uses resources in one OpenGENI resources and it was successfully executed. Test Case Tickets:
    2121
    2222 - 104 Sliverstatus is inaccurate and nodes are shown as ready while they are not ready for use.
     
    2424 - 110 There is no bandwidth shaping, experimenter specified capacity is ignored.
    2525
    26 === GR-EXP-3: GRAM Single Site 100 VM Test ===
     26=== OG-EXP-3: OpenGENI Single Site 100 VM Test ===
    2727
    2828Test Case Status: "Pass: most criteria"
     
    3333
    3434
    35 === GR-EXP-4: GRAM Multi-site Acceptance Test ===
     35=== OG-EXP-4: OpenGENI Multi-site Acceptance Test ===
    3636
    3737Test Case Status: "Pass". 
     
    4444 - 110 There is no bandwidth shaping, experimenter specified capacity is ignored.
    4545
    46 === GR-EXP-5: GRAM Network Resources Acceptance Test ===
     46=== OG-EXP-5: OpenGENI Network Resources Acceptance Test ===
    4747
    4848This test has not been executed yet, need site resources to be added to Clemson rack !OpenFlow switch.
    4949
    50 === GR-EXP-6: GRAM and Meso-scale Multi-site !OpenFlow Acceptance Test ===
     50=== OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test ===
    5151Test Case Status: "In Progress". 
    5252
    5353This test case status is being executed with local OpenFlow connections and over stitched OpenFlow connections to InstaGENI sites.  There is no testing with ExoGENI due to a bug in ExoGENI code that prevents user from specifying an !OpenFlow Controller. !OpenFlow controllers used successfully VMOC (patched POX) and !FloodLight. Test Case Tickets:
    5454
    55  - 114 GRAM OpenFlow handshake failure with POX experimenter controllers - POX requires patching
     55 - 114 OpenGENI OpenFlow handshake failure with POX experimenter controllers - POX requires patching
    5656 
    57 === GR-EXP-7: Click Router Experiment Acceptance Test ===
     57=== OG-EXP-7: Click Router Experiment Acceptance Test ===
    5858
    5959This test is not planned at this time.
     
    6161== Administration Acceptance Tests ==
    6262
    63 === GR-ADM-1: Rack Receipt and Inventory Test ===
     63=== OG-ADM-1: Rack Receipt and Inventory Test ===
    6464
    6565Test Case Status: "In Progress". 
     
    6767Most of this test cannot be completed at BBN, there was no rack delivery process and there is no GMOC monitoring:
    6868
    69  - 49   There is no GMOC data collection or GRAM monitoring data reporting
     69 - 49   There is no GMOC data collection or OpenGENI monitoring data reporting
    7070
    71 === GR-ADM-2: Rack Administrator Access Test ===
     71=== OG-ADM-2: Rack Administrator Access Test ===
    7272
    7373Test Case Status: "In Progress"
     
    7777 - 65  Remote console not available via web interface, some iDRAC access via SSH.
    7878
    79 === GR-ADM-3: Full Rack Reboot Test ===
     79=== OG-ADM-3: Full Rack Reboot Test ===
    8080
    8181Test Case Status: "Pass". 
     
    8383This test was executed and verified full rack power down and power up, along with process health at bootup.  No issues remain.
    8484
    85 === GR-ADM-4: Emergency Stop Test ===
     85=== OG-ADM-4: Emergency Stop Test ===
    8686
    8787This test is not planned at this time because it requires requires GMOC interaction and monitoring.  Without monitoring GMOC cannot participate in an emergency exercise:
    8888
    89  - 49   There is no GMOC data collection or GRAM monitoring data reporting
     89 - 49   There is no GMOC data collection or OpenGENI monitoring data reporting
    9090 
    91 === GR-ADM-5: Software Update Test ===
     91=== OG-ADM-5: Software Update Test ===
    9292
    9393Test Case Status: "In Progress"
    9494
    95 GRAM Debian package creation for initial install and upgrades tested.
     95OpenGENI Debian package creation for initial install and upgrades tested.
    9696
    9797
    98 === GR-ADM-6: Control Network Disconnection Test ===
     98=== OG-ADM-6: Control Network Disconnection Test ===
    9999
    100100Test Case Status: "In Progress"
     
    102102This test requires monitoring which is not available:
    103103
    104  - 49   There is no GMOC data collection or GRAM monitoring data reporting
     104 - 49   There is no GMOC data collection or OpenGENI monitoring data reporting
    105105
    106 === GR-ADM-7: Documentation Review Test ===
     106=== OG-ADM-7: Documentation Review Test ===
    107107
    108108Test Case Status: "In Progress"
     
    110110Significant improvement in documentation since the last evaluation. This documentation test case checks for the existence of 14 separate documents. Each are listed below  with the overall status and associated ticket.
    111111
    112   1. GRAM Rack Installation Documentation  - Pass: most criteria - 74       
    113   2. GRAM Rack Part List Documentation  - Pass: most criteria - 75               
    114   3. GRAM Software Components Identification Documentation - Pass               
    115   4. GRAM and OS Upgrade Plans  - Pass - 77               
    116   5. GRAM Software Components Functional Description and Access to Source code - Pass             
    117   6. GRAM Rack Resources Usage and Policy Documentation - In Progress - 79             
    118   7. GRAM Rack Resources Administration - Pass: most criteria - 80           
    119   8. GRAM Rack Administrative procedure for user accounts - Pass: most criteria           
    120   9. GRAM Rack Administrative procedure for rack operations - Pass             
    121   10. GRAM Rack Administrative procedure for rack scheduled shutdown - Pass           
    122   11. GRAM Rack Administrative procedure for Emergency Stop - In Progress - 84             
    123   12. GRAM Rack Administrative procedure for health status - Fail         
    124   13. GRAM Rack Site Operations documentation  - Fail - 86     
    125   14. Site Specific GRAM Deployment Data  - Fail - 87           
     112  1. OpenGENI Rack Installation Documentation  - Pass: most criteria - 74       
     113  2. OpenGENI Rack Part List Documentation  - Pass: most criteria - 75               
     114  3. OpenGENI Software Components Identification Documentation - Pass               
     115  4. OpenGENI and OS Upgrade Plans  - Pass - 77               
     116  5. OpenGENI Software Components Functional Description and Access to Source code - Pass             
     117  6. OpenGENI Rack Resources Usage and Policy Documentation - In Progress - 79             
     118  7. OpenGENI Rack Resources Administration - Pass: most criteria - 80           
     119  8. OpenGENI Rack Administrative procedure for user accounts - Pass: most criteria           
     120  9. OpenGENI Rack Administrative procedure for rack operations - Pass             
     121  10. OpenGENI Rack Administrative procedure for rack scheduled shutdown - Pass           
     122  11. OpenGENI Rack Administrative procedure for Emergency Stop - In Progress - 84             
     123  12. OpenGENI Rack Administrative procedure for health status - Fail         
     124  13. OpenGENI Rack Site Operations documentation  - Fail - 86     
     125  14. Site Specific OpenGENI Deployment Data  - Fail - 87           
    126126
    127127
     
    129129== Monitoring Acceptance Tests ==
    130130
    131 === GR-MON-1: Control Network Software and VLAN Inspection Test ===
     131=== OG-MON-1: Control Network Software and VLAN Inspection Test ===
    132132
    133133Test Case Status: "In Progress"
    134134
    135 Most checks were successful, except for GRAM git access is BBN internal only.
     135Most checks were successful, except for OpenGENI git access is BBN internal only.
    136136
    137  - 66 GRAM git access is BBN internal only, external access will be needed after initial evaluation
     137 - 66 OpenGENI git access is BBN internal only, external access will be needed after initial evaluation
    138138
    139 === GR-MON-2: GENI Software Configuration Inspection Test ===
     139=== OG-MON-2: GENI Software Configuration Inspection Test ===
    140140
    141141This test was deemed "Pass". No issue were found.
    142142
    143 === GR-MON-3: GENI Active Experiment Inspection Test ===
     143=== OG-MON-3: GENI Active Experiment Inspection Test ===
    144144
    145145Test Case Status: "In Progress"
     
    147147Test is in progress, no issue were found so far.
    148148
    149 === GR-MON-4: Infrastructure Device Performance Test ===
     149=== OG-MON-4: Infrastructure Device Performance Test ===
    150150
    151151Test Case Status: "Blocked"
     
    153153Cannot be executed without monitoring data collection:
    154154
    155  - 49 There is no GMOC data collection or GRAM monitoring data reporting
     155 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting
    156156
    157 === GR-MON-5: GMOC Data Collection Test ===
     157=== OG-MON-5: GMOC Data Collection Test ===
    158158
    159159Test Case Status: "Blocked"
     
    161161Cannot be executed without monitoring data collection:
    162162
    163  - 49 There is no GMOC data collection or GRAM monitoring data reporting
     163 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting