Changes between Version 1 and Version 2 of GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014
- Timestamp:
- 05/23/14 12:21:41 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014
v1 v2 1 = GRAMIntermediate Acceptance Test Report - April 2014 =1 = OpenGENI Intermediate Acceptance Test Report - April 2014 = 2 2 3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/ GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan]. Results are grouped by major area including: Experimenter, Administrative and Monitoring Acceptance tests. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. Once again there has been significant improvement since the last evaluation results captured in the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Most tickets reported in earlier evaluation have been successfully addressed.3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan GRAM Acceptance Test Plan]. Results are grouped by major area including: Experimenter, Administrative and Monitoring Acceptance tests. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. Once again there has been significant improvement since the last evaluation results captured in the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Most tickets reported in earlier evaluation have been successfully addressed. 4 4 5 5 == Experimenter Acceptance Tests == 6 6 7 === GR-EXP-1: Bare Metal Support Acceptance Test ===7 === OG-EXP-1: Bare Metal Support Acceptance Test === 8 8 9 9 Test Case Status: "Fail" … … 11 11 Test case fails due to lack of supported for bare metal nodes and for Windows OS. Test Case Tickets: 12 12 13 - 51 Bare metal support feature is missing for GR-EXP-114 - 70 MS Windows support requires handling of account setup in GRAM13 - 51 Bare metal support feature is missing for OG-EXP-1 14 - 70 MS Windows support requires handling of account setup in OpenGENI 15 15 16 === GR-EXP-2: GRAMSingle Site Acceptance Test ===16 === OG-EXP-2: OpenGENI Single Site Acceptance Test === 17 17 18 18 Test Case Status: "Pass: most criteria" 19 19 20 This test uses resources in one GRAMresources and it was successfully executed. Test Case Tickets:20 This test uses resources in one OpenGENI resources and it was successfully executed. Test Case Tickets: 21 21 22 22 - 104 Sliverstatus is inaccurate and nodes are shown as ready while they are not ready for use. … … 24 24 - 110 There is no bandwidth shaping, experimenter specified capacity is ignored. 25 25 26 === GR-EXP-3: GRAMSingle Site 100 VM Test ===26 === OG-EXP-3: OpenGENI Single Site 100 VM Test === 27 27 28 28 Test Case Status: "Pass: most criteria" … … 33 33 34 34 35 === GR-EXP-4: GRAMMulti-site Acceptance Test ===35 === OG-EXP-4: OpenGENI Multi-site Acceptance Test === 36 36 37 37 Test Case Status: "Pass". … … 44 44 - 110 There is no bandwidth shaping, experimenter specified capacity is ignored. 45 45 46 === GR-EXP-5: GRAMNetwork Resources Acceptance Test ===46 === OG-EXP-5: OpenGENI Network Resources Acceptance Test === 47 47 48 48 This test has not been executed yet, need site resources to be added to Clemson rack !OpenFlow switch. 49 49 50 === GR-EXP-6: GRAMand Meso-scale Multi-site !OpenFlow Acceptance Test ===50 === OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test === 51 51 Test Case Status: "In Progress". 52 52 53 53 This test case status is being executed with local OpenFlow connections and over stitched OpenFlow connections to InstaGENI sites. There is no testing with ExoGENI due to a bug in ExoGENI code that prevents user from specifying an !OpenFlow Controller. !OpenFlow controllers used successfully VMOC (patched POX) and !FloodLight. Test Case Tickets: 54 54 55 - 114 GRAMOpenFlow handshake failure with POX experimenter controllers - POX requires patching55 - 114 OpenGENI OpenFlow handshake failure with POX experimenter controllers - POX requires patching 56 56 57 === GR-EXP-7: Click Router Experiment Acceptance Test ===57 === OG-EXP-7: Click Router Experiment Acceptance Test === 58 58 59 59 This test is not planned at this time. … … 61 61 == Administration Acceptance Tests == 62 62 63 === GR-ADM-1: Rack Receipt and Inventory Test ===63 === OG-ADM-1: Rack Receipt and Inventory Test === 64 64 65 65 Test Case Status: "In Progress". … … 67 67 Most of this test cannot be completed at BBN, there was no rack delivery process and there is no GMOC monitoring: 68 68 69 - 49 There is no GMOC data collection or GRAMmonitoring data reporting69 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting 70 70 71 === GR-ADM-2: Rack Administrator Access Test ===71 === OG-ADM-2: Rack Administrator Access Test === 72 72 73 73 Test Case Status: "In Progress" … … 77 77 - 65 Remote console not available via web interface, some iDRAC access via SSH. 78 78 79 === GR-ADM-3: Full Rack Reboot Test ===79 === OG-ADM-3: Full Rack Reboot Test === 80 80 81 81 Test Case Status: "Pass". … … 83 83 This test was executed and verified full rack power down and power up, along with process health at bootup. No issues remain. 84 84 85 === GR-ADM-4: Emergency Stop Test ===85 === OG-ADM-4: Emergency Stop Test === 86 86 87 87 This test is not planned at this time because it requires requires GMOC interaction and monitoring. Without monitoring GMOC cannot participate in an emergency exercise: 88 88 89 - 49 There is no GMOC data collection or GRAMmonitoring data reporting89 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting 90 90 91 === GR-ADM-5: Software Update Test ===91 === OG-ADM-5: Software Update Test === 92 92 93 93 Test Case Status: "In Progress" 94 94 95 GRAMDebian package creation for initial install and upgrades tested.95 OpenGENI Debian package creation for initial install and upgrades tested. 96 96 97 97 98 === GR-ADM-6: Control Network Disconnection Test ===98 === OG-ADM-6: Control Network Disconnection Test === 99 99 100 100 Test Case Status: "In Progress" … … 102 102 This test requires monitoring which is not available: 103 103 104 - 49 There is no GMOC data collection or GRAMmonitoring data reporting104 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting 105 105 106 === GR-ADM-7: Documentation Review Test ===106 === OG-ADM-7: Documentation Review Test === 107 107 108 108 Test Case Status: "In Progress" … … 110 110 Significant improvement in documentation since the last evaluation. This documentation test case checks for the existence of 14 separate documents. Each are listed below with the overall status and associated ticket. 111 111 112 1. GRAMRack Installation Documentation - Pass: most criteria - 74113 2. GRAMRack Part List Documentation - Pass: most criteria - 75114 3. GRAMSoftware Components Identification Documentation - Pass115 4. GRAMand OS Upgrade Plans - Pass - 77116 5. GRAMSoftware Components Functional Description and Access to Source code - Pass117 6. GRAMRack Resources Usage and Policy Documentation - In Progress - 79118 7. GRAMRack Resources Administration - Pass: most criteria - 80119 8. GRAMRack Administrative procedure for user accounts - Pass: most criteria120 9. GRAMRack Administrative procedure for rack operations - Pass121 10. GRAMRack Administrative procedure for rack scheduled shutdown - Pass122 11. GRAMRack Administrative procedure for Emergency Stop - In Progress - 84123 12. GRAMRack Administrative procedure for health status - Fail124 13. GRAMRack Site Operations documentation - Fail - 86125 14. Site Specific GRAMDeployment Data - Fail - 87112 1. OpenGENI Rack Installation Documentation - Pass: most criteria - 74 113 2. OpenGENI Rack Part List Documentation - Pass: most criteria - 75 114 3. OpenGENI Software Components Identification Documentation - Pass 115 4. OpenGENI and OS Upgrade Plans - Pass - 77 116 5. OpenGENI Software Components Functional Description and Access to Source code - Pass 117 6. OpenGENI Rack Resources Usage and Policy Documentation - In Progress - 79 118 7. OpenGENI Rack Resources Administration - Pass: most criteria - 80 119 8. OpenGENI Rack Administrative procedure for user accounts - Pass: most criteria 120 9. OpenGENI Rack Administrative procedure for rack operations - Pass 121 10. OpenGENI Rack Administrative procedure for rack scheduled shutdown - Pass 122 11. OpenGENI Rack Administrative procedure for Emergency Stop - In Progress - 84 123 12. OpenGENI Rack Administrative procedure for health status - Fail 124 13. OpenGENI Rack Site Operations documentation - Fail - 86 125 14. Site Specific OpenGENI Deployment Data - Fail - 87 126 126 127 127 … … 129 129 == Monitoring Acceptance Tests == 130 130 131 === GR-MON-1: Control Network Software and VLAN Inspection Test ===131 === OG-MON-1: Control Network Software and VLAN Inspection Test === 132 132 133 133 Test Case Status: "In Progress" 134 134 135 Most checks were successful, except for GRAMgit access is BBN internal only.135 Most checks were successful, except for OpenGENI git access is BBN internal only. 136 136 137 - 66 GRAMgit access is BBN internal only, external access will be needed after initial evaluation137 - 66 OpenGENI git access is BBN internal only, external access will be needed after initial evaluation 138 138 139 === GR-MON-2: GENI Software Configuration Inspection Test ===139 === OG-MON-2: GENI Software Configuration Inspection Test === 140 140 141 141 This test was deemed "Pass". No issue were found. 142 142 143 === GR-MON-3: GENI Active Experiment Inspection Test ===143 === OG-MON-3: GENI Active Experiment Inspection Test === 144 144 145 145 Test Case Status: "In Progress" … … 147 147 Test is in progress, no issue were found so far. 148 148 149 === GR-MON-4: Infrastructure Device Performance Test ===149 === OG-MON-4: Infrastructure Device Performance Test === 150 150 151 151 Test Case Status: "Blocked" … … 153 153 Cannot be executed without monitoring data collection: 154 154 155 - 49 There is no GMOC data collection or GRAMmonitoring data reporting155 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting 156 156 157 === GR-MON-5: GMOC Data Collection Test ===157 === OG-MON-5: GMOC Data Collection Test === 158 158 159 159 Test Case Status: "Blocked" … … 161 161 Cannot be executed without monitoring data collection: 162 162 163 - 49 There is no GMOC data collection or GRAMmonitoring data reporting163 - 49 There is no GMOC data collection or OpenGENI monitoring data reporting