Changes between Version 1 and Version 2 of GENIRacksHome/OpenGENIRacks/AcceptanceTestReportDec2013
- Timestamp:
- 05/23/14 12:26:19 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GENIRacksHome/OpenGENIRacks/AcceptanceTestReportDec2013
v1 v2 1 = GRAMIntermediate Acceptance Test Report - December 2013 =1 = OpenGENI Intermediate Acceptance Test Report - December 2013 = 2 2 3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/ GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan]. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusDec2013 GRAM Acceptance Test Status - December 2013] page. As can be seen from the page there has been significant improvement since the last evaluation results captured in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportMay2013 Acceptance Test Report - May 2013] (May 2013). Also many tickets reported in earlier evaluation have been successfully addressed. Following is a list of the most critical issues as of 2013-12-18:3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan GRAM Acceptance Test Plan]. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusDec2013 GRAM Acceptance Test Status - December 2013] page. As can be seen from the page there has been significant improvement since the last evaluation results captured in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportMay2013 Acceptance Test Report - May 2013] (May 2013). Also many tickets reported in earlier evaluation have been successfully addressed. Following is a list of the most critical issues as of 2013-12-18: 4 4 5 5 - ticket 100 There is no connectivity between racks 6 - ticket 49 There is no GMOC data collection or GRAM monitoring data reporting for GR-MON-4/56 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting for OG-MON-4/5 7 7 - ticket 51 Bare metal support feature is missing 8 - ticket 99 GRAMSoftware packaging8 - ticket 99 OpenGENI Software packaging 9 9 - ticket 65 Remote console not available via web interface, some iDRAC access via SSH. 10 - ticket 66 GRAMgit access is BBN internal only, external access will be needed after initial eval10 - ticket 66 OpenGENI git access is BBN internal only, external access will be needed after initial eval 11 11 12 12 13 13 == Experimenter Acceptance Tests == 14 14 15 === GR-EXP-1: Bare Metal Support Acceptance Test ===15 === OG-EXP-1: Bare Metal Support Acceptance Test === 16 16 17 17 This test case was deemed "Fail" due to the fact that there is still no supported for bare metal nodes and there is no support for windows. Modified the test case to replace all dedicated nodes (aka bare metal) with VMs to get test coverage.Issues captured in tickets: 18 18 19 - ticket 51 Bare metal support feature is missing for GR-EXP-120 - ticket 70 MS Windows support requires handling of account setup in GRAM19 - ticket 51 Bare metal support feature is missing for OG-EXP-1 20 - ticket 70 MS Windows support requires handling of account setup in OpenGENI 21 21 22 === GR-EXP-2: GRAMSingle Site Acceptance Test ===22 === OG-EXP-2: OpenGENI Single Site Acceptance Test === 23 23 24 24 This test was deemed "Pass". No issues were found. 25 25 26 === GR-EXP-3: GRAMSingle Site 100 VM Test ===26 === OG-EXP-3: OpenGENI Single Site 100 VM Test === 27 27 28 28 This test was deemed "Pass: most criteria". No problems were found, but only two scenario were tested due to lack of time. 29 29 30 === GR-EXP-4: GRAMMulti-site Acceptance Test ===30 === OG-EXP-4: OpenGENI Multi-site Acceptance Test === 31 31 32 32 This test was deemed "Blocked" due to the lack of connectivity between racks. … … 34 34 - ticket 100 There is no connectivity between racks 35 35 36 === GR-EXP-5: GRAMNetwork Resources Acceptance Test ===36 === OG-EXP-5: OpenGENI Network Resources Acceptance Test === 37 37 38 38 This test was deemed "Blocked" due to the lack of connectivity between racks. … … 40 40 - ticket 100 There is no connectivity between racks 41 41 42 === GR-EXP-6: GRAMand Meso-scale Multi-site !OpenFlow Acceptance Test ===42 === OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test === 43 43 44 44 This test was deemed "Blocked" due to the lack of connectivity between racks. … … 46 46 - ticket 100 There is no connectivity between racks 47 47 48 === GR-EXP-7: Click Router Experiment Acceptance Test ===48 === OG-EXP-7: Click Router Experiment Acceptance Test === 49 49 50 50 This test was deemed "Blocked" due to the lack of connectivity between racks. … … 54 54 == Administration Acceptance Tests == 55 55 56 === GR-ADM-1: Rack Receipt and Inventory Test ===56 === OG-ADM-1: Rack Receipt and Inventory Test === 57 57 58 58 This test was deemed "Completed" and needing further testing because there was no rack delivery process and there is no GMOC monitoring: 59 59 60 - ticket 49 There is no GMOC data collection or GRAMmonitoring data reporting60 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting 61 61 62 === GR-ADM-2: Rack Administrator Access Test ===62 === OG-ADM-2: Rack Administrator Access Test === 63 63 64 64 This test was deemed "Pass: most criteria" because the site Administrator account instructions are incomplete and because remote access not fully validated. … … 68 68 69 69 70 === GR-ADM-3: Full Rack Reboot Test ===70 === OG-ADM-3: Full Rack Reboot Test === 71 71 72 72 This test was deemed "In progress" because it was expected to have major impact on resource recovery. Delayed until January 2014. Will verify ticket #83 when executed. 73 73 74 74 75 === GR-ADM-4: Emergency Stop Test ===75 === OG-ADM-4: Emergency Stop Test === 76 76 77 77 This test was deemed "In Progress because it was expected to have major impact on resource recovery. Delayed until January 2014. Will verify ticket #84 when executed. 78 78 79 79 80 === GR-ADM-5: Software Update Test ===80 === OG-ADM-5: Software Update Test === 81 81 82 This test was deemed "Complete" and needs to be re-executed due to the expected changes to the GRAMpackage which will introduce support for update operation, currently only debian package for initial install.82 This test was deemed "Complete" and needs to be re-executed due to the expected changes to the OpenGENI package which will introduce support for update operation, currently only debian package for initial install. 83 83 84 - ticket 99 No GRAMpackage update; First install GRAM package not tested84 - ticket 99 No OpenGENI package update; First install GRAM package not tested 85 85 86 === GR-ADM-6: Control Network Disconnection Test ===86 === OG-ADM-6: Control Network Disconnection Test === 87 87 88 88 This test was deemed "In Progress because it was expected to have major impact on resource recovery and because there is no GMOC monitoring: 89 89 90 - ticket 49 There is no GMOC data collection or GRAMmonitoring data reporting90 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting 91 91 92 === GR-ADM-7: Documentation Review Test ===92 === OG-ADM-7: Documentation Review Test === 93 93 94 94 Significant improvement in documentation, but this test is deemed "In Progress". This documentation test case checks for the existence of 14 separate documents. Each are listed below with the overall status and associated ticket. 95 95 96 96 97 1. GRAMRack Installation Documentation - Pass: most criteria - ticket 7498 2. GRAMRack Part List Documentation - Pass: most criteria - ticket 7599 3. GRAMSoftware Components Identification Documentation - Pass100 4. GRAMand OS Upgrade Plans - In Progress - ticket 77101 5. GRAMSoftware Components Functional Description and Access to Source code - Pass102 6. GRAMRack Resources Usage and Policy Documentation - Fail - ticket 79103 7. GRAMRack Resources Administration - Pass: most criteria - ticket 80104 8. GRAMRack Administrative procedure for user accounts - Pass: most criteria - ticket 81105 9. GRAMRack Administrative procedure for rack operations - Fail - ticket 82106 10. GRAMRack Administrative procedure for rack scheduled shutdown - In Progress - ticket 83107 11. GRAMRack Administrative procedure for Emergency Stop - In Progress - ticket 84108 12. GRAMRack Administrative procedure for health status - Fail - ticket 85109 13. GRAMRack Site Operations documentation - Fail - ticket 86110 14. Site Specific GRAMDeployment Data - Fail - ticket 8797 1. OpenGENI Rack Installation Documentation - Pass: most criteria - ticket 74 98 2. OpenGENI Rack Part List Documentation - Pass: most criteria - ticket 75 99 3. OpenGENI Software Components Identification Documentation - Pass 100 4. OpenGENI and OS Upgrade Plans - In Progress - ticket 77 101 5. OpenGENI Software Components Functional Description and Access to Source code - Pass 102 6. OpenGENI Rack Resources Usage and Policy Documentation - Fail - ticket 79 103 7. OpenGENI Rack Resources Administration - Pass: most criteria - ticket 80 104 8. OpenGENI Rack Administrative procedure for user accounts - Pass: most criteria - ticket 81 105 9. OpenGENI Rack Administrative procedure for rack operations - Fail - ticket 82 106 10. OpenGENI Rack Administrative procedure for rack scheduled shutdown - In Progress - ticket 83 107 11. OpenGENI Rack Administrative procedure for Emergency Stop - In Progress - ticket 84 108 12. OpenGENI Rack Administrative procedure for health status - Fail - ticket 85 109 13. OpenGENI Rack Site Operations documentation - Fail - ticket 86 110 14. Site Specific OpenGENI Deployment Data - Fail - ticket 87 111 111 112 112 == Monitoring Acceptance Tests == 113 113 114 === GR-MON-1: Control Network Software and VLAN Inspection Test ===114 === OG-MON-1: Control Network Software and VLAN Inspection Test === 115 115 116 This test was deemed "Pass: met most criteria". All steps were successful, except for GRAMgit access is BBN internal only.116 This test was deemed "Pass: met most criteria". All steps were successful, except for OpenGENI git access is BBN internal only. 117 117 118 - ticket 66 GRAMgit access is BBN internal only, external access will be needed after initial evaluation118 - ticket 66 OpenGENI git access is BBN internal only, external access will be needed after initial evaluation 119 119 120 === GR-MON-2: GENI Software Configuration Inspection Test ===120 === OG-MON-2: GENI Software Configuration Inspection Test === 121 121 122 122 This test was deemed "Pass". No issue were found. 123 123 124 === GR-MON-3: GENI Active Experiment Inspection Test ===124 === OG-MON-3: GENI Active Experiment Inspection Test === 125 125 126 126 This test was deemed "Pass". No issue were found. 127 127 128 === GR-MON-4: Infrastructure Device Performance Test ===128 === OG-MON-4: Infrastructure Device Performance Test === 129 129 130 130 This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: 131 131 132 - ticket 49 There is no GMOC data collection or GRAMmonitoring data reporting132 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting 133 133 134 === GR-MON-5: GMOC Data Collection Test ===134 === OG-MON-5: GMOC Data Collection Test === 135 135 136 136 This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: 137 137 138 - ticket 49 There is no GMOC data collection or GRAMmonitoring data reporting138 - ticket 49 There is no GMOC data collection or OpenGENI monitoring data reporting