Changes between Version 1 and Version 2 of GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014Final
- Timestamp:
- 05/23/14 12:24:46 (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014Final
v1 v2 1 = GRAM Intermediate Acceptance Test Report - April 2014 =1 = OpenGENI AM Intermediate Acceptance Test Report - April 2014 = 2 2 3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/ GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan]. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. As can be seen from the page there has been significant improvement since the last evaluation results captured in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Also many tickets reported in earlier evaluation have been successfully addressed. Following is a list of the most critical issues as of 2014-05-19:3 This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan GRAM Acceptance Test Plan]. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. As can be seen from the page there has been significant improvement since the last evaluation results captured in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Also many tickets reported in earlier evaluation have been successfully addressed. Following is a list of the most critical issues as of 2014-05-19: 4 4 5 - ticket 123 - GRAMrack Dell Switches require that ETH_Typer is wild carded on flow mods6 - ticket 122 - Poor performance to remote IG with !OpenFlow over stitched connection 7 - ticket 121 - Need !OpenFlow Campus Resources connected to Clemson rack 8 - ticket 119 - Longer project and slice names causes GRAMinternal error for !OpenStack9 - ticket 110 - No bandwidth shaping available on GRAM nodes10 - ticket 107 - mismatch in VLAN allocation VS reported in sliver manifest11 - ticket 103 - Tools listinst.sh and vmlist.sh return no information when there are active VMs12 - ticket 94 - Deleting sliver that does not exist at the GRAM aggregate does not return error13 - ticket 93 - Five VM sliver manifest has unexpected format and does not pass rspec lint. 14 - ticket 92 - Createsliver does not return after accepting requests, seems to return when request completed. 5 - ticket 123 - OpenGENI rack Dell Switches require that ETH_Typer is wild carded on flow mods 6 - ticket 122 - Poor performance to remote IG with !OpenFlow over stitched connection 7 - ticket 121 - Need !OpenFlow Campus Resources connected to Clemson rack 8 - ticket 119 - Longer project and slice names causes OpenGENI internal error for !OpenStack 9 - ticket 110 - No bandwidth shaping available on OpenGENI nodes 10 - ticket 107 - mismatch in VLAN allocation VS reported in sliver manifest 11 - ticket 103 - Tools listinst.sh and vmlist.sh return no information when there are active VMs 12 - ticket 94 - Deleting sliver that does not exist at the OpenGENI aggregate does not return error 13 - ticket 93 - Five VM sliver manifest has unexpected format and does not pass rspec lint. 14 - ticket 92 - Createsliver does not return after accepting requests, seems to return when request completed. 15 15 16 16 17 17 == Experimenter Acceptance Tests == 18 18 19 === GR-EXP-1: Bare Metal Support Acceptance Test ===19 === OG-EXP-1: Bare Metal Support Acceptance Test === 20 20 21 21 This test case was deemed "Fail" due to the fact that there is still no supported for bare metal nodes and there is no support for windows. Modified the test case to replace all dedicated nodes (aka bare metal) with VMs to get test coverage.Issues captured in tickets: 22 22 23 - ticket 51 - Bare metal support feature is missing for GR-EXP-124 - ticket 70 - MS Windows support requires handling of account setup in GRAM23 - ticket 51 - Bare metal support feature is missing for OG-EXP-1 24 - ticket 70 - MS Windows support requires handling of account setup in OpenGENI 25 25 26 === GR-EXP-2: GRAMSingle Site Acceptance Test ===26 === OG-EXP-2: OpenGENI Single Site Acceptance Test === 27 27 28 28 This test was deemed "Pass". No issues were found. 29 29 30 === GR-EXP-3: GRAMSingle Site 100 VM Test ===30 === OG-EXP-3: OpenGENI Single Site 100 VM Test === 31 31 32 32 This test was deemed "Pass: most criteria". No problems were found, but only two scenario were tested due to lack of time. 33 33 34 === GR-EXP-4: GRAMMulti-site Acceptance Test ===34 === OG-EXP-4: OpenGENI Multi-site Acceptance Test === 35 35 36 - ticket 110 - No bandwidth shaping available on GRAMnodes36 - ticket 110 - No bandwidth shaping available on OpenGENI nodes 37 37 38 === GR-EXP-5: GRAMNetwork Resources Acceptance Test ===38 === OG-EXP-5: OpenGENI Network Resources Acceptance Test === 39 39 - ticket 121 - Need !OpenFlow Campus Resources connected to Clemson rack 40 40 41 === GR-EXP-6: GRAMand Meso-scale Multi-site !OpenFlow Acceptance Test ===41 === OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test === 42 42 43 43 44 === GR-EXP-7: Click Router Experiment Acceptance Test ===44 === OG-EXP-7: Click Router Experiment Acceptance Test === 45 45 46 46 This layer two test was not attempted during this test effort. … … 48 48 == Administration Acceptance Tests == 49 49 50 === GR-ADM-1: Rack Receipt and Inventory Test ===50 === OG-ADM-1: Rack Receipt and Inventory Test === 51 51 52 52 This test was deemed "Completed" and needing further testing because there was no rack delivery process and there is no GMOC monitoring: 53 53 54 - ticket 49 - There is no GMOC data collection or GRAMmonitoring data reporting54 - ticket 49 - There is no GMOC data collection or OpenGENI monitoring data reporting 55 55 56 === GR-ADM-2: Rack Administrator Access Test ===56 === OG-ADM-2: Rack Administrator Access Test === 57 57 58 58 This test was deemed "Pass: most criteria" because the site Administrator account instructions are incomplete and because remote access not fully validated. … … 61 61 62 62 63 === GR-ADM-3: Full Rack Reboot Test ===63 === OG-ADM-3: Full Rack Reboot Test === 64 64 65 65 This test was run successfully and using existing documentation was able to complete a full reboot and have the aggregate running and available upon test completion. 66 66 67 67 68 === GR-ADM-4: Emergency Stop Test ===68 === OG-ADM-4: Emergency Stop Test === 69 69 70 This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the GRAMRacks at Clemson.70 This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the OpenGENI Racks at Clemson. 71 71 72 72 73 === GR-ADM-5: Software Update Test ===73 === OG-ADM-5: Software Update Test === 74 74 75 This test was run without any problem, GRAMsoftware packages was installed as part of an upgrade.75 This test was run without any problem, OpenGENI software packages was installed as part of an upgrade. 76 76 77 === GR-ADM-6: Control Network Disconnection Test ===77 === OG-ADM-6: Control Network Disconnection Test === 78 78 79 This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the GRAMRacks at Clemson.79 This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the OpenGENI Racks at Clemson. 80 80 81 === GR-ADM-7: Documentation Review Test ===81 === OG-ADM-7: Documentation Review Test === 82 82 83 83 Significant improvement in documentation. This documentation test case checks for the existence of 14 separate documents. Each are listed below with the overall status and associated ticket. 84 84 85 1. GRAMRack Installation Documentation - Pass: most criteria - ticket 7486 2. GRAMRack Part List Documentation - Pass: most criteria - ticket 7587 3. GRAMSoftware Components Identification Documentation - Pass88 4. GRAMand OS Upgrade Plans - Pass89 5. GRAMSoftware Components Functional Description and Access to Source code - Pass90 6. GRAMRack Resources Usage and Policy Documentation - In Progress - ticket 7991 7. GRAMRack Resources Administration - Pass: most criteria - ticket 8092 8. GRAMRack Administrative procedure for user accounts - Pass93 9. GRAMRack Administrative procedure for rack operations - Pass94 10. GRAMRack Administrative procedure for rack scheduled shutdown - Pass95 11. GRAMRack Administrative procedure for Emergency Stop - In Progress - ticket 8496 12. GRAMRack Administrative procedure for health status - Fail - ticket 8597 13. GRAMRack Site Operations documentation - Fail - ticket 8698 14. Site Specific GRAMDeployment Data - Pass85 1. OpenGENI Rack Installation Documentation - Pass: most criteria - ticket 74 86 2. OpenGENI Rack Part List Documentation - Pass: most criteria - ticket 75 87 3. OpenGENI Software Components Identification Documentation - Pass 88 4. OpenGENI and OS Upgrade Plans - Pass 89 5. OpenGENI Software Components Functional Description and Access to Source code - Pass 90 6. OpenGENI Rack Resources Usage and Policy Documentation - In Progress - ticket 79 91 7. OpenGENI Rack Resources Administration - Pass: most criteria - ticket 80 92 8. OpenGENI Rack Administrative procedure for user accounts - Pass 93 9. OpenGENI Rack Administrative procedure for rack operations - Pass 94 10. OpenGENI Rack Administrative procedure for rack scheduled shutdown - Pass 95 11. OpenGENI Rack Administrative procedure for Emergency Stop - In Progress - ticket 84 96 12. OpenGENI Rack Administrative procedure for health status - Fail - ticket 85 97 13. OpenGENI Rack Site Operations documentation - Fail - ticket 86 98 14. Site Specific OpenGENI Deployment Data - Pass 99 99 100 100 == Monitoring Acceptance Tests == 101 101 102 === GR-MON-1: Control Network Software and VLAN Inspection Test ===102 === OG-MON-1: Control Network Software and VLAN Inspection Test === 103 103 104 This test was deemed "Pass: met most criteria". All steps were successful, except for GRAMgit access is BBN internal only.104 This test was deemed "Pass: met most criteria". All steps were successful, except for OpenGENI git access is BBN internal only. 105 105 106 - ticket 66 - GRAMgit access is BBN internal only, external access will be needed after initial evaluation106 - ticket 66 - OpenGENI git access is BBN internal only, external access will be needed after initial evaluation 107 107 108 === GR-MON-2: GENI Software Configuration Inspection Test ===108 === OG-MON-2: GENI Software Configuration Inspection Test === 109 109 110 110 This test was deemed "Pass". No issue were found. 111 111 112 === GR-MON-3: GENI Active Experiment Inspection Test ===112 === OG-MON-3: GENI Active Experiment Inspection Test === 113 113 114 114 This test was deemed "Pass". No issue were found. 115 115 116 === GR-MON-4: Infrastructure Device Performance Test ===116 === OG-MON-4: Infrastructure Device Performance Test === 117 117 118 118 This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: 119 119 120 - ticket 49 - There is no GMOC data collection or GRAMmonitoring data reporting120 - ticket 49 - There is no GMOC data collection or OpenGENI monitoring data reporting 121 121 122 === GR-MON-5: GMOC Data Collection Test ===122 === OG-MON-5: GMOC Data Collection Test === 123 123 124 124 This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: 125 125 126 - ticket 49 - There is no GMOC data collection or GRAMmonitoring data reporting126 - ticket 49 - There is no GMOC data collection or OpenGENI monitoring data reporting