| | 1 | = GRAM Intermediate Acceptance Test Report - April 2014 = |
| | 2 | |
| | 3 | This pages captures the findings for each of the test cases in the [wiki:GENIRacksHome/AcceptanceTests/GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan]. Status for the current evaluation can be found in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014 GRAM Acceptance Test Status - April 2014] page. As can be seen from the page there has been significant improvement since the last evaluation results captured in the [wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportDec2013 Acceptance Test Report - December 2013]. Also many tickets reported in earlier evaluation have been successfully addressed. Following is a list of the most critical issues as of 2014-05-19: |
| | 4 | |
| | 5 | - ticket 123 - GRAM rack Dell Switches require that ETH_Typer is wild carded on flow mods |
| | 6 | - ticket 122 - Poor performance to remote IG with !OpenFlow over stitched connection |
| | 7 | - ticket 121 - Need !OpenFlow Campus Resources connected to Clemson rack |
| | 8 | - ticket 119 - Longer project and slice names causes GRAM internal error for !OpenStack |
| | 9 | - ticket 110 - No bandwidth shaping available on GRAM nodes |
| | 10 | - ticket 107 - mismatch in VLAN allocation VS reported in sliver manifest |
| | 11 | - ticket 103 - Tools listinst.sh and vmlist.sh return no information when there are active VMs |
| | 12 | - ticket 94 - Deleting sliver that does not exist at the GRAM aggregate does not return error |
| | 13 | - ticket 93 - Five VM sliver manifest has unexpected format and does not pass rspec lint. |
| | 14 | - ticket 92 - Createsliver does not return after accepting requests, seems to return when request completed. |
| | 15 | |
| | 16 | |
| | 17 | == Experimenter Acceptance Tests == |
| | 18 | |
| | 19 | === GR-EXP-1: Bare Metal Support Acceptance Test === |
| | 20 | |
| | 21 | This test case was deemed "Fail" due to the fact that there is still no supported for bare metal nodes and there is no support for windows. Modified the test case to replace all dedicated nodes (aka bare metal) with VMs to get test coverage.Issues captured in tickets: |
| | 22 | |
| | 23 | - ticket 51 - Bare metal support feature is missing for GR-EXP-1 |
| | 24 | - ticket 70 - MS Windows support requires handling of account setup in GRAM |
| | 25 | |
| | 26 | === GR-EXP-2: GRAM Single Site Acceptance Test === |
| | 27 | |
| | 28 | This test was deemed "Pass". No issues were found. |
| | 29 | |
| | 30 | === GR-EXP-3: GRAM Single Site 100 VM Test === |
| | 31 | |
| | 32 | This test was deemed "Pass: most criteria". No problems were found, but only two scenario were tested due to lack of time. |
| | 33 | |
| | 34 | === GR-EXP-4: GRAM Multi-site Acceptance Test === |
| | 35 | |
| | 36 | - ticket 110 - No bandwidth shaping available on GRAM nodes |
| | 37 | |
| | 38 | === GR-EXP-5: GRAM Network Resources Acceptance Test === |
| | 39 | - ticket 121 - Need !OpenFlow Campus Resources connected to Clemson rack |
| | 40 | |
| | 41 | === GR-EXP-6: GRAM and Meso-scale Multi-site !OpenFlow Acceptance Test === |
| | 42 | |
| | 43 | |
| | 44 | === GR-EXP-7: Click Router Experiment Acceptance Test === |
| | 45 | |
| | 46 | This layer two test was not attempted during this test effort. |
| | 47 | |
| | 48 | == Administration Acceptance Tests == |
| | 49 | |
| | 50 | === GR-ADM-1: Rack Receipt and Inventory Test === |
| | 51 | |
| | 52 | This test was deemed "Completed" and needing further testing because there was no rack delivery process and there is no GMOC monitoring: |
| | 53 | |
| | 54 | - ticket 49 - There is no GMOC data collection or GRAM monitoring data reporting |
| | 55 | |
| | 56 | === GR-ADM-2: Rack Administrator Access Test === |
| | 57 | |
| | 58 | This test was deemed "Pass: most criteria" because the site Administrator account instructions are incomplete and because remote access not fully validated. |
| | 59 | |
| | 60 | - ticket 65 - Remote console not available via web interface, some iDRAC access via SSH. |
| | 61 | |
| | 62 | |
| | 63 | === GR-ADM-3: Full Rack Reboot Test === |
| | 64 | |
| | 65 | This test was run successfully and using existing documentation was able to complete a full reboot and have the aggregate running and available upon test completion. |
| | 66 | |
| | 67 | |
| | 68 | === GR-ADM-4: Emergency Stop Test === |
| | 69 | |
| | 70 | This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the GRAM Racks at Clemson. |
| | 71 | |
| | 72 | |
| | 73 | === GR-ADM-5: Software Update Test === |
| | 74 | |
| | 75 | This test was run without any problem, GRAM software packages was installed as part of an upgrade. |
| | 76 | |
| | 77 | === GR-ADM-6: Control Network Disconnection Test === |
| | 78 | |
| | 79 | This test was not run because it requires GMOC interactions and monitoring, neither of which are available at this time for the GRAM Racks at Clemson. |
| | 80 | |
| | 81 | === GR-ADM-7: Documentation Review Test === |
| | 82 | |
| | 83 | Significant improvement in documentation. This documentation test case checks for the existence of 14 separate documents. Each are listed below with the overall status and associated ticket. |
| | 84 | |
| | 85 | 1. GRAM Rack Installation Documentation - Pass: most criteria - ticket 74 |
| | 86 | 2. GRAM Rack Part List Documentation - Pass: most criteria - ticket 75 |
| | 87 | 3. GRAM Software Components Identification Documentation - Pass |
| | 88 | 4. GRAM and OS Upgrade Plans - Pass |
| | 89 | 5. GRAM Software Components Functional Description and Access to Source code - Pass |
| | 90 | 6. GRAM Rack Resources Usage and Policy Documentation - In Progress - ticket 79 |
| | 91 | 7. GRAM Rack Resources Administration - Pass: most criteria - ticket 80 |
| | 92 | 8. GRAM Rack Administrative procedure for user accounts - Pass |
| | 93 | 9. GRAM Rack Administrative procedure for rack operations - Pass |
| | 94 | 10. GRAM Rack Administrative procedure for rack scheduled shutdown - Pass |
| | 95 | 11. GRAM Rack Administrative procedure for Emergency Stop - In Progress - ticket 84 |
| | 96 | 12. GRAM Rack Administrative procedure for health status - Fail - ticket 85 |
| | 97 | 13. GRAM Rack Site Operations documentation - Fail - ticket 86 |
| | 98 | 14. Site Specific GRAM Deployment Data - Pass |
| | 99 | |
| | 100 | == Monitoring Acceptance Tests == |
| | 101 | |
| | 102 | === GR-MON-1: Control Network Software and VLAN Inspection Test === |
| | 103 | |
| | 104 | This test was deemed "Pass: met most criteria". All steps were successful, except for GRAM git access is BBN internal only. |
| | 105 | |
| | 106 | - ticket 66 - GRAM git access is BBN internal only, external access will be needed after initial evaluation |
| | 107 | |
| | 108 | === GR-MON-2: GENI Software Configuration Inspection Test === |
| | 109 | |
| | 110 | This test was deemed "Pass". No issue were found. |
| | 111 | |
| | 112 | === GR-MON-3: GENI Active Experiment Inspection Test === |
| | 113 | |
| | 114 | This test was deemed "Pass". No issue were found. |
| | 115 | |
| | 116 | === GR-MON-4: Infrastructure Device Performance Test === |
| | 117 | |
| | 118 | This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: |
| | 119 | |
| | 120 | - ticket 49 - There is no GMOC data collection or GRAM monitoring data reporting |
| | 121 | |
| | 122 | === GR-MON-5: GMOC Data Collection Test === |
| | 123 | |
| | 124 | This test was deemed "Blocked" because it cannot be executed without monitoring data collection at the GMOC. Ticket tracking missing GMOC support: |
| | 125 | |
| | 126 | - ticket 49 - There is no GMOC data collection or GRAM monitoring data reporting |
