Changes between Initial Version and Version 1 of GENIRacksHome/ExogeniRacks/AcceptanceTestStatus/EG-ADM-4

02/26/13 17:50:07 (9 years ago)



  • GENIRacksHome/ExogeniRacks/AcceptanceTestStatus/EG-ADM-4

    v1 v1  
     3= Detailed test plan for EG-ADM-4: Emergency Stop Test =
     5''This page is GPO's working page for performing EG-ADM-4.  It is public for informational purposes, but it is not an official status report.  See [wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus] for the current status of ExoGENI acceptance tests.''
     7''Last substantive edit of this page: 2013-02-26''
     9= Page format =
     11 * The status chart summarizes the state of this test
     12 * The high-level description from test plan contains text copied exactly from the public test plan and acceptance criteria pages.
     13 * The steps contain things I will actually do/verify:
     14   * Steps may be composed of related substeps where I find this useful for clarity
     15   * Each step is identified as either "(prep)" or "(verify)":
     16     * Prep steps are just things we have to do.  They're not tests of the rack, but are prerequisites for subsequent verification steps
     17     * Verify steps are steps in which we will actually look at rack output and make sure it is as expected.  They contain a '''Using:''' block, which lists the steps to run the verification, and an '''Expect:''' block which lists what outcome is expected for the test to pass.
     19= Status of test =
     21See [wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus#Legend] for the meanings of test states.
     23|| '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments''' ||
     24|| 1          || [[Color(#63B8FF,In Progress)]] ||                      ||                    || tupty has reviewed GMOC doc, working on GPO doc  ||
     25|| 2          || [[Color(#63B8FF,In Progress)]] ||                      ||                    || tupty has proposed a schedule  ||
     26|| 3          ||                                ||                      ||                    ||   ||
     27|| 4          ||                                ||                      ||                    ||   ||
     28|| 5A         ||                                ||                      ||                    ||   ||
     29|| 5B         ||                                ||                      ||                    ||   ||
     30|| 5C         ||                                ||                      ||                    ||   ||
     31|| 6          ||                                ||                      ||                    ||   ||
     33= High-level description from test plan =
     35In this test, an ES (Emergency Stop) drill is performed on a sliver in the rack.
     37== Procedure ==
     39 * A site administrator reviews the local site ES procedure, GMOC ES procedure, and sliver shut down procedure, and verifies that these documents combined fully document the campus side of the ES procedure.
     40 * A second administrator (or the GPO) submits an ES request to GMOC, referencing activity from a public IP address assigned to a compute sliver in the rack that is part of the test experiment.
     41 * GMOC and the first site administrator perform an ES drill in which the site administrator successfully shuts down the sliver in coordination with GMOC.
     42 * GMOC completes the ES workflow, including updating/closing GMOC tickets.
     44== Criteria to verify as part of this test ==
     46 * VI.07. A public document explains the requirements that site administrators have to the GENI community, including how to join required mailing lists, how to keep their support contact information up-to-date, how and under what circumstances to work with Legal, Law Enforcement and Regulatory(LLR) Plan, how to best contact the rack vendor with operational problems, what information needs to be provided to GMOC to support emergency stop, and how to interact with GMOC when an Emergency Stop request is received. (F.3, C.3.d)
     47 * VI.17. A procedure is documented for performing a shut down operation on any type of sliver on the rack, in support of an Emergency Stop request. (C.3.d)
     48 * VII.18. Given a public IP address and port, an exclusive VLAN, a sliver name, or a piece of user-identifying information such as e-mail address or username, a site administrator or GMOC operator can identify the email address, username, and affiliation of the experimenter who controlled that resource at a particular time. (D.7)
     49 * VII.19. GMOC and a site administrator can perform a successful Emergency Stop drill in which slivers containing compute and OpenFlow-controlled network resources are shut down. (C.3.d)
     51= Step 1 (prep): Site administrator reviews local site ES procedure, GMOC ES procedure, and ExoGENI sliver shut down procedure =
     53The site administrator should review the local site ES procedure, the ES procedure provided by the GMOC, and the ExoGENI sliver shut down procedure.  All of these procedures should make sense together, and the site administrator should follow the local site ES procedure for the test.  The site administrator should identify parts of the local procedure where they need to take action on the aggregate, and they should reference the ExoGENI sliver shut down procedure for that part of the test.  He or she should also identify where the local site procedure requires interfacing with the GMOC.  The parts identified by the site administrator should be verified with the GMOC and with the ExoGENI team.
     55= Step 2 (prep): GPO, GMOC, and ExoGENI team coordinate a time to run an ES test =
     57The GPO will coordinate with parties at the GMOC and on the ExoGENI team to identify when an ES test can be run.  This test will focus primarily on the interactions with the site administator(s) and performing the procedures documented by the rack team.  The following roles will need to be defined for this test:
     58 * '''GMOC Coordinator''': person from the GMOC who coordinates the ES activity on the GMOC's side
     59 * '''ExoGENI Contact''': person from the ExoGENI team who can be around if there are questions about the document or sliver shut down procedure
     60 * '''ES Initiator''': GPO person who initiates an Emergency Stop request
     61 * '''Experimenter''': GPO person who has created a sliver
     62 * '''Site Administrator''': GPO person who is acting as the site administrator of the GPO ExoGENI rack
     64= Step 3 (prep): Experimenter sets up a slice =
     66The experimenter will set up a slice that includes a sliver on the GPO ExoGENI rack.  The sliver should be a VM that is attached to the shared mesoscale VLAN, and it should be sending traffic that is visible through monitoring.
     68= Step 4 (prep): ES initiated =
     70The ES Initiator contacts the GMOC Coordinator to initiate an ES request describing the slice name or URN.  The GMOC walks quickly walks through their procedure, skipping more formal steps as needed, in order to contact the aggregate operator primary contact.
     72= Step 5: Site Administrator receives ES request =
     74== Step 5A (verify): Data passed from GMOC to Site Administrator is in expected format ==
     77 * Local site ES procedure
     78 * Documented ExoGENI sliver shut down procedure
     79 * GMOC monitoring tools
     82 * The GMOC sends a request with slice-specific or sliver-specific data in a format that can be fed into the shut down procedure
     83 * There is a step in the local site ES procedure for the Site Administrator to acknowledge that the GMOC's request is being processed
     84 * The Site Administrator can identify the experimenter's email address, username, and affiliation with the information provided by the GMOC and GMOC monitoring tools
     86== Step 5B (verify): Shut down procedure can be followed to successfully shut down a sliver ==
     89 * Documented ExoGENI sliver shut down procedure
     90 * Administrative tools to shut down a sliver
     91 * GMOC monitoring tools
     94 * The shut down procedure includes the complete set of steps shut down a sliver in the rack
     95 * Following the shut down procedure results in a sliver being deactivated on a rack
     96 * Experimental traffic from the sliver is no longer being sent
     98== Step 5C (verify): Documented procedure includes a step to follow up with GMOC ==
     101 * Local site ES procedure
     104 * There is a step for the site administrator to follow up with the GMOC that a sliver has been shut down
     106= Step 6 (verify): Sliver shut down procedure includes a clean-up step (if necessary) =
     109 * Documented ExoGENI sliver shut down procedure
     112 * Ensure the ExoGENI sliver shut down procedure contains a recovery step describing what to do if the shut down affects other experimenters.