Changes between Version 1 and Version 2 of GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan


Ignore:
Timestamp:
05/23/14 14:25:32 (10 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan

    v1 v2  
    33= GENI Rack Aggregate Manager Acceptance Test Plan =
    44
    5 This page captures the GENI Racks Acceptance Test Plan to be executed for the BBN GENI Rack Aggregate Manager (GRAM). This test plan is based on the [http://groups.geni.net/geni/wiki/GeniRacks GENI Racks Requirements] and it outlines all features that are normally validated for GENI Racks. The goal of this effort is to capture the state of the current features and to generate a list of missing feature that are required to meet all GENI Racks requirements.
    6 
    7 The BBN GRAM Acceptance Test effort will generate the following:
    8  - a [wiki:GENIRacksHome/AcceptanceTests/GRAMAcceptanceTestsPlan GRAM Acceptance Test Plan], this document written on 05/16/2013.
    9  - tickets to track  GRAM issues discovered in testing  - are internal only
    10  - an '''[wiki:GENIRacksHome/GRAMRacks/AcceptanceTestStatusApr2014  Acceptance Test Status - April 2014]''' page where test status and logs are found.
    11  - an intermediate '''[wiki:GENIRacksHome/GRAMRacks/AcceptanceTestReportApr2014 Acceptance Test Report - April 2014]''' page to summarize state.
     5This page captures the GENI Racks Acceptance Test Plan to be executed for the BBN GENI Rack Aggregate Manager (OpenGENI). This test plan is based on the [http://groups.geni.net/geni/wiki/GeniRacks GENI Racks Requirements] and it outlines all features that are normally validated for GENI Racks. The goal of this effort is to capture the state of the current features and to generate a list of missing feature that are required to meet all GENI Racks requirements.
     6
     7The BBN OpenGENI Acceptance Test effort will generate the following:
     8 - a [wiki:GENIRacksHome/AcceptanceTests/OpenGENIAcceptanceTestsPlan OpenGENI Acceptance Test Plan], this document written on 05/16/2013.
     9 - tickets to track  OpenGENI issues discovered in testing  - are internal only
     10 - an '''[wiki:GENIRacksHome/OpenGENIRacks/AcceptanceTestStatusApr2014  Acceptance Test Status - April 2014]''' page where test status and logs are found.
     11 - an intermediate '''[wiki:GENIRacksHome/OpenGENIRacks/AcceptanceTestReportApr2014 Acceptance Test Report - April 2014]''' page to summarize state.
    1212
    1313== Assumptions and Dependencies ==
    … …  
    1515The following assumptions are made for all tests described in this plan:
    1616
    17  * GRAM Clearing house credentials will be used for all tests.
    18  * GRAM is the slice authority for all tests in this plan.
     17 * OpenGENI Clearing house credentials will be used for all tests.
     18 * OpenGENI is the slice authority for all tests in this plan.
    1919 * No tests using the GENI Clearinghouse and GPO ProtoGENI credentials are planned for the initial tests.
    20  * Resources for each test will be requested from the GRAM Aggregate Manager.
     20 * Resources for each test will be requested from the OpenGENI Aggregate Manager.
    2121 * Compute resources are VMs unless otherwise stated, there are no dedicated devices available for this initial evaluation.
    2222 * All Aggregate Manager requests are made via the Omni command line tool which uses the GENI AM API.
    2323 * In all scenarios, one experiment is always equal to one slice.
    24  * Currently there is only one GRAM rack, which has the following test implications. All scenarios that are meant to be run on 1 rack will be run within 1 VM.  All scenarios that are meant to have multiple racks, will use VMs on multiple servers.
    25  * GRAM will be used as the interface to the rack !OpenFlow resources in the !OpenFlow test cases.
    26 
    27 It is expected that the GRAM Aggregate Manager will provide an interface into the (VLAN-based Multiplexed !OpenFlow Controller) VMOC aggregate. If the GRAM interface to VMOC is not available, tests will be executed by submitting requests directly to VMOC.
    28 
    29 If the GRAM solution does not provide support for experimenters uploading custom VM images to the rack, any test case using custom images will modified to use available images for the rack. The ability to upload a custom VM image to the GRAM rack will be tested when it becomes available. 
     24 * Currently there is only one OpenGENI rack, which has the following test implications. All scenarios that are meant to be run on 1 rack will be run within 1 VM.  All scenarios that are meant to have multiple racks, will use VMs on multiple servers.
     25 * OpenGENI will be used as the interface to the rack !OpenFlow resources in the !OpenFlow test cases.
     26
     27It is expected that the OpenGENI Aggregate Manager will provide an interface into the (VLAN-based Multiplexed !OpenFlow Controller) VMOC aggregate. If the OpenGENI interface to VMOC is not available, tests will be executed by submitting requests directly to VMOC.
     28
     29If the OpenGENI solution does not provide support for experimenters uploading custom VM images to the rack, any test case using custom images will modified to use available images for the rack. The ability to upload a custom VM image to the OpenGENI rack will be tested when it becomes available. 
    3030
    3131 
    … …  
    4141This section describes each acceptance test by defining its goals, topology, and outline test procedure.  Test cases are listed by priority in sections below.  The cases that verify the largest number of requirement criteria are typically listed at a higher priority. The prerequisite tests are usually executed first to verify that baseline monitoring and administrative functions are available. This allows the execution of the experimenter test cases. Additional monitoring and administrative tests described in later sections that are also run before the completion of the acceptance test effort. 
    4242
    43 For the GRAM Acceptance Test evaluation some of these administrative and monitoring features may not be available but the tests are still planned in order to capture the availability of expected features.
     43For the OpenGENI Acceptance Test evaluation some of these administrative and monitoring features may not be available but the tests are still planned in order to capture the availability of expected features.
    4444
    4545== Administration Prerequisite Tests ==
    … …  
    4848
    4949
    50 === GR-ADM-1: Rack Receipt and Inventory Test ===
     50=== OG-ADM-1: Rack Receipt and Inventory Test ===
    5151
    5252This acceptance test uses BBN as an example site because it requires physical access to the rack.  The goal of this test is to verify that administrators can integrate the rack into a standard local procedures for systems hosted by the site.
    … …  
    5858 * Power and wire the BBN rack
    5959 * Administrator configures the gramm.gpolab.bbn.com DNS namespace and 192.1.242.128/25 IP space, and enters all public IP addresses used by the rack into DNS.
    60  * Administrator requests and receives administrator accounts on the rack and receive read access to all GRAM monitoring of the rack.
     60 * Administrator requests and receives administrator accounts on the rack and receive read access to all OpenGENI monitoring of the rack.
    6161 * Administrator inventories the physical rack contents, network connections and VLAN configuration, and power connectivity, using standard operational inventories.
    62  * Administrator, GRAM team, and GMOC share information about contact information and change control procedures, and GRAM operators subscribe to GENI operations mailing lists and submit their contact information to GMOC.
     62 * Administrator, OpenGENI team, and GMOC share information about contact information and change control procedures, and OpenGENI operators subscribe to GENI operations mailing lists and submit their contact information to GMOC.
    6363 * Administrator reviews the documented parts list, power requirements, physical and logical network connectivity requirements, and site administrator community requirements, verifying that these documents should be sufficient for a new site to use when setting up a rack.
    6464
    6565
    66 === GR-ADM-2: Rack Administrator Access Test ===
     66=== OG-ADM-2: Rack Administrator Access Test ===
    6767
    6868This test verifies local and remote administrative access to rack devices.
    … …  
    8181   * Verify that you cannot login to the device via an unencrypted login protocol.
    8282   * Use the "enable" command or equivalent to verify privileged access.
    83  3. Verify that GRAM remote console solution for rack hosts can be used to access the consoles all server hosts and experimental hosts:
     83 3. Verify that OpenGENI remote console solution for rack hosts can be used to access the consoles all server hosts and experimental hosts:
    8484   * Login via SSH or other encrypted protocol.
    8585   * Verify that you cannot login via an unencrypted login protocol.
    … …  
    9090These tests verify the availability of information needed to determine rack state, and needed to debug problems during experimental testing. Also verified is the ability to determine the rack components' test-readiness.  Additional monitoring tests are defined in a later section to complete the validation in this section.
    9191
    92 === GR-MON-1: Control Network Software and VLAN Inspection Test ===
     92=== OG-MON-1: Control Network Software and VLAN Inspection Test ===
    9393
    9494This test inspects the state of the rack control network, infrastructure nodes, and system software.
    … …  
    100100 * A site administrator reviews the MAC address table on the control plane switch, and verifies that all entries are identifiable and expected.
    101101
    102 === GR-MON-2: GENI Software Configuration Inspection Test ===
     102=== OG-MON-2: GENI Software Configuration Inspection Test ===
    103103
    104104This test inspects the state of the GENI AM software in use on the rack.
    … …  
    106106==== Procedure ====
    107107
    108  * A site administrator uses available system data sources (process listings, monitoring output, system logs, etc) and/or AM administrative interfaces to determine the configuration of GRAM resources:
     108 * A site administrator uses available system data sources (process listings, monitoring output, system logs, etc) and/or AM administrative interfaces to determine the configuration of OpenGENI resources:
    109109   * How many experimental nodes are available for bare metal use, how many are configured as !OpenStack containers, and how many are configured as !PlanetLab containers.
    110110   * What operating system each !OpenStack container makes available for experimental VMs.
    111111   * How many unbound VLANs are in the rack's available pool.
    112    * Whether the GRAM and !OpenFlow AMs trust the pgeni.gpolab.bbn.com slice authority, which will be used for testing. Note, that the pgeni.gpolab.bbn.com slice authority is not used in this test, a local slice authority is used for the initial evaluation.
    113  * A site administrator uses available system data sources to determine the configuration of !OpenFlow resources according to VMOC and GRAM.
    114 
    115 === GR-MON-3: GENI Active Experiment Inspection Test ===
     112   * Whether the OpenGENI and !OpenFlow AMs trust the pgeni.gpolab.bbn.com slice authority, which will be used for testing. Note, that the pgeni.gpolab.bbn.com slice authority is not used in this test, a local slice authority is used for the initial evaluation.
     113 * A site administrator uses available system data sources to determine the configuration of !OpenFlow resources according to VMOC and OpenGENI.
     114
     115=== OG-MON-3: GENI Active Experiment Inspection Test ===
    116116
    117117This test inspects the state of the rack data plane and control networks when experiments are running, and verifies that a site administrator can find information about running experiments.
    … …  
    135135== Experimenter Acceptance Tests ==
    136136
    137 For the GRAM Acceptance Test evaluation some of these topologies normally validated in GENI Racks are not possible. This effort has only one rack available.  Each test case is described as originally intended, but additionally there are details to show how the test case is modified for the initial GRAM evaluation.  Topologies not be available but the tests are still planned as intended in order to capture the availability of expected features.
    138 
    139 === GR-EXP-1: Bare Metal Support Acceptance Test ===
     137For the OpenGENI Acceptance Test evaluation some of these topologies normally validated in GENI Racks are not possible. This effort has only one rack available.  Each test case is described as originally intended, but additionally there are details to show how the test case is modified for the initial OpenGENI evaluation.  Topologies not be available but the tests are still planned as intended in order to capture the availability of expected features.
     138
     139=== OG-EXP-1: Bare Metal Support Acceptance Test ===
    140140
    141141Bare metal nodes are exclusive dedicated physical nodes that are used throughout the experimenter test cases.  This section outlines features to be verified which are not explicitly validated in other scenarios:
    … …  
    144144 2. Obtain 2 licensed recent Microsoft OS images for physical nodes from the site (BBN).
    145145 3. Reserve and boot 2 physical nodes using Microsoft image.
    146  4. Obtain a recent Linux OS image for physical nodes from the GRAM list.
     146 4. Obtain a recent Linux OS image for physical nodes from the OpenGENI list.
    147147 5. Reserve and boot a physical node using this Linux OS image.
    148148 6. Release physical node resource.
    … …  
    154154
    155155
    156 === GR-EXP-2: GRAM Single Site Acceptance Test ===
    157 
    158 This one site test is run on the BBN GRAM rack and it includes two experiments. Each experiment requests local compute resources, which generate bidirectional traffic over a Layer 2 data plane network connection. The goals of this test are to verify basic operations of VMs and data flows within one rack; verify the ability to request a publically routable IP address and public TCP/UDP port mapping for a control interface on a compute resource; and verify the ability to add a customized image for the rack.
     156=== OG-EXP-2: OpenGENI Single Site Acceptance Test ===
     157
     158This one site test is run on the BBN OpenGENI rack and it includes two experiments. Each experiment requests local compute resources, which generate bidirectional traffic over a Layer 2 data plane network connection. The goals of this test are to verify basic operations of VMs and data flows within one rack; verify the ability to request a publically routable IP address and public TCP/UDP port mapping for a control interface on a compute resource; and verify the ability to add a customized image for the rack.
    159159
    160160
    … …  
    163163This test uses this topology:
    164164
    165 [[Image(GRAMSingleSiteAcceptanceTest.jpg)]]
     165[[Image(OpenGENISingleSiteAcceptanceTest.jpg)]]
    166166
    167167''Note:''  The diagram shows the logical end-points for each experiment traffic exchange. The VMs may or may not be on different experiment nodes.
    … …  
    169169For the initial evaluation there are no bare metal nodes, so the test case is modified to have only VMs. Here is the actual topology run:
    170170
    171 [[Image(GRAMSingleSiteAcceptanceTest-actual.jpg)]]
     171[[Image(OpenGENISingleSiteAcceptanceTest-actual.jpg)]]
    172172
    173173''Evaluation Note:'' Test case is described for the original test case, actual procedure will be captures as part of test details available from the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatus Acceptance Test Status] page.
    … …  
    177177This test has these prerequisites:
    178178
    179  * GRAM makes available at least two Linux distributions and a FreeBSD image. If the not available, test will be run with available images.
    180  * Two GPO customized Ubuntu image snapshots are available and have been manually uploaded by the rack administrator using available GRAM documentation. One Ubuntu image is for the VM and one Ubuntu image is for the physical node in this test. Physical node are not available, so VM will be used.
     179 * OpenGENI makes available at least two Linux distributions and a FreeBSD image. If the not available, test will be run with available images.
     180 * Two GPO customized Ubuntu image snapshots are available and have been manually uploaded by the rack administrator using available OpenGENI documentation. One Ubuntu image is for the VM and one Ubuntu image is for the physical node in this test. Physical node are not available, so VM will be used.
    181181 * Traffic generation tools may be part of image or may be installed at experiment runtime.
    182  * Administrative accounts have been created for GPO staff on the BBN GRAM rack.
     182 * Administrative accounts have been created for GPO staff on the BBN OpenGENI rack.
    183183 * GENI Experimenter1 and Experimenter2 accounts exist at the GPO PG Clearinghouse.
    184184 * If available, use baseline Monitoring to ensure that any problems are quickly identified.
    … …  
    189189Do the following:
    190190
    191  1. As Experimenter1, request !ListResources from BBN GRAM.
     191 1. As Experimenter1, request !ListResources from BBN OpenGENI.
    192192 2. Review advertisement RSpec for a list of OS images which can be loaded, and identify available resources.
    193193 3. Verify that the GPO Ubuntu customized image is available in the advertisement RSpec.
    … …  
    197197 7. Log in to each of the systems, and send traffic to the other system sharing a VLAN.
    198198 8. Using root privileges on one of the VMs load a Kernel module. If not supported on !OpenStack nodes, testing will proceed past this step.
    199  9. Run a netcat listener and bind to port XYZ on each of the VMs in the BBN GRAM rack.
    200  10. Send traffic to port XYZ on each of the VMs in the GRAM rack over the control network from any commodity Internet host.
    201  11. As Experimenter2, request !ListResources from Site2 GRAM.
     199 9. Run a netcat listener and bind to port XYZ on each of the VMs in the BBN OpenGENI rack.
     200 10. Send traffic to port XYZ on each of the VMs in the OpenGENI rack over the control network from any commodity Internet host.
     201 11. As Experimenter2, request !ListResources from Site2 OpenGENI.
    202202 12. Define a request RSpec for two physical nodes, both using the uploaded GPO Ubuntu images. If not available, VMs and other images will be used.
    203203 13. Create the second slice.
    … …  
    210210 20. Stop traffic and delete slivers.
    211211
    212 === GR-EXP-3: GRAM Single Site 100 VM Test ===
    213 
    214 This one site test runs on the BBN GRAM rack and includes various scenarios to validate compute resource requirements for VMs. The goal of this test is not to validate the GRAM limits, but simply to verify that the GRAM rack can provide 100 VMs with its experiment nodes under various scenarios, including:
     212=== OG-EXP-3: OpenGENI Single Site 100 VM Test ===
     213
     214This one site test runs on the BBN OpenGENI rack and includes various scenarios to validate compute resource requirements for VMs. The goal of this test is not to validate the OpenGENI limits, but simply to verify that the OpenGENI rack can provide 100 VMs with its experiment nodes under various scenarios, including:
    215215
    216216 * Scenario 5: 100 Slices with 1 VM each
    … …  
    226226This test uses this topology:
    227227
    228 [[Image(GRAMSingleSiteLimitsTest.jpg)]]
     228[[Image(OpenGENISingleSiteLimitsTest.jpg)]]
    229229
    230230==== Prerequisites ====
    … …  
    232232
    233233 * Traffic generation tools may be part of image or installed at experiment runtime.
    234  * Administrative accounts exist for GPO staff on the BBN GRAM rack.
     234 * Administrative accounts exist for GPO staff on the BBN OpenGENI rack.
    235235 * GENI Experimenter1 account exists at GPO PG Clearinghouse.
    236236 * If available, baseline Monitoring is used to ensure that any problems are quickly identified.
    … …  
    240240Do the following:
    241241
    242  1. As Experimenter1, request !ListResources from BBN GRAM.
     242 1. As Experimenter1, request !ListResources from BBN OpenGENI.
    243243 2. Review !ListResources output, and identify available resources.
    244244 3. Write the Scenario 1 RSpec that requests 100 VMs evenly distributed across the experiment nodes using the default image.
    … …  
    258258
    259259
    260 === GR-EXP-4: GRAM Multi-site Acceptance Test ===
     260=== OG-EXP-4: OpenGENI Multi-site Acceptance Test ===
    261261
    262262This test normally includes two sites and two experiments. Only one rack is available, so the test case will be modified to run within one rack, but VMs will be requested on separate servers.  Each of the compute resources will exchange traffic.  In addition, the VMsin Experiment2 will use multiple data interfaces. Normally, all site-to-site experiments  take place over a wide-area Layer 2 data plane network connection via Internet2 or NLR using VLANs allocated by the AM, that is not the case for the initial test evaluation, all connections will be within the rack.  The goal of this test is to verify basic operations of VMs and data flows between rack resources.
    … …  
    266266This test uses this topology:
    267267
    268 [[Image(GRAMMultiSiteAcceptanceTest.jpg)]]
     268[[Image(OpenGENIMultiSiteAcceptanceTest.jpg)]]
    269269
    270270
    271271For the initial evaluation there is only one rack, so the test case is modified to have VMs on different servers rather than different racks. Here is the actual topology run:
    272272
    273 [[Image(GRAMMultiSiteAcceptanceTest-actual.jpg)]]
     273[[Image(OpenGENIMultiSiteAcceptanceTest-actual.jpg)]]
    274274
    275275''Evaluation Note:'' Test case is described for the original test case, actual procedure will be captures as part of test details available from the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatus Acceptance Test Status] page.
    … …  
    280280This test has these prerequisites:
    281281
    282  * If available, BBN GRAM connectivity statistics will be monitored.
    283  * Administrative accounts have been created for GPO staff at the BBN GRAM rack.
     282 * If available, BBN OpenGENI connectivity statistics will be monitored.
     283 * Administrative accounts have been created for GPO staff at the BBN OpenGENI rack.
    284284 * The VLANs used will be allocated by the rack AM.
    285285 * If available, baseline Monitoring is used to ensure that any problems are quickly identified.
    286  * GRAM manages private address allocation for the endpoints in this test.
     286 * OpenGENI manages private address allocation for the endpoints in this test.
    287287 * Normal network aggregate requirement for the availability of the ION AM do not apply to current evaluation.
    288288
    … …  
    291291Do the following:
    292292
    293  1. As Experimenter1, Request !ListResources from BBN GRAM.
    294  2. Request !ListResources for second GRAM AM (does not exist, thus skipping step).
     293 1. As Experimenter1, Request !ListResources from BBN OpenGENI.
     294 2. Request !ListResources for second OpenGENI AM (does not exist, thus skipping step).
    295295 3. Review !ListResources output from both AMs. (only one am used in this initial evaluation).
    296  4. Define a request RSpec for VMs at BBN GRAM to be on separate VM servers.
    297  5. Define a request RSpec for a VM at remote GRAM for an unbound exclusive non-!OpenFlow VLAN to connect the 2 endpoints. 
     296 4. Define a request RSpec for VMs at BBN OpenGENI to be on separate VM servers.
     297 5. Define a request RSpec for a VM at remote OpenGENI for an unbound exclusive non-!OpenFlow VLAN to connect the 2 endpoints. 
    298298 6. Create the first slice.
    299  7. Create a sliver at each GRAM aggregate using the RSpecs defined above.
     299 7. Create a sliver at each OpenGENI aggregate using the RSpecs defined above.
    300300 8. Log in to each of the systems, and send traffic to the other system, leave traffic running.
    301  9. As Experimenter2, Request !ListResources from BBN GRAM, (skipping second remote GRAM)
    302  10. Define an request RSpec for one VM and one bare metal node in the BBN GRAM rack. Each resource should have two logical interfaces and a 3rd VLAN for the local connection.
    303  11. Define a request RSpec to add two VMs at Site2 and two VLANs to connect the BBN GRAM to the Site2 GRAM. (Modified for one aggregate)
     301 9. As Experimenter2, Request !ListResources from BBN OpenGENI, (skipping second remote OpenGENI)
     302 10. Define an request RSpec for one VM and one bare metal node in the BBN OpenGENI rack. Each resource should have two logical interfaces and a 3rd VLAN for the local connection.
     303 11. Define a request RSpec to add two VMs at Site2 and two VLANs to connect the BBN OpenGENI to the Site2 OpenGENI. (Modified for one aggregate)
    304304 12. Create a second slice.
    305  13. In the second slice, create a sliver at each GRAM aggregate using the RSpecs defined above. (Modified for one aggregate)
     305 13. In the second slice, create a sliver at each OpenGENI aggregate using the RSpecs defined above. (Modified for one aggregate)
    306306 14. Log in to each of the end-point systems, and send traffic to the other end-point system which shares the same VLAN.
    307307 15. Verify traffic handling per experiment, VM isolation, and MAC address assignment.
    … …  
    314314
    315315
    316 === GR-EXP-5: GRAM Network Resources Acceptance Test ===
    317 
    318 A three site experiment where the only GRAM resources used are !OpenFlow network resources. All compute resources are outside the GRAM rack. The experiment will use the GRAM Aggregate Manager to request the rack data plane resources. The GRAM AM configures the GRAM site !OpenFlow switch. The goal of this test is to verify !OpenFlow operations and integration with meso-scale compute resources and other compute resources external to the GRAM rack.
     316=== OG-EXP-5: OpenGENI Network Resources Acceptance Test ===
     317
     318A three site experiment where the only OpenGENI resources used are !OpenFlow network resources. All compute resources are outside the OpenGENI rack. The experiment will use the OpenGENI Aggregate Manager to request the rack data plane resources. The OpenGENI AM configures the OpenGENI site !OpenFlow switch. The goal of this test is to verify !OpenFlow operations and integration with meso-scale compute resources and other compute resources external to the OpenGENI rack.
    319319
    320320==== Test Topology ====
    321321
    322 [[Image(GRAMOFNetworkResourceAcceptanceTest.jpg)]]
     322[[Image(OpenGENIOFNetworkResourceAcceptanceTest.jpg)]]
    323323
    324324''Note:'' The NLR and Internet2 !OpenFlow VLANs are the [wiki:NetworkCore GENI Network Core] static VLANs.
    … …  
    326326For the initial evaluation there is only one rack, so the test case is modified to have VMs on different servers rather than different racks. Here is the actual topology run:
    327327
    328 [[Image(GRAMOFNetworkResourceAcceptanceTest-actual.jpg)]]
     328[[Image(OpenGENIOFNetworkResourceAcceptanceTest-actual.jpg)]]
    329329
    330330
    331331==== Prerequisites ====
    332332
    333  - A GPO site network is connected to the GRAM !OpenFlow switch. 
    334  - GRAM VMOC is running and can manage the GRAM !OpenFlow switch
     333 - A GPO site network is connected to the OpenGENI !OpenFlow switch. 
     334 - OpenGENI VMOC is running and can manage the OpenGENI !OpenFlow switch
    335335 - An !OpenFlow controller is run by the experimenter and is accessible via DNS hostname (or IP address) and TCP port.
    336336 - Two meso-scale remote sites make compute resources and !OpenFlow meso-scale resources available for this test.
    337  - GMOC data collection for the meso-scale and GRAM rack resources is functioning for the !OpenFlow and traffic measurements required in this test.
     337 - GMOC data collection for the meso-scale and OpenGENI rack resources is functioning for the !OpenFlow and traffic measurements required in this test.
    338338
    339339''Evaluation Note:'' GMOC data collection is not available for the initial evaluation.  Remote meso-scale sites are not possible for the initial evaluation and will be replaced by local rack nodes.
    … …  
    343343 1. As Experimenter1, Determine BBN compute resources and define RSpec.
    344344 2. Determine remote meso-scale compute resources and define RSpec.  (Modified for one aggregate and no meso-scale)
    345  3. Define a request RSpec for !OpenFlow network resources at the BBN GRAM AM.
     345 3. Define a request RSpec for !OpenFlow network resources at the BBN OpenGENI AM.
    346346 4. Define a request RSpec for !OpenFlow network resources at the remote I2 Meso-scale site. (Rack nodes will replace remote meso-scale.)
    347347 5. Define a request RSpec for the !OpenFlow Core resources
    … …  
    349349 7. Create a sliver for the BBN compute resources.
    350350 8. Create a sliver at the I2 meso-scale site using VMOC at site.  (Modified for one aggregate and no meso-scale)
    351  9. Create a sliver at of the BBN GRAM AM.
     351 9. Create a sliver at of the BBN OpenGENI AM.
    352352 10. Create a sliver for the !OpenFlow resources in the core. (Modified for one aggregate and no meso-scale)
    353353 11. Create a sliver for the meso-scale compute resources. (Modified for one aggregate and no meso-scale)
    … …  
    357357 14. As Experimenter2, determine BBN compute resources and define RSpec.
    358358 15. Determine remote meso-scale compute resources and define RSpec.
    359  16. Define a request RSpec for !OpenFlow network resources at the BBN GRAM AM.
     359 16. Define a request RSpec for !OpenFlow network resources at the BBN OpenGENI AM.
    360360 17. Define a request RSpec for !OpenFlow network resources at the remote NLR Meso-scale site. (Rack nodes will replace remote meso-scale.)
    361361 18. Define a request RSpec for the !OpenFlow Core resources (No core resources will be used in initial evaluation)
    … …  
    363363 20. Create a sliver for the BBN compute resources.
    364364 21. Create a sliver at the meso-scale site using FOAM at site.
    365  22. Create a sliver at of the BBN GRAM AM.
     365 22. Create a sliver at of the BBN OpenGENI AM.
    366366 23. Create a sliver for the !OpenFlow resources in the core.
    367367 24. Create a sliver for the meso-scale compute resources.
    … …  
    372372 29. Stop traffic and delete slivers.
    373373
    374 === GR-EXP-6: GRAM and Meso-scale Multi-site !OpenFlow Acceptance Test ===
    375 
    376 This test case normally includes three sites and three experiments, using resources in the BBN and Site2 GRAM racks as well as meso-scale resources, where the network resources are the core !OpenFlow-controlled VLANs. Each of the compute resources will exchange traffic with the others in its slice, over a wide-area Layer 2 data plane network connection, using Internet2 and NLR VLANs. In particular, the following slices will be set up for this test:
    377   * Slice 1: One GRAM VM at each of BBN and Site2.
    378   * Slice 2: Two GRAM VMs at Site2 and one VM and one bare metal node at BBN.
    379   * Slice 3: An GRAM VM at BBN, a PG node at BBN, and a meso-scale Wide-Area ProtoGENI (WAPG) node.
     374=== OG-EXP-6: OpenGENI and Meso-scale Multi-site !OpenFlow Acceptance Test ===
     375
     376This test case normally includes three sites and three experiments, using resources in the BBN and Site2 OpenGENI racks as well as meso-scale resources, where the network resources are the core !OpenFlow-controlled VLANs. Each of the compute resources will exchange traffic with the others in its slice, over a wide-area Layer 2 data plane network connection, using Internet2 and NLR VLANs. In particular, the following slices will be set up for this test:
     377  * Slice 1: One OpenGENI VM at each of BBN and Site2.
     378  * Slice 2: Two OpenGENI VMs at Site2 and one VM and one bare metal node at BBN.
     379  * Slice 3: An OpenGENI VM at BBN, a PG node at BBN, and a meso-scale Wide-Area ProtoGENI (WAPG) node.
    380380
    381381The above topology will be requested within one rack.
    … …  
    385385This test uses this topology:
    386386
    387 [[Image(GRAMMultiSiteOpenFlowAcceptanceTest.jpg)]]
     387[[Image(OpenGENIMultiSiteOpenFlowAcceptanceTest.jpg)]]
    388388
    389389Note: The two Site2 VMs in Experiment2 must be on the same experiment node. This is not the case for other experiments.
    … …  
    392392For the initial evaluation there is only one rack, so the test case is modified to have VMs on different servers rather than different racks. Here is the actual topology run:
    393393
    394 [[Image(GRAMMultiSiteOpenFlowAcceptanceTest-actual.jpg)]]
     394[[Image(OpenGENIMultiSiteOpenFlowAcceptanceTest-actual.jpg)]]
    395395
    396396''Evaluation Note:'' Test case is described for the original test case, actual procedure will be captures as part of test details available from the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatus Acceptance Test Status] page.
    … …  
    401401
    402402 * Meso-scale sites are available for testing
    403  * BBN GRAM connectivity statistics are monitored at the GPO GRAM Monitoring site.
     403 * BBN OpenGENI connectivity statistics are monitored at the GPO OpenGENI Monitoring site.
    404404 * GENI Experimenter1, Experimenter2 and Experimenter3 accounts exist.
    405405 * This test will be scheduled at a time when site contacts are available to address any problems.
    406  * Both GRAM aggregates can link to static VLANs.  (Modified for one aggregate)
     406 * Both OpenGENI aggregates can link to static VLANs.  (Modified for one aggregate)
    407407 * Site's !OpenFlow VLAN is implemented and is known for this test. (Use VMOC allocated OF VLANs)
    408408 * If available, baseline Monitoring is in place at each site, to ensure that any problems are quickly identified.
    409  * GMOC data collection for the meso-scale and GRAM rack resources is functioning for the !OpenFlow and traffic measurements required in this test.
     409 * GMOC data collection for the meso-scale and OpenGENI rack resources is functioning for the !OpenFlow and traffic measurements required in this test.
    410410 * An !OpenFlow controller is run by the experimenter and is accessible via DNS hostname (or IP address) and TCP port.
    411411 * a PG !OpenFlow site is also added to the setup described in the diagram
    412412
    413413
    414 ''Evaluation Note:''  There is no GMOC data colleciton and PG Site  for initial GRAM evaluation.
     414''Evaluation Note:''  There is no GMOC data colleciton and PG Site  for initial OpenGENI evaluation.
    415415
    416416==== Procedure ====
    … …  
    418418Do the following:
    419419
    420  1. As Experimenter1, request !ListResources from BBN GRAM, Site2 GRAM, and from VMOC at I2 and NLR Site.
     420 1. As Experimenter1, request !ListResources from BBN OpenGENI, Site2 OpenGENI, and from VMOC at I2 and NLR Site.
    421421 2. Review !ListResources output from all AMs.
    422  3. Define a request RSpec for a VM at the BBN GRAM.
    423  4. Define a request RSpec for a VM at the Site2 GRAM. (only one site used)
     422 3. Define a request RSpec for a VM at the BBN OpenGENI.
     423 4. Define a request RSpec for a VM at the Site2 OpenGENI. (only one site used)
    424424 5. Define request RSpecs for !OpenFlow resources from BBN FOAM to access GENI !OpenFlow core resources.  (only one site used)
    425425 6. Define request RSpecs for !OpenFlow core resources at I2 FOAM (only one site used)
    … …  
    428428 9. Create a sliver in the first slice at each AM, using the RSpecs defined above.
    429429 10. Log in to each of the systems, verify IP address assignment. Send traffic to the other system, leave traffic running.
    430  11. As Experimenter2, define a request RSpec for one VM and one physical node at BBN GRAM.
    431  12. Define a request RSpec for two VMs on the same experiment node at Site2 GRAM.  (only one site used)
     430 11. As Experimenter2, define a request RSpec for one VM and one physical node at BBN OpenGENI.
     431 12. Define a request RSpec for two VMs on the same experiment node at Site2 OpenGENI.  (only one site used)
    432432 11. Define request RSpecs for !OpenFlow resources from BBN FOAM to access GENI !OpenFlow core resources.  (only one site used)
    433433 14. Define request RSpecs for !OpenFlow core resources at I2 FOAM. (only one site used)
    … …  
    436436 17. Create a sliver in the second slice at each AM, using the RSpecs defined above.
    437437 18. Log in to each of the systems in the slice, and send traffic to each other systems; leave traffic running
    438  19. As Experimenter3, request !ListResources from BBN GRAM, BBN meso-scale FOAM, and FOAM at Meso-scale Site (Internet2 Site BBN and NLR site).  (only one site used)
     438 19. As Experimenter3, request !ListResources from BBN OpenGENI, BBN meso-scale FOAM, and FOAM at Meso-scale Site (Internet2 Site BBN and NLR site).  (only one site used)
    439439 20. Review !ListResources output from all AMs.
    440  21. Define a request RSpec for a VM at the BBN GRAM.
     440 21. Define a request RSpec for a VM at the BBN OpenGENI.
    441441 22. Define a request RSpec for a compute resource at the BBN meso-scale site.  (only one site used)
    442442 23. Define a request RSpec for a compute resource at a meso-scale site. (only one site used)
    443  24. Define request RSpecs for !OpenFlow resources to allow connection from !OpenFlow BBN GRAM to Meso-scale !OpenFlow sites(BBN and second site TBD) (I2 and NLR).  (only one site used)
     443 24. Define request RSpecs for !OpenFlow resources to allow connection from !OpenFlow BBN OpenGENI to Meso-scale !OpenFlow sites(BBN and second site TBD) (I2 and NLR).  (only one site used)
    444444 25. If PG access to !OpenFlow is available, define a request RSpec for the PG !OpenFlow resource. (only one site used)
    445445 26. Create a third slice.
    446  27. Create slivers that connects the Internet2 Meso-scale !OpenFlow site to the BBN GRAM Site, and the BBN Meso-scale site; and if available, to PG node.
    447  28. Log in to each of the compute resources in the slice, configure data plane network interfaces on any non-GRAM resources as necessary, and send traffic to each other systems; leave traffic running.
     446 27. Create slivers that connects the Internet2 Meso-scale !OpenFlow site to the BBN OpenGENI Site, and the BBN Meso-scale site; and if available, to PG node.
     447 28. Log in to each of the compute resources in the slice, configure data plane network interfaces on any non-OpenGENI resources as necessary, and send traffic to each other systems; leave traffic running.
    448448 29. Verify that all three experiment continue to run without impacting each other's traffic, and that data is exchanged over the path along which data is supposed to flow.
    449449 30. Review baseline monitoring statistics and checks.
    450  31. As site administrator, identify all controllers that the BBN GRAM !OpenFlow switch is connected to.
     450 31. As site administrator, identify all controllers that the BBN OpenGENI !OpenFlow switch is connected to.
    451451 32. As Experimenter3, verify that traffic only flows on the network resources assigned to slivers as specified by the controller.
    452452 33. Verify that no default controller, switch fail-open behavior, or other resource other than experimenters' controllers, can control how traffic flows on network resources assigned to experimenters' slivers.
    … …  
    463463 1. Verify access to documentation about which !OpenFlow actions can be performed in hardware.
    464464
    465 === GR-EXP-7: Click Router Experiment Acceptance Test ===
    466 
    467 This test case uses a [http://read.cs.ucla.edu/click/click Click] modular router experiment with GRAM VM nodes. The scenario uses 2 VMs as hosts and 4 VMs as Click Routers and is based on the following [http://groups.geni.net/geni/wiki/ClickExampleExperiment Click example] experiment, although unlike the example, this test case uses VMs and it runs the Click router module in user space. 
     465=== OG-EXP-7: Click Router Experiment Acceptance Test ===
     466
     467This test case uses a [http://read.cs.ucla.edu/click/click Click] modular router experiment with OpenGENI VM nodes. The scenario uses 2 VMs as hosts and 4 VMs as Click Routers and is based on the following [http://groups.geni.net/geni/wiki/ClickExampleExperiment Click example] experiment, although unlike the example, this test case uses VMs and it runs the Click router module in user space. 
    468468
    469469==== Test Topology ====
    … …  
    471471This test uses this topology:
    472472
    473 [[Image(GRAMClickRouterAcceptanceTest.jpg)]]
     473[[Image(OpenGENIClickRouterAcceptanceTest.jpg)]]
    474474
    475475Note: Two VMs will be requested on the same physical worker node at each rack site for the user-level Click Router .
    … …  
    477477For the initial evaluation there is only one rack, so the test case is modified to have VMs on different servers rather than different racks. Here is the actual topology run:
    478478
    479 [[Image(GRAMClickRouterAcceptanceTest-actual.jpg)]]
     479[[Image(OpenGENIClickRouterAcceptanceTest-actual.jpg)]]
    480480
    481481''Evaluation Note:'' Test case is described for the original test case, actual procedure will be captures as part of test details available from the [https://superior.bbn.com/trac/bbn-rack/wiki/AcceptanceTestStatus Acceptance Test Status] page. The test case will be run within one rack.
    … …  
    491491Do the following:
    492492
    493     1. As Experimenter1, request ListResources from BBN GRAM
     493    1. As Experimenter1, request ListResources from BBN OpenGENI
    494494   2. Review ListResources
    495    3. Define a request RSpec for six VMs at BBN GRAM
     495   3. Define a request RSpec for six VMs at BBN OpenGENI
    496496   4. Create slice
    497497   5. Create a sliver
    … …  
    507507These tests will be performed as needed after the administration baseline tests complete successfully.  For example, the Software Update Test will be performed at least once when the rack team provides new software for testing.  We expect these tests to be interspersed with other tests in this plan at times that are agreeable to the GPO and the participants, not just run in a block at the end of testing.  The goal of these tests is to verify that sites have adequate documentation, procedures, and tools to satisfy all GENI site requirements.
    508508
    509 === GR-ADM-3: Full Rack Reboot Test ===
     509=== OG-ADM-3: Full Rack Reboot Test ===
    510510
    511511In this test, a full rack reboot is performed as a drill of a procedure which a site administrator may need to perform for site maintenance.
    … …  
    513513''Note: this test must be run using the BBN rack because it requires physical access.''
    514514
    515 ''Evaluation note:'' Can this be executed for the BBN GRAM rack?
     515''Evaluation note:'' Can this be executed for the BBN OpenGENI rack?
    516516
    517517==== Procedure ====
    … …  
    521521 3. Power on all devices, bring all logical components back online, and use monitoring and comprehensive health tests to verify that the rack is healthy again.
    522522
    523 === GR-ADM-4: Emergency Stop Test ===
     523=== OG-ADM-4: Emergency Stop Test ===
    524524
    525525In this test, an Emergency Stop drill is performed on a sliver in the rack.
    … …  
    528528
    529529 * GMOC's updated Emergency Stop procedure is approved and published on a public wiki.
    530  * GRAM's procedure for performing a shutdown operation on any type of sliver in an GRAM rack is published on a public wiki or on a protected wiki that all GRAM site administrators (including GPO) can access.
     530 * OpenGENI's procedure for performing a shutdown operation on any type of sliver in an OpenGENI rack is published on a public wiki or on a protected wiki that all OpenGENI site administrators (including GPO) can access.
    531531 * An Emergency Stop test is scheduled at a convenient time for all participants and documented in GMOC ticket(s).
    532  * A test experiment is running that involves a slice with connections to at least one GRAM rack compute resource.
     532 * A test experiment is running that involves a slice with connections to at least one OpenGENI rack compute resource.
    533533
    534534''Evaluation note:'' Emergency stop is not expected to be supported for the initial evaluation.
    … …  
    542542 * GMOC completes the Emergency Stop workflow, including updating/closing GMOC tickets.
    543543
    544 === GR-ADM-5: Software Update Test ===
     544=== OG-ADM-5: Software Update Test ===
    545545
    546546In this test, we update software on the rack as a test of the software update procedure.
    … …  
    548548==== Prerequisites ====
    549549
    550 Minor updates of system packages for all infrastructure OSes, GRAM local AM software, and VMOC are available to be installed on the rack.  This test may need to be scheduled to take advantage of a time when these updates are available.
     550Minor updates of system packages for all infrastructure OSes, OpenGENI local AM software, and VMOC are available to be installed on the rack.  This test may need to be scheduled to take advantage of a time when these updates are available.
    551551
    552552==== Procedure ====
    … …  
    557557   * At least one update of a standard (non-GENI) system package on the VMOC VM.
    558558   * At least one update of a standard (non-GENI) system package on the VM server host OS.
    559    * An update of GRAM local AM software on control node.
     559   * An update of OpenGENI local AM software on control node.
    560560   * An update of VMOC software
    561561 * The admin confirms that the software updates completed successfully
    562562 * The admin updates any appropriate version tracking documentation or runs appropriate tool checks indicated by the version tracking procedure.
    563563
    564 === GR-ADM-6: Control Network Disconnection Test ===
     564=== OG-ADM-6: Control Network Disconnection Test ===
    565565
    566566In this test, we disconnect parts of the rack control network or its dependencies to test partial rack functionality in an outage situation.
    … …  
    571571
    572572 * Simulate an outage of ???? by inserting a firewall rule on the BBN router blocking the rack from reaching it.  Verify that an administrator can still access the rack, that rack monitoring to GMOC continues through the outage, and that some experimenter operations still succeed.
    573  * Simulate an outage of each of the rack server host and control plane switch by disabling their respective interfaces on the BBN's control network switch.  Verify that GPO, GRAM, and GMOC monitoring all see the outage.
    574 
    575 ''Evaluation Note:'' The simulated outage does not apply to initial evaluation, there will be no monitoring by GMOC. Also there is no GRAM SNMP polling. 
    576 
    577 
    578 === GR-ADM-7: Documentation Review Test ===
     573 * Simulate an outage of each of the rack server host and control plane switch by disabling their respective interfaces on the BBN's control network switch.  Verify that GPO, OpenGENI, and GMOC monitoring all see the outage.
     574
     575''Evaluation Note:'' The simulated outage does not apply to initial evaluation, there will be no monitoring by GMOC. Also there is no OpenGENI SNMP polling. 
     576
     577
     578=== OG-ADM-7: Documentation Review Test ===
    579579
    580580Although this is not a single test ''per-se'', this section lists required documents that the rack teams will write.  Draft documents should be delivered prior to testing of the functional areas to which they apply.  Final documents must be deliveredto be made available for non-developer sites.  Final documents will be public, unless there is some specific reason a particular document cannot be public (e.g. a security concern from a GENI rack site). 
    … …  
    611611== Additional Monitoring Acceptance Tests ==
    612612
    613 These tests will be performed as needed after the monitoring baseline tests complete successfully.  For example, the GMOC data collection test will be performed during the GRAM Network Resources Acceptance test, where we already use the GMOC for meso-scale !OpenFlow monitoring.  We expect these tests to be interspersed with other tests in this plan at times that are agreeable to the GPO and the participants, not just run in a block at the end of testing.  The goal of these tests is to verify that sites have adequate tools to view and share GENI rack data that satisfies all GENI monitoring requirements.
    614 
    615 === GR-MON-4: Infrastructure Device Performance Test ===
     613These tests will be performed as needed after the monitoring baseline tests complete successfully.  For example, the GMOC data collection test will be performed during the OpenGENI Network Resources Acceptance test, where we already use the GMOC for meso-scale !OpenFlow monitoring.  We expect these tests to be interspersed with other tests in this plan at times that are agreeable to the GPO and the participants, not just run in a block at the end of testing.  The goal of these tests is to verify that sites have adequate tools to view and share GENI rack data that satisfies all GENI monitoring requirements.
     614
     615=== OG-MON-4: Infrastructure Device Performance Test ===
    616616
    617617This test verifies that the rack head node performs well enough to run all the services it needs to run.
    … …  
    619619==== Procedure ====
    620620
    621 While experiments involving GRAM-controlled !OpenFlow slivers and compute slivers are running:
     621While experiments involving OpenGENI-controlled !OpenFlow slivers and compute slivers are running:
    622622 * View !OpenFlow control monitoring at GMOC and verify that no monitoring data is missing
    623623 * View VLAN 1750 data plane monitoring, which pings the rack's interface on VLAN 1750, and verify that packets are not being dropped
    … …  
    627627
    628628
    629 === GR-MON-5: GMOC Data Collection Test ===
     629=== OG-MON-5: GMOC Data Collection Test ===
    630630
    631631This test verifies the rack's submission of monitoring data to GMOC.
    … …  
    635635==== Procedure ====
    636636
    637 View the dataset collected at GMOC for the BBN and Site2 GRAM racks.  For each piece of required data, attempt to verify that:
     637View the dataset collected at GMOC for the BBN and Site2 OpenGENI racks.  For each piece of required data, attempt to verify that:
    638638 * The data is being collected and accepted by GMOC and can be viewed at gmoc-db.grnoc.iu.edu
    639  * The data's "site" tag indicates that it is being reported for the GRAM rack located at the `gpolab` or GRAM site2 site (as appropriate for that rack).
     639 * The data's "site" tag indicates that it is being reported for the OpenGENI rack located at the `gpolab` or OpenGENI site2 site (as appropriate for that rack).
    640640 * The data has been reported within the past 10 minutes.
    641641 * For each piece of data, either verify that it is being collected at least once a minute, or verify that it requires more complicated processing than a simple file read to collect, and thus can be collected less often.
    642642
    643643Verify that the following pieces of data are being reported:
    644  * Is each of the rack GRAM and VMOC AMs reachable via the GENI AM API right now?
     644 * Is each of the rack OpenGENI and VMOC AMs reachable via the GENI AM API right now?
    645645 * Is each compute or unbound VLAN resource at each rack AM online?  Is it available or in use?
    646646 * Sliver count and percentage of rack compute and unbound VLAN resources in use.
    … …  
    670670== Issue Tracking ==
    671671 1. All issues discovered in acceptance testing regardless of priority are to be tracked in a bug tracking system.
    672  2. The bug tracking system to be used is the [https://superior.bbn.com/trac/bbn-rack/query?status=accepted&status=assigned&status=new&status=reopened&component=test  GRAM trac] using the "test" component.
     672 2. The bug tracking system to be used is the [https://superior.bbn.com/trac/bbn-rack/query?status=accepted&status=assigned&status=new&status=reopened&component=test  OpenGENI trac] using the "test" component.
    673673 3. All types of issues encountered (documentation error, software bug, missing features, missing documentation, etc.) are to tracked.
    674674 4. All unresolved issues will be reviewed and published at the end of the acceptance test as part of the acceptance test report.
    … …  
    682682== Test Case Naming ==
    683683
    684 The test case in this plan follow a naming convention that uses ''GR-XXX-Y'' where ''GR'' is GRAM and ''XXX'' may equal any of the following: ''ADM'' for Administrative or ''EXP'' for Experimenter or ''MON'' for Monitoring. The final component of the test case name is the ''Y'', which is the test case number.   
     684The test case in this plan follow a naming convention that uses ''OG-XXX-Y'' where ''GR'' is OpenGENI and ''XXX'' may equal any of the following: ''ADM'' for Administrative or ''EXP'' for Experimenter or ''MON'' for Monitoring. The final component of the test case name is the ''Y'', which is the test case number.   
    685685
    686686= Requirements Validation =