wiki:GENIRacksHome/CiscoGENIRacks/AcceptanceTestPlan

Version 3 (modified by lnevers@bbn.com, 9 years ago) (diff)

--

Cisco GENI Racks Interoperability and OpenFlow Acceptance Test Plan

The Cisco GENI Racks Interoperability and OpenFlow Acceptance Test Plan is based on the ExoGENI System Acceptance Test Plan test cases. Test cases have been renamed for the Cisco rack and some have been combined to generate a more efficient test effort. The goal of this plan is to validate the [ http://groups.geni.net/geni/wiki/GeniRacks GENI Rack Requirements].

This plan defines tests that cover the following types of requirements: Integration (C), Monitoring(D), Experimenter(G) and Local Aggregate (F) requirements. The GENI AM API Acceptance tests suite covers Software (A and B) requirements that are not covered in this plan. This plan covers high-priority functional tests; ExoGENI racks support more functions than those highlighted here. Issues from acceptance testing will be tracked in ExoGENI trac tickets system. Test Status can be found in the Cisco GENI Acceptance Test Status page.

Assumptions and Dependencies

The following assumptions are made for all tests described in this plan:

  • GENI Clearinghouse credentials are used for all tests.
  • GPO Clearinghouse is the Slice Authority for all tests.
  • Resources for each test will be requested from the local broker whenever possible.
  • Compute resources are VMs unless otherwise stated.
  • All Service Manager (SM) requests MUST be made via the GENI AM API.
  • In all scenarios, one experiment is always equal to one slice.
  • FOAM will be used as the OpenFlow aggregate manager for Meso-scale resources in the OpenFlow test cases.

The following technical dependencies will be verified before test cases are executed:

  • ORCA RSpec/NDL conversion service is available to convert GENI requests.

Test Traffic Profile:

  • Experiment traffic includes UDP and TCP data streams that run for 60 seconds
  • Traffic exchange is used to verify that the appropriate data paths are used and that traffic is delivered successfully for each test described.
  • Performance measurement is not a goal of these acceptance tests.

Acceptance Tests Descriptions

This section describes each acceptance test by defining its goals, topology, and outline test procedure. Main area of acceptance include: GENI AM API compliance, Resource Allocation, ExoGENI Stitching Support, GENI Stitching Support, OpenFlow Support, Administrative functions and Monitoring. Attempts will also be made at characterizing rack performance.

CG-EXP-1: GENI AM API Compliance

This tests will validate compliance to GENI AM API by:

CG-EXP-2: Resource Allocation

This test validate ability to allocate all resource types specified in the GENI Rack Requirements including:

  • Request VMs and exchange traffic between allocated end-points
  • Request bare Metal and exchange traffic between allocated end-points

CG-EXP-2: ExoGENI Stitching Support

This test case validate the ability to use ExoGENI Stitching to other ExoGENI racks. Attempts will be made to connect resources at:

  • co-located Cisco rack - NCSU to NCSU2
  • remote ExoGENI racks:
    • NCSU2 to WVN
    • NCSU to WVN
    • NCSU2 to WVN
    • NCSU to GPO EG
    • NCSU2 to GPO EG

CG-EXP-4: GENI Stitching Support

Support for GENI Network Stitching will also be validated by verifying that the following connected are available:

  • NCSU2 to NCSU
  • NCSU2 to GPO EG
  • NCSU2 to ION Site
  • NCSU2 to AL2S Site
  • Multiple network aggregates (NYSERNet IG<ION-AL2S->NCSU2<-AL2S->Stanford-IG)

CG-EXP-5: OpenFlow Support

These tests can only be run on NCSU2, which supports !Open Flow. Testing will validate the following:

  • Support for OpenFlow Commands:

o getversion o listresources

  • Support for OpenFlow RSpecs:

o Advertisement o Manifest o Request

  • Shared OF VLAN Interoperability to EG and IG sites
  • OpenFlow over Stitched connection to remote InstaGENI
  • OpenFlow over Stitched connection to remote ExoGENI

CG-EXP-6: Limits Tests

Various resource allocation limits tests will be run in both NCSU and NCSU2. Scenarios will be expanded as status is determined for this area. Following are the types of tests that are executed to verify that the requirements for 100 VMs is met by this rack:

  • 100 VM - 50 slices w/2 VMs
  • 100 VMs - 10 slices w/10 VMs
  • 100 VMs - 4 slices w/25 VMs

CG-ADM-1 Administrative Tests

This test validates administrative access for NCSU2 and NCSU Cisco racks. Cisco GENI Racks Administrator accounts can be obtained as described in the ExoGENI Rack Operators page where accounts creation and usage is documented.

Administrative tests will verify SSH access to each rack component and "administrative functions" access (root) to each of the following:

  • head node
  • access from head node to each worker node and bare metal (if available)
  • management switch
  • dataplane switch
  • FOAM and FlowVisor

CG-MON-1: Monitoring Functions

Validation of GENI Monitoring support will be attempted, if the rack has implemented support for the GENI Operational Monitoring