= Stitching Functional Tests =

This page captures status and execution details for the Stitching Functional tests.   For overall status see the [http://groups.geni.net/geni/wiki/GeniNetworkStitchingTestStatus GENI Network Stitching Test Status] page and for test details see  the [http://groups.geni.net/geni/wiki/GeniNetworkStitchingTestPlan GENI Network Stitching Test Plan] page. 

''Last update: 06/06/13''

|| '''Function'''        || '''State'''              || ''' Ticket ''' || '''Comments''' ||
|| GENI AM API Functions ||  || || ||
|| RSpec Support         ||  || || ||
||Negative and Boundary  ||  || || ||


[[BR]]
|| '''State Legend'''                     || '''Description'''                                                    ||
|| [[Color(green,Pass)]]                  || Test completed and met all criteria                                  ||
|| [[Color(#98FB98,Pass: most criteria)]] || Test completed and met most criteria. Exceptions documented          ||
|| [[Color(red,Fail)]]                    || Test completed and failed to meet criteria.                          ||
|| [[Color(yellow,Complete)]]             || Test completed but will require re-execution due to expected changes ||
|| [[Color(orange,Blocked)]]              || Blocked by ticketed issue(s).                                        ||
|| [[Color(#63B8FF,In Progress)]]         || Currently under test.                                                ||
[[BR]]

= Test Execution =

Assumptions: 
 - It is expected that aggregates will not support GENI AM API V3 before GEC17, and that the GCF tool (stitcher.py) will negotiate to GENI AM API V2.  
 - Both GENI portal credential and pgeni.gpolab.bbn.com credentials are used throughout. 
 - All tests in this page are executed using a stitched sliver from IG GPO to IG Utah, which includes the following aggregates:
       * https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am,urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
       * http://geni-am.net.internet2.edu:12346,urn:publicid:IDN+ion.internet2.edu+authority+cm
       * https://www.emulab.net:12369/protogeni/xmlrpc/am,urn:publicid:IDN+emulab.net+authority+cm
       * https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am,urn:publicid:IDN+utah.geniracks.net+authority+cm


== GENI AM API Functions ==

The following GENI AM API functions are verified in managing stitching resources using GENI AM API V3. It is expected that GCF tool (stitcher.py) will negotiate to V2 for aggregates that do not support V3.
      * List Resources for aggregate 
      * Getversion 
      * Create Sliver (allocate, provion, perform operational action start/stop/restart)
      * List resources for sliver (describe)
      * Sliver status  
      * Renew sliver 
      * Delete sliver
      * Shutdown Sliver



== GENI RSpec Support ==

GENI RSpec support tests will validate that each of the RSpec (Advertisement, Manifest, and Request) provide complete and accurate information.

'''Stitching Extensions:''' [[BR]]

    - Request RSpec with stitching extension -> to verify that extension is used
    - Request RSpec without stitching extension -> to verify that one is created
    - Stitching Path support:
      * Point-to-point path only
      * Multiple point-to-point 

'''Requests RSpec validation:''' [[BR]]

   - RSpec Versions support (V3)
   - Routing profile options to include or exclude:
     * hop_include_list
     * hop_exclude_list
     * VLAN tags (exclude only)
  - Verify '''only''' bandwidth and VLAN are negotiable.


'''RSpec Input Validation:'' [[BR]]

  - Request link with only 1 component_manager tag
  - Request link with 2 interface_ref elements but no component_manager elements
  - Request link with 2 local interface_ref elements and 1 remote interface_ref element
  - Request link with an interface_ref whose client_id is not the ID for any interface in the RSpec
  - Request stitching extension with a path that has 0 hops
  - Request capacity that is below minimum and above maximum reservable Capacity
  - Invalid Path options (not sure what this is and how to do it ?)
  - Invalid Routing profiles (not sure what this is and how to do it ?)
  - Include/exclude invalid VLANs

 
'''Advertisement RSpecs:''' [[BR]]

Advertisement RSpecs will be reviewed to verify that they accurately capture:
 - Stitching elements (nodes, ports, links, capacity)
 - VLAN IDs and their availability
 - VLAN Bandwidth capability
 

== Negative and Boundary tests ==

This is a starting list of scenarios that will validate negative and boundary conditions:
 - At ION, InstaGENI and ExoGENI, allocate resources that are not advertised, verify failure.
 - At ION, InstaGENI and ExoGENI, allocate VLAN tags that are already in use, verify failure.
 - Kill an ION circuit manually, verify recovery and logging of event.
 - Request a VLAN that is already in use, verify handling.
 - Create a request race condition where two slices (Slice1 and Slice2) request the same resources (AM1 <->VLAN1<->AM2), but Slice1 gets VLAN1 at AM1 and Slice2 gets VLAN1 at AM2. Verify results tools handle the results and properly handle resources.
 - Pseudo Loop Scenario:  Request PG Utah to ION to IG GPO. Then request a 2nd interface at PG Utah node to ION to same interface on same node at IG GPO.  If that fails, then request a 2nd interface on that node at IG GPO - that should work.