wiki:GeniNetworkStitchingTestStatus/Functional

Version 8 (modified by lnevers@bbn.com, 7 years ago) (diff)

--

Stitching Functional Tests

This page captures status and execution details for the Stitching Functional tests. For overall status see the GENI Network Stitching Test Status page and for test details see the GENI Network Stitching Test Plan page.

Last update: 06/06/13

Function State Ticket Comments
GENI AM API Functions
RSpec Support
Negative and Boundary


State Legend Description
Color(green,Pass)? Test completed and met all criteria
Color(#98FB98,Pass: most criteria)? Test completed and met most criteria. Exceptions documented
Color(red,Fail)? Test completed and failed to meet criteria.
Color(yellow,Complete)? Test completed but will require re-execution due to expected changes
Color(orange,Blocked)? Blocked by ticketed issue(s).
Color(#63B8FF,In Progress)? Currently under test.


Test Execution

Assumptions:

GENI AM API Functions

The following GENI AM API functions are verified in managing stitching resources using GENI AM API V3. The GCF tool (stitcher.py) negotiates to V2 when an aggregate does not support V3.

List Resources for aggregate

List resources for the 4 aggregate listed in the assumptions and verify that Advertisement:

  • can pass rspelint
  • contains expected VLAN information (vlanRangeAvailability, vlanTranslation).

GPO InstaGENI:

$ omni.py listresources -a ig-gpo -V3 -o
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework portal
INFO:omni:Substituting AM nickname ig-gpo with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
INFO:omni:Saving output to a file.
INFO:omni:Substituting AM nickname ig-gpo with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
WARNING:omni:Requested API version 3, but AM https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0 uses version 2. Same aggregate talks API v3 at a different URL: https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/3.0
INFO:omni:Switching AM URL to match requested version
INFO:omni:Listed advertised resources at 1 out of 1 possible aggregates.
INFO:omni:Substituting AM nickname ig-gpo with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
INFO:omni: (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=0290a8d62a7817406203ef2fd86ed99f)
INFO:omni:Writing to 'rspec-instageni-gpolab-bbn-com.xml'
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed listresources:

  Options as run:
		aggregate: ['ig-gpo']
		api_version: 3
		framework: portal
		output: True
		project: ln-prj

  Args: listresources

  Result Summary: Queried resources from 1 of 1 aggregate(s).
Wrote rspecs from 1 aggregate(s) to 1 file(s)
Saved listresources RSpec from 'urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm' (url 'https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/3.0') to file rspec-instageni-gpolab-bbn-com.xml;  
INFO:omni: ============================================================

$ rspeclint rspec-instageni-gpolab-bbn-com-protogeniv3.xml
Ignoring node rspec_opstate with unknown namespace http://www.geni.net/resources/rspec/ext/opstate/1

$ egrep -i "vlanRangeAvailability|vlanTranslation" rspec-instageni-gpolab-bbn-com-protogeniv3.xml 
                            <vlanRangeAvailability>                    3747-3749                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    2644-2649                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  

Internet2:

$ omni.py listresources -a i2 -V2 -o
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework portal
INFO:omni:Substituting AM nickname i2 with URL http://geni-am.net.internet2.edu:12346, URN urn:publicid:IDN+ion.internet2.edu+authority+cm
INFO:omni:Saving output to a file.
INFO:omni:Substituting AM nickname i2 with URL http://geni-am.net.internet2.edu:12346, URN urn:publicid:IDN+ion.internet2.edu+authority+cm
INFO:omni:Listed advertised resources at 1 out of 1 possible aggregates.
INFO:omni:Substituting AM nickname i2 with URL http://geni-am.net.internet2.edu:12346, URN urn:publicid:IDN+ion.internet2.edu+authority+cm
INFO:omni:Writing to 'rspec-ion-internet2-edu.xml'
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed listresources:

  Options as run:
		aggregate: ['i2']
		framework: portal
		output: True
		project: ln-prj

  Args: listresources

  Result Summary: Queried resources from 1 of 1 aggregate(s).
Wrote rspecs from 1 aggregate(s) to 1 file(s)
Saved listresources RSpec from 'urn:publicid:IDN+ion.internet2.edu+authority+cm' (url 'http://geni-am.net.internet2.edu:12346') to file rspec-ion-internet2-edu.xml;  
INFO:omni: ============================================================

$ rspeclint rspec-ion-internet2-edu.xml
Line 6: Failed validation with root at element: Schemas validity error : Element '{http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}scheduledServices': This element is not expected. Expected is one of ( {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}scheduledservices, {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}negotiatedservices, {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}lifetime, {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}node, {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}port, {http://hpn.east.isi.edu/rspec/ext/stitch/0.1/}link ).
: stitching

$ egrep -i "vlanRangeAvailability|vlanTranslation" rspec-ion-internet2-edu.xml 
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    3020-3030,3100-3109,3200-3219                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    670,3726-3750                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>                  
                             <vlanRangeAvailability>                    2-4094                  </vlanRangeAvailability>                  
                             <vlanTranslation>                    true                  </vlanTranslation>            

Utah ProtoGENI:

$ omni.py listresources -a pg-utah -V3 -o
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework portal
INFO:omni:Substituting AM nickname pg-utah with URL https://www.emulab.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+emulab.net+authority+cm
INFO:omni:Saving output to a file.
INFO:omni:Substituting AM nickname pg-utah with URL https://www.emulab.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+emulab.net+authority+cm
WARNING:omni:Requested API version 3, but AM https://www.emulab.net:12369/protogeni/xmlrpc/am/2.0 uses version 2. Same aggregate talks API v3 at a different URL: https://www.emulab.net:12369/protogeni/xmlrpc/am/3.0
INFO:omni:Switching AM URL to match requested version
INFO:omni:Listed advertised resources at 1 out of 1 possible aggregates.
INFO:omni:Substituting AM nickname pg-utah with URL https://www.emulab.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+emulab.net+authority+cm
INFO:omni: (PG log url - look here for details on any failures: https://www.emulab.net/spewlogfile.php3?logfile=0c869084faca0cc290a5f7f17364933c)
INFO:omni:Writing to 'rspec-emulab-net.xml'
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed listresources:

  Options as run:
		aggregate: ['pg-utah']
		api_version: 3
		framework: portal
		output: True
		project: ln-prj

  Args: listresources

  Result Summary: Queried resources from 1 of 1 aggregate(s).
Wrote rspecs from 1 aggregate(s) to 1 file(s)
Saved listresources RSpec from 'urn:publicid:IDN+emulab.net+authority+cm' (url 'https://www.emulab.net:12369/protogeni/xmlrpc/am/3.0') to file rspec-emulab-net.xml;  
INFO:omni: ============================================================

$ rspeclint rspec-emulab-net.xml 
Ignoring node rspec_opstate with unknown namespace http://www.geni.net/resources/rspec/ext/opstate/1

$ egrep -i "vlanRangeAvailability|vlanTranslation" rspec-emulab-net.xml        
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  
                            <vlanRangeAvailability>                    750-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  

Utah InstaGENI:

$ omni.py listresources -a ig-utah -V3 -o
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework portal
INFO:omni:Substituting AM nickname ig-utah with URL https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+utah.geniracks.net+authority+cm
INFO:omni:Saving output to a file.
INFO:omni:Substituting AM nickname ig-utah with URL https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+utah.geniracks.net+authority+cm
WARNING:omni:Requested API version 3, but AM https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/2.0 uses version 2. Same aggregate talks API v3 at a different URL: https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/3.0
INFO:omni:Switching AM URL to match requested version
INFO:omni:Listed advertised resources at 1 out of 1 possible aggregates.
INFO:omni:Substituting AM nickname ig-utah with URL https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+utah.geniracks.net+authority+cm
INFO:omni: (PG log url - look here for details on any failures: https://boss.utah.geniracks.net/spewlogfile.php3?logfile=7ebb578814d5e76b124dbdd2690a75f1)
INFO:omni:Writing to 'rspec-utah-geniracks-net.xml'
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed listresources:

  Options as run:
		aggregate: ['ig-utah']
		api_version: 3
		framework: portal
		output: True
		project: ln-prj

  Args: listresources

  Result Summary: Queried resources from 1 of 1 aggregate(s).
Wrote rspecs from 1 aggregate(s) to 1 file(s)
Saved listresources RSpec from 'urn:publicid:IDN+utah.geniracks.net+authority+cm' (url 'https://boss.utah.geniracks.net:12369/protogeni/xmlrpc/am/3.0') to file rspec-utah-geniracks-net.xml;  
INFO:omni: ============================================================

$ rspeclint rspec-utah-geniracks-net.xml         
Ignoring node rspec_opstate with unknown namespace http://www.geni.net/resources/rspec/ext/opstate/1

$ egrep -i "vlanRangeAvailability|vlanTranslation" rspec-utah-geniracks-net.xml 
                            <vlanRangeAvailability>                    256-1000                  </vlanRangeAvailability>                  
                            <vlanTranslation>                    false                  </vlanTranslation>                  

Getversion

Create Sliver

(allocate, provision, perform operational action start/stop/restart)

List resources for sliver

(describe)

Sliver status

Renew sliver

Delete sliver

Shutdown Sliver

GENI RSpec Support

GENI RSpec support tests will validate that each of the RSpec (Advertisement, Manifest, and Request) provide complete and accurate information.

Stitching Extensions:

  • Request RSpec with stitching extension -> to verify that extension is used
  • Request RSpec without stitching extension -> to verify that one is created
  • Stitching Path support:
    • Point-to-point path only
    • Multiple point-to-point

Requests RSpec validation:

  • RSpec Versions support (V3)
  • Routing profile options to include or exclude:
    • hop_include_list
    • hop_exclude_list
    • VLAN tags (exclude only)
  • Verify only bandwidth and VLAN are negotiable.

RSpec Input Validation:

  • Request link with only 1 component_manager tag
  • Request link with 2 interface_ref elements but no component_manager elements
  • Request link with 2 local interface_ref elements and 1 remote interface_ref element
  • Request link with an interface_ref whose client_id is not the ID for any interface in the RSpec
  • Request stitching extension with a path that has 0 hops
  • Request capacity that is below minimum and above maximum reservable Capacity
  • Invalid Path options (not sure what this is and how to do it ?)
  • Invalid Routing profiles (not sure what this is and how to do it ?)
  • Include/exclude invalid VLANs

Advertisement RSpecs:

Advertisement RSpecs will be reviewed to verify that they accurately capture:

  • Stitching elements (nodes, ports, links, capacity)
  • VLAN IDs and their availability
  • VLAN Bandwidth capability

Negative and Boundary tests

This is a starting list of scenarios that will validate negative and boundary conditions:

  • At ION, InstaGENI and ExoGENI, allocate resources that are not advertised, verify failure.
  • At ION, InstaGENI and ExoGENI, allocate VLAN tags that are already in use, verify failure.
  • Kill an ION circuit manually, verify recovery and logging of event.
  • Request a VLAN that is already in use, verify handling.
  • Create a request race condition where two slices (Slice1 and Slice2) request the same resources (AM1 <->VLAN1<->AM2), but Slice1 gets VLAN1 at AM1 and Slice2 gets VLAN1 at AM2. Verify results tools handle the results and properly handle resources.
  • Pseudo Loop Scenario: Request PG Utah to ION to IG GPO. Then request a 2nd interface at PG Utah node to ION to same interface on same node at IG GPO. If that fails, then request a 2nd interface on that node at IG GPO - that should work.

Attachments (1)

Download all attachments as: .zip