wiki:GENIRacksHome/CiscoGENIRacks/AcceptanceTestStatus/CG-EXP-1

Version 16 (modified by lnevers@bbn.com, 5 years ago) (diff)

--

CG-EXP-1: GENI AM API Compliance

GENI AM API v2 tests for NCSU2 and NCSU Cisco racks.

Test Status

This section captures the status for each step in the acceptance test plan.

Test State Tickets/Comments
Omni Commands unknown component id
NCSU Compliance
NCSU2 Compliance


State Legend Description
Color(green,Pass)? Test completed and met all criteria
Color(#98FB98,Pass: most criteria)? Test completed and met most criteria. Exceptions documented
Color(red,Fail)? Test completed and failed to meet criteria.
Color(yellow,Complete)? Test completed but will require re-execution due to expected changes
Color(orange,Blocked)? Blocked by ticketed issue(s).
Color(#63B8FF,In Progress)? Currently under test.


Omni Commands

Collect getversion information to verify that NCSU is running the same Orca version as NCSU2.

$ omni getversion -a ncsu-eg -o ; omni getversion -a ncsu2-eg -o
$ egrep orca_version getversion-exogeni-net-ncsu*
getversion-exogeni-net-ncsu2vmsite.json:    'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'}
getversion-exogeni-net-ncsuvmsite.json:    'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'}

Both racks are running the same version.

Verify compute resources available in NCSU2 rack:

$ omni listresources -a ncsu2-eg -o; egrep "node c" rspec-exogeni-net-ncsu2vmsite.xml 
<node component_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+authority+am" component_name="orca-vm-cloud" exclusive="false">
<node component_id="urn:publicid:IDN+exogeni.net:ncsu2Net+node+orca-transit-net-vlan" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2Net+authority+am" component_name="orca-transit-net-vlan" exclusive="false">

Verify compute resources available in NCSU rack:

$ omni listresources -a ncsu-eg -o; egrep "node c" rspec-exogeni-net-ncsuvmsite.xml 
<node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+unknown" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="unknown" exclusive="false">
<node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="orca-vm-cloud" exclusive="false">

Note: What is "unknown" component_id for both NCSU and NCSU2?

Note: No Bare Metal nodes available

Running rspeclint on the Advertisement generated by listresources for NCSU2 passes:

$  rspeclint rspec-exogeni-net-ncsu2vmsite.xml 

The rspeclint does not pass for NCSU:

$ rspeclint rspec-exogeni-net-ncsuvmsite.xml 
Line 7: Failed validation with root at element: Schemas validity error : 
Element '{http://www.geni.net/resources/rspec/3}hardware_type': 
The attribute 'name' is required but missing. : rspec

Note: The "<hardware_type/>" on line 10 should be removed from the advertisement.

Note: NCSU2 advertises stitching, NCSU does not... investigating.

CG-EXP-1a GENI AM API Compliance - NCSU2

This test verifies compliance for the NCSU2 Cisco rack.

NCSU2 Unbound Resources

The AM API Acceptance test results for NCSU2:

lnevers@sendaria:~/gcf-2.7/acceptance_tests/AM_API$ ./am_api_accept.py -a am-undertest --un-bound
E.E..........
======================================================================
ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 803, in subtest_CreateSliverWorkflow
    numslivers, manifest, slivers = self.subtest_generic_CreateSliver( slicename, doProvision, doPOA, expectedExpiration=sliceExpiration )
  File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver
    numslivers, manifest = self.subtest_CreateSliver( slicename )
  File "./am_api_accept.py", line 1610, in subtest_CreateSliver
    text, manifest = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver
    (result, message) = self._retrieve_value(result, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1.
 Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions..

======================================================================
ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1141, in test_CreateSliverWorkflow_multiSlice
    createReturn = self.subtest_generic_CreateSliver( slicenames[i], expectedExpiration=sliceExpiration )
  File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver
    numslivers, manifest = self.subtest_CreateSliver( slicename )
  File "./am_api_accept.py", line 1610, in subtest_CreateSliver
    text, manifest = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver
    (result, message) = self._retrieve_value(result, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1.
 Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions..

----------------------------------------------------------------------
Ran 13 tests in 210.520s

FAILED (errors=2)

Note: Lack or VLAN resources is reported, but could not duplicate... investigating

NCSU2 Bound Resource

The results:

$ ./am_api_accept.py -a am-undertest  -V2
F.F..........
======================================================================
FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow
    "Created sliver")
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest
    str(msg)+": Request RSpec and Manifest RSpec " \
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not.

======================================================================
FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice
    "but did not." % slicenames[i])
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1605420' expected to have same component_ids but did not.

----------------------------------------------------------------------
Ran 13 tests in 247.947s

FAILED (failures=2)

Note: may be related to "unknown" component_id

CG-EXP-1b GENI AM API Compliance - NCSU

This test verifies compliance for the NCSU Cisco rack.

NCSU Unbound Resources

The AM API Acceptance test results:

$ ./am_api_accept.py -a am-undertest --un-bound
E.E..........
======================================================================
ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 929, in subtest_CreateSliverWorkflow
    self.subtest_generic_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many
    self.subtest_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many
    self.subtest_RenewSliver( slicename, twodays )
  File "./am_api_accept.py", line 1375, in subtest_RenewSliver
    text, (succList, failList) = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver
    (res, message) = self._retrieve_value(res, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid..

======================================================================
ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1267, in test_CreateSliverWorkflow_multiSlice
    self.subtest_generic_RenewSliver_many( slicenames[i] )
  File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many
    self.subtest_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many
    self.subtest_RenewSliver( slicename, twodays )
  File "./am_api_accept.py", line 1375, in subtest_RenewSliver
    text, (succList, failList) = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver
    (res, message) = self._retrieve_value(res, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid..

----------------------------------------------------------------------
Ran 13 tests in 674.790s

FAILED (errors=2)

Note: New issue... Expiration should not be causing issues....Investigating

NCSU Bound Compute Resource

The results:

$ ./am_api_accept.py -a am-undertest           
F.F..........
======================================================================
FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow
    "Created sliver")
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest
    str(msg)+": Request RSpec and Manifest RSpec " \
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not.

======================================================================
FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice
    "but did not." % slicenames[i])
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1539130' expected to have same component_ids but did not.

----------------------------------------------------------------------
Ran 13 tests in 247.056s
FAILED (failures=2)

Note: may be related to "unknown" component_id