CG-EXP-1: GENI AM API Compliance

​GENI AM API v2 tests for NCSU2 and NCSU Cisco racks.

Test Status

This section captures the status for each step in the acceptance test plan.

Test	State	Tickets/Comments
Omni Commands		Advertisement problem: unknown component id;
NCSU2 Compliance		Advertisement problem: NCSU2-only no VLAN failure even though resources are available?
NCSU Compliance		Advertisement problem: 2)NCSU-only unkown hardware type

State Legend	Description
Color(green,Pass)?	Test completed and met all criteria
Color(#98FB98,Pass: most criteria)?	Test completed and met most criteria. Exceptions documented
Color(red,Fail)?	Test completed and failed to meet criteria.
Color(yellow,Complete)?	Test completed but will require re-execution due to expected changes
Color(orange,Blocked)?	Blocked by ticketed issue(s).
Color(#63B8FF,In Progress)?	Currently under test.

Omni Commands

Collect getversion information to verify that NCSU is running the same Orca version as NCSU2.

$ omni getversion -a ncsu-eg -o ; omni getversion -a ncsu2-eg -o
$ egrep orca_version getversion-exogeni-net-ncsu*
getversion-exogeni-net-ncsu2vmsite.json:    'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'}
getversion-exogeni-net-ncsuvmsite.json:    'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'}

Both racks are running the same version.

Verify compute resources available in NCSU2 rack:

$ omni listresources -a ncsu2-eg -o; egrep "node c" rspec-exogeni-net-ncsu2vmsite.xml 
<node component_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+authority+am" component_name="orca-vm-cloud" exclusive="false">
<node component_id="urn:publicid:IDN+exogeni.net:ncsu2Net+node+orca-transit-net-vlan" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2Net+authority+am" component_name="orca-transit-net-vlan" exclusive="false">

Verify compute resources available in NCSU rack:

$ omni listresources -a ncsu-eg -o; egrep "node c" rspec-exogeni-net-ncsuvmsite.xml 
<node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+unknown" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="unknown" exclusive="false">
<node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="orca-vm-cloud" exclusive="false">

Note: What is "unknown" component_id for both NCSU and NCSU2?

Note: No Bare Metal nodes available

Running rspeclint on the Advertisement generated by listresources for NCSU2 passes:

$  rspeclint rspec-exogeni-net-ncsu2vmsite.xml 

The rspeclint does not pass for NCSU:

$ rspeclint rspec-exogeni-net-ncsuvmsite.xml 
Line 7: Failed validation with root at element: Schemas validity error : 
Element '{http://www.geni.net/resources/rspec/3}hardware_type': 
The attribute 'name' is required but missing. : rspec

Note: The "<hardware_type/>" on line 10 should be removed from the advertisement.

Note: NCSU2 advertises stitching, NCSU does not... investigating.

CG-EXP-1a GENI AM API Compliance - NCSU2

This test verifies compliance for the NCSU2 Cisco rack.

NCSU2 Unbound Resources

The AM API Acceptance test results for NCSU2:

lnevers@sendaria:~/gcf-2.7/acceptance_tests/AM_API$ ./am_api_accept.py -a am-undertest --un-bound
E.E..........
======================================================================
ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 803, in subtest_CreateSliverWorkflow
    numslivers, manifest, slivers = self.subtest_generic_CreateSliver( slicename, doProvision, doPOA, expectedExpiration=sliceExpiration )
  File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver
    numslivers, manifest = self.subtest_CreateSliver( slicename )
  File "./am_api_accept.py", line 1610, in subtest_CreateSliver
    text, manifest = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver
    (result, message) = self._retrieve_value(result, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1.
 Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions..

======================================================================
ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1141, in test_CreateSliverWorkflow_multiSlice
    createReturn = self.subtest_generic_CreateSliver( slicenames[i], expectedExpiration=sliceExpiration )
  File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver
    numslivers, manifest = self.subtest_CreateSliver( slicename )
  File "./am_api_accept.py", line 1610, in subtest_CreateSliver
    text, manifest = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver
    (result, message) = self._retrieve_value(result, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1.
 Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions..

----------------------------------------------------------------------
Ran 13 tests in 210.520s

FAILED (errors=2)

Note: Lack or VLAN resources is reported, but could not duplicate... investigating

NCSU2 Bound Resource

The results:

$ ./am_api_accept.py -a am-undertest  -V2
F.F..........
======================================================================
FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow
    "Created sliver")
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest
    str(msg)+": Request RSpec and Manifest RSpec " \
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not.

======================================================================
FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice
    "but did not." % slicenames[i])
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1605420' expected to have same component_ids but did not.

----------------------------------------------------------------------
Ran 13 tests in 247.947s

FAILED (failures=2)

Note: may be related to "unknown" component_id

CG-EXP-1b GENI AM API Compliance - NCSU

This test verifies compliance for the NCSU Cisco rack.

NCSU Unbound Resources

The AM API Acceptance test results:

$ ./am_api_accept.py -a am-undertest --un-bound
E.E..........
======================================================================
ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 929, in subtest_CreateSliverWorkflow
    self.subtest_generic_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many
    self.subtest_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many
    self.subtest_RenewSliver( slicename, twodays )
  File "./am_api_accept.py", line 1375, in subtest_RenewSliver
    text, (succList, failList) = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver
    (res, message) = self._retrieve_value(res, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid..

======================================================================
ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1267, in test_CreateSliverWorkflow_multiSlice
    self.subtest_generic_RenewSliver_many( slicenames[i] )
  File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many
    self.subtest_RenewSliver_many( slicename )
  File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many
    self.subtest_RenewSliver( slicename, twodays )
  File "./am_api_accept.py", line 1375, in subtest_RenewSliver
    text, (succList, failList) = self.call(omniargs, self.options_copy)
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call
    ret_val = omni.call( cmd, options=options, verbose=True )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call
    return API_call( framework, config, args, opts, verbose=verbose )
  File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call
    result = handler._handle(args)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle
    (message, val) = getattr(self.amhandler,call)(args[1:])
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver
    (res, message) = self._retrieve_value(res, message, self.framework)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value
    self._raise_omni_error(message, AMAPIError, result)
  File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error
    raise err, (msg, triple)
AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid..

----------------------------------------------------------------------
Ran 13 tests in 674.790s

FAILED (errors=2)

Note: New issue... Expiration should not be causing issues....Investigating

NCSU Bound Compute Resource

The results:

$ ./am_api_accept.py -a am-undertest           
F.F..........
======================================================================
FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 773, in test_CreateSliver
    self.subtest_CreateSliverWorkflow()
  File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow
    "Created sliver")
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest
    str(msg)+": Request RSpec and Manifest RSpec " \
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not.

======================================================================
FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice
    "but did not." % slicenames[i])
  File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual
    raise NotEqualComponentIDsError, msg
NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1539130' expected to have same component_ids but did not.

----------------------------------------------------------------------
Ran 13 tests in 247.056s
FAILED (failures=2)

Note: may be related to "unknown" component_id