Opened 10 years ago
Last modified 9 years ago
#1156 new
Unable to stitch multiple link from GPO ExoGENI rack
Reported by: | lnevers@bbn.com | Owned by: | lnevers@bbn.com |
---|---|---|---|
Priority: | major | Milestone: | |
Component: | STITCHING | Version: | SPIRAL6 |
Keywords: | Network Stitching | Cc: | exogeni-design@geni.net, divyashri.bhat@gmail.com |
Dependencies: |
Description
When creating a stitched sliver from EG GPO to IG Utah with 2 links between the 2 nodes, the following error is reported:
10:07:24 INFO stitch.Aggregate: Stitcher doing createsliver at https://bbn-hn.exogeni.net:11443/orca/xmlrpc 10:07:28 ERROR omni: {'output': 'ERROR: Exception encountered: orca.ndl.NdlException: Interface bbnNet.rdf#BbnNet/IBM/G8052/GigabitEthernet /1/2/ethernet should be owned by exactly two entities (a node and a connection/vlan), instead: 4', 'code': {'geni_code': 2}} 10:07:28 INFO stitch.Aggregate: Got AMAPIError doing createsliver lndivy at <Aggregate urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am>: AMAPIError: Error from Aggregate: code 2: ERROR: Exception encountered: orca.ndl.NdlException: Interface bbnNet.rdf#BbnNet/IBM/G8052/GigabitEthernet/1/2/ethernet should be owned by exactly two entities (a node and a connection/vlan), instead: 4.
The original RSpec (stitch-eg-gpo-ig-gpo-2l.rspec) and the request generated by the stitcher and submitted to EG aggregate (ndivy-createsliver-request-11-exogeni-net-bbnvmsite.xml) are attached to this ticket.
Attachments (3)
Change History (9)
Changed 10 years ago by
Attachment: | lndivy-createsliver-request-11-exogeni-net-bbnvmsite.xml added |
---|
Changed 10 years ago by
Attachment: | stitch-eg-gpo-ig-utah-2l.rspec added |
---|
comment:1 Changed 10 years ago by
Changed 10 years ago by
Attachment: | Stitch5nodes-ln2.rspec added |
---|
comment:2 Changed 10 years ago by
When I tried this again after the recent maintenance on ExoGENI racks, this is what I get. It seems the link between IG-Utah and EG-GPO is being rejected.
dbhat@emmy9:~/Omni/gcf/src$ python stitcher.py createsliver testexostitch Stitch5nodes-ln2.rspec 14:04:43 INFO stitcher: Loading agg_nick_cache file '/home/dbhat/.gcf/agg_nick_cache' 14:04:43 INFO stitcher: Loading config file /home/dbhat/.gcf/omni_config 14:04:43 INFO stitcher: Using control framework portal 14:04:43 INFO stitcher: Checking that slice testexostitch is valid... 14:04:43 INFO stitcher: Slice urn:publicid:IDN+ch.geni.net:GIMITesting+slice+testexostitch expires on 2014-01-24 19:00:08 UTC 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+emulab.net+authority+cm> speaks AM API v3, but sticking with v2 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+utah.geniracks.net+authority+cm> speaks AM API v3, but sticking with v2 14:04:45 INFO stitcher: Stitched reservation will include resources from these aggregates: 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+am> 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am> 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+emulab.net+authority+cm> 14:04:45 INFO stitcher: <Aggregate urn:publicid:IDN+utah.geniracks.net+authority+cm> 14:04:45 INFO stitch.Aggregate: Writing to '/tmp/testexostitch-createsliver-request-11-exogeni-net-bbnvmsite.xml' 14:04:45 INFO stitch.Aggregate: Stitcher doing createsliver at https://bbn-hn.exogeni.net:11443/orca/xmlrpc 14:04:49 ERROR omni: {'output': 'ERROR: Exception encountered: orca.ndl.NdlException: Interface bbnNet.rdf#BbnNet/IBM/G8052/GigabitEthernet/1/2/ethernet is not really owned by this resource ab7ed642-f739-4116-b2eb-4b6efdb7b028#link1', 'code': {'geni_code': 2}} 14:04:49 INFO stitch.Aggregate: Got AMAPIError doing createsliver testexostitch at <Aggregate urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am>: AMAPIError: Error from Aggregate: code 2: ERROR: Exception encountered: orca.ndl.NdlException: Interface bbnNet.rdf#BbnNet/IBM/G8052/GigabitEthernet/1/2/ethernet is not really owned by this resource ab7ed642-f739-4116-b2eb-4b6efdb7b028#link1. 14:04:50 WARNING stitcher: Stitching failed but will retry: Circuit reservation failed at <Aggregate urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am> (AMAPIError: Error from Aggregate: code 2: ERROR: Exception encountered: orca.ndl.NdlException: Interface bbnNet.rdf#BbnNet/IBM/G8052/GigabitEthernet/1/2/ethernet is not really owned by this resource ab7ed642-f739-4116-b2eb-4b6efdb7b028#link1.). Try again from the SCS 14:04:50 INFO stitcher: Calling SCS for the 2rd time... 14:04:51 INFO stitcher: Pausing for 600 seconds for Aggregates to free up resources...
comment:3 Changed 10 years ago by
Owner: | changed from somebody to lnevers@bbn.com |
---|
comment:4 Changed 10 years ago by
This one will not get fixed until after recovery code is working (may be even after GEC19, depending on how things go). The fix is in the controller code and we don't have the cycles to do it right now, as we are focusing on recovery.
comment:5 follow-up: 6 Changed 9 years ago by
Support multiple GENI stitched links is available at the ucdavis-eg site.
First attempt to test fix failed due to stitcher bug (http://trac.gpolab.bbn.com/gcf/ticket/805) which is solved in gcf-2.9.
Using an early version of GCF 2.9, I was able to create a sliver with 2 links and exchange traffic. Also created a 5 node stitched star topology (4 stitched links) with ucdavis-eg as the center node. Was able to exchange traffic on all links.
Keeping ticket open until fix is installed at all EG racks and GCF 2.9 is released.
comment:6 Changed 9 years ago by
Replying to lnevers@…:
Keeping ticket open until fix is installed at all EG racks and GCF 2.9 is released.
The support for multiple GENI stitched links is on all ExoGENI racks. All that remains to do is to make GCF 2.9 available to experimenters.
This issue was discovered while investigating the errors reported by Divy for a 5 node stitched topology with 1 node at IG Utah and 4 nodes at ExoGENI GPO. Error differed, and failures changed with each attempt, will verify the support of multiple links by the ExoGENI GPO rack by verifying solution with Divy's RSpec, attached (Stitch5nodes-ln2.rspec).