Opened 6 years ago

Closed 4 years ago

#1094 closed (wontfix)

'Path Setup failed with error from PSSSetup' reported for I2/ION aggregate

Reported by: lnevers@bbn.com Owned by: ckotil@grnoc.iu.edu
Priority: major Milestone:
Component: I2AM Version: SPIRAL5
Keywords: Network Stitching Cc: xyang@maxgigapop.net
Dependencies:

Description

Creating a sliver from IG GPO to IG Utah. Createsliver reports a PATH_SETUP_FAILED error for Internet2/ION aggregate. The request RSpec is attached, the output from stitcher:

08:54:37 INFO     stitch.Aggregate: Writing to '/tmp/demo-createsliver-request-11-ion-internet2-edu.xml'
08:54:37 INFO     stitch.Aggregate: 
	Stitcher doing createsliver at http://geni-am.net.internet2.edu:12346
08:55:22 INFO     stitch.Aggregate: DCN AM <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm>: must wait for status ready....
08:55:22 INFO     stitch.Aggregate: Pause to let circuit become ready...

08:56:29 WARNING  stitch.Aggregate: sliverstatus 22201 is (still) failed at <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm>. Delete and retry.
08:56:29 WARNING  stitch.Aggregate:   Status had error message: Coordinator(PATH_SETUP_FAILED): 'Path Setup failed with error from PSSSetup'
08:56:35 INFO     stitch.launcher: Will put <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm> back in the pool to allocate. Got Sliver status for circuit 22201 was (still): failed: Coordinator(PATH_SETUP_FAILED): 'Path Setup failed with error from PSSSetup'
08:56:35 INFO     stitch.launcher: Pausing for 120 seconds for Aggregates to free up resources...

08:58:35 INFO     stitch.Aggregate: Writing to '/tmp/demo-createsliver-request-12-ion-internet2-edu.xml'
08:58:35 INFO     stitch.Aggregate: 
	Stitcher doing createsliver at http://geni-am.net.internet2.edu:12346
08:59:20 INFO     stitch.Aggregate: DCN AM <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm>: must wait for status ready....
08:59:20 INFO     stitch.Aggregate: Pause to let circuit become ready...
09:00:27 WARNING  stitch.Aggregate: sliverstatus 22211 is (still) failed at <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm>. Delete and retry.
09:00:27 WARNING  stitch.Aggregate:   Status had error message: Coordinator(PATH_SETUP_FAILED): 'Path Setup failed with error from PSSSetup'
09:00:33 INFO     stitch.launcher: Will put <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+cm> back in the pool to allocate. Got Sliver status for circuit 22211 was (still): failed: Coordinator(PATH_SETUP_FAILED): 'Path Setup failed with error from PSSSetup'

Attachments (2)

demo-createsliver-request-11-ion-internet2-edu.xml (6.3 KB) - added by lnevers@bbn.com 6 years ago.
demo-createsliver-request-12-ion-internet2-edu.xml (6.3 KB) - added by lnevers@bbn.com 6 years ago.

Download all attachments as: .zip

Change History (9)

Changed 6 years ago by lnevers@bbn.com

Changed 6 years ago by lnevers@bbn.com

comment:1 Changed 6 years ago by xyang@maxgigapop.net

Cc: xyang@maxgigapop.net added; ckotil@grnoc.iu.edu removed
Owner: changed from xyang@maxgigapop.net to ckotil@grnoc.iu.edu

Reassign to Chad to check with NOC.

comment:2 Changed 6 years ago by lnevers@bbn.com

Will re-test when the ION update is applied. Email from Chad explaining about the update:

On 7/3/13 11:16 AM, Chad Kotil wrote:

Luisa, We have identified the problem. There was an interface change at atla and wash which has not been applied to ION yet. Essentially anything taking the path rtr.atla ae8.10 or rtr.wash ae8.10 will fail until the fix is applied. There is a systems ticket tracking the issue. I have asked that an update be sent to you and Xi when it has been fixed. Should be some time later today. Im out of the office this week so I will be slower than usual to respond. If you need immediate help, please contact the NOC. Thanks, --Chad

comment:3 Changed 6 years ago by lnevers@bbn.com

Tried after getting an update about the fix being installed. No longer see the path error, but there are no VLANs available at rtr.newy:ae0:bbn, so will try again later. Following is the email form the GlobalNOC:

On 7/3/13 11:33 AM, Jason Al-Mansor wrote:

> This should be fixed now. I created a test circuit between our test
> points in ATLA and WASH successfully. Please verify that this is working
> correctly for you now.
>
>
> Jason Al-Mansor
> Systems Engineer
> GlobalNOC at Indiana University
> jalmanso@grnoc.iu.edu
>
> On 7/3/13 11:15 AM, Footprints Systems Support Project wrote:
>> When replying, type your text above this line.
>> ----------------------------------------------
>>
>> System Support ticket 19118 was updated on 07/03/2013 at 15:15:12 UTC.
>>
>> TITLE:        Internet2 ION Topology File Correction Request
>> SUBMITTER:    noc@net.internet2.edu
>> CREATED:      07/02/2013 20:09:14
>>
>> CI:           4-Normal
>> TICKET TYPE:  Problem/Request
>> STATUS:       Engineering
>> ASSIGNEES:    System Engineering, Jason Al-Mansor 
>> LAST CHANGE:  Appended a Description, Incoming mail: From: ckotil@grnoc.iu.edu; To: systems-support@tick.globalnoc.iu.edu.
>>
>> SUMMARY OF CURRENT STATUS
>>
>> I2 NetEng has requested for the Topology File for
>> rtr.wash and rtr.atla to be updated to use the proper
>> ae8.30.
>>
>> DESCRIPTION
>>
>> Geni folks are reporting that this is causing problems for their testing.
>> I tracked the issue down to this 'corruption'. Change ae8.10 to ae8.30 for rtr.atla , and probably wash too, and fix up the IPs.
>> When this is done please send a note to:
>>
>> lnevers@bbn.com
>> xyang@maxgigapop.net
>>
>> Thanks,
>> --Chad
>>
>> On Jul 2, 2013, at 4:15 PM, Footprints Systems Support Project <systems-support@tick.globalnoc.iu.edu> wrote:
>>
>>> [Duplicate message snipped]
>>
>>
>> Ticket URL:  https://tick.globalnoc.iu.edu/MRcgi/MRlogin.pl?DL=19118DA101
>>
>

comment:4 Changed 6 years ago by lnevers@bbn.com

Still unable to verify the fix to the path. Had several attempts and all fail with:

VLAN PCE(PCE_CREATE_FAILED): 'There are no VLANs available on link 
ion.internet2.edu:rtr.newy:ae0:bbn  on reservation ion.internet2.edu-22331 
in VLAN PCE'

comment:5 Changed 6 years ago by xyang@maxgigapop.net

Two slices had taken 3748 (ig-gpo-ig-utah ) and 3749 (pg-utah-ig-gpo). Only 3747 was left. It may take SCS several times to hit that left one.

comment:6 Changed 6 years ago by Aaron Helsinger

Component: STITCHINGI2AM

comment:7 Changed 4 years ago by hdempsey@bbn.com

Resolution: wontfix
Status: newclosed

ION AM has been decomissioned.

Note: See TracTickets for help on using tickets.