Deleting a failed slivers at ION does not clean up circuits that had successed

[lnevers@bbn.com]

Hello,

While working on reproducing ticket #1353, noticed a problem with OSCAR circuits not being deleted. The sequence used to see the problem:

1. In GENI portal, create stitched sliver "iozcelik1" and leave it running.
2. With omni create a stitched sliver "lnilker" using the same RSpec as portal.
3. The omni sliver fails due to lack of resources at GPO, stitcher request delete at all sites.
4. Deleted portal sliver "iozcelik1"
5. Create the omni sliver again "lnilker", at this point I can see that ION has the old sliver record for the sliver that should have been deleted.
6. Checked sliver status for the ION "lnilker" sliver:

         "geni_urn": "urn:publicid:IDN+ion.internet2.edu+sliver+lnilker_vlan_ion.internet2.edu-143451",
         "geni_error": "",
         "geni_status": "ready"
   
         "geni_urn": "urn:publicid:IDN+ion.internet2.edu+sliver+lnilker_vlan_ion.internet2.edu-143461",
         "geni_error": "VLAN cancelled by rollback from contingent failure",
         "geni_status": "failed"
   
         "geni_urn": "urn:publicid:IDN+ion.internet2.edu+sliver+lnilker_vlan_ion.internet2.edu-143471",
         "geni_error": "VLAN PCE(PCE_CREATE_FAILED): 'There are no VLANs available on link ion.internet2.edu:rtr.newy:et-5/0/0:gpo-ig  on reservation ion.internet2.edu-143471 in VLAN PCE'",
         "geni_status": "failed"
   
         "geni_urn": "urn:publicid:IDN+ion.internet2.edu+sliver+lnilker_vlan_ion.internet2.edu-143481",
         "geni_error": "VLAN cancelled by rollback from contingent failure",
         "geni_status": "failed"
   
         "geni_urn": "urn:publicid:IDN+ion.internet2.edu+sliver+lnilker_vlan_ion.internet2.edu-143491",
         "geni_error": "VLAN cancelled by rollback from contingent failure",
         "geni_status": "failed"
   

7. Checked OSCAR web interface and found the active GRI ion.internet2.edu-143451.
8. Issued a deletesliver for "lnilker" at ion aggregate.
9. Issues sliverstatus at ION to verify sliver cleanup. Result "No existing record" as expected.
10. Checked OSCAR, and the GRI ion.internet2.edu-143451 is still active and not deleted.

Looking at 6-10 above, one can see if the a sliver has an overall status of "failed" but some of the circuits are "ready" requesting the deletion of the sliver does not clean up the circuit which was active.

Assigning it to Xi, please re-assign if this is an ION problem disagree.

Luisa

[lnevers@bbn.com]

This problem is still happening and I have been deleting my left-over circuits from time to time. Occurrence is very low.