Opened 10 years ago
Last modified 10 years ago
#1197 new
MAX and ION aggregates report Internal API error for out-of-sync credentials sometimes
Reported by: | lnevers@bbn.com | Owned by: | xyang@maxgigapop.net |
---|---|---|---|
Priority: | major | Milestone: | |
Component: | STITCHING | Version: | SPIRAL6 |
Keywords: | Network Stitching | Cc: | tmack@CS.Princeton.EDU |
Dependencies: |
Description
Both ION and MAX aggregates have reported the following type of exception:
14:24:26 INFO stitch.Aggregate: Writing to '/tmp/ig-nyse-ig-miss-createsliver-request-11-ion-internet2-edu.xml' 14:24:26 INFO stitch.Aggregate: Stitcher doing createsliver at http://geni-am.net.internet2.edu:12346 14:24:26 ERROR omni: {'output': "Internal API error: 'NoneType' object is not iterable", 'geni_api': 2, 'code': {'am_type': 'sfa', 'geni_code': 5, 'am_code': 5}, 'value': ''} 14:24:26 INFO stitch.Aggregate: Got AMAPIError doing createsliver ig-nyse-ig-miss at <Aggregate urn:publicid:IDN+ion.internet2.edu+authority+am>: AMAPIError: Error from Aggregate: code 5. sfa AM code: 5: Internal API error: 'NoneType' object is not iterable.
There is no manifest files for the above failure. The problem does not occur on the next try.
On 2/10/14 4:53 PM, Xi Yang wrote:
> I looked back into the sfa.log at MAX AM. The trace stack is below: > 2014-02-06 13:25:17,573 - ERROR - XmlrpcApi.handle has caught Exception BEG TRACEBACK > Traceback (most recent call last): > File "/usr/lib/python2.5/site-packages/sfa/server/xmlrpcapi.py", line 153, in handle > result = self.call(source, method, *args) > File "/usr/lib/python2.5/site-packages/sfa/server/xmlrpcapi.py", line 125, in call > return function(*args) > File "/usr/lib/python2.5/site-packages/sfa/util/method.py", line 79, in __call__ > result = self.call(*args, **kwds) > File "/usr/lib/python2.5/site-packages/sfa/methods/CreateSliver.py", line 58, in call > result = self.api.manager.CreateSliver(self.api, slice_xrn, creds, rspec, users, options) > File "/usr/lib/python2.5/site-packages/sfa/managers/aggregate_manager_max.py", line 380, in CreateSliver > ret = self.create_slice(api, slice_xrn, creds, rspec_string, users) > File "/usr/lib/python2.5/site-packages/sfa/managers/aggregate_manager_max.py", line 247, in create_slice > self.prepare_slice(api, xrn, cred, users) > File "/usr/lib/python2.5/site-packages/sfa/managers/aggregate_manager_max.py", line 175, in prepare_slice > site = slices.verify_site(hrn, slice_record, peer, sfa_peer) > File "/usr/lib/python2.5/site-packages/sfa/plc/plslices.py", line 288, in verify_site > sites = self.driver.shell.GetSites(login_base) > File "/usr/lib/python2.5/site-packages/sfa/plc/plshell.py", line 83, in func > result=getattr(self.proxy, actual_name)(self.plauth, *args, **kwds) > File "/usr/share/plc_api/PLC/Shell.py", line 52, in __call__ > return self.func(*args, **kwds) > File "/usr/share/plc_api/PLC/Method.py", line 97, in __call__ > result = self.call(*args, **kwds) > File "/usr/share/plc_api/PLC/Methods/GetSites.py", line 33, in call > return Sites(self.api, site_filter, return_fields) > File "/usr/share/plc_api/PLC/Sites.py", line 274, in __init__ > self.selectall(sql) > File "/usr/share/plc_api/PLC/Table.py", line 426, in selectall > for row in self.api.db.selectall(sql, params): > File "/usr/share/plc_api/PLC/PostgreSQL.py", line 208, in selectall > labels = [column[0] for column in self.description] > TypeError: 'NoneType' object is not iterable > > This problem is similar to what I have seen with first-timer project. It is an SFA-MyPLC problem that I have no idea how it happens. > Looks like SFA has some issue when out-of-sync site credential between the Portal clearing house and MyPLC in some rare cases. > I have no clue on either the cause or the pattern.
Note: See
TracTickets for help on using
tickets.
Adding Tony Mack (tmack@CS.Princeton.EDU) to the CC: as requested by Xi.