Opened 12 years ago

Closed 12 years ago

#91 closed (fixed)

gmoc.py XML data submission fails with an XML validation error

Reported by: chaos@bbn.com Owned by: somebody
Priority: major Milestone: gmoc.py data submission debugging
Component: Clients Version:
Keywords: v0.6.3 Cc:
Dependencies:

Description

When i try to report FOAM data using the new gmoc.py, the code: 

data = gmoc.GMOCClient(
         serviceURL = config['GMOC_REL_URL'],
         username = config['SITENAME'],
         password = gmoc_passwd,
       )
if len(sys.argv) > 1 and sys.argv[1] == '-d':
  data.debugLevel = gmoc.GMOC_DEBUG_VERBOSE
...
result = data.store(pop)
if result != 0:
  print "Attempted to submit relational data, but received: %s" % result
  sys.exit(1)

run with the -d argument, yields: 

<?xml version="1.0" ?>
<gmoc_topology time="1347415973" version="4">
        <pop name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab"/>
        <aggregate name="foam5.gpolab.bbn.com:3626" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" type="foam" version="0.8.2">
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1348992000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr:b72b0a06-beb9-4be2-94ba-cfbd8654ad82" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr" slice_uuid="None" state="Up" uuid="b72b0a06-beb9-4be2-94ba-cfbd8654ad82">
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" type="flowspace"/>
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1348992000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath:9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath" slice_uuid="None" state="Up" uuid="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a">
                        <resource_mapping local_name="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
        </aggregate>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
        <net_topology/>
</gmoc_topology>

Supplied XML does not validate against GMOC data exchange schema

Attempted to submit relational data, but received: 16

Change History (8)

comment:1 Changed 12 years ago by chaos@bbn.com

Keywords: v0.6.3 added

comment:2 Changed 12 years ago by chaos@bbn.com

Milestone: → gmoc.py data submission debugging

comment:3 Changed 12 years ago by chaos@bbn.com

I got another, different, example of this today, when trying to report site POP/ORG/LOCATION/CONTACT info: codepad is at ​http://codepad.org/uiEh0rp1.

comment:4 Changed 12 years ago by chaos@bbn.com

Kevin put a new schema file in place, and now i'm getting an internal server error when i try to submit either foam relations or site metadata. Here's the site metadata example, since it's shorter: 

tau-verde,[~],11:15(1)$ sudo -u ganglia ./report_site_metadata -d
<?xml version="1.0" ?>
<gmoc_topology time="1348604127" version="4">
        <location name="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St">
                <address city="Cambridge" country="US" postal_code="02138" state="MA" street1="10 Moulton St."/>
        </location>
        <contact email="gpo-infra@geni.net" given_name="GPO" last_name="Infrastructure Group" urn="urn:publicid:IDN+gmoc.geni.net+contact+gpo-infra_geni.net"/>
        <organization location="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St" name="urn:publicid:IDN+gmoc.geni.net+organization+BBN" primary_contact="urn:publicid:IDN+gmoc.geni.net+contact+gpo-infra_geni.net" type="campus"/>
        <pop location="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St" name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" operator="urn:publicid:IDN+gmoc.geni.net+organization+BBN"/>
        <net_topology/>
</gmoc_topology>

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 root@localhost and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.2.3 (Red Hat) Server at gmoc-db2.grnoc.iu.edu Port 443</address>
</body></html>

Result: 16

comment:5 Changed 12 years ago by chaos@bbn.com

Ah, in fact, all submissions to gmoc-db2 are now failing, including the cron submissions which used the older client. For instance: 

From: ganglia@boss.pgeni1.gpolab.bbn.com (Cron Daemon)
Date: Tue, 25 Sep 2012 11:15:02 -0400 (EDT)
To: ganglia@boss.pgeni1.gpolab.bbn.com
Subject: [gpo-ops] Cron <ganglia@nicea>
        /usr/sysadmin/bin/report_protogeni_relations

500 Internal Server Error: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">   
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 root@localhost and inform them of the time the error occurred,                 
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.2.3 (Red Hat) Server at gmoc-db2.grnoc.iu.edu Port            
+443</address>
</body></html>
 at /usr/sysadmin/bin/report_protogeni_relations line 9
        main::__ANON__('500 Internal Server Error: <!DOCTYPE HTML PUBLIC "-//IETF//DT...') called at /usr/sysadmin/bin/report_protogeni_relations line 198

comment:6 Changed 12 years ago by chaos@bbn.com

With version 0.6.4, site metadata submission now works: 

tau-verde,[~],13:10(0)$ sudo -u ganglia ./report_site_metadata -d
<?xml version="1.0" ?>
<gmoc_topology time="1348611033" version="4">
        <location name="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St">
                <address city="Cambridge" country="US" postal_code="02138" state="MA" street1="10 Moulton St."/>
        </location>
        <contact email="gpo-infra@geni.net" given_name="GPO" last_name="Infrastructure Group" urn="urn:publicid:IDN+gmoc.geni.net+contact+gpo-infra_geni.net"/>
        <organization location="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St" name="urn:publicid:IDN+gmoc.geni.net+organization+BBN" primary_contact="urn:publicid:IDN+gmoc.geni.net+contact+gpo-infra_geni.net" type="campus"/>
        <pop location="urn:publicid:IDN+gmoc.geni.net+location+10_Moulton_St" name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" operator="urn:publicid:IDN+gmoc.geni.net+organization+BBN"/>
</gmoc_topology>

tau-verde,[~],13:10(0)$

comment:7 Changed 12 years ago by chaos@bbn.com

With version 0.6.4, foam relational submission still fails for me, with a different error now: 

tau-verde,[~],13:10(0)$ sudo -u ganglia ./report_foam_relations -d
<?xml version="1.0" ?>
<gmoc_topology time="1348611077" version="4">
        <pop name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab"/>
        <aggregate name="foam5.gpolab.bbn.com:3626" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" type="foam" version="0.8.2">
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1362114000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr:b72b0a06-beb9-4be2-94ba-cfbd8654ad82" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr" slice_uuid="None" state="Up" uuid="b72b0a06-beb9-4be2-94ba-cfbd8654ad82">
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" type="flowspace"/>
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1362114000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath:9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath" slice_uuid="None" state="Up" uuid="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a">
                        <resource_mapping local_name="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
        </aggregate>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
</gmoc_topology>

Use of uninitialized value in hash element at xchange2db3 line 386.
DBD::mysql::st execute failed: Cannot add or update a child row: a foreign key constraint fails (`gmoc/point_of_presence`, CONSTRAINT `address_point_of_presence_fk` FOREIGN KEY (`address_id`) REFERENCES `address` (`address_id`) ON DELETE NO ACTION ON UPDATE NO ACTION) at xchange2db3 line 390.
Could not execute statement INSERT IGNORE INTO point_of_presence (name, address_id, operator_org_id, admin_org_id) VALUES (?, ?, ?, ?).


Attempted to submit relational data, but received: 16

comment:8 Changed 12 years ago by chaos@bbn.com

Resolution: → fixed
Status: new → closed

Kevin found that the POP database table was being keyed by POP and org both, so submitting a stub POP with no organization was failing. We discussed this and agreed that the intention is that POP names are globally unique (the operating organization of a POP is just the primary contact for that POP --- it's fine to have individual SAs/AMs/resources in a POP which are operated by someone else, and that is why each entity has its own POP and organization defined). Kevin made that change, and now i get: 

tau-verde,[~],11:11(0)$ sudo -u ganglia ./report_foam_relations -d
<?xml version="1.0" ?>
<gmoc_topology time="1349122476" version="4">
        <pop name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab"/>
        <aggregate name="foam5.gpolab.bbn.com:3626" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" type="foam" version="0.8.2">
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1362114000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr:b72b0a06-beb9-4be2-94ba-cfbd8654ad82" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-montreve-macaddr" slice_uuid="None" state="Up" uuid="b72b0a06-beb9-4be2-94ba-cfbd8654ad82">
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" type="flowspace"/>
                        <resource_mapping local_name="b72b0a06-beb9-4be2-94ba-cfbd8654ad82" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
                <sliver approved="1" created="1342549645" creator="urn:publicid:IDN+gmoc.geni.net+contact+site-gpolab-grandfathered-slice" expires="1362114000" local_name="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath:9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+chaos-poblano1101-datapath" slice_uuid="None" state="Up" uuid="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a">
                        <resource_mapping local_name="9c3476a5-dfe9-4d8c-9d2d-3b20d846f36a" resource="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" type="flowspace"/>
                </sliver>
        </aggregate>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
        <resource aggregate="foam5.gpolab.bbn.com:3626" description="" name="urn:publicid:IDN+openflow:foam:foam5.gpolab.bbn.com+datapath+04:4d:00:26:f1:3f:3b:00" organization="urn:publicid:IDN+gmoc.geni.net+organization+BBN" pop="urn:publicid:IDN+gmoc.geni.net+pop+gpolab" state="Unknown" type="datapath"/>
</gmoc_topology>

foam5.gpolab.bbn.com

or, in non-debug mode: 

tau-verde,[~],11:14(0)$ sudo -u ganglia ./report_foam_relations   
foam5.gpolab.bbn.com
tau-verde,[~],11:16(0)$

That's exactly what i want, so i think that's it for this ticket. If data submission isn't getting into the DB correctly, i'll create a new ticket for that.

Note: See TracTickets for help on using tickets.