Opened 11 years ago
Closed 11 years ago
#74 closed (fixed)
InstaGENI Compute resource aggregate in GMOC no longer shows Slivers and Resources data
| Reported by: | lnevers@bbn.com | Owned by: | sedwards@bbn.com |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | AM | Version: | SPIRAL5 |
| Keywords: | Cc: | ||
| Dependencies: |
Description
The GPO InstaGENI rack (instageni.gpolab.bbn.com:12369) is now part of the the list of aggregates at:
But there are no details for the for "Slivers" and "Resources", see :
Also checked the Utah aggregate (utah.geniracks.net:12369) and it also has no "Slivers" and no "Resources".
This problem may be related to the addition of port 12369 URL change. Sarah is investigating.
Change History (9)
comment:1 Changed 11 years ago by
| Status: | new → assigned |
|---|
comment:2 Changed 11 years ago by
comment:3 Changed 11 years ago by
One more data point...
I found a resource that looks like it is an IG node on the GPO rack: https://gmoc-db.grnoc.iu.edu/protected-openid/index.pl?method=node_details;node=urn%3Apublicid%3AIDN%2Binstageni.gpolab.bbn.com%2Bnode%2Bpc1
But it doesn't like it has an aggregate at all.
Also, we switched the reported name on the instageni.gpolab.com rack pretty quickly after reporting it at all. Whereas utah.geniracks.net had been reporting for awhile before we made the switch.
comment:4 Changed 11 years ago by
I spoke to the GMOC folks and they are going to look at this soon and at least nail down the issue before the next release (which is supposed to be this week).
I created a GMOC ticket here: http://groups.geni.net/gmoc/ticket/146
comment:5 Changed 11 years ago by
Mitch looked into what was going on and reminded me of some salient facts which gave me an idea. I'll try that out and report back.
comment:6 Changed 11 years ago by
Ok. Fixed some stuff with my code but when I submit I get an error. Chaos is actively debugging with the GMOC folks. So I'll work on debugging this a bit later. I'll also may need to update my version of gmoc.py.
<?xml version="1.0" ?> <gmoc_topology time="1358904776" version="4"> <pop name="urn:publicid:IDN+gmoc.geni.net+pop+gpolab"/> <aggregate name="instageni.gpolab.bbn.com:12369"> <sliver approved="false" created="1356059087" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs" expires="1359491450" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+144" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs" slice_uuid="ee66ba9f-00fe-11e1-bf86-000c29f89f7b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+144" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc3" type="host"/> </sliver> <sliver approved="false" created="1357668991" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491485" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+252" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemoprimary" slice_uuid="aa964ac2-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+252" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> <sliver approved="false" created="1357669288" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491485" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+256" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemobackup" slice_uuid="94207f53-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+256" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358860100" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491497" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-EXP-7" slice_uuid="1f103145-64a5-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> <sliver approved="false" created="1355445356" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs" expires="1359491530" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+10" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15" slice_uuid="ff9a2477-a41c-11e0-acc6-000c29f89f7b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+10" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> <sliver approved="false" created="1355451614" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs" expires="1359491530" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+14" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16" slice_uuid="30ec474a-a41d-11e0-acc6-000c29f89f7b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+14" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> <sliver approved="false" created="1357324609" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491546" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+211" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon" slice_uuid="88a518f0-af53-11e1-8a62-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+211" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1357668991" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491546" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+252" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemoprimary" slice_uuid="aa964ac2-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+252" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> <sliver approved="false" created="1357669288" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491546" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+256" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemobackup" slice_uuid="94207f53-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+256" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358719104" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491558" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+713" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-EXP-5-exp1" slice_uuid="1db446f5-635d-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+713" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358860100" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491558" local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-EXP-7" slice_uuid="1f103145-64a5-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+766" resource="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" type="vm"/> </sliver> </aggregate> </gmoc_topology> DBD::mysql::st execute failed: Duplicate entry '-1-1578-301' for key 1 at xchange2db3 line 1046. Could not execute statement INSERT INTO sliver_resource_map (end_version_id, sliver_id, resource_id, name, start_version_id, type) VALUES (-1, ?, ?, ?, ?, ?). <?xml version="1.0" ?> <gmoc_topology time="1358904777" version="4"> <pop name="urn:publicid:IDN+gmoc.geni.net+pop+Utah"/> <aggregate name="utah.geniracks.net:12369"> <sliver approved="false" created="1355931143" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs" expires="1359491450" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15762" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16" slice_uuid="30ec474a-a41d-11e0-acc6-000c29f89f7b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15762" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> </sliver> <sliver approved="false" created="1355932347" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs" expires="1359491450" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15770" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15" slice_uuid="ff9a2477-a41c-11e0-acc6-000c29f89f7b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15770" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> </sliver> <sliver approved="false" created="1348838938" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491466" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+11075" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755" slice_uuid="483eb6c2-7ffd-11e1-8136-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+11075" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1357669388" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491485" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15882" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemobackup" slice_uuid="94207f53-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15882" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358860132" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491497" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-EXP-7" slice_uuid="1f103145-64a5-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> </sliver> <sliver approved="false" created="1358879176" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491497" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-202527" slice_uuid="dcdfe7b7-64d1-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358879277" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491497" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-2026360" slice_uuid="05de9c5f-64d2-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> </sliver> <sliver approved="false" created="1357850511" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491515" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15997" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemoprimary" slice_uuid="aa964ac2-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15997" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> </sliver> <sliver approved="false" created="1357669388" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491546" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15882" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tipdemobackup" slice_uuid="94207f53-59cb-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+15882" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358860132" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491558" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-EXP-7" slice_uuid="1f103145-64a5-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16304" resource="urn:publicid:IDN+utah.geniracks.net+node+pc5" type="vm"/> </sliver> <sliver approved="false" created="1358879176" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491558" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-202527" slice_uuid="dcdfe7b7-64d1-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16317" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> <sliver approved="false" created="1358879277" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expires="1359491558" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-2026360" slice_uuid="05de9c5f-64d2-11e2-bf18-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+16326" resource="urn:publicid:IDN+utah.geniracks.net+node+pc3" type="vm"/> </sliver> <sliver approved="false" created="1351079507" creator="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty" expires="1359491575" local_name="urn:publicid:IDN+utah.geniracks.net+sliver+13957" slice_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon" slice_uuid="88a518f0-af53-11e1-8a62-000c29cca98b" state="Unknown" uuid=""> <resource_mapping local_name="urn:publicid:IDN+utah.geniracks.net+sliver+13957" resource="urn:publicid:IDN+utah.geniracks.net+node+pc1" type="vm"/> </sliver> </aggregate> </gmoc_topology> DBD::mysql::st execute failed: Duplicate entry '-1-1582-182' for key 1 at xchange2db3 line 1046. Could not execute statement INSERT INTO sliver_resource_map (end_version_id, sliver_id, resource_id, name, start_version_id, type) VALUES (-1, ?, ?, ?, ?, ?).
comment:7 Changed 11 years ago by
Chaos looked over this and we agree there seems to be a problem with the DB at the GMOC end.
We'll propose debugging this after the DB cutover early next week.
comment:8 Changed 11 years ago by
Did a few more rounds of bug fixes. I'm able to submit code again, but it's not showing up in the UI interface and Chaos can't download it.
I sent mail to the GMOC guys and will follow up tomorrow.
comment:9 Changed 11 years ago by
| Resolution: | → fixed |
|---|---|
| Status: | assigned → closed |
This problem is resolved. Resolution verified by IG-CT-5 Monitoring tests for GPO, Northwestern and Kentucky. Closing ticket.
Note: See
TracTickets for help on using
tickets.
Ok. So I originally thought this was going to be a problem with my aggregate/resource code which Leigh is running. However, closer inspection leads me to believe this may be a subtle gmoc client data submission bug.
In particular, the UI doesn't show there being any resources for these two aggregates:
These formerly were reported as:
In particular I now notice that the old aggregate
utah.geniracks.net:80hasn't updated for awhile: https://gmoc-db.grnoc.iu.edu/protected-openid/index.pl?method=aggregate_details;aggregate=utah.geniracks.net%3A80But the resources for it are listed as having reported today: https://gmoc-db.grnoc.iu.edu/protected-openid/index.pl?method=aggregate_details;aggregate=utah.geniracks.net%3A80
Likewise the old aggregate
instageni.gpolab.bbn.com:80hasn't updated in awhile: https://gmoc-db.grnoc.iu.edu/protected-openid/index.pl?method=aggregate_details;aggregate=instageni.gpolab.bbn.com%3A80HOWEVER there are no resources for it listed at all: https://gmoc-db.grnoc.iu.edu/protected-openid/index.pl?method=aggregate_details;aggregate=instageni.gpolab.bbn.com%3A80
The first case (the
utah.geniracks.netcase) makes me think the link between the aggregate and the resources wasn't updated when the aggregate name changed. However, the second case doesn't clearly support that thinking.I remember that we had some discussions about things like what can be updated once it's reported when Kevin built the client so this may be somewhat expected behavior. That said, I'm not sure how to proceed and get the resources linked to the new aggregate. I've emailed Kevin and Mitch to get them to poke at the DB and see if they can provide additional guidance here.