Creatiing 2 slivers with 25 VMs each results in numerous errors for VMs and links

[lnevers@bbn.com]

In attempting to run EG-EXP-3 "ExoGENI Single Site 100 VM Test" started by requesting one slice with two slivers:

• sliver 1: 25 VMs in the BBN rack from the BBN SM with 38 links
• Sliver 2: 25 VMs in the BBN rack from the ExoSM with 38 links

Attaching the sliverstatus and the listresources for each sliver, and the Request RSpec used.

Following are the failures for sliver 1 (25 VMs from BBN SM):

'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan72',
{'geni_error': 'Reservation 22e9d243-0c96-4e5d-94f3-10601a69d641 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 42AA4F6C [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan59',
{'geni_error': 'Reservation 78a199c4-2d95-46d8-8f21-5b42dacf7847 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 448CE9CA [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-10',
{'geni_error': 'Reservation d51fdd91-4c87-43a2-95a3-22a791a90971 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: D6F50072 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan58',
{'geni_error': 'Reservation 51a7d774-a5ef-427c-83e8-729163758cfa (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 953672C0 [1]: unable to create instance: exit code 1, Traceback (most recent call last):\n  File "/usr/bin/euca-describ
e-images", line 172, in <module>\n    main()\n  File "/usr/bin/euca-describe-images", line 166, in main\n    euca.display_error_and_exit(\'%s\' % ex)\n  File "/usr/lib/python2
.6/site-packages/euca2ools/__init__.py", line 1436, in display_error_and_exit\n    dom = minidom.parseString(msg)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/mini
dom.py", line 1925, in parseString\n    return expatbuilder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseSt
ring\n    return builder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString\n    parser.Parse(string, True
)\nxml.parsers.expat.ExpatError: mismatched tag: line 6, column 2\nTue Aug 14 03:49:20 UTC 2012: start.sh: check_image_size returned: check_image_size unable to read manifest 
for ami-0000000f\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-14',
{'geni_error': 'Reservation bbb85391-380c-4353-9333-d37d3412f8b6 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 1B07D53E [1]: unable to create instance: exit code 1, Traceback (most recent call last):\n  File "/usr/bin/euca-describ
e-images", line 172, in <module>\n    main()\n  File "/usr/bin/euca-describe-images", line 166, in main\n    euca.display_error_and_exit(\'%s\' % ex)\n  File "/usr/lib/python2
.6/site-packages/euca2ools/__init__.py", line 1436, in display_error_and_exit\n    dom = minidom.parseString(msg)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/mini
dom.py", line 1925, in parseString\n    return expatbuilder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseSt
ring\n    return builder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString\n    parser.Parse(string, True
)\nxml.parsers.expat.ExpatError: mismatched tag: line 6, column 2\nTue Aug 14 03:52:46 UTC 2012: start.sh: check_image_size returned: check_image_size unable to read manifest 
for ami-0000000f\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-15',
{'geni_error': 'Reservation 500d185f-ed57-46d3-a42c-97cc29ce4d40 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: B2FA5629 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-12',
{'geni_error': 'Reservation 98c31c9a-69f2-4655-b0e1-7be2f0b4a847 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: C0099945 [1]: unable to create instance: exit code 1, Traceback (most recent call last):\n  File "/usr/bin/euca-describ
e-images", line 172, in <module>\n    main()\n  File "/usr/bin/euca-describe-images", line 166, in main\n    euca.display_error_and_exit(\'%s\' % ex)\n  File "/usr/lib/python2
.6/site-packages/euca2ools/__init__.py", line 1436, in display_error_and_exit\n    dom = minidom.parseString(msg)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/mini
dom.py", line 1925, in parseString\n    return expatbuilder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseSt
ring\n    return builder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString\n    parser.Parse(string, True
)\nxml.parsers.expat.ExpatError: mismatched tag: line 6, column 2\nTue Aug 14 04:38:20 UTC 2012: start.sh: check_image_size returned: check_image_size unable to read manifest 
for ami-0000000f\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan3',
{'geni_error': 'Reservation ca829f28-2d64-42a2-ab11-5b2351d4c44f (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 52A14EC1 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-19',
{'geni_error': 'Reservation 55442230-8eca-4990-bfed-69eb1e6eabdf (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 5B68C25E [1]: unable to create instance: exit code 1, \n',
{'geni_error': 'Reservation 7e69f3a5-4569-439b-ab27-57d6fc1af12d (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: FE3C261 [1]: unable to create instance: exit code 1, Traceback (most recent call last):\n  File "/usr/bin/euca-describe
-images", line 172, in <module>\n    main()\n  File "/usr/bin/euca-describe-images", line 166, in main\n    euca.display_error_and_exit(\'%s\' % ex)\n  File "/usr/lib/python2.
6/site-packages/euca2ools/__init__.py", line 1436, in display_error_and_exit\n    dom = minidom.parseString(msg)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/minid
om.py", line 1925, in parseString\n    return expatbuilder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseStr
ing\n    return builder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString\n    parser.Parse(string, True)
\nxml.parsers.expTue Aug 14 03:55:35 UTC 2012: start.sh: check_image_size returned: check_image_size unable to read manifest for ami-0000000f\nat.ExpatError: mismatched tag: l
ine 6, column 2\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-7',
{'geni_error': 'Reservation 1995bc0d-2dcc-47a4-b038-d6234f3389b2 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: A2409ABE [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-9',
{'geni_error': 'Reservation febef6c5-3e03-4125-af31-24c4a8c4b733 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: C653E3B7 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-3',
{'geni_error': 'Reservation 3896a844-36e9-448a-b7a8-6cc35498f562 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: F07DE0CF [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-4',
{'geni_error': 'Reservation 15804822-899b-4190-a06d-963c2c2c2ed2 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 603480F0 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-5',
{'geni_error': 'Reservation bf166cfa-8812-474a-aa8a-e47966751b8b (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 7CFC92B1 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-6',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan68',
{'geni_error': 'Reservation d87f14ca-2fa6-4337-a896-2e61890461d4 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: B505D15C [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-0',
{'geni_error': 'Reservation 2efb6912-34ed-46d1-821f-8591e2bd7461 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 57EA14F3 [1]: unable to create instance: exit code 1, Traceback (most recent call last):\n  File "/usr/bin/euca-describ
e-images", line 172, in <module>\n    main()\n  File "/usr/bin/euca-describe-images", line 166, in main\n    euca.display_error_and_exit(\'%s\' % ex)\n  File "/usr/lib/python2
.6/site-packages/euca2ools/__init__.py", line 1436, in display_error_and_exit\n    dom = minidom.parseString(msg)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/mini
dom.py", line 1925, in parseString\n    return expatbuilder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseSt
ring\n    return builder.parseString(string)\n  File "/usr/lib64/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString\n    parser.Parse(string, True
)\nxml.parsers.expat.ExpatError: mismatched tag: line 6, column 2\nTue Aug 14 04:41:32 UTC 2012: start.sh: check_image_size returned: check_image_size unable to read manifest 
for ami-0000000f\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-20',
{'geni_error': 'Reservation f173c56c-f4d8-4f93-ba1c-31e63356a4a7 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 303DEDE1 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-2',
{'geni_error': 'Reservation d70f6665-4fe1-4e3d-be8e-887fa68c05e2 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 9E99360A [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-21',
{'geni_error': 'Reservation 788f1c07-d9bf-4876-8200-c0841d436045 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: DA49E0B1 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-22',
{'geni_error': 'Reservation 8a007e2d-402f-4db1-b575-43fcdcc1d57d (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 37E91344 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-1',
{'geni_error': 'Reservation b8dad923-c840-4ea4-9ec8-c170b603c9ff (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 471BC0D6 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-23',
{'geni_error': 'Reservation f67a73c9-023f-4a8e-ba07-57c25334c123 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: B41F78EA [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-24',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-26',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#lan25',
{'geni_error': 'Reservation ef6bd285-16f4-4283-82f6-455f8cb92a54 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 66808A22 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+bb74c216-117c-4345-81d7-04f19c9884a6#VM-27',

Following are the failures for sliver 2 (25 VMs from the ExoSM):

'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan26',
{'geni_error': 'Reservation 20e7cf93-c1a5-42b1-bb4b-81a1c7b29346 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: C5A72220 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan29',
{'geni_error': 'Reservation 7d8a2fe8-9781-40b7-bfa4-6d58157ce9c9 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 38CE1339 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-6',
{'geni_error': 'Reservation feebd7da-db12-400c-b887-2723e34c3dc3 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 2265B6A [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan25',
{'geni_error': 'Reservation 8e8b63c0-dfd4-434c-90d7-8d7598f617d8 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 15D95B5F [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-7',
{'geni_error': 'Reservation eb427777-0024-439b-9285-b5a4b629e94b (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 20C21370 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-2',
{'geni_error': 'Reservation e5be372b-1fa1-4f6f-ab5c-bd5cfd6e8372 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 5A461D4B [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-1',
{'geni_error': 'Reservation 443b5971-ecda-4f2c-ab25-7cd62c8e961e (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 31FB83C7 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-4',
{'geni_error': 'Reservation 69c8a703-1dd9-4cb8-8d22-0ffcf7b6ec22 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 6CBAE023 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan4',
{'geni_error': 'Reservation a652ff97-76b8-4fca-8160-7e5e5b74c61c (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 70DCDB86 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan77',
{'geni_error': 'Reservation 8e00c67a-f4e1-4c9e-a1ef-ff0e66a9de34 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 448F6581 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan79',
{'geni_error': 'Reservation a13ba0d9-cea4-4879-966b-c77108236edf (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: C943080B [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan75',
{'geni_error': 'Reservation d836f9fd-3a2a-45bb-9043-9c1c6c8e8cac (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: A0D41FF9 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-12',
{'geni_error': 'Reservation 90428493-4249-4445-9faa-ad8f38075d73 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 3B35EC73 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-24',
{'geni_error': 'Reservation ac6ddfa7-6411-4614-8e2c-9ddffaa5bca4 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: E4377B59 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-13',
{'geni_error': 'Reservation f67e091f-e8d5-44ef-832b-a4625320bf9d (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 529C98DD [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-23',
{'geni_error': 'Reservation d31ef5d7-2938-4c0b-8e29-dca13703017c (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 8EF10F35 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-10',
{'geni_error': 'Reservation 8d464e50-df32-41b5-aae4-5dabcd4fbcc3 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 9706D569 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-22',
{'geni_error': 'Reservation 5c335e52-6c2f-4e90-97fa-3dabb2fe30de (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: B8C23FCA [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-11',
{'geni_error': 'Reservation 05b30a54-b15f-4326-8b6f-ad2f7155bf7e (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: EDDA3AB3 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-21',
{'geni_error': 'Reservation 9069e2b2-973f-497d-890a-0ac433351cef (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: BBA45969 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-16',
{'geni_error': 'Reservation 353df287-9de0-41c3-8060-6d0de8136800 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 73E92E89 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-27',
{'geni_error': 'Reservation 780a278d-828e-4f6a-80f5-29534fe12ba0 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 3ADAAA83 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-26',
{'geni_error': 'Reservation 34268ae9-008d-4cda-8443-6b125a24c17a (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: C5EEAFDE [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-14',
{'geni_error': 'Reservation 2a55572d-0da1-4fbc-8c08-f854feee4ece (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: EFA39C1A [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan80',
{'geni_error': 'Reservation 36fac56a-7f3e-42fc-8ac2-9af083b8f926 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Exception during join for unit: 50597CA0 The following error occurred while executing this line:\n/opt/orca-12080/tomcat/webapps/orca/WEB-INF/handl
ers/providers/ben.no-na.tasks.xml:747: /opt/orca-12080/tomcat/webapps/orca/WEB-INF/handlers/providers/ben.no-na.tasks.xml:759: [OpenFlow.CreateSliceTask] An error occurred: Fa
iled to invoke method createSlice in class org.flowvisor.api.FVUserAPIXMLRPCImpl: A slice with this controller: bbn-hn.exogeni.net:50078 already exists, this slice (orca-73) w
ill not be created\n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan84',
{'geni_error': "Reservation c7f17db2-7467-4892-93f1-a6519d67cd64 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Exception during join for unit: 6C1635BA The following error occurred while executing this line:\n/opt/orca-12080/tomcat/webapps/orca/WEB-INF/handl
ers/providers/ben.no-na.tasks.xml:747: /opt/orca-12080/tomcat/webapps/orca/WEB-INF/handlers/providers/ben.no-na.tasks.xml:759: [OpenFlow.CreateSliceTask] An error occurred: Fa
iled to read server's response: Connection refused\n",
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan59',
{'geni_error': "Reservation dbd0c901-b333-427e-a963-d249a5de0e1e (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Exception during join for unit: 9E15A560 The following error occurred while executing this line:\n/opt/orca-12080/tomcat/webapps/orca/WEB-INF/handl
ers/providers/ben.no-na.tasks.xml:747: /opt/orca-12080/tomcat/webapps/orca/WEB-INF/handlers/providers/ben.no-na.tasks.xml:759: [OpenFlow.CreateSliceTask] An error occurred: Fa
iled to read server's response: Connection refused\n",
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan54',
{'geni_error': "Reservation 6a75cfb3-dce8-4c9f-8a55-81f860abb4ae (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Exception during join for unit: A2340EFC The following error occurred while executing this line:\n/opt/orca-12080/tomcat/webapps/orca/WEB-INF/handl
ers/providers/ben.no-na.tasks.xml:747: /opt/orca-12080/tomcat/webapps/orca/WEB-INF/handlers/providers/ben.no-na.tasks.xml:759: [OpenFlow.CreateSliceTask] An error occurred: Fa
iled to read server's response: Connection refused\n",
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan58',
{'geni_error': "Reservation bf891bec-7453-431c-922e-3f94cc7c959e (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Exception during join for unit: 43C5F8C7 The following error occurred while executing this line:\n/opt/orca-12080/tomcat/webapps/orca/WEB-INF/handl
ers/providers/ben.no-na.tasks.xml:747: /opt/orca-12080/tomcat/webapps/orca/WEB-INF/handlers/providers/ben.no-na.tasks.xml:759: [OpenFlow.CreateSliceTask] An error occurred: Fa
iled to read server's response: Connection refused\n",
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#lan55',
{'geni_error': 'Reservation 924a3845-2fa7-4e91-a361-8e6ef081072a (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver) is in state [Failed,None], er
r=resources failed to join: Error during join for unit: 6506F559 [1]: unable to create instance: exit code 1, \n',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ca9e4a3-185b-4adf-8c10-5c52095b681c#VM-0',
 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+25sliver'}

[lnevers@bbn.com]

Tried to create experiments with 25 VMs grid as described in this ticket. First experiment at ExoSM ran into resource availability issue and was deleted.

The second 25 VM grid experiment was created at BBN SM and sliverstatus was still configuring 40 minutes after sliver creation. Checked again on sliver 70 minutes after creation and found that sliver had failures. Twenty of the VM showed the following error:

"geni_error": "Reservation 59105c68-b74a-42a3-a339-302ba387acc8 
(Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+ln25) is in state 
[Failed,None], err=resources failed to join: Error during join for unit: 
2FB7B3C4 [1]: unable to create instance: exit code 1, \n", 

All 38 LANs were active.

[vjo@duke.edu]

Louisa,

Looks like some of your VMs did not have images specified. That would explain why your VMs failed.

Please retry, with images specified.

[lnevers@bbn.com]

I added the image and re-created the sliver 25 minutes ago... waiting.

[lnevers@bbn.com]

DONE!!! Sliver is ready, and all nodes are "Active". Yeah!!

I was able to login to all nodes but I do not have IP addresses defined in my RSpec, so I cannot verify connectivity.

(enhancement :-) It would be nice if addresses were assigned (as with !InstaGENI) when none are defined for an interface.

[vjo@duke.edu]

So - can we close this?

I.e. shouldn't the request for enhancement be its own ticket? ;)

[lnevers@bbn.com]

Yes, this ticket should be closed and enhancement will be written up as separate ticket.