Opened 10 years ago

Closed 10 years ago

#50 closed (fixed)

13 node linear sliver fails with geni_status: 'failed'

Reported by: lnevers@bbn.com Owned by: somebody
Priority: major Milestone: EG-EXP-3
Component: Experiment Version: SPIRAL4
Keywords: vm support Cc:
Dependencies:

Description

Since linear topologies work, I attempted to request all available resources at the ExoGENI rack via the BBN SM. The slice name used is "18vmlinear-local", although only 13 VM were found to be available (cannot tell before the request).

  1. Tried to request 18 VM linear topology which failed with:
     Request id: 8de94436-170e-44ab-86d5-7f99fe88597c
     Embedding workflow ERROR: 1:Insufficient resources or Unknown domain: 
     http://geni-orca.renci.org/owl/bbnvmsite.rdf#bbnvmsite/Domain/vm:vm:13! 
     Requested: vm:18 
    
  2. Then requested 13 nodes in a linear sliver which was successful:
     $ ./src/omni.py -a exobbn createsliver 18vmlinear-local ./exorspec/exo-13vm-linear.rspec
    INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+18vmlinear-local expires on 2012-06-13 03:50:07 UTC
    INFO:omni:Substituting AM nickname exobbn with URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_U
    RN
    INFO:omni:Substituting AM nickname exobbn with URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_U
    RN
    INFO:omni:Creating sliver(s) from rspec file ./exorspec/exo-13vm-linear.rspec for slice urn:publicid:IDN+pgeni.gpolab
    .bbn.com+slice+18vmlinear-local
    INFO:omni:Asked https://bbn-hn.exogeni.net:11443/orca/xmlrpc to reserve resources. Result:
    INFO:omni:<?xml version="1.0" ?>
    INFO:omni:<!-- Reserved resources for:
    	Slice: 18vmlinear-local
    	At AM:
    	URL: https://bbn-hn.exogeni.net:11443/orca/xmlrpc
     -->
    INFO:omni:<rspec type="manifest" xmlns="http://www.protogeni.net/resources/rspec/2"
    ... <lines> deleted...
    
    
    Note: Attaching sliver manifest RSpec.
  1. The sliver status was checked immediately after the sliver creation and it showed the error:
    INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+18vmlinear-local expires on 2012-06-13 03:50:07 UTC
    INFO:omni:Substituting AM nickname exobbn with URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc, 
    URN unspecified_AM_URN
    INFO:omni:Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+18vmlinear-local:
    INFO:omni:Sliver status for Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+18vmlinear-local at AM URL https://bbn-
    hn.exogeni.net:11443/orca/xmlrpc
    INFO:omni:{'geni_resources': '',
     'geni_status': 'failed',
     'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+18vmlinear-local'}
    
    

Also attaching the sliver status. Let sliver running.

Attachments (7)

18vmlinear-local-rspec-bbn-hn-exogeni-net-11443-orca.xml (16.4 KB) - added by lnevers@bbn.com 10 years ago.
18vmlinear-local-sliverstatus-bbn-hn-exogeni-net-11443-orca.json (126 bytes) - added by lnevers@bbn.com 10 years ago.
exo-13vm-linear.rspec (11.1 KB) - added by lnevers@bbn.com 10 years ago.
allVMlinear-rspec-bbn-hn-exogeni-net-11443-orca.xml (15.9 KB) - added by lnevers@bbn.com 10 years ago.
13vmlinear-rspec-bbn-hn-exogeni-net-11443-orca.xml (18.5 KB) - added by lnevers@bbn.com 10 years ago.
13linear-rspec-bbn-hn-exogeni-net-11443-orca.xml (25.1 KB) - added by lnevers@bbn.com 10 years ago.
13linear-sliverstatus-bbn-hn-exogeni-net-11443-orca.json (6.0 KB) - added by lnevers@bbn.com 10 years ago.

Download all attachments as: .zip

Change History (41)

Changed 10 years ago by lnevers@bbn.com

Changed 10 years ago by lnevers@bbn.com

Attachment: exo-13vm-linear.rspec added

comment:1 Changed 10 years ago by ahelsing@bbn.com

I notice a couple things about this manifest

  • many links are missing the vlantag element
  • many nodes have empty services tag

I suspect those two things are symptoms of whatever is failing.

comment:2 Changed 10 years ago by ibaldin@renci.org

Head node at BBN ran out of file descriptors. We've cleaned up. We do not know what has caused it - there were some old processes hanging around from previous redeployments that have now been terminated and could've been the cause.

comment:3 Changed 10 years ago by lnevers@bbn.com

Delete the earlier failed sliver and then proceeded to execute the following:

  1. Request 18 VM linear topology, to determine available VMs (13 VMs)
  2. Created a sliver with 13 VM linear (slice=Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allVMlinear)
  3. Sliver status returned:
       INFO:omni:Sliver status for Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allVMlinear 
       at AM URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc
       INFO:omni:{'geni_resources': '',
       'geni_status': 'failed',
       'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allVMlinear'}
    

Attaching sliver manifest to ticket.

Changed 10 years ago by lnevers@bbn.com

comment:4 Changed 10 years ago by lnevers@bbn.com

Deleted the previous failed sliver, then created a 13VMs sliver named "13vmlinear".

The create sliver was successful. Checked the sliver status which was "configuring" for a few minutes and then a failure was reported:

INFO:omni:Sliver status for Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13vmlinear 
at AM URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc
INFO:omni:{'geni_resources': '',
 'geni_status': 'failed',
 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13vmlinear'}

In previous attempt the failure was immediate, while now it takes a few minutes..some progress.

Attaching the sliver manifest.

Changed 10 years ago by lnevers@bbn.com

comment:5 Changed 10 years ago by ibaldin@renci.org

Give it another try. You had an old half-built slice hanging around with 5 vms. I cleaned it all up. PLEASE BE SURE TO CLOSE THE SLICE EVEN IF IT FAILS, SINCE OTHERWISE SLIVERS OF IT THAT DID SUCCEED (VLANs or VMs) WILL REMAIN ACTIVE UNTIL THEY EXPIRE.

comment:6 Changed 10 years ago by lnevers@bbn.com

I usually delete all slivers after failures, unless I am leaving them up for investigation.

I did set up a 5 node sliver yesterday which resulted in the "Unknown slice URN" error . At the time I did not delete the sliver because because the aggregate did not know the slice urn.

BTW, I just submitted a sliver request to the BBN Rack SM and it has been "configuring for approx 8 minute. I will follow up with the results.

comment:7 Changed 10 years ago by ibaldin@renci.org

It is coming up.

comment:8 Changed 10 years ago by ibaldin@renci.org

I take that back. One vlan failed again (which was the failure last time too).

comment:9 Changed 10 years ago by ibaldin@renci.org

This is a problem with floodlight not starting reliably. Please try again - I put in a temporary fix.

comment:10 Changed 10 years ago by lnevers@bbn.com

First deleted the previous sliver, then created a new slice named "allvmlinear". Created a sliver requesting all VMS, which failed as follows:

$ ./src/omni.py -a exobbn createsliver allvmlinear ./exorspec/exo-18vm-linear.rspec 

INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allvmlinear expires within 1 day on 2012-06-13 14:45:38 UTC
INFO:omni:Substituting AM nickname exobbn with URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname exobbn with URL https://bbn-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./exorspec/exo-18vm-linear.rspec for 
slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allvmlinear
INFO:omni:Asked https://bbn-hn.exogeni.net:11443/orca/xmlrpc to reserve resources. Result:
INFO:omni:<!-- Reserved resources for:
	Slice: allvmlinear
	At AM:
	URL: https://bbn-hn.exogeni.net:11443/orca/xmlrpc
 -->
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:

  Options as run:
		aggregate: exobbn
		framework: pgeni
		native: True

  Args: createsliver allvmlinear ./exorspec/exo-18vm-linear.rspec

  Result Summary: Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+allvmlinear 
expires within 1 day(s) on 2012-06-13 14:45:38 UTC
Asked https://bbn-hn.exogeni.net:11443/orca/xmlrpc to reserve resources. No manifest 
Rspec returned. Credendial Exception: javax.security.auth.login.CredentialException: 
No credential was found with appropriate privileges. 

comment:11 Changed 10 years ago by ibaldin@renci.org

This is the one I said earlier - it failed because of one VLAN not coming up. I closed all reservations in this slice, please try again.

comment:12 Changed 10 years ago by ibaldin@renci.org

everything except one vm is up. openstack is not letting it come up for some reason. 30 minutes after the start the timeout will kick in and orca will kill this vm and try it again.

comment:13 Changed 10 years ago by lnevers@bbn.com

Thanks Ilia, I had been watching the sliver status for the past 16 minutes. Should I let it run until the timeout?

comment:14 Changed 10 years ago by lnevers@bbn.com

I am able to login and use the 17 VMs in the slive manifest. I assume the 18th will be added when/if successfully started.

comment:15 Changed 10 years ago by lnevers@bbn.com

Here are the results:

  • Sliver created at 14:47
  • geni_status remained "configuring" until 15:17
  • geni_status became 'ready' at 15:18:13
  • geni_status became 'failed' at 15:18:35

After the geni_status 'failed' is reported, I can still use the 17 VMs.

What would you like for me to do next, Ilia?

comment:16 Changed 10 years ago by ibaldin@renci.org

OpenStack? failed to bring up the last vm - I believe it is a transient OpenStack? failure (we have seen it in the past). We can't do anything about them other than wait for a newer version of OpenStack?. Please delete and try again.

comment:17 Changed 10 years ago by ibaldin@renci.org

The slice status is determined as a conjunction of all reservation statuses. If any of reservations fail, the slice is flagged as failed.

comment:18 Changed 10 years ago by ibaldin@renci.org

Looks like it worked this time.

As I said - transient OpenStack? behavior is a problem and the only solution is to try a few times (if you are sure the image is good).

comment:19 Changed 10 years ago by lnevers@bbn.com

YES!!!! I have a sliver with 18 VMs in a linear topology from the local rack SM.

I will re-run this test a few more times and then update the ticket (or close if no issue).

comment:20 Changed 10 years ago by ibaldin@renci.org

Just to summarize:

1) Original failure caused by head node running out of file descriptors. Not clear what caused it, we will keep on watching for it. There were traces of old processes that should have been terminated during last redeploy that could've been the cause.

2) There were vlan failiures caused by unreliable floodlight startup. I deployed a small fix (increased timeout) and committed a change to orca code

3) There was transient OpenStack? vm failure. The only fix is trying again (and waiting for a better OpenStack? version).

comment:21 Changed 10 years ago by lnevers@bbn.com

Here are results for 3 attempts, where each sliver requested 18 VMs from BBN rack SM:

  • Created sliver lnexo1, sliver ready in 3 minutes, able to login to each host - OK
  • Deleted sliver lnexo1.
  • Created sliver lnexo2, sliver failed in 15 minutes - FAIL
  • Deleted sliver lnexo2.
  • Created sliver lnexo3, sliver was in "configuring" state for over 36 minutes, deleted the sliver. Should this have timeout out in 30 minutes?

comment:22 Changed 10 years ago by ibaldin@renci.org

I can't tell now. I don't see any new failures in the logs, so I'm not sure what happened. This will have to wait until the rack is back tomorrow.

comment:23 Changed 10 years ago by lnevers@bbn.com

Re-ran this test by requesting a 13 node linear topology to the BBN SM which generated this failure:

Asked https://bbn-hn.exogeni.net:11443/orca/xmlrpc to reserve resources. No manifest Rspec returned. 
Error encountered converting manifest to RSpec: ERROR encountered while converting the manifest 
due to null pointer, inspect the logs 

Note: This converter problem had been fixed on Tuesday, but now shows up for this sliver and not others.

After the sliver named 13linear is created there are inconsistencies.

The sliverstatus shows 'geni_status'='ready', and the 13 VMs allocated

$ egrep VM ./13linear-sliverstatus-bbn-hn-exogeni-net-11443-orca.json 
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-1',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-3',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-2',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-8',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-9',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-4',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-5',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-6',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-7',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-10',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-11',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-12',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+196dab99-e4dc-41d0-8a1f-64048367037e#VM-13',

But the listresources only show 11 hosts (note the listresources was collected 10 minutes after the geni_status was ready) :

$ egrep hostname 13vmlinear-rspec-bbn-hn-exogeni-net-11443-orca.xml    
<login authentication="ssh-keys" hostname="192.1.242.28" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.44" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.46" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.38" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.29" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.45" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.37" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.22" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.26" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.42" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.27" port="22" username="root"/>      

Attaching the Request RSpec, the listresources (manifest) RSpec, and the sliverstatus output.

Changed 10 years ago by lnevers@bbn.com

Changed 10 years ago by lnevers@bbn.com

comment:24 Changed 10 years ago by lnevers@bbn.com

I have tried an 18 node linear topology sliver from the BBN SM and it fails. The failed sliver status and urn:

 'geni_status': 'failed',
 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnexo'}

I realize this original ticket was written for a 13 VM linear topology sliver, and the current issue may be unrelated. If needed, I can write a new ticket, please let me know.

Also, I am including a capture of email exchanges outside the ticket about the original failure in this ticket:


On 7/5/12 4:32 PM, Ilia Baldine wrote:

Don't kill this slice please


On 7/5/12 4:49 PM, Ilia Baldine wrote:

Try again on the same slice. I've updated the converter.


On 7/5/12 5:13 PM, Ilia Baldine wrote:

I see a slice (13-node) created at 5:05 that is active. Anything else?


On Jul 5, 2012, at 5:17 PM, Luisa Nevers wrote:

The request just returned a failure:

Result Summary: Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear expires on 2012-07-06 21:05:53 UTC Asked https://bbn-hn.exogeni.net:11443/orca/xmlrpc to reserve resources. No manifest Rspec returned. Error encountered converting manifest to RSpec: Unable to contact/use any of specified NDL-RSPEC converters at

http://geni.renci.org:12080/ndl-conversion/,http://bbn-hn.exogeni.net:14080/ndl-conversion/


On 7/5/12 5:18 PM, Ilia Baldine wrote:

They both seem to have run out of memory. Try requesting manifest now. I restarted them. I may need to tweak the amount of virtual memory Java uses for conversion.


On 7/5/12 5:20 PM, Ilia Baldine wrote:

I was talking about the converters. The Java containers running them have run out of memory.


On 7/5/12 5:30 PM, Ilia Baldine wrote:

Too many moving pieces. I will tweak the converter settings tonight to increase the memory available to them to make sure this doesn't happen.


On Jul 5, 2012, at 5:29 PM, Luisa Nevers wrote:

It now works, I request 13 and get 13 in the listresources.


On 7/5/12 5:47 PM, Ilia Baldine wrote:

I've added parameters to tomcat start scripts on both converters that should help with this. I've also restarted them.

comment:25 in reply to:  24 Changed 10 years ago by lnevers@bbn.com

Replying to lnevers@bbn.com:

I have tried an 18 node linear topology sliver from the BBN SM and it fails. The failed sliver status and urn:

 'geni_status': 'failed',
 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnexo'}

I had not noticed the following error because the output had scrolled of the screen:

                    {'geni_error': 'Reservation 7b3f4c19-46a8-4705-aa08-ab9fac5625c2 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnexo) is in state [Failed,None], err=resources failed to join: Error during join for unit: 3E2D1B98 [1]: unable to create instance: exit code 1, \n',
                     'geni_status': 'Failed',

comment:26 Changed 10 years ago by ibaldin@renci.org

This is a transient OpenStack? error in one of the VMs. All I can say is 'try again please'. We hope the newer version of OpenStack? proves more reliable.

comment:28 Changed 10 years ago by lnevers@bbn.com

Ilia, I took a quick look and at the new page and have some feedback:

  1. The text "Errors from the first category" and "Errors from the second category" should mention the category name (transient/persistent), as I scrolled down to read the second section, I no longer recalled the order they were listed in at the top of the page.
  1. There is no mapping between the types of errors (transient/persistent) and the phases (embedding/allocation/provisioning). The page tells me about the phases that occur while creating a slice, but does not relate the phases (embedding/allocation/provisioning) to the types of errors (transient/persistent).
  1. It would be helpful to show an example for the persistent errors. We know what the errors look like and it makes it easier for the experimenter searching for help when they run into the problem.

comment:29 Changed 10 years ago by lnevers@bbn.com

2nd attempt, same failure:

{'geni_error': 'Reservation b27bb316-c041-4077-a243-0ef059660483 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnexo) is in state [Failed,None], err=resources failed to join: Error during join for unit: 395A6DA3 [1]: unable to create instance: exit code 1, \n',
'geni_status': 'Failed',
'geni_urn': 'urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+fd120d4d-446c-4e07-a684-7ff7be1102c5#host-10',

3rd attempt, worked!!!

comment:30 Changed 10 years ago by lnevers@bbn.com

Attempted to re-ran the 13 node linear scenario and it failed. First a capture of the createsliver:

$ omni.py createsliver -a eg-sm 13linear ./exorspec/exo-13vm-linear2.rspec 
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear expires on 2012-12-05 21:41:24 UTC
INFO:omni:Substituting AM nickname eg-sm with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname eg-sm with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./exorspec/exo-13vm-linear2.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear
INFO:omni:Got return from CreateSliver for slice 13linear at https://geni.renci.org:11443/orca/xmlrpc:
INFO:omni:<?xml version="1.0" ?>
INFO:omni:  <!-- Reserved resources for:
	Slice: 13linear
	at AM:
	URN: unspecified_AM_URN
	URL: https://geni.renci.org:11443/orca/xmlrpc
 -->
INFO:omni:  <rspec type="manifest" xmlns="http://www.geni.net/resources/rspec/3" xmlns:ns2="http://hpn.east.isi.edu/rspec/ext/stitch/0.1/" xmlns:ns3="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1" xmlns:ns4="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/manifest.xsd http://hpn.east.isi.edu/rspec/ext/stitch/0.1/ http://hpn.east.isi.edu/rspec/ext/stitch/0.1/stitch-schema.xsd http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1/slice_info.xsd?format=raw http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1/sliver_info.xsd?format=raw">  
      <node client_id="host-10" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-10">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-10:if1">      
                  <ip address="172.16.10.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-10:if0">      
                  <ip address="172.16.9.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-4" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-4">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-4:if1">      
                  <ip address="172.16.4.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-4:if0">      
                  <ip address="172.16.3.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-2" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-2">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-2:if0">      
                  <ip address="172.16.1.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-2:if1">      
                  <ip address="172.16.2.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-9" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-9">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-9:if1">      
                  <ip address="172.16.9.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-9:if0">      
                  <ip address="172.16.8.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-13" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-13">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-13:if0">      
                  <ip address="172.16.12.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-7" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-7">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-7:if0">      
                  <ip address="172.16.6.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-7:if1">      
                  <ip address="172.16.7.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-11" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-11">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-11:if0">      
                  <ip address="172.16.10.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-11:if1">      
                  <ip address="172.16.11.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-12" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-12">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-12:if0">      
                  <ip address="172.16.11.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-12:if1">      
                  <ip address="172.16.12.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-1" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-1">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-1:if0">      
                  <ip address="172.16.1.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-8" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-8">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-8:if0">      
                  <ip address="172.16.7.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-8:if1">      
                  <ip address="172.16.8.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-3" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-3">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-3:if0">      
                  <ip address="172.16.2.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-3:if1">      
                  <ip address="172.16.3.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-6" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-6">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-6:if0">      
                  <ip address="172.16.5.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-6:if1">      
                  <ip address="172.16.6.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <node client_id="host-5" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_name="orca-vm-cloud" exclusive="true" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-5">    
            <sliver_type name="m1.small">      
                  <disk_image name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.5.xml" version="104ea3d824906f0e13cebb89b14df232290553b1 "/>      
            </sliver_type>    
            <services/>    
            <interface client_id="host-5:if0">      
                  <ip address="172.16.4.2" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <interface client_id="host-5:if1">      
                  <ip address="172.16.5.1" netmask="255.255.255.0" type="ipv4"/>      
            </interface>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </node>  
      <link client_id="lan25" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan25" vlantag="unknown">    
            <interface_ref client_id="host-6:if1"/>    
            <interface_ref client_id="host-7:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan210" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan210" vlantag="unknown">    
            <interface_ref client_id="host-11:if1"/>    
            <interface_ref client_id="host-12:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan28" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan28" vlantag="unknown">    
            <interface_ref client_id="host-10:if0"/>    
            <interface_ref client_id="host-9:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan22" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan22" vlantag="unknown">    
            <interface_ref client_id="host-3:if1"/>    
            <interface_ref client_id="host-4:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan26" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan26" vlantag="unknown">    
            <interface_ref client_id="host-8:if0"/>    
            <interface_ref client_id="host-7:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan211" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan211" vlantag="unknown">    
            <interface_ref client_id="host-12:if1"/>    
            <interface_ref client_id="host-13:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan23" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan23" vlantag="unknown">    
            <interface_ref client_id="host-5:if0"/>    
            <interface_ref client_id="host-4:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan27" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan27" vlantag="unknown">    
            <interface_ref client_id="host-8:if1"/>    
            <interface_ref client_id="host-9:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan29" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan29" vlantag="unknown">    
            <interface_ref client_id="host-11:if0"/>    
            <interface_ref client_id="host-10:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan24" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan24" vlantag="unknown">    
            <interface_ref client_id="host-6:if0"/>    
            <interface_ref client_id="host-5:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan20" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan20" vlantag="unknown">    
            <interface_ref client_id="host-2:if0"/>    
            <interface_ref client_id="host-1:if0"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <link client_id="lan21" sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan21" vlantag="unknown">    
            <interface_ref client_id="host-3:if0"/>    
            <interface_ref client_id="host-2:if1"/>    
            <ns4:geni_sliver_info creation_time="2012-12-04T15:43:55.044-05:00" creator_urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers" expiration_time="2012-12-18T14:43:55.044-05:00" start_time="2012-12-04T15:43:55.044-05:00" state="Nascent"/>    
      </link>  
      <ns3:geni_slice_info state="unknown" urn="urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear" uuid="a3b379cc-4206-4ad8-869b-95b2b65c51ea"/>  
</rspec>
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:

  Options as run:
		aggregate: ['eg-sm']
		framework: pg

  Args: createsliver 13linear ./exorspec/exo-13vm-linear2.rspec

  Result Summary: Got Reserved resources RSpec from geni-renci-org-11443-orca 
INFO:omni: ============================================================

Here is the failure in the sliverstatus:

$ omni.py sliverstatus -a eg-sm 13linear                                   
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear expires on 2012-12-05 21:41:24 UTC
INFO:omni:Substituting AM nickname eg-sm with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN
INFO:omni:Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear:
INFO:omni:Sliver status for Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear at AM URL https://geni.renci.org:11443/orca/xmlrpc
INFO:omni:{
  "geni_status": "failed", 
  "geni_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear", 
  "geni_resources": [
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-11", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-10", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-13", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-12", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan25", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan26", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan23", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan24", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan29", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan27", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan28", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan21", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan22", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan20", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-1", 
      "geni_error": "Reservation 0e294f19-58f4-4d1b-a92c-ea4a3368d905 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:0e294f19-58f4-4d1b-a92c-ea4a3368d905\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:0e294f19-58f4-4d1b-a92c-ea4a3368d905", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-2", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-3", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-4", 
      "geni_error": "", 
      "geni_status": "Ticketed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-5", 
      "geni_error": "Reservation 6de29f56-0fb7-4304-bbf8-c2716ffe0ade (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:6de29f56-0fb7-4304-bbf8-c2716ffe0ade\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:6de29f56-0fb7-4304-bbf8-c2716ffe0ade", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-6", 
      "geni_error": "Reservation 7c8cce97-7702-4d19-aa10-8fcde66ef593 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:7c8cce97-7702-4d19-aa10-8fcde66ef593\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:7c8cce97-7702-4d19-aa10-8fcde66ef593", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-7", 
      "geni_error": "Reservation df8862d2-4a81-4d97-9da8-5a6b852f351e (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:df8862d2-4a81-4d97-9da8-5a6b852f351e\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:df8862d2-4a81-4d97-9da8-5a6b852f351e", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-8", 
      "geni_error": "Reservation 74abe4b4-308c-4666-9c48-1a7dad2c0127 (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:74abe4b4-308c-4666-9c48-1a7dad2c0127\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:74abe4b4-308c-4666-9c48-1a7dad2c0127", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#host-9", 
      "geni_error": "Reservation 60f932f2-c93c-4603-b590-dc75a2a53b0c (Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear) is in state [Failed,None]\n\nLast ticket update: Insufficient resources for specified start time, Failing reservation:60f932f2-c93c-4603-b590-dc75a2a53b0c\n\nTicket events\nInsufficient resources for specified start time, Failing reservation:60f932f2-c93c-4603-b590-dc75a2a53b0c", 
      "geni_status": "Failed"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan210", 
      "geni_error": "", 
      "geni_status": "Active"
    }, 
    {
      "orca_expires": "Tue Dec 18 14:43:55 EST 2012", 
      "geni_urn": "urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+3d5a7123-744c-4ea1-9c39-980f6b5a6b9f#lan211", 
      "geni_error": "", 
      "geni_status": "Active"
    }
  ]
}
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed sliverstatus:

  Options as run:
		aggregate: ['eg-sm']
		framework: pg

  Args: sliverstatus 13linear

  Result Summary: Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+13linear expires on 2012-12-05 21:41:24 UTC
Returned status of slivers on 1 of 1 possible aggregates. 
INFO:omni: ============================================================

comment:31 Changed 10 years ago by ibaldin@renci.org

I don't think this is a bug. People are using racks and contention for resources is possible. As you can tell from the error message the problem was not enough resources at that time.

comment:32 Changed 10 years ago by ibaldin@renci.org

I just created a linear 13-node slice successfully.

comment:33 Changed 10 years ago by lnevers@bbn.com

ok, I will try again to setup a 13 node linear setup and will update the ticket when done.

comment:34 Changed 10 years ago by lnevers@bbn.com

Resolution: fixed
Status: newclosed

Successfully created a 13 VM Linear sliver, was able to login to each node and exchange traffic. Closing ticket.

Note: See TracTickets for help on using tickets.