Opened 12 years ago

Closed 12 years ago

#10 closed (fixed)

A 2 VM sliver (1@InstaGENI & 1@PG Utah) shows pg_status 'notready' 40 minutes after slivercreate

Reported by: lnevers@bbn.com Owned by: somebody
Priority: major Milestone:
Component: AM Version: SPIRAL4
Keywords: AM API Cc:
Dependencies:

Description

The slice used for this experiment is urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+1nsta-1pg and was created at around 4:21 pm eastern.

Twenty minutes after the two sliver were created at InstaGENI AM and PG Utah AM, the pg_status' was still 'notready'.

Change History (9)

comment:1 Changed 12 years ago by lnevers@bbn.com

Summary: A 2 VM sliver (1@InstaGENI & 1@PG Utah) shows pg_status 'notready' 20 minutes after slivercreateA 2 VM sliver (1@InstaGENI & 1@PG Utah) shows pg_status 'notready' 40 minutes after slivercreate

Still in the same state after 40 minutes.

comment:2 Changed 12 years ago by lnevers@bbn.com

On 5/9/12 5:14 PM, Leigh Stoller wrote:

Still in the same state after 40 minutes.

Yep, something is whacky. Please leave this slice active.

Thanks! Lbs

Any update on this ticket?

comment:3 Changed 12 years ago by lnevers@bbn.com

Re-ran the experiment with 1 VM @instageni and 1 VM @ PG Utah, and again got the 2 nodes that were requested.

Connected to each host and pinged the remote, and found poor connectivity. On InstaGENI Rack host:

[lnevers@VM-0 ~]$ ping 192.168.3.2
PING 192.168.3.2 (192.168.3.2) 56(84) bytes of data.
<<output deleted>>>
--- 192.168.3.2 ping statistics ---
65 packets transmitted, 16 received, 75% packet loss, time 64001ms
rtt min/avg/max/mdev = 0.172/34.236/544.708/131.803 ms

On PG Utah Rack:

[lnevers@VM ~]$ ping 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
<<<output deleted>>>>
--- 192.168.3.1 ping statistics ---
138 packets transmitted, 19 received, 86% packet loss, time 137009ms
rtt min/avg/max/mdev = 0.204/123.744/1629.581/381.678 ms, pipe 2
[lnevers@VM ~]$ 

comment:4 Changed 12 years ago by lnevers@bbn.com

Synching up ticket with email exchange:

On 5/10/12 12:46 PM, Leigh Stoller wrote:

Could you please elaborate on the fix.

Hi. I forgot to kill dhcpd on the control node, so we had two dhcpds running, which is very bad for Emulab.

Lbs

On 5/10/12 1:02 PM, Leigh Stoller wrote:

Re-ran the experiment with 1 VM @instageni and 1 VM @ PG Utah, and again

got the 2 nodes that were requested.

Connected to each host and pinged the remote, and found poor connectivity.

Yep, looks bad. Please leave this active so I can poke around.

Lbs

comment:5 Changed 12 years ago by lnevers@bbn.com

Re-ran the experiment with 1 VM @InstaGENi and 1 VM @ PG Utah.

Both InstaGENI and PG Utah slivers were created at the same time without error. The InstaGENI VM was available and ready for login in a few minutes, but 40 minutes after creating the sliver, the PG Utah node is still note ready.

 'geni_status': 'unknown',
 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+1nsta-1pg',
 'pg_expires': '2012-05-25T17:31:49',
 'pg_status': 'notready',

Unable to verify issue resolution.

comment:6 Changed 12 years ago by lnevers@bbn.com

Re-ran this scenario at approx 10 AM and was able to successfully set up the sliver, slice named "1vminsta-1vmpgutah2".

Once logged in allocated VMs, could not exchange any traffic between the InstaGENI VM endpoint and the PG Utah VM endpoint. Also noticed that the InstaGENI VM did not use the hostname defined in the request RSpec. The command prompt showed:

"[lnevers@unknown ~]$ "

rather than the expected

"[lnevers@VM-0 ~]$ "

Attaching sliver manifests, and leaving the slivers running.

comment:7 Changed 12 years ago by lnevers@bbn.com

Resolution: fixed
Status: newclosed

On 6/6/12 1:38 PM, Leigh Stoller wrote:

Re-ran this scenario at approx 10 AM and was able to successfully set up the sliver, slice named "1vminsta-1vmpgutah2".

Once logged in allocated VMs, could not exchange any traffic between the InstaGENI VM endpoint and the PG Utah VM endpoint.

This should be fixed ... It was a firewall problem.

Also noticed that the InstaGENI VM did not use the hostname defined in the request RSpec. The command prompt showed:

"[lnevers@unknown ~]$ "

I logged in and it looked okay:

[root@VM-0 ~]# hostname VM-0.1vminsta-1vmpgutah2.pgeni-gpolab-bbn-com.utah.geniracks.net

I am now able to exchange traffic between the two endpoints. Problem is solved, Closing ticket.

Note: See TracTickets for help on using tickets.