Opened 11 years ago

Closed 10 years ago

#171 closed (fixed)

bbnadmins do not have sudo on BBN rack worker nodes

Reported by: Owned by:
Priority: minor Milestone:
Component: Administration Version: SPIRAL5
Keywords: Cc:


I noticed that bbnadmins no longer have sudo on rack worker nodes. I see:

bbn-w2,[~],20:44(0)$ groups
nonrenci bbnadmins

bbn-w2,[~],20:44(0)$ sudo -l
[sudo] password for chaos: 
Sorry, user chaos may not run sudo on bbn-w2.

bbn-w2,[~],20:44(0)$ sudo whoami
[sudo] password for chaos: 
chaos is not in the sudoers file.  This incident will be reported.

Josh reported similar output from sudo -l on bbn-w1.

I'm pretty sure this used to work for us. Can someone take a look?


Change History (7)

comment:1 Changed 11 years ago by

Owner: changed from somebody to

comment:2 Changed 11 years ago by

Please do not classify this as a 'major' flaw. No one has lost anything in the way of functionality.

A few months ago, we discovered a problem with having sudo in ldap on the worker nodes. The neuca client on the worker nodes polls sudo *a lot*. More than once a second, at times. This was effectively DDoSing the site LDAP replica servers (I first noticed when I couldn't explain the slow response times in ssh logins.)

Then, in an unrelated set of circumstances, ExoGENI Ops decided to remove the restriction to ban the /bin/su command from sudo. Doing any kind of real work was just getting ridiculous when having type 'sudo' in front of a zillion commands. Most of us were just bypassing it anyway via root passwords or shell escapes.

The result is that you can now 'sudo su - ' from a head node if you are a site admin.

Once you are root on your site's rack head node, you can, obviously, ssh without passwords (using keys) directly to any worker node, where you will be root.

We fixed the worker node sudo/ldap/DDoS problem by just removing the ldap database line from nsswitch.conf on those nodes:

[root@bbn-w4 ~]# grep sudoers /etc/nsswitch.conf sudoers: files [root@bbn-w4 ~]#

There isn't any net loss of functionality that I can see, and the result is that we aren't crushing LDAP replicas and sending more messages than rsyslog can possibly forward. Now, is it the ideal solution that I want long term? I don't know about that. I think we could potentially get somewhere with SSSD and credential caching. But we won't have SSSD until we upgrade all our infrastructure to CentOS 6.4, and even then, credential caching is a double-edged sword.

comment:3 Changed 11 years ago by

Priority: majorminor

Downgrading to minor in deference to Jonathan. I'll verify that the other functionality works and (presumably) close on Monday.

comment:4 Changed 11 years ago by

Okay, indeed:

sudo ssh -i /opt/orca-12080/xcat/id_rsa root@bbn-w1

works. I think this should be on the ExoGENI wiki somewhere, so that site admins can find out where the SSH key file to use is.

comment:5 Changed 11 years ago by

Adding a note to track where on the FIU rack the ssh key file is located:

 sudo ssh -i /etc/orca/am+broker-12080/xcat/id_rsa  root@fiu-w2 

comment:6 Changed 11 years ago by

On 5/2/13 12:30 PM, Jonathan Mills wrote:

So this is a change that came with ORCA 4.0 and is not specific to FIU.

Adding to ticket, to capture that this is the new "and only" location for the ssh keys.

comment:7 Changed 10 years ago by

Resolution: fixed
Status: newclosed

This is no longer an issue, able to login to the bare metal nodes without specifying a path to the identity file. Closing ticket.

Note: See TracTickets for help on using tickets.