[[PageOutline]] = Detailed test plan for EG-MON-1: Control Network Software and VLAN Inspection Test = ''This page is GPO's working page for performing EG-MON-1. It is public for informational purposes, but it is not an official status report. See [wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus] for the current status of ExoGENI acceptance tests.'' ''Last update: 2012-12-18'' == Page format == * The status chart summarizes the state of this test * The high-level description from test plan contains text copied exactly from the public test plan and acceptance criteria pages. * The steps contain things i will actually do/verify: * Steps may be composed of related substeps where i find this useful for clarity * Each step is either a preparatory step (identified by "(prep)") or a verification step (the default): * Preparatory steps are just things we have to do. They're not tests of the rack, but are prerequisites for subsequent verification steps * Verification steps are steps in which we will actually look at rack output and make sure it is as expected. They contain a '''Using:''' block, which lists the steps to run the verification, and an '''Expect:''' block which lists what outcome is expected for the test to pass. == Status of test == Meaning of states: * [[Color(lightgreen,Pass)]]: Step is completed and passed (for a verification step), or is completed (for a prep step) * [[Color(red,Fail)]]: Step is completed and failed, and is not being revisited * [[Color(#63B8FF,In Progress)]]: We are currently testing or iterating on this step * [[Color(orange,Blocked)]]: Step is blocked by some other step or activity || '''Step''' || '''State''' || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets''' || || 1 || [[Color(orange,Blocked)]] || || exoticket:93 [[BR]] exoticket:96 || exoticket:95 [[BR]] exoticket:98 || || 2 || [[Color(lightgreen,Pass)]] || 2012-08-15 || || || || 3 || [[Color(lightgreen,Pass)]] || 2012-10-10 || || exoticket:10 || || 4 || [[Color(lightgreen,Pass)]] || 2012-10-10 || || exoticket:10 || == High-level description from test plan == This test inspects the state of the rack control network, infrastructure nodes, and system software. === Procedure === * A site administrator enumerates processes on each of the head node and an !OpenStack worker node which listen for network connections from other nodes, identifies what version of what software package is in use for each, and verifies that we know the source of each piece of software and could get access to its source code. * A site administrator reviews the configuration of the rack management switch and verifies that each worker node's control interfaces are on the expected VLANs for that worker node's function (!OpenStack or bare metal). * A site administrator reviews the MAC address table on the management switch, and verifies that all entries are identifiable and expected. === Criteria to verify as part of this test === * VI.09. A public document explains how to identify the software versions and system file configurations running on the rack, and how to get information about recent changes to the rack software and configuration. (F.5) * VI.11. A public document describes the GENI software running on the rack, and explains how to get access to the source code of each piece of GENI software. (F.6) * VII.03. Site administrators can understand the expected control and dataplane network behavior of their rack. (F.2) * VII.04. Site administrators can view and investigate current system and network activity on their rack. (F.2) * VII.06. A site administrator can verify the control software and configurations on the rack at some point in time. (F.5) * VII.08. A site administrator can get access to source code for the version of each piece of GENI code installed on their site rack at some point in time. (F.6) * VII.09. A site administrator can determine the MAC addresses of all physical host interfaces, all network device interfaces, all active experimental VMs, and all recently-terminated experimental VMs. (C.3.f) * VII.10. A site administrator can locate current and recent CPU and memory utilization for each rack network device, and can find recent changes or errors in a log. (D.6.a) * VII.12. For each infrastructure and experimental host, a site administrator can locate current and recent uptime, CPU, disk, and memory utilization, interface traffic counters, process counts, and active user counts. (D.6.b) * VII.13. A site administrator can locate recent syslogs for all infrastructure and experimental hosts. (D.6.b) == Step 1: identify network-listening software on the head node == === Overview of Step 1 === '''Using:''' * Using netstat, enumerate processes on bbn-hn which listen for network connections from outside the node * For each process found: * Use the command-line or `/proc` to determine what executable file is running * Use RPM tools to determine whether the executable file is part of an RPM * Otherwise, use documentation or iterate with the ExoGENI team to determine the origin of the software * For each RPM found, identify a location from which a source RPM for that package can be obtained * For each non-RPM software source found, identify a location from which the source code for that version can be obtained. '''Verify:''' * The source of each network-listening file can be identified * RPM source packages can be found for each RPM-sourced package * The source code and identifiable version (e.g. a git tag) can be found for each non-RPM software source === Results of Step 1 from 2012-08-03 === A list of processes that are listening: {{{ [11:11:51] jbs@bbn-hn:/home/jbs +$ sudo netstat -anp | egrep ':\*.+LISTEN' | egrep -v '(127.0.0.1|192.168.103.10|10.100.0.1|10.103.0.1)' > netstat-raw.txt [11:17:13] jbs@bbn-hn:/home/jbs +$ cat netstat-raw.txt tcp 0 0 0.0.0.0:8811 0.0.0.0:* LISTEN 29140/python tcp 0 0 192.1.242.3:8139 0.0.0.0:* LISTEN 724/ruby tcp 0 0 0.0.0.0:36843 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:875 0.0.0.0:* LISTEN 3258/rpc.rquotad tcp 0 0 0.0.0.0:8812 0.0.0.0:* LISTEN 29162/python tcp 0 0 0.0.0.0:9292 0.0.0.0:* LISTEN 28780/python tcp 0 0 0.0.0.0:47308 0.0.0.0:* LISTEN 2806/rpc.statd tcp 0 0 0.0.0.0:8813 0.0.0.0:* LISTEN 29185/python tcp 0 0 0.0.0.0:8781 0.0.0.0:* LISTEN 28606/python tcp 0 0 0.0.0.0:8814 0.0.0.0:* LISTEN 29205/python tcp 0 0 0.0.0.0:8782 0.0.0.0:* LISTEN 28701/python tcp 0 0 0.0.0.0:8815 0.0.0.0:* LISTEN 29205/python tcp 0 0 0.0.0.0:8783 0.0.0.0:* LISTEN 28814/python tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 2780/rpcbind tcp 0 0 0.0.0.0:8816 0.0.0.0:* LISTEN 29185/python tcp 0 0 0.0.0.0:8784 0.0.0.0:* LISTEN 28878/python tcp 0 0 0.0.0.0:14000 0.0.0.0:* LISTEN 19259/java tcp 0 0 0.0.0.0:8817 0.0.0.0:* LISTEN 29162/python tcp 0 0 0.0.0.0:8785 0.0.0.0:* LISTEN 28878/python tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 4123/epmd tcp 0 0 0.0.0.0:10514 0.0.0.0:* LISTEN 2717/rsyslogd tcp 0 0 0.0.0.0:8818 0.0.0.0:* LISTEN 29140/python tcp 0 0 0.0.0.0:8786 0.0.0.0:* LISTEN 28814/python tcp 0 0 0.0.0.0:8787 0.0.0.0:* LISTEN 28701/python tcp 0 0 0.0.0.0:8788 0.0.0.0:* LISTEN 28606/python tcp 0 0 0.0.0.0:54004 0.0.0.0:* LISTEN 3274/rpc.mountd tcp 0 0 0.0.0.0:51413 0.0.0.0:* LISTEN 2378/java tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN 23122/vsftpd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3634/sshd tcp 0 0 0.0.0.0:8791 0.0.0.0:* LISTEN 28903/python tcp 0 0 0.0.0.0:11000 0.0.0.0:* LISTEN 29224/java tcp 0 0 0.0.0.0:8792 0.0.0.0:* LISTEN 28923/python tcp 0 0 0.0.0.0:49784 0.0.0.0:* LISTEN 3274/rpc.mountd tcp 0 0 0.0.0.0:8793 0.0.0.0:* LISTEN 28943/python tcp 0 0 0.0.0.0:3001 0.0.0.0:* LISTEN 23124/xcatd tcp 0 0 0.0.0.0:8794 0.0.0.0:* LISTEN 28965/python tcp 0 0 0.0.0.0:3002 0.0.0.0:* LISTEN 23127/xcatd tcp 0 0 0.0.0.0:8795 0.0.0.0:* LISTEN 28965/python tcp 0 0 0.0.0.0:8796 0.0.0.0:* LISTEN 28943/python tcp 0 0 192.1.242.3:636 0.0.0.0:* LISTEN 3199/slapd tcp 0 0 0.0.0.0:8797 0.0.0.0:* LISTEN 28923/python tcp 0 0 0.0.0.0:8798 0.0.0.0:* LISTEN 28903/python tcp 0 0 0.0.0.0:3551 0.0.0.0:* LISTEN 3660/apcupsd tcp 0 0 0.0.0.0:12000 0.0.0.0:* LISTEN 19470/java tcp 0 0 0.0.0.0:53504 0.0.0.0:* LISTEN 3274/rpc.mountd tcp 0 0 0.0.0.0:8801 0.0.0.0:* LISTEN 28987/python tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:8802 0.0.0.0:* LISTEN 29016/python tcp 0 0 0.0.0.0:8803 0.0.0.0:* LISTEN 29042/python tcp 0 0 0.0.0.0:8804 0.0.0.0:* LISTEN 29084/python tcp 0 0 0.0.0.0:43972 0.0.0.0:* LISTEN 4163/beam.smp tcp 0 0 0.0.0.0:8805 0.0.0.0:* LISTEN 29084/python tcp 0 0 192.1.242.3:3333 0.0.0.0:* LISTEN 28639/python tcp 0 0 0.0.0.0:8773 0.0.0.0:* LISTEN 27286/nginx tcp 0 0 192.1.242.3:389 0.0.0.0:* LISTEN 3199/slapd tcp 0 0 0.0.0.0:8806 0.0.0.0:* LISTEN 29042/python tcp 0 0 0.0.0.0:8807 0.0.0.0:* LISTEN 29016/python tcp 0 0 0.0.0.0:9191 0.0.0.0:* LISTEN 28826/python tcp 0 0 0.0.0.0:7911 0.0.0.0:* LISTEN 27903/dhcpd tcp 0 0 0.0.0.0:8808 0.0.0.0:* LISTEN 28987/python tcp 0 0 0.0.0.0:3626 0.0.0.0:* LISTEN 27286/nginx tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 3948/mysqld tcp 0 0 :::14443 :::* LISTEN 19259/java tcp 0 0 :::51339 :::* LISTEN 3274/rpc.mountd tcp 0 0 :::46254 :::* LISTEN - tcp 0 0 :::111 :::* LISTEN 2780/rpcbind tcp 0 0 :::50000 :::* LISTEN 24491/java tcp 0 0 :::12080 :::* LISTEN 19470/java tcp 0 0 :::8080 :::* LISTEN 9114/java tcp 0 0 :::80 :::* LISTEN 8477/httpd tcp 0 0 :::50001 :::* LISTEN 1380/java tcp 0 0 :::13009 :::* LISTEN 19625/java tcp 0 0 :::10514 :::* LISTEN 2717/rsyslogd tcp 0 0 :::11443 :::* LISTEN 29224/java tcp 0 0 :::50004 :::* LISTEN 9566/java tcp 0 0 :::51413 :::* LISTEN 2378/java tcp 0 0 ::1:53 :::* LISTEN 13783/named tcp 0 0 :::22 :::* LISTEN 3634/sshd tcp 0 0 :::45430 :::* LISTEN 2806/rpc.statd tcp 0 0 :::55000 :::* LISTEN 24491/java tcp 0 0 :::13080 :::* LISTEN 19625/java tcp 0 0 :::55001 :::* LISTEN 1380/java tcp 0 0 :::14009 :::* LISTEN 19259/java tcp 0 0 ::1:25 :::* LISTEN 4095/master tcp 0 0 ::1:6010 :::* LISTEN 16835/sshd tcp 0 0 ::1:6011 :::* LISTEN 32641/sshd tcp 0 0 :::12443 :::* LISTEN 19470/java tcp 0 0 :::443 :::* LISTEN 8477/httpd tcp 0 0 :::33595 :::* LISTEN 3274/rpc.mountd tcp 0 0 :::55004 :::* LISTEN 9566/java tcp 0 0 :::55740 :::* LISTEN 9114/java tcp 0 0 :::6556 :::* LISTEN 3642/xinetd tcp 0 0 :::6557 :::* LISTEN 22989/xinetd tcp 0 0 :::14080 :::* LISTEN 19259/java tcp 0 0 :::11009 :::* LISTEN 29224/java tcp 0 0 :::2049 :::* LISTEN - tcp 0 0 :::13443 :::* LISTEN 19625/java tcp 0 0 :::58949 :::* LISTEN 3274/rpc.mountd tcp 0 0 :::11080 :::* LISTEN 29224/java tcp 0 0 :::5672 :::* LISTEN 4163/beam.smp tcp 0 0 :::12009 :::* LISTEN 19470/java tcp 0 0 :::11081 :::* LISTEN 2378/java tcp 0 0 :::6633 :::* LISTEN 9114/java }}} That's 103 things, which seems like a lot, but I gather that this is normal. Ah, and there's some duplication in there. Stripping that into a list of PIDs: {{{ [11:22:25] jbs@bbn-hn:/home/jbs +$ awk '{print $NF}' netstat-raw.txt | sed -r -e 's|/.*||' | sort -n | uniq > netstat-pids.txt [11:23:04] jbs@bbn-hn:/home/jbs +$ cat netstat-pids.txt - 724 1380 2378 2717 2780 2806 3199 3258 3274 3634 3642 3660 3948 4095 4123 4163 8477 9114 9566 13783 16835 19259 19470 19625 22989 23122 23124 23127 24491 27286 27903 28606 28639 28701 28780 28814 28826 28878 28903 28923 28943 28965 28987 29016 29042 29084 29140 29162 29185 29205 29224 32641 }}} For some of those, the PID is '-': {{{ [11:25:15] jbs@bbn-hn:/home/jbs +$ egrep -- - netstat-raw.txt tcp 0 0 0.0.0.0:36843 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN - tcp 0 0 :::46254 :::* LISTEN - tcp 0 0 :::2049 :::* LISTEN - }}} lsof doesn't know what they are either: {{{ [11:26:32] jbs@bbn-hn:/home/jbs +$ sudo lsof -i | grep 2049 [11:27:15] jbs@bbn-hn:/home/jbs +$ sudo lsof -i | grep 46254 [11:27:26] jbs@bbn-hn:/home/jbs +$ sudo lsof -i | grep 36843 }}} We think those are probably NFS-related things, which perhaps are listening from a kernel module rather than a process. Leaving those aside, here are the others: {{{ [11:40:08] jbs@bbn-hn:/home/jbs +$ for pid in $(cat netstat-pids.txt) ; do ps -efwww | egrep "^[^ ]+ +$pid" ; done >& netstat-commands.txt [11:40:11] jbs@bbn-hn:/home/jbs +$ cat netstat-commands.txt root 724 1 0 Jul03 ? 02:44:09 /usr/bin/ruby /usr/sbin/puppetd 2006 1380 1 0 Jul26 ? 00:06:48 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55001 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50001 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 2378 2372 0 Jul20 ? 00:08:06 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml root 2717 1 0 Jun13 ? 00:09:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 4 rpc 2780 1 0 Jun13 ? 00:00:02 rpcbind rpcuser 2806 1 0 Jun13 ? 00:00:00 rpc.statd ldap 3199 1 0 Jun13 ? 00:10:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf root 3258 1 0 Jun13 ? 00:00:00 rpc.rquotad root 3274 1 0 Jun13 ? 00:00:00 rpc.mountd root 3634 1 0 Jun13 ? 00:01:34 /usr/sbin/sshd root 3642 1 0 Jun13 ? 00:00:18 xinetd -stayalive -pidfile /var/run/xinetd.pid root 3660 1 0 Jun13 ? 00:04:13 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid mysql 3948 3706 0 Jun13 ? 02:05:09 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=8192 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306 root 4095 1 0 Jun13 ? 00:00:28 /usr/libexec/postfix/master rabbitmq 4123 1 0 Jun13 ? 00:00:16 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon rabbitmq 4163 4162 0 Jun13 ? 00:18:00 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn" root 8477 1 0 Jun22 ? 00:00:42 /usr/sbin/httpd 491 9114 1 3 Jun27 ? 1-10:41:10 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml 2006 9566 1 0 Aug01 ? 00:01:35 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-5- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55004 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50004 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar named 13783 1 0 Jun22 ? 00:01:04 /usr/sbin/named -u named jonmills 16835 16833 0 00:09 ? 00:00:00 sshd: jonmills@pts/2 2006 19259 1 0 Jul05 ? 00:24:49 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start 2006 19470 1 0 Jul20 ? 01:02:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start 2006 19625 1 0 Jul20 ? 00:29:16 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start rack_bbn 22989 1 0 Jun18 ? 00:00:24 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf root 23122 1 0 Jun22 ? 00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf root 23124 1 0 Jun22 ? 00:00:06 xcatd: SSL listener root 23127 23124 0 Jun22 ? 00:00:01 xcatd: install monitor 2006 24491 1 0 Aug01 ? 00:01:25 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55000 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50000 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar root 27286 1 0 Jun18 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf dhcpd 27903 1 0 Jun22 ? 00:02:12 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006 nova 28606 28603 0 Jul06 ? 00:06:51 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8781.log --ec2_listen_port=8781 --osapi_listen_port=8788 nova 28639 1 0 Jul06 ? 00:00:07 /usr/bin/python /usr/bin/nova-objectstore --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-objectstore.log --pidfile=/var/run/nova/nova-objectstore.pid nova 28701 28699 0 Jul06 ? 00:06:46 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8782.log --ec2_listen_port=8782 --osapi_listen_port=8787 glance 28780 1 0 Jul06 ? 00:00:54 /usr/bin/python /usr/bin/glance-api --config-file=/etc/glance/glance-api.conf --log-config=/etc/glance/logging-api.conf nova 28814 28812 0 Jul06 ? 00:06:32 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8783.log --ec2_listen_port=8783 --osapi_listen_port=8786 glance 28826 1 0 Jul06 ? 00:00:50 /usr/bin/python /usr/bin/glance-registry --config-file=/etc/glance/glance-registry.conf -v --log-config=/etc/glance/logging-registry.conf nova 28878 28876 0 Jul06 ? 00:06:09 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8784.log --ec2_listen_port=8784 --osapi_listen_port=8785 nova 28903 28901 0 Jul06 ? 00:06:12 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8791.log --ec2_listen_port=8791 --osapi_listen_port=8798 nova 28923 28921 0 Jul06 ? 00:06:49 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8792.log --ec2_listen_port=8792 --osapi_listen_port=8797 nova 28943 28941 0 Jul06 ? 00:06:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8793.log --ec2_listen_port=8793 --osapi_listen_port=8796 nova 28965 28963 0 Jul06 ? 00:07:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8794.log --ec2_listen_port=8794 --osapi_listen_port=8795 nova 28987 28985 0 Jul06 ? 00:06:01 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8801.log --ec2_listen_port=8801 --osapi_listen_port=8808 nova 29016 29014 0 Jul06 ? 00:06:33 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8802.log --ec2_listen_port=8802 --osapi_listen_port=8807 nova 29042 29040 0 Jul06 ? 00:09:05 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8803.log --ec2_listen_port=8803 --osapi_listen_port=8806 nova 29084 29080 0 Jul06 ? 00:06:34 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8804.log --ec2_listen_port=8804 --osapi_listen_port=8805 nova 29140 29138 0 Jul06 ? 00:06:11 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8811.log --ec2_listen_port=8811 --osapi_listen_port=8818 nova 29162 29160 0 Jul06 ? 00:06:29 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8812.log --ec2_listen_port=8812 --osapi_listen_port=8817 nova 29185 29183 0 Jul06 ? 00:06:42 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8813.log --ec2_listen_port=8813 --osapi_listen_port=8816 nova 29205 29203 0 Jul06 ? 00:06:47 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8814.log --ec2_listen_port=8814 --osapi_listen_port=8815 2006 29224 1 0 Jul20 ? 00:29:38 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start jonmills 32641 32639 0 10:09 ? 00:00:00 sshd: jonmills@pts/1 }}} Just to sanity check that we haven't missed anything: {{{ [11:40:14] jbs@bbn-hn:/home/jbs +$ wc -l netstat-* 52 netstat-commands.txt 53 netstat-pids.txt 103 netstat-raw.txt 208 total }}} The change from raw to pids is because of duplicates, and from pids to commands is because of the '-' PID dropping out. So, let's start plowing through those. This is from an RPM: {{{ root 724 1 0 Jul03 ? 02:44:09 /usr/bin/ruby /usr/sbin/puppetd [11:41:39] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/puppetd puppet-2.7.18-1.el6.noarch }}} Lumping these Floodlight things together, Java is from an RPM, but floodlight.jar isn't: {{{ 2006 1380 1 0 Jul26 ? 00:06:48 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55001 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50001 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 9566 1 0 Aug01 ? 00:01:35 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-5- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55004 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50004 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 24491 1 0 Aug01 ? 00:01:25 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55000 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50000 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar [15:56:51] jbs@bbn-hn:/home/jbs +$ which java java is /usr/java/default/bin/java [15:56:52] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/java/default/bin/java jdk-1.6.0_30-fcs.x86_64 [15:56:56] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/floodlight/floodlight.jar file /opt/floodlight/floodlight.jar is not owned by any package }}} Java is from an RPM (see above), imageproxy isn't: {{{ 2006 2378 2372 0 Jul20 ? 00:08:06 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml [15:57:00] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/imageproxy file /opt/imageproxy is not owned by any package }}} This is from an RPM: {{{ root 2717 1 0 Jun13 ? 00:09:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 4 [15:59:03] jbs@bbn-hn:/home/jbs +$ rpm -qf /sbin/rsyslogd rsyslog-4.6.2-12.el6.x86_64 }}} This is from an RPM: {{{ rpc 2780 1 0 Jun13 ? 00:00:02 rpcbind [16:00:13] jbs@bbn-hn:/home/jbs +$ which rpcbind rpcbind is /sbin/rpcbind [16:00:28] jbs@bbn-hn:/home/jbs +$ rpm -qf /sbin/rpcbind rpcbind-0.2.0-8.el6.x86_64 }}} This is from an RPM: {{{ rpcuser 2806 1 0 Jun13 ? 00:00:00 rpc.statd [16:00:32] jbs@bbn-hn:/home/jbs +$ which rpc.statd rpc.statd is /sbin/rpc.statd [16:00:52] jbs@bbn-hn:/home/jbs +$ rpm -qf /sbin/rpc.statd nfs-utils-1.2.3-15.el6.x86_64 }}} This is from an RPM: {{{ ldap 3199 1 0 Jun13 ? 00:10:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf [16:00:58] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/slapd openldap-servers-2.4.23-20.el6.x86_64 }}} This is from an RPM: {{{ root 3258 1 0 Jun13 ? 00:00:00 rpc.rquotad [16:01:28] jbs@bbn-hn:/home/jbs +$ which rpc.rquotad rpc.rquotad is /usr/sbin/rpc.rquotad [16:01:34] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/rpc.rquotad quota-3.17-16.el6.x86_64 }}} This is from an RPM: {{{ root 3274 1 0 Jun13 ? 00:00:00 rpc.mountd [16:01:40] jbs@bbn-hn:/home/jbs +$ which rpc.mountd rpc.mountd is /usr/sbin/rpc.mountd [16:02:24] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/rpc.mountd nfs-utils-1.2.3-15.el6.x86_64 }}} This is from an RPM: {{{ root 3634 1 0 Jun13 ? 00:01:34 /usr/sbin/sshd [16:06:46] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/sshd openssh-server-5.3p1-70.el6_2.2.x86_64 }}} This is from an RPM: {{{ root 3642 1 0 Jun13 ? 00:00:18 xinetd -stayalive -pidfile /var/run/xinetd.pid [16:07:15] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/xinetd xinetd-2.3.14-33.el6.x86_64 }}} This is from an RPM: {{{ root 3660 1 0 Jun13 ? 00:04:13 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid [16:07:45] jbs@bbn-hn:/home/jbs +$ rpm -qf /sbin/apcupsd apcupsd-3.14.8-4.el6.x86_64 }}} This is from an RPM: {{{ mysql 3948 3706 0 Jun13 ? 02:05:09 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=8192 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306 [16:08:22] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/libexec/mysqld mysql-server-5.1.61-1.el6_2.1.x86_64 }}} This is from an RPM: {{{ root 4095 1 0 Jun13 ? 00:00:28 /usr/libexec/postfix/master [16:08:28] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/libexec/postfix/master postfix-2.6.6-2.2.el6_1.x86_64 }}} This is from an RPM: {{{ rabbitmq 4123 1 0 Jun13 ? 00:00:16 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon [16:09:09] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/lib64/erlang/erts-5.8.5/bin/epmd erlang-erts-R14B-04.1.el6.x86_64 }}} This is from an RPM: {{{ rabbitmq 4163 4162 0 Jun13 ? 00:18:00 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn" [16:09:46] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/lib64/erlang/erts-5.8.5/bin/beam.smp erlang-erts-R14B-04.1.el6.x86_64 [16:10:42] jbs@bbn-hn:/home/jbs +$ rpm -qf /var/lib/rabbitmq rabbitmq-server-2.6.1-1.el6.noarch }}} (I don't really know what that is; I didn't check every path or file on that commandline, but I'm optimistic that they're all from RPMs.) This is from an RPM: {{{ root 8477 1 0 Jun22 ? 00:00:42 /usr/sbin/httpd [16:13:19] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/httpd httpd-2.2.15-15.el6.centos.1.x86_64 }}} Java is from an RPM (see above), and so is FlowVisor: {{{ 491 9114 1 3 Jun27 ? 1-10:41:10 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml [16:14:02] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/flowvisor/libexec/flowvisor/flowvisor.jar flowvisor-0.8.2-1.x86_64 }}} This is from an RPM: {{{ named 13783 1 0 Jun22 ? 00:01:04 /usr/sbin/named -u named [16:15:45] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/named bind-9.7.3-8.P3.el6_2.2.x86_64 }}} This is from an RPM: {{{ jonmills 16835 16833 0 00:09 ? 00:00:00 sshd: jonmills@pts/2 [16:16:07] jbs@bbn-hn:/home/jbs +$ which sshd sshd is /usr/sbin/sshd [16:16:12] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/sshd openssh-server-5.3p1-70.el6_2.2.x86_64 }}} Java is from an RPM (see above), I don't think the other stuff here is: {{{ 2006 19259 1 0 Jul05 ? 00:24:49 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start [16:16:15] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar file /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package [16:17:07] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-aux/tomcat-7/bin file /opt/orca-aux/tomcat-7/bin is not owned by any package [16:17:10] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-aux/tomcat-7 file /opt/orca-aux/tomcat-7 is not owned by any package [16:17:11] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-aux file /opt/orca-aux is not owned by any package [16:18:40] jbs@bbn-hn:/home/jbs +$ rpm -qa | grep -i orca || echo no match no match }}} Java is from an RPM (see above), I don't think the other stuff here is: {{{ 2006 19470 1 0 Jul20 ? 01:02:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start [16:19:43] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-12080/tomcat/bin/tomcat-juli.jar file /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package [16:19:47] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-12080/tomcat/bin file /opt/orca-12080/tomcat/bin is not owned by any package [16:19:49] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-12080 file /opt/orca-12080 is not owned by any package [16:19:51] jbs@bbn-hn:/home/jbs +$ rpm -qa | grep -i orca || echo no match no match }}} Java is from an RPM (see above), I don't think the other stuff here is: {{{ 2006 19625 1 0 Jul20 ? 00:29:16 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start [16:21:25] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-13080/tomcat/bin/tomcat-juli.jar file /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package [16:21:38] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-13080/tomcat/bin file /opt/orca-13080/tomcat/bin is not owned by any package [16:21:40] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-13080/tomcat file /opt/orca-13080/tomcat is not owned by any package [16:21:42] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-13080 file /opt/orca-13080 is not owned by any package }}} This is not from an RPM: {{{ rack_bbn 22989 1 0 Jun18 ? 00:00:24 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf [16:21:43] jbs@bbn-hn:/home/jbs +$ rpm -qf /omd/sites/rack_bbn/var/tmp/xinetd file /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package }}} This is from an RPM: {{{ root 23122 1 0 Jun22 ? 00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf [16:23:13] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/vsftpd vsftpd-2.2.2-6.el6_2.1.x86_64 }}} This is from an RPM: {{{ root 23124 1 0 Jun22 ? 00:00:06 xcatd: SSL listener [16:23:32] jbs@bbn-hn:/home/jbs +$ which xcatd xcatd is /opt/xcat/sbin/xcatd [16:23:35] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/xcat/sbin/xcatd xCAT-server-2.6.9-snap201111090305.noarch }}} This is from an RPM: {{{ root 23127 23124 0 Jun22 ? 00:00:01 xcatd: install monitor [16:23:32] jbs@bbn-hn:/home/jbs +$ which xcatd xcatd is /opt/xcat/sbin/xcatd [16:23:35] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/xcat/sbin/xcatd xCAT-server-2.6.9-snap201111090305.noarch }}} This is from an RPM: {{{ root 27286 1 0 Jun18 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf [16:23:40] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/nginx nginx-0.8.54-1.el6.x86_64 }}} This is from an RPM: {{{ dhcpd 27903 1 0 Jun22 ? 00:02:12 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006 [16:24:50] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/dhcpd dhcp-4.1.1-25.P1.el6_2.1.x86_64 }}} Lumping these nova things together, Python is from an RPM, as are nova-api and nova-objectstore: {{{ nova 28606 28603 0 Jul06 ? 00:06:51 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8781.log --ec2_listen_port=8781 --osapi_listen_port=8788 nova 28639 1 0 Jul06 ? 00:00:07 /usr/bin/python /usr/bin/nova-objectstore --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-objectstore.log --pidfile=/var/run/nova/nova-objectstore.pid nova 28701 28699 0 Jul06 ? 00:06:46 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8782.log --ec2_listen_port=8782 --osapi_listen_port=8787 nova 28814 28812 0 Jul06 ? 00:06:32 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8783.log --ec2_listen_port=8783 --osapi_listen_port=8786 nova 28878 28876 0 Jul06 ? 00:06:09 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8784.log --ec2_listen_port=8784 --osapi_listen_port=8785 nova 28903 28901 0 Jul06 ? 00:06:12 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8791.log --ec2_listen_port=8791 --osapi_listen_port=8798 nova 28923 28921 0 Jul06 ? 00:06:49 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8792.log --ec2_listen_port=8792 --osapi_listen_port=8797 nova 28943 28941 0 Jul06 ? 00:06:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8793.log --ec2_listen_port=8793 --osapi_listen_port=8796 nova 28965 28963 0 Jul06 ? 00:07:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8794.log --ec2_listen_port=8794 --osapi_listen_port=8795 nova 28987 28985 0 Jul06 ? 00:06:01 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8801.log --ec2_listen_port=8801 --osapi_listen_port=8808 nova 29016 29014 0 Jul06 ? 00:06:33 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8802.log --ec2_listen_port=8802 --osapi_listen_port=8807 nova 29042 29040 0 Jul06 ? 00:09:05 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8803.log --ec2_listen_port=8803 --osapi_listen_port=8806 nova 29084 29080 0 Jul06 ? 00:06:34 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8804.log --ec2_listen_port=8804 --osapi_listen_port=8805 nova 29140 29138 0 Jul06 ? 00:06:11 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8811.log --ec2_listen_port=8811 --osapi_listen_port=8818 nova 29162 29160 0 Jul06 ? 00:06:29 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8812.log --ec2_listen_port=8812 --osapi_listen_port=8817 nova 29185 29183 0 Jul06 ? 00:06:42 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8813.log --ec2_listen_port=8813 --osapi_listen_port=8816 nova 29205 29203 0 Jul06 ? 00:06:47 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8814.log --ec2_listen_port=8814 --osapi_listen_port=8815 [16:27:28] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/python python-2.6.6-29.el6.x86_64 [16:27:31] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/nova-api openstack-nova-api-2011.3-b1281neuca2.noarch [16:27:34] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/nova-objectstore openstack-nova-objectstore-2011.3-b1281neuca2.noarch }}} Lumping these glance things together, all of these are from RPMs: {{{ glance 28780 1 0 Jul06 ? 00:00:54 /usr/bin/python /usr/bin/glance-api --config-file=/etc/glance/glance-api.conf --log-config=/etc/glance/logging-api.conf glance 28826 1 0 Jul06 ? 00:00:50 /usr/bin/python /usr/bin/glance-registry --config-file=/etc/glance/glance-registry.conf -v --log-config=/etc/glance/logging-registry.conf [16:28:17] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/python python-2.6.6-29.el6.x86_64 [16:28:18] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/glance-api openstack-glance-2011.3-b1277.noarch [16:28:20] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/bin/glance-registry openstack-glance-2011.3-b1277.noarch }}} Java is from an RPM (see above), I don't think the other stuff here is: {{{ 2006 29224 1 0 Jul20 ? 00:29:38 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start [16:29:14] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-11080/tomcat/bin/tomcat-juli.jar file /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package [16:29:15] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-11080/tomcat/bin file /opt/orca-11080/tomcat/bin is not owned by any package [16:29:17] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-11080/tomcat file /opt/orca-11080/tomcat is not owned by any package [16:29:18] jbs@bbn-hn:/home/jbs +$ rpm -qf /opt/orca-11080 file /opt/orca-11080 is not owned by any package [16:29:19] jbs@bbn-hn:/home/jbs +$ rpm -qa | grep -i orca || echo no match no match }}} This is from an RPM: {{{ jonmills 32641 32639 0 10:09 ? 00:00:00 sshd: jonmills@pts/1 [16:29:51] jbs@bbn-hn:/home/jbs +$ which sshd sshd is /usr/sbin/sshd [16:29:52] jbs@bbn-hn:/home/jbs +$ rpm -qf /usr/sbin/sshd openssh-server-5.3p1-70.el6_2.2.x86_64 }}} So, that's these RPMs: {{{ puppet-2.7.18-1.el6.noarch jdk-1.6.0_30-fcs.x86_64 rsyslog-4.6.2-12.el6.x86_64 rpcbind-0.2.0-8.el6.x86_64 nfs-utils-1.2.3-15.el6.x86_64 openldap-servers-2.4.23-20.el6.x86_64 quota-3.17-16.el6.x86_64 nfs-utils-1.2.3-15.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 xinetd-2.3.14-33.el6.x86_64 apcupsd-3.14.8-4.el6.x86_64 mysql-server-5.1.61-1.el6_2.1.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 erlang-erts-R14B-04.1.el6.x86_64 rabbitmq-server-2.6.1-1.el6.noarch httpd-2.2.15-15.el6.centos.1.x86_64 flowvisor-0.8.2-1.x86_64 bind-9.7.3-8.P3.el6_2.2.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 vsftpd-2.2.2-6.el6_2.1.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xCAT-server-2.6.9-snap201111090305.noarch nginx-0.8.54-1.el6.x86_64 dhcp-4.1.1-25.P1.el6_2.1.x86_64 python-2.6.6-29.el6.x86_64 openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch python-2.6.6-29.el6.x86_64 openstack-glance-2011.3-b1277.noarch openstack-glance-2011.3-b1277.noarch openssh-server-5.3p1-70.el6_2.2.x86_64 }}} Sorted and uniq-ed: {{{ apcupsd-3.14.8-4.el6.x86_64 bind-9.7.3-8.P3.el6_2.2.x86_64 dhcp-4.1.1-25.P1.el6_2.1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 flowvisor-0.8.2-1.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 jdk-1.6.0_30-fcs.x86_64 mysql-server-5.1.61-1.el6_2.1.x86_64 nfs-utils-1.2.3-15.el6.x86_64 nginx-0.8.54-1.el6.x86_64 openldap-servers-2.4.23-20.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 openstack-glance-2011.3-b1277.noarch openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch postfix-2.6.6-2.2.el6_1.x86_64 puppet-2.7.18-1.el6.noarch python-2.6.6-29.el6.x86_64 quota-3.17-16.el6.x86_64 rabbitmq-server-2.6.1-1.el6.noarch rpcbind-0.2.0-8.el6.x86_64 rsyslog-4.6.2-12.el6.x86_64 vsftpd-2.2.2-6.el6_2.1.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xinetd-2.3.14-33.el6.x86_64 }}} And these things which aren't from RPMs: {{{ /opt/floodlight/floodlight.jar is not owned by any package /opt/imageproxy is not owned by any package /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package /opt/orca-aux/tomcat-7/bin is not owned by any package /opt/orca-aux/tomcat-7 is not owned by any package /opt/orca-aux is not owned by any package /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package /opt/orca-12080/tomcat/bin is not owned by any package /opt/orca-12080 is not owned by any package /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package /opt/orca-13080/tomcat/bin is not owned by any package /opt/orca-13080/tomcat is not owned by any package /opt/orca-13080 is not owned by any package /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package /opt/orca-11080/tomcat/bin is not owned by any package /opt/orca-11080/tomcat is not owned by any package /opt/orca-11080 is not owned by any package }}} The xinetd in /omd is a copy of the stock xinetd: {{{ [12:27:40] jbs@bbn-hn:/home/jbs +$ md5sum /omd/sites/rack_bbn/var/tmp/xinetd /usr/sbin/xinetd 1c0ef402cd3ad81e01cda637a6c02104 /omd/sites/rack_bbn/var/tmp/xinetd 1c0ef402cd3ad81e01cda637a6c02104 /usr/sbin/xinetd }}} So that essentialy is from an RPM. ExoGENI ticket [exoticket:93] is tracking the task of identifying the versions and sources of the others. As to the RPMs, these look like standard CentOS: {{{ [11:58:34] jbs@bbn-hn:/home/jbs +$ yum list installed apcupsd-3.14.8-4.el6.x86_64 bind-9.7.3-8.P3.el6_2.2.x86_64 dhcp-4.1.1-25.P1.el6_2.1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 mysql-server-5.1.61-1.el6_2.1.x86_64 nfs-utils-1.2.3-15.el6.x86_64 nginx-0.8.54-1.el6.x86_64 openldap-servers-2.4.23-20.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 python-2.6.6-29.el6.x86_64 quota-3.17-16.el6.x86_64 rabbitmq-server-2.6.1-1.el6.noarch rpcbind-0.2.0-8.el6.x86_64 rsyslog-4.6.2-12.el6.x86_64 vsftpd-2.2.2-6.el6_2.1.x86_64 xinetd-2.3.14-33.el6.x86_64 Loaded plugins: fastestmirror, refresh-packagekit, security Loading mirror speeds from cached hostfile * base: mirror.dattobackup.com * epel: mirror.metrocast.net * extras: mirror.dattobackup.com * updates: mirror.metrocast.net Installed Packages apcupsd.x86_64 3.14.8-4.el6 @epel dhcp.x86_64 12:4.1.1-25.P1.el6_2.1 @CentOS Updates/6.2 erlang-erts.x86_64 R14B-04.1.el6 @UIedited_1/6.2 httpd.x86_64 2.2.15-15.el6.centos.1 @CentOS Updates/6.2 mysql-server.x86_64 5.1.61-1.el6_2.1 @CentOS Updates/6.2 nfs-utils.x86_64 1:1.2.3-15.el6 @anaconda-CentOS-201112091719.x86_64/6.2 nginx.x86_64 0.8.54-1.el6 @UIedited_1/6.2 openldap-servers.x86_64 2.4.23-20.el6 @base openssh-server.x86_64 5.3p1-70.el6_2.2 @CentOS Updates/6.2 postfix.x86_64 2:2.6.6-2.2.el6_1 @anaconda-CentOS-201112091719.x86_64/6.2 python.x86_64 2.6.6-29.el6 @anaconda-CentOS-201112091719.x86_64/6.2 quota.x86_64 1:3.17-16.el6 @anaconda-CentOS-201112091719.x86_64/6.2 rabbitmq-server.noarch 2.6.1-1.el6 @UIedited_1/6.2 rpcbind.x86_64 0.2.0-8.el6 @anaconda-CentOS-201112091719.x86_64/6.2 rsyslog.x86_64 4.6.2-12.el6 @anaconda-CentOS-201112091719.x86_64/6.2 vsftpd.x86_64 2.2.2-6.el6_2.1 @CentOS Updates/6.2 xinetd.x86_64 2:2.3.14-33.el6 @anaconda-CentOS-201112091719.x86_64/6.2 }}} These look like not: {{{ [11:58:38] jbs@bbn-hn:/home/jbs +$ yum list installed flowvisor-0.8.2-1.x86_64 jdk-1.6.0_30-fcs.x86_64 openstack-glance-2011.3-b1277.noarch openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch puppet-2.7.18-1.el6.noarch xCAT-server-2.6.9-snap201111090305.noarch Loaded plugins: fastestmirror, refresh-packagekit, security Loading mirror speeds from cached hostfile * base: mirror.dattobackup.com * epel: mirror.metrocast.net * extras: mirror.dattobackup.com * updates: mirror.metrocast.net Installed Packages flowvisor.x86_64 0.8.2-1 @exogeni jdk.x86_64 2000:1.6.0_30-fcs @ExoGENI/6.2 openstack-glance.noarch 1:2011.3-b1277 @ExoGENI-noarch/6.2 openstack-nova-api.noarch 1:2011.3-b1281neuca2 @exogeni-noarch openstack-nova-objectstore.noarch 1:2011.3-b1281neuca2 @exogeni-noarch puppet.noarch 2.7.18-1.el6 @puppetlabs-products xCAT-server.noarch 4:2.6.9-snap201111090305 @xCAT/6.2 }}} I can find the name of a source RPM for each of those: {{{ [12:12:44] jbs@bbn-hn:/home/jbs +$ for package in flowvisor-0.8.2-1.x86_64 jdk-1.6.0_30-fcs.x86_64 openstack-glance-2011.3-b1277.noarch openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch puppet-2.7.18-1.el6.noarch xCAT-server-2.6.9-snap201111090305.noarch ; do rpm -q $package --qf 'Name : %{NAME}\nSource RPM : %{SOURCERPM}\n\n' ; done Name : flowvisor Source RPM : flowvisor-0.8.2-1.src.rpm Name : jdk Source RPM : jdk-1.6.0_30-fcs.src.rpm Name : openstack-glance Source RPM : openstack-glance-2011.3-b1277.src.rpm Name : openstack-nova-api Source RPM : openstack-nova-2011.3-b1281neuca2.src.rpm Name : openstack-nova-objectstore Source RPM : openstack-nova-2011.3-b1281neuca2.src.rpm Name : puppet Source RPM : puppet-2.7.18-1.el6.src.rpm Name : xCAT-server Source RPM : xCAT-server-2.6.9-snap201111090305.src.rpm }}} So, now to connect that with the repo URLs... That's a somewhat manual process, as you need to fish the repo URL out of the relevant file in /etc/yum.repos.d, but those seem to be: {{{ [14:19:44] jbs@bbn-hn:/home/jbs +$ grep baseurl /etc/yum.repos.d/exogeni.repo baseurl=http://software.exogeni.net/repo/exogeni/6/current/$basearch baseurl=http://software.exogeni.net/repo/exogeni/6/current/noarch [14:19:55] jbs@bbn-hn:/home/jbs +$ grep baseurl /etc/yum.repos.d/puppetlabs.repo baseurl=http://yum.puppetlabs.com/el/6/products/$basearch baseurl=http://yum.puppetlabs.com/el/6/dependencies/$basearch baseurl=http://yum.puppetlabs.com/el/6/products/SRPMS baseurl=http://yum.puppetlabs.com/el/6/dependencies/SRPMS }}} Hmm, I don't see a repo for xCAT anywhere: {{{ [14:20:23] jbs@bbn-hn:/home/jbs +$ grep -i xcat /etc/yum.repos.d/* || echo no match no match }}} ExoGENI ticket [exoticket:98] is tracking the task of making sure that repo exists. Meanwhile, http://software.exogeni.net/repo/exogeni doesn't seem to have an obvious SRPM location, so I've created ExoGENI ticket [exoticket:96] for that. Puppet does have SRPMs, as the baseurl listings above imply, and http://yum.puppetlabs.com/el/6/products/SRPMS/puppet-2.7.18-1.el6.src.rpm has one for Puppet 2.7.18, so that seems good. Aside from the open tickets (and the one question to exogeni-design), I think this step is now complete. === Results of Step 1 from 2012-12-18 === We went through this test again on Dec 18th, after the !OpenStack upgrade on the BBN rack. A list of processes that are listening: {{{ [15:54:51] jbs@bbn-hn:/home/jbs +$ sudo netstat -anp | egrep ':\*.+LISTEN' | egrep -v '(127.0.0.1|192.168.103.10|10.100.0.1|10.103.0.1)' > netstat-raw.txt [15:55:09] jbs@bbn-hn:/home/jbs +$ cat netstat-raw.txt tcp 0 0 0.0.0.0:8773 0.0.0.0:* LISTEN 8162/python tcp 0 0 192.1.242.3:3333 0.0.0.0:* LISTEN 8021/python tcp 0 0 0.0.0.0:29509 0.0.0.0:* LISTEN 4614/beam.smp tcp 0 0 0.0.0.0:35557 0.0.0.0:* LISTEN 3697/rpc.mountd tcp 0 0 192.1.242.3:389 0.0.0.0:* LISTEN 3628/slapd tcp 0 0 0.0.0.0:8774 0.0.0.0:* LISTEN 8162/python tcp 0 0 0.0.0.0:8775 0.0.0.0:* LISTEN 8162/python tcp 0 0 0.0.0.0:9191 0.0.0.0:* LISTEN 8831/python tcp 0 0 0.0.0.0:7911 0.0.0.0:* LISTEN 4470/dhcpd tcp 0 0 0.0.0.0:8776 0.0.0.0:* LISTEN 8162/python tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 8655/python tcp 0 0 0.0.0.0:30024 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:49258 0.0.0.0:* LISTEN 5306/../jre/bin/jav tcp 0 0 0.0.0.0:3626 0.0.0.0:* LISTEN 4839/nginx tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 4433/mysqld tcp 0 0 192.1.242.3:8139 0.0.0.0:* LISTEN 15118/ruby tcp 0 0 0.0.0.0:875 0.0.0.0:* LISTEN 3693/rpc.rquotad tcp 0 0 0.0.0.0:9292 0.0.0.0:* LISTEN 8813/python tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 3243/rpcbind tcp 0 0 0.0.0.0:14000 0.0.0.0:* LISTEN 30677/java tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 4576/epmd tcp 0 0 0.0.0.0:10514 0.0.0.0:* LISTEN 3101/rsyslogd tcp 0 0 0.0.0.0:37555 0.0.0.0:* LISTEN 3697/rpc.mountd tcp 0 0 10.101.0.1:53 0.0.0.0:* LISTEN 26601/named tcp 0 0 192.1.242.3:53 0.0.0.0:* LISTEN 26601/named tcp 0 0 0.0.0.0:51413 0.0.0.0:* LISTEN 7695/java tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN 4893/vsftpd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 4107/sshd tcp 0 0 0.0.0.0:11000 0.0.0.0:* LISTEN 9667/java tcp 0 0 0.0.0.0:3001 0.0.0.0:* LISTEN 4895/xcatd tcp 0 0 0.0.0.0:3002 0.0.0.0:* LISTEN 4898/xcatd tcp 0 0 0.0.0.0:39034 0.0.0.0:* LISTEN 3697/rpc.mountd tcp 0 0 0.0.0.0:25978 0.0.0.0:* LISTEN 3261/rpc.statd tcp 0 0 192.1.242.3:636 0.0.0.0:* LISTEN 3628/slapd tcp 0 0 0.0.0.0:35357 0.0.0.0:* LISTEN 8655/python tcp 0 0 0.0.0.0:3071 0.0.0.0:* LISTEN 5306/../jre/bin/jav tcp 0 0 0.0.0.0:3551 0.0.0.0:* LISTEN 4147/apcupsd tcp 0 0 0.0.0.0:12000 0.0.0.0:* LISTEN 31265/java tcp 0 0 0.0.0.0:9696 0.0.0.0:* LISTEN 8683/python tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN - tcp 0 0 :::28231 :::* LISTEN 3261/rpc.statd tcp 0 0 :::11080 :::* LISTEN 9667/java tcp 0 0 :::5672 :::* LISTEN 4614/beam.smp tcp 0 0 :::54697 :::* LISTEN 19396/java tcp 0 0 :::57673 :::* LISTEN 19278/java tcp 0 0 :::12009 :::* LISTEN 31265/java tcp 0 0 :::11081 :::* LISTEN 7695/java tcp 0 0 :::6633 :::* LISTEN 3767/java tcp 0 0 :::27945 :::* LISTEN 3697/rpc.mountd tcp 0 0 :::14443 :::* LISTEN 30677/java tcp 0 0 :::43467 :::* LISTEN 3697/rpc.mountd tcp 0 0 :::111 :::* LISTEN 3243/rpcbind tcp 0 0 :::12080 :::* LISTEN 31265/java tcp 0 0 :::80 :::* LISTEN 4798/httpd tcp 0 0 :::8080 :::* LISTEN 3767/java tcp 0 0 :::59697 :::* LISTEN 19396/java tcp 0 0 :::13009 :::* LISTEN 30574/java tcp 0 0 :::10514 :::* LISTEN 3101/rsyslogd tcp 0 0 :::11443 :::* LISTEN 9667/java tcp 0 0 :::39604 :::* LISTEN 3697/rpc.mountd tcp 0 0 :::51413 :::* LISTEN 7695/java tcp 0 0 :::22 :::* LISTEN 4107/sshd tcp 0 0 :::13080 :::* LISTEN 30574/java tcp 0 0 :::21144 :::* LISTEN 3767/java tcp 0 0 :::14009 :::* LISTEN 30677/java tcp 0 0 ::1:25 :::* LISTEN 4546/master tcp 0 0 ::1:6010 :::* LISTEN 6868/sshd tcp 0 0 :::12443 :::* LISTEN 31265/java tcp 0 0 :::443 :::* LISTEN 4798/httpd tcp 0 0 :::6556 :::* LISTEN 4115/xinetd tcp 0 0 :::6557 :::* LISTEN 4091/xinetd tcp 0 0 :::14080 :::* LISTEN 30677/java tcp 0 0 :::52673 :::* LISTEN 19278/java tcp 0 0 :::11009 :::* LISTEN 9667/java tcp 0 0 :::22017 :::* LISTEN - tcp 0 0 :::2049 :::* LISTEN - tcp 0 0 :::13443 :::* LISTEN 30574/java }}} Stripping that into a list of PIDs: {{{ [15:55:17] jbs@bbn-hn:/home/jbs +$ awk '{print $NF}' netstat-raw.txt | sed -r -e 's|/.*||' | sort -n | uniq > netstat-pids.txt [15:55:58] jbs@bbn-hn:/home/jbs +$ cat netstat-pids.txt - 3101 3243 3261 3628 3693 3697 3767 4091 4107 4115 4147 4433 4470 4546 4576 4614 4798 4839 4893 4895 4898 5306 6868 7695 8021 8162 8655 8683 8813 8831 9667 15118 19278 19396 26601 30574 30677 31265 }}} For some of those, the PID is '-': {{{ [15:56:22] jbs@bbn-hn:/home/jbs +$ egrep -- - netstat-raw.txt tcp 0 0 0.0.0.0:30024 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN - tcp 0 0 :::22017 :::* LISTEN - tcp 0 0 :::2049 :::* LISTEN - }}} We think those are probably NFS-related things, which perhaps are listening from a kernel module rather than a process. Leaving those aside, here are the others: {{{ [15:56:22] jbs@bbn-hn:/home/jbs +$ for pid in $(cat netstat-pids.txt) ; do ps -efwww | egrep "^[^ ]+ +$pid" ; done >& netstat-commands.txt [15:56:56] jbs@bbn-hn:/home/jbs +$ cat netstat-commands.txt root 3101 1 0 Dec06 ? 00:17:01 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5 rpc 3243 1 0 Dec06 ? 00:00:00 rpcbind rpcuser 3261 1 0 Dec06 ? 00:00:00 rpc.statd ldap 3628 1 2 Dec06 ? 07:54:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf root 3693 1 0 Dec06 ? 00:00:00 rpc.rquotad root 3697 1 0 Dec06 ? 00:00:00 rpc.mountd 491 3767 1 0 Dec06 ? 01:04:15 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml rack_bbn 4091 1 0 Dec06 ? 00:00:04 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf root 4107 1 0 Dec06 ? 00:00:05 /usr/sbin/sshd root 4115 1 0 Dec06 ? 00:00:02 xinetd -stayalive -pidfile /var/run/xinetd.pid root 4147 1 0 Dec06 ? 00:03:22 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid mysql 4433 4194 1 Dec06 ? 03:45:51 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=25000 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306 dhcpd 4470 1 0 Dec06 ? 00:01:34 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006 bond2.1008 root 4546 1 0 Dec06 ? 00:00:05 /usr/libexec/postfix/master rabbitmq 4576 1 0 Dec06 ? 00:00:05 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon rabbitmq 4614 4613 2 Dec06 ? 06:26:09 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn" root 4798 1 0 Dec06 ? 00:00:22 /usr/sbin/httpd root 4839 1 0 Dec06 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf root 4893 1 0 Dec06 ? 00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf root 4895 1 0 Dec06 ? 00:00:00 xcatd: SSL listener root 4898 4895 0 Dec06 ? 00:00:00 xcatd: install monitor root 5306 5300 0 Dec06 ? 00:08:52 ../jre/bin/java -classpath ../jre/lib/rt.jar:../jre/lib/jsse.jar:../jre/lib/jce.jar:mail.jar:Framework.jar -Djava.library.path=. Framework.FrameworkManager jonmills 6868 6864 0 Dec11 ? 00:00:06 sshd: jonmills@pts/2 2006 7695 7661 0 Dec06 ? 00:15:09 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml nova 8021 1 0 Dec10 ? 00:00:01 /usr/bin/python /usr/bin/nova-objectstore --config-file /etc/nova/nova.conf --logfile /var/log/nova/objectstore.log nova 8162 1 0 Dec10 ? 00:00:01 /usr/bin/python /usr/bin/nova-api --config-file /etc/nova/nova.conf --logfile /var/log/nova/api.log keystone 8655 1 0 Dec06 ? 01:02:13 /usr/bin/python /usr/bin/keystone-all --config-file /etc/keystone/keystone.conf quantum 8683 1 0 Dec06 ? 00:05:49 python /usr/bin/quantum-server --config-file /etc/quantum/quantum.conf --log-file /var/log/quantum/server.log glance 8813 1 0 Dec06 ? 00:34:10 /usr/bin/python /usr/bin/glance-api --config-file /etc/glance/glance-api.conf --debug --verbose glance 8831 1 0 Dec06 ? 00:07:38 /usr/bin/python /usr/bin/glance-registry --config-file /etc/glance/glance-registry.conf --debug --verbose 2006 9667 1 0 Dec06 ? 00:20:15 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx2g -Djava.awt.headless=true -XX:PermSize=96m -XX:MaxPermSize=128m -XX:+UseParallelOldGC -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start root 15118 1 0 Dec15 ? 00:22:47 /usr/bin/ruby /usr/bin/puppet agent 2006 19278 1 0 Dec17 ? 00:01:34 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=57673 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=52673 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 19396 1 0 Dec17 ? 00:01:36 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=59697 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=54697 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar named 26601 1 0 Dec10 ? 00:02:21 /usr/sbin/named -u named 2006 30574 1 0 Dec14 ? 00:06:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start 2006 30677 1 0 Dec13 ? 00:05:00 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start 2006 31265 1 0 Dec14 ? 00:09:48 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start }}} Just to sanity check that we haven't missed anything: {{{ [15:57:02] jbs@bbn-hn:/home/jbs +$ wc -l netstat-* 38 netstat-commands.txt 39 netstat-pids.txt 77 netstat-raw.txt 154 total }}} The change from raw to pids is because of duplicates, and from pids to commands is because of the '-' PID dropping out. So, let's start plowing through those. These things have the full path to an executable in the process listing, so we can easily round up what RPM they're from: {{{ root 3101 1 0 Dec06 ? 00:17:01 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5 ldap 3628 1 2 Dec06 ? 07:54:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf rack_bbn 4091 1 0 Dec06 ? 00:00:04 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf root 4107 1 0 Dec06 ? 00:00:05 /usr/sbin/sshd root 4147 1 0 Dec06 ? 00:03:22 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid mysql 4433 4194 1 Dec06 ? 03:45:51 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=25000 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306 dhcpd 4470 1 0 Dec06 ? 00:01:34 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006 bond2.1008 root 4546 1 0 Dec06 ? 00:00:05 /usr/libexec/postfix/master rabbitmq 4576 1 0 Dec06 ? 00:00:05 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon rabbitmq 4614 4613 2 Dec06 ? 06:26:09 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn" root 4798 1 0 Dec06 ? 00:00:22 /usr/sbin/httpd root 4893 1 0 Dec06 ? 00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf named 26601 1 0 Dec10 ? 00:02:21 /usr/sbin/named -u named }}} Thus: {{{ [16:11:13] jbs@bbn-hn:/home/jbs +$ for file in /sbin/rsyslogd /usr/sbin/slapd /usr/sbin/sshd /omd/sites/rack_bbn/var/tmp/xinetd /sbin/apcupsd /usr/libexec/mysqld /usr/sbin/dhcpd /usr/libexec/postfix/master /usr/lib64/erlang/erts-5.8.5/bin/epmd /usr/lib64/erlang/erts-5.8.5/bin/beam.smp /usr/sbin/httpd /usr/sbin/vsftpd /usr/sbin/named ; do rpm -qf $file ; done rsyslog-5.8.10-2.el6.x86_64 openldap-servers-2.4.23-26.el6_3.2.x86_64 openssh-server-5.3p1-81.el6.x86_64 file /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package apcupsd-3.14.10-1.el6.x86_64 mysql-server-5.1.61-4.el6.x86_64 dhcp-4.1.1-31.0.1.P1.el6.centos.1.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 erlang-erts-R14B-04.1.el6.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 vsftpd-2.2.2-11.el6.x86_64 bind-9.8.2-0.10.rc1.el6_3.5.x86_64 }}} These things are Python or Ruby programs, so we can check each program (and the interpreters themselves), and see what RPM they're from: {{{ nova 8021 1 0 Dec10 ? 00:00:01 /usr/bin/python /usr/bin/nova-objectstore --config-file /etc/nova/nova.conf --logfile /var/log/nova/objectstore.log nova 8162 1 0 Dec10 ? 00:00:01 /usr/bin/python /usr/bin/nova-api --config-file /etc/nova/nova.conf --logfile /var/log/nova/api.log keystone 8655 1 0 Dec06 ? 01:02:13 /usr/bin/python /usr/bin/keystone-all --config-file /etc/keystone/keystone.conf glance 8813 1 0 Dec06 ? 00:34:10 /usr/bin/python /usr/bin/glance-api --config-file /etc/glance/glance-api.conf --debug --verbose glance 8831 1 0 Dec06 ? 00:07:38 /usr/bin/python /usr/bin/glance-registry --config-file /etc/glance/glance-registry.conf --debug --verbose root 15118 1 0 Dec15 ? 00:22:47 /usr/bin/ruby /usr/bin/puppet agent quantum 8683 1 0 Dec06 ? 00:05:49 python /usr/bin/quantum-server --config-file /etc/quantum/quantum.conf --log-file /var/log/quantum/server.log }}} Thus: {{{ [16:15:34] jbs@bbn-hn:/home/jbs +$ for file in /usr/bin/python /usr/bin/ruby /usr/bin/nova-objectstore /usr/bin/nova-api /usr/bin/keystone-all /usr/bin/glance-api /usr/bin/glance-registry /usr/bin/puppet /usr/bin/quantum-server ; do rpm -qf $file ; done python-2.6.6-29.el6_3.3.x86_64 ruby-1.8.7.352-7.el6_2.x86_64 openstack-nova-objectstore-2012.1.1-exogeni6.el6.noarch openstack-nova-api-2012.1.1-exogeni6.el6.noarch openstack-keystone-2012.1.2-4.el6.noarch openstack-glance-2012.1.1-1.el6.noarch openstack-glance-2012.1.1-1.el6.noarch puppet-3.0.1-1.el6.noarch openstack-quantum-2012.1-8.el6.noarch }}} These things are Java programs, so we can check of the jar files (and Java itself), and see what RPM they're from: {{{ 2006 9667 1 0 Dec06 ? 00:20:15 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx2g -Djava.awt.headless=true -XX:PermSize=96m -XX:MaxPermSize=128m -XX:+UseParallelOldGC -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start 491 3767 1 0 Dec06 ? 01:04:15 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml 2006 7695 7661 0 Dec06 ? 00:15:09 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml 2006 19278 1 0 Dec17 ? 00:01:34 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=57673 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=52673 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 19396 1 0 Dec17 ? 00:01:36 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=59697 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=54697 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar 2006 30574 1 0 Dec14 ? 00:06:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start 2006 30677 1 0 Dec13 ? 00:05:00 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start 2006 31265 1 0 Dec14 ? 00:09:48 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start }}} Thus: {{{ [16:28:37] jbs@bbn-hn:/home/jbs +$ for file in /usr/java/latest/bin/java /opt/orca-11080/tomcat/bin/bootstrap.jar /opt/orca-11080/tomcat/bin/tomcat-juli.jar /opt/flowvisor/libexec/flowvisor/openflow.jar /opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar /opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar /opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar /opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar /opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar /opt/flowvisor/libexec/flowvisor/jsse.jar /opt/flowvisor/libexec/flowvisor/asm-3.0.jar /opt/flowvisor/libexec/flowvisor/cglib-2.2.jar /opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar /opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar /opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar /opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar /opt/flowvisor/libexec/flowvisor/flowvisor.jar /opt/imageproxy/axis2 /opt/imageproxy/axis2/conf /usr/java/latest/lib/tools.jar /opt/imageproxy/axis2/lib/activation-1.1.jar /opt/imageproxy/axis2/lib/antlr-2.7.7.jar /opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar /opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar /opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar /opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar /opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar /opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar /opt/imageproxy/axis2/lib/bcel-5.1.jar /opt/imageproxy/axis2/lib/commons-cli-1.2.jar /opt/imageproxy/axis2/lib/commons-codec-1.3.jar /opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar /opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar /opt/imageproxy/axis2/lib/commons-io-1.4.jar /opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar /opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar /opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar /opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar /opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar /opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar /opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar /opt/imageproxy/axis2/lib/httpcore-4.0.jar /opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar /opt/imageproxy/axis2/lib/jaxb-api-2.1.jar /opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar /opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar /opt/imageproxy/axis2/lib/jaxen-1.1.1.jar /opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar /opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar /opt/imageproxy/axis2/lib/jibx-bind-1.2.jar /opt/imageproxy/axis2/lib/jibx-run-1.2.jar /opt/imageproxy/axis2/lib/jsr311-api-1.0.jar /opt/imageproxy/axis2/lib/juli-6.0.16.jar /opt/imageproxy/axis2/lib/log4j-1.2.15.jar /opt/imageproxy/axis2/lib/mail-1.4.jar /opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar /opt/imageproxy/axis2/lib/neethi-3.0.2.jar /opt/imageproxy/axis2/lib/regexp-1.2.jar /opt/imageproxy/axis2/lib/tribes-6.0.16.jar /opt/imageproxy/axis2/lib/woden-api-1.0M9.jar /opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar /opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar /opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar /opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar /opt/imageproxy/axis2/lib/xalan-2.7.0.jar /opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar /opt/imageproxy/axis2/lib/xml-resolver-1.2.jar /opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar /opt/floodlight/floodlight.jar /opt/floodlight/floodlight.jar /opt/orca-13080/tomcat/bin/bootstrap.jar /opt/orca-13080/tomcat/bin/tomcat-juli.jar /opt/orca-aux/tomcat-7/bin/bootstrap.jar /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar /opt/orca-12080/tomcat/bin/bootstrap.jar /opt/orca-12080/tomcat/bin/tomcat-juli.jar ; do rpm -qf $file ; done jdk-1.6.0_30-fcs.x86_64 file /opt/orca-11080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 file /opt/imageproxy/axis2 is not owned by any package file /opt/imageproxy/axis2/conf is not owned by any package jdk-1.6.0_30-fcs.x86_64 file /opt/imageproxy/axis2/lib/activation-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/antlr-2.7.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/bcel-5.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-cli-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-codec-1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-io-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/httpcore-4.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-api-2.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxen-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-bind-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-run-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jsr311-api-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/juli-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/log4j-1.2.15.jar is not owned by any package file /opt/imageproxy/axis2/lib/mail-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar is not owned by any package file /opt/imageproxy/axis2/lib/neethi-3.0.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/regexp-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/tribes-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-api-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar is not owned by any package file /opt/imageproxy/axis2/lib/xalan-2.7.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/xml-resolver-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar is not owned by any package file /opt/floodlight/floodlight.jar is not owned by any package file /opt/floodlight/floodlight.jar is not owned by any package file /opt/orca-13080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/bootstrap.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package file /opt/orca-12080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package }}} This one is slightly more complicated, because all of the jar files have relative paths: {{{ root 5306 5300 0 Dec06 ? 00:08:52 ../jre/bin/java -classpath ../jre/lib/rt.jar:../jre/lib/jsse.jar:../jre/lib/jce.jar:mail.jar:Framework.jar -Djava.library.path=. Framework.FrameworkManager }}} Let's look in /proc for $PWD for that process: {{{ [16:33:11] jbs@bbn-hn:/home/jbs +$ sudo cat /proc/5306/environ TERM=linuxLD_PRELOAD=../jre/lib/i386/libjsig.soOLDPWD=/LD_LIBRARY_PATH=/usr/local/MegaRAID Storage Manager/jre/lib/amd64/server:/usr/local/MegaRAID Storage Manager/jre/lib/amd64:/usr/local/MegaRAID Storage Manager/jre/../lib/amd64:/usr/local/MegaRAID Storage Manager/Framework:/opt/lsi/Apache:/opt/lsi/Pegasus:/opt/lsi/opensslMSM_PRODUCT=MSMPATH=/sbin:/usr/sbin:/bin:/usr/binrunlevel=3RUNLEVEL=3LANGSH_SOURCED=1PWD=/usr/local/MegaRAID Storage Manager/FrameworkLANG=en_US.UTF-8previous=NPREVLEVEL=NCONSOLETYPE=vtSHLVL=3UPSTART_INSTANCE=MSM_HOME=/usr/local/MegaRAID Storage ManagerUPSTART_EVENTS=runlevelUPSTART_JOB=rc_=../jre/bin/java }}} So that's {{{PWD=/usr/local/MegaRAID Storage Manager/Framework}}}, and therefore: {{{ [16:35:12] jbs@bbn-hn:/home/jbs +$ for file in ../jre/lib/rt.jar ../jre/lib/jsse.jar ../jre/lib/jce.jar mail.jar Framework.jar ; do rpm -qf "/usr/local/MegaRAID Storage Manager/Framework"/$file ; done MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch }}} That leaves these guys without full pathnames: {{{ rpc 3243 1 0 Dec06 ? 00:00:00 rpcbind rpcuser 3261 1 0 Dec06 ? 00:00:00 rpc.statd root 3693 1 0 Dec06 ? 00:00:00 rpc.rquotad root 3697 1 0 Dec06 ? 00:00:00 rpc.mountd root 4115 1 0 Dec06 ? 00:00:02 xinetd -stayalive -pidfile /var/run/xinetd.pid root 4839 1 0 Dec06 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf root 4895 1 0 Dec06 ? 00:00:00 xcatd: SSL listener root 4898 4895 0 Dec06 ? 00:00:00 xcatd: install monitor jonmills 6868 6864 0 Dec11 ? 00:00:06 sshd: jonmills@pts/2 }}} We can find the full pathnames for them, and then round up what RPM they're from: {{{ [16:40:12] jbs@bbn-hn:/home/jbs +$ for file in $(for command in rpcbind rpc.statd rpc.rquotad rpc.mountd xinetd nginx xcatd xcatd sshd ; do which $command | awk '{print $NF}' ; done) ; do rpm -qf $file ; done rpcbind-0.2.0-9.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 quota-3.17-16.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 xinetd-2.3.14-35.el6_3.x86_64 nginx-1.0.15-3.el6.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xCAT-server-2.6.9-snap201111090305.noarch openssh-server-5.3p1-81.el6.x86_64 }}} Putting all of that output together: {{{ rsyslog-5.8.10-2.el6.x86_64 openldap-servers-2.4.23-26.el6_3.2.x86_64 openssh-server-5.3p1-81.el6.x86_64 file /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package apcupsd-3.14.10-1.el6.x86_64 mysql-server-5.1.61-4.el6.x86_64 dhcp-4.1.1-31.0.1.P1.el6.centos.1.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 erlang-erts-R14B-04.1.el6.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 vsftpd-2.2.2-11.el6.x86_64 bind-9.8.2-0.10.rc1.el6_3.5.x86_64 python-2.6.6-29.el6_3.3.x86_64 ruby-1.8.7.352-7.el6_2.x86_64 openstack-nova-objectstore-2012.1.1-exogeni6.el6.noarch openstack-nova-api-2012.1.1-exogeni6.el6.noarch openstack-keystone-2012.1.2-4.el6.noarch openstack-glance-2012.1.1-1.el6.noarch openstack-glance-2012.1.1-1.el6.noarch puppet-3.0.1-1.el6.noarch openstack-quantum-2012.1-8.el6.noarch jdk-1.6.0_30-fcs.x86_64 file /opt/orca-11080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 flowvisor-0.8.2-2.x86_64 file /opt/imageproxy/axis2 is not owned by any package file /opt/imageproxy/axis2/conf is not owned by any package jdk-1.6.0_30-fcs.x86_64 file /opt/imageproxy/axis2/lib/activation-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/antlr-2.7.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/bcel-5.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-cli-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-codec-1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-io-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/httpcore-4.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-api-2.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxen-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-bind-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-run-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jsr311-api-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/juli-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/log4j-1.2.15.jar is not owned by any package file /opt/imageproxy/axis2/lib/mail-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar is not owned by any package file /opt/imageproxy/axis2/lib/neethi-3.0.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/regexp-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/tribes-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-api-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar is not owned by any package file /opt/imageproxy/axis2/lib/xalan-2.7.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/xml-resolver-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar is not owned by any package file /opt/floodlight/floodlight.jar is not owned by any package file /opt/floodlight/floodlight.jar is not owned by any package file /opt/orca-13080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/bootstrap.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package file /opt/orca-12080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch MegaRAID_Storage_Manager-11.08.03-03.noarch rpcbind-0.2.0-9.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 quota-3.17-16.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 xinetd-2.3.14-35.el6_3.x86_64 nginx-1.0.15-3.el6.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xCAT-server-2.6.9-snap201111090305.noarch openssh-server-5.3p1-81.el6.x86_64 }}} Sorted and uniq-ed, here are the RPMs: {{{ MegaRAID_Storage_Manager-11.08.03-03.noarch apcupsd-3.14.10-1.el6.x86_64 bind-9.8.2-0.10.rc1.el6_3.5.x86_64 dhcp-4.1.1-31.0.1.P1.el6.centos.1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 flowvisor-0.8.2-2.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 jdk-1.6.0_30-fcs.x86_64 mysql-server-5.1.61-4.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 nginx-1.0.15-3.el6.x86_64 openldap-servers-2.4.23-26.el6_3.2.x86_64 openssh-server-5.3p1-81.el6.x86_64 openstack-glance-2012.1.1-1.el6.noarch openstack-keystone-2012.1.2-4.el6.noarch openstack-nova-api-2012.1.1-exogeni6.el6.noarch openstack-nova-objectstore-2012.1.1-exogeni6.el6.noarch openstack-quantum-2012.1-8.el6.noarch postfix-2.6.6-2.2.el6_1.x86_64 puppet-3.0.1-1.el6.noarch python-2.6.6-29.el6_3.3.x86_64 quota-3.17-16.el6.x86_64 rpcbind-0.2.0-9.el6.x86_64 rsyslog-5.8.10-2.el6.x86_64 ruby-1.8.7.352-7.el6_2.x86_64 vsftpd-2.2.2-11.el6.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xinetd-2.3.14-35.el6_3.x86_64 }}} And here are the files that aren't from an RPM: {{{ file /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package file /opt/floodlight/floodlight.jar is not owned by any package file /opt/imageproxy/axis2 is not owned by any package file /opt/imageproxy/axis2/conf is not owned by any package file /opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/activation-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/antlr-2.7.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/bcel-5.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-cli-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-codec-1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-io-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/httpcore-4.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-api-2.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxen-1.1.1.jar is not owned by any package file /opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar is not owned by any package file /opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-bind-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jibx-run-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/jsr311-api-1.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/juli-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/log4j-1.2.15.jar is not owned by any package file /opt/imageproxy/axis2/lib/mail-1.4.jar is not owned by any package file /opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar is not owned by any package file /opt/imageproxy/axis2/lib/neethi-3.0.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/regexp-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/tribes-6.0.16.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-api-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar is not owned by any package file /opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar is not owned by any package file /opt/imageproxy/axis2/lib/xalan-2.7.0.jar is not owned by any package file /opt/imageproxy/axis2/lib/xml-resolver-1.2.jar is not owned by any package file /opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar is not owned by any package file /opt/orca-11080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package file /opt/orca-12080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package file /opt/orca-13080/tomcat/bin/bootstrap.jar is not owned by any package file /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/bootstrap.jar is not owned by any package file /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package }}} Let's look at the files that aren't from an RPM first. The xinetd in /omd is a copy of the stock xinetd: {{{ [16:47:17] jbs@bbn-hn:/home/jbs +$ md5sum /omd/sites/rack_bbn/var/tmp/xinetd /usr/sbin/xinetd 4c016506d2a3996cf6967fef958e254f /omd/sites/rack_bbn/var/tmp/xinetd 4c016506d2a3996cf6967fef958e254f /usr/sbin/xinetd }}} Since /usr/sbin/xinetd is from an RPM (see above), this effectively is too. ExoGENI ticket [exoticket:93] is tracking the task of identifying the versions and sources of the others. Next, the RPMs. Here's a list of what repo each came from: {{{ [16:51:11] jbs@bbn-hn:/home/jbs +$ yum list installed MegaRAID_Storage_Manager-11.08.03-03.noarch apcupsd-3.14.10-1.el6.x86_64 bind-9.8.2-0.10.rc1.el6_3.5.x86_64 dhcp-4.1.1-31.0.1.P1.el6.centos.1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 flowvisor-0.8.2-2.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 jdk-1.6.0_30-fcs.x86_64 mysql-server-5.1.61-4.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 nginx-1.0.15-3.el6.x86_64 openldap-servers-2.4.23-26.el6_3.2.x86_64 openssh-server-5.3p1-81.el6.x86_64 openstack-glance-2012.1.1-1.el6.noarch openstack-keystone-2012.1.2-4.el6.noarch openstack-nova-api-2012.1.1-exogeni6.el6.noarch openstack-nova-objectstore-2012.1.1-exogeni6.el6.noarch openstack-quantum-2012.1-8.el6.noarch postfix-2.6.6-2.2.el6_1.x86_64 puppet-3.0.1-1.el6.noarch python-2.6.6-29.el6_3.3.x86_64 quota-3.17-16.el6.x86_64 rpcbind-0.2.0-9.el6.x86_64 rsyslog-5.8.10-2.el6.x86_64 ruby-1.8.7.352-7.el6_2.x86_64 vsftpd-2.2.2-11.el6.x86_64 xCAT-server-2.6.9-snap201111090305.noarch xinetd-2.3.14-35.el6_3.x86_64 Loaded plugins: fastestmirror, priorities, refresh-packagekit, security Loading mirror speeds from cached hostfile * base: mirror.atlanticmetro.net * epel: mirror.metrocast.net * passenger: mirror.hmdc.harvard.edu * puppetlabs-deps: software.exogeni.net * puppetlabs-products: software.exogeni.net * updates: software.exogeni.net 76 packages excluded due to repository priority protections Installed Packages MegaRAID_Storage_Manager.noarch 11.08.03-03 installed apcupsd.x86_64 3.14.10-1.el6 @epel bind.x86_64 32:9.8.2-0.10.rc1.el6_3.5 @updates dhcp.x86_64 12:4.1.1-31.0.1.P1.el6.centos.1 @updates erlang-erts.x86_64 R14B-04.1.el6 @UIedited_1/6.2 flowvisor.x86_64 0.8.2-2 @exogeni httpd.x86_64 2.2.15-15.el6.centos.1 @CentOS Updates/6.2 jdk.x86_64 2000:1.6.0_30-fcs @ExoGENI/6.2 mysql-server.x86_64 5.1.61-4.el6 @base nfs-utils.x86_64 1:1.2.3-26.el6 @base nginx.x86_64 1.0.15-3.el6 @epel openldap-servers.x86_64 2.4.23-26.el6_3.2 @updates openssh-server.x86_64 5.3p1-81.el6 @base openstack-glance.noarch 2012.1.1-1.el6 @epel openstack-keystone.noarch 2012.1.2-4.el6 @openstack-noarch openstack-nova-api.noarch 2012.1.1-exogeni6.el6 @exogeni-noarch openstack-nova-objectstore.noarch 2012.1.1-exogeni6.el6 @exogeni-noarch openstack-quantum.noarch 2012.1-8.el6 @openstack-noarch postfix.x86_64 2:2.6.6-2.2.el6_1 @anaconda-CentOS-201112091719.x86_64/6.2 puppet.noarch 3.0.1-1.el6 @puppetlabs-products python.x86_64 2.6.6-29.el6_3.3 @updates quota.x86_64 1:3.17-16.el6 @anaconda-CentOS-201112091719.x86_64/6.2 rpcbind.x86_64 0.2.0-9.el6 @base rsyslog.x86_64 5.8.10-2.el6 @base ruby.x86_64 1.8.7.352-7.el6_2 @base vsftpd.x86_64 2.2.2-11.el6 @base xCAT-server.noarch 4:2.6.9-snap201111090305 @xCAT/6.2 xinetd.x86_64 2:2.3.14-35.el6_3 @updates }}} "epel", "updates", "UIedited_1/6.2", "CentOS Updates/6.2", "base", and "anaconda-CentOS-201112091719.x86_64/6.2" are all stanard RHEL repos, which have SRPMs in standard locations. Here are the others from that list: {{{ MegaRAID_Storage_Manager.noarch 11.08.03-03 installed flowvisor.x86_64 0.8.2-2 @exogeni jdk.x86_64 2000:1.6.0_30-fcs @ExoGENI/6.2 openstack-keystone.noarch 2012.1.2-4.el6 @openstack-noarch openstack-nova-api.noarch 2012.1.1-exogeni6.el6 @exogeni-noarch openstack-nova-objectstore.noarch 2012.1.1-exogeni6.el6 @exogeni-noarch openstack-quantum.noarch 2012.1-8.el6 @openstack-noarch puppet.noarch 3.0.1-1.el6 @puppetlabs-products xCAT-server.noarch 4:2.6.9-snap201111090305 @xCAT/6.2 }}} I can find the name of a source RPM for each of those: {{{ [16:56:29] jbs@bbn-hn:/home/jbs +$ for package in MegaRAID_Storage_Manager.noarch flowvisor.x86_64 jdk.x86_64 openstack-keystone.noarch openstack-nova-api.noarch openstack-nova-objectstore.noarch openstack-quantum.noarch puppet.noarch xCAT-server.noarch ; do rpm -q $package --qf 'Name : %{NAME}\nSource RPM : %{SOURCERPM}\n\n' ; done Name : MegaRAID_Storage_Manager Source RPM : MegaRAID_Storage_Manager-11.08.03-03.src.rpm Name : flowvisor Source RPM : flowvisor-0.8.2-2.src.rpm Name : jdk Source RPM : jdk-1.6.0_30-fcs.src.rpm Name : openstack-keystone Source RPM : openstack-keystone-2012.1.2-4.el6.src.rpm Name : openstack-nova-api Source RPM : openstack-nova-2012.1.1-exogeni6.el6.src.rpm Name : openstack-nova-objectstore Source RPM : openstack-nova-2012.1.1-exogeni6.el6.src.rpm Name : openstack-quantum Source RPM : openstack-quantum-2012.1-8.el6.src.rpm Name : puppet Source RPM : puppet-3.0.1-1.el6.src.rpm Name : xCAT-server Source RPM : xCAT-server-2.6.9-snap201111090305.src.rpm }}} So, now to connect that with the repo URLs... That's a somewhat manual process, as you need to fish the repo URL out of the relevant file in /etc/yum.repos.d, and then poke around on the web sites. The base URLs: {{{ [16:58:10] jbs@bbn-hn:/home/jbs +$ grep baseurl /etc/yum.repos.d/exogeni.repo baseurl=http://software.exogeni.net/repo/exogeni/6/current/$basearch baseurl=http://software.exogeni.net/repo/exogeni/6/current/noarch [16:58:13] jbs@bbn-hn:/home/jbs +$ grep baseurl /etc/yum.repos.d/openstack.repo baseurl=http://software.exogeni.net/repo/openstack/$basearch baseurl=http://software.exogeni.net/repo/openstack/noarch [16:58:15] jbs@bbn-hn:/home/jbs +$ grep baseurl /etc/yum.repos.d/puppetlabs.repo baseurl=http://software.exogeni.net/repo/puppet/products/$basearch baseurl=http://software.exogeni.net/repo/puppet/dependencies/$basearch baseurl=http://software.exogeni.net/repo/puppet/devel/$basearch baseurl=http://software.exogeni.net/repo/puppet/products/SRPMS baseurl=http://software.exogeni.net/repo/puppet/dependencies/SRPMS baseurl=http://software.exogeni.net/repo/puppet/devel/SRPMS [16:58:16] jbs@bbn-hn:/home/jbs +$ grep -h baseurl /etc/yum.repos.d/xCAT-*.repo baseurl=http://xcat.sourceforge.net/yum/2.6/xcat-core baseurl=http://xcat.sourceforge.net/yum/xcat-dep/rh6/x86_64 }}} Based on those, I found these SRPM URLs: {{{ http://software.exogeni.net/repo/exogeni/6/current/SRPMS/flowvisor-0.8.2-2.src.rpm http://software.exogeni.net/repo/exogeni/6/current/SRPMS/openstack-nova-2012.1.1-exogeni6.el6.src.rpm http://software.exogeni.net/repo/exogeni/6/current/SRPMS/openstack-quantum-2012.1-8.el6.src.rpm http://software.exogeni.net/repo/puppet/products/SRPMS/puppet-3.0.1-1.el6.src.rpm }}} I couldn't find these: * MegaRAID_Storage_Manager-11.08.03-03.src.rpm -- where is this from? * openstack-keystone-2012.1.2-4.el6.src.rpm -- this seems like it should be in the ExoGENI SRPM repo, but isn't. * jdk-1.6.0_30-fcs.src.rpm -- that's ok, though, there's a README file in the ExoGENI SRPM repo explaining why not. * xCAT-server-2.6.9-snap201111090305.src.rpm -- http://xcat.sourceforge.net/yum/2.6/core-snap-srpms has xCAT-rmc-2.6.9-snap201110210637.src.rpm, but that's a different file. I've updated ExoGENI ticket [exoticket:96] to track those down. 2013-01-21 update: I got the wrong SRPM for openstack-quantum: There's an openstack-quantum-neuca SRPM in the ExoGENI repo, but not openstack-quantum. That's actually in the same repo as openstack-keystone, which, as the `yum list` output says, is http://software.exogeni.net/repo/openstack/noarch/, and indeed, there are http://software.exogeni.net/repo/openstack/noarch/openstack-keystone-2012.1.2-4.el6.noarch.rpm and http://software.exogeni.net/repo/openstack/noarch/openstack-quantum-cisco-2012.1-8.el6.noarch.rpm there. I also found http://xcat.sourceforge.net/yum/2.6/core-snap-srpms/xCAT-server-2.6.10-snap201201050319.src.rpm; Victor suggested "I think you were examining the repo during an update". Victor suggested creating a readme file for MegaRAID, and I agreed that this sounds good. == Step 2: identify network-listening software on an !OpenStack worker node == === Overview of Step 2 === '''Using:''' * Using netstat, enumerate processes on bbn-w1 which listen for network connections from outside the node * For each process found: * Use the command-line or `/proc` to determine what executable file is running * Use RPM tools to determine whether the executable file is part of an RPM * Otherwise, use documentation or iterate with the ExoGENI team to determine the origin of the software * For each RPM found, identify a location from which a source RPM for that package can be obtained * For each non-RPM software source found, identify a location from which the source code for that version can be obtained. '''Verify:''' * The source of each network-listening file can be identified * RPM source packages can be found for each RPM-sourced package * The source code and identifiable version (e.g. a git tag) can be found for each non-RPM software source === Results of Step 2 from 2012-08-15 === A list of processes that are listening: {{{ [11:20:57] jbs@bbn-w1:/home/jbs +$ sudo netstat -anp | egrep ':\*.+LISTEN' | egrep -v '(127.0.0.1|192.168.103.10|10.100.0.1|10.103.0.1)' > netstat-raw.txt [11:21:22] jbs@bbn-w1:/home/jbs +$ cat netstat-raw.txt tcp 0 0 0.0.0.0:8139 0.0.0.0:* LISTEN 2572/ruby tcp 0 0 0.0.0.0:5900 0.0.0.0:* LISTEN 18815/qemu-kvm tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1983/rpcbind tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 2272/sshd tcp 0 0 0.0.0.0:33762 0.0.0.0:* LISTEN 2001/rpc.statd tcp 0 0 :::111 :::* LISTEN 1983/rpcbind tcp 0 0 :::22 :::* LISTEN 2272/sshd tcp 0 0 ::1:25 :::* LISTEN 2374/master tcp 0 0 :::6556 :::* LISTEN 2280/xinetd tcp 0 0 :::48004 :::* LISTEN 2001/rpc.statd }}} Stripping that into a list of PIDs: {{{ [11:21:36] jbs@bbn-w1:/home/jbs +$ awk '{print $NF}' netstat-raw.txt | sed -r -e 's|/.*||' | sort -n | uniq > netstat-pids.txt [11:22:03] jbs@bbn-w1:/home/jbs +$ cat netstat-pids.txt 1983 2001 2272 2280 2374 2572 18815 }}} Here are the commands: {{{ [11:22:38] jbs@bbn-w1:/home/jbs +$ for pid in $(cat netstat-pids.txt) ; do ps -efwww | egrep "^[^ ]+ +$pid" ; done >& netstat-commands.txt [11:23:47] jbs@bbn-w1:/home/jbs +$ cat netstat-commands.txt rpc 1983 1 0 Jul17 ? 00:00:01 rpcbind rpcuser 2001 1 0 Jul17 ? 00:00:00 rpc.statd root 2272 1 0 Jul17 ? 00:00:00 /usr/sbin/sshd root 2280 1 0 Jul17 ? 00:00:05 xinetd -stayalive -pidfile /var/run/xinetd.pid root 2374 1 0 Jul17 ? 00:00:10 /usr/libexec/postfix/master root 2572 1 0 Jul17 ? 01:55:23 /usr/bin/ruby /usr/sbin/puppetd qemu 18815 1 0 Aug14 ? 00:01:20 /usr/libexec/qemu-kvm -S -M pc-0.14 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name instance-00000d78 -uuid efb7e543-0ed8-9dac-76dd-12345b4be3a9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000d78.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -kernel /var/lib/nova/instances/instance-00000d78/kernel -initrd /var/lib/nova/instances/instance-00000d78/ramdisk -append root=/dev/vda console=ttyS0 selinux=0 -drive file=/var/lib/nova/instances/instance-00000d78/disk,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/nova/instances/instance-00000d78/disk.local,if=none,id=drive-virtio-disk1,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=22,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:16:3e:69:b0:db,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:e6:5e:1a,bus=pci.0,addr=0x4 -chardev file,id=charserial0,path=/var/lib/nova/instances/instance-00000d78/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 }}} Just to sanity check that we haven't missed anything: {{{ [11:23:51] jbs@bbn-w1:/home/jbs +$ wc -l netstat-* 7 netstat-commands.txt 7 netstat-pids.txt 10 netstat-raw.txt 24 total }}} The change from raw to pids is because of duplicates: Two for 1983, two for 2001, and two for 2272. Let's plow through each of those. This is from an RPM: {{{ rpc 1983 1 0 Jul17 ? 00:00:01 rpcbind [11:27:39] jbs@bbn-w1:/home/jbs +$ which rpcbind rpcbind is /sbin/rpcbind [11:27:41] jbs@bbn-w1:/home/jbs +$ rpm -qf /sbin/rpcbind rpcbind-0.2.0-8.el6.x86_64 }}} This is from an RPM: {{{ rpcuser 2001 1 0 Jul17 ? 00:00:00 rpc.statd [11:27:45] jbs@bbn-w1:/home/jbs +$ which rpc.statd rpc.statd is /sbin/rpc.statd [11:28:02] jbs@bbn-w1:/home/jbs +$ rpm -qf /sbin/rpc.statd nfs-utils-1.2.3-15.el6.x86_64 }}} This is from an RPM: {{{ root 2272 1 0 Jul17 ? 00:00:00 /usr/sbin/sshd [11:28:35] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/sbin/sshd openssh-server-5.3p1-70.el6_2.2.x86_64 }}} This is from an RPM: {{{ root 2280 1 0 Jul17 ? 00:00:05 xinetd -stayalive -pidfile /var/run/xinetd.pid [11:28:50] jbs@bbn-w1:/home/jbs +$ which xinetd xinetd is /usr/sbin/xinetd [11:28:52] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/sbin/xinetd xinetd-2.3.14-33.el6.x86_64 }}} This is from an RPM: {{{ root 2374 1 0 Jul17 ? 00:00:10 /usr/libexec/postfix/master [11:28:56] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/libexec/postfix/master postfix-2.6.6-2.2.el6_1.x86_64 }}} These are from RPMs: {{{ root 2572 1 0 Jul17 ? 01:55:23 /usr/bin/ruby /usr/sbin/puppetd [11:29:12] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/bin/ruby ruby-1.8.7.352-4.el6_2.x86_64 [11:29:29] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/sbin/puppetd puppet-2.6.14-1.el6.noarch }}} This is not from an RPM; but there seems to be a qemu-kvm RPM; but it doesn't contain any files: {{{ qemu 18815 1 0 Aug14 ? 00:01:20 /usr/libexec/qemu-kvm -S -M pc-0.14 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name instance-00000d78 -uuid efb7e543-0ed8-9dac-76dd-12345b4be3a9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000d78.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -kernel /var/lib/nova/instances/instance-00000d78/kernel -initrd /var/lib/nova/instances/instance-00000d78/ramdisk -append root=/dev/vda console=ttyS0 selinux=0 -drive file=/var/lib/nova/instances/instance-00000d78/disk,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/nova/instances/instance-00000d78/disk.local,if=none,id=drive-virtio-disk1,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=22,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:16:3e:69:b0:db,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:e6:5e:1a,bus=pci.0,addr=0x4 -chardev file,id=charserial0,path=/var/lib/nova/instances/instance-00000d78/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 [11:29:34] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/libexec/qemu-kvm file /usr/libexec/qemu-kvm is not owned by any package [11:30:58] jbs@bbn-w1:/home/jbs +$ rpm -q qemu-kvm qemu-kvm-0.15.0-3.el6.x86_64 [11:31:15] jbs@bbn-w1:/home/jbs +$ rpm -ql qemu-kvm (contains no files) }}} Ah: {{{ [11:31:17] jbs@bbn-w1:/home/jbs +$ rpm -qi qemu-kvm Name : qemu-kvm Relocations: (not relocatable) Version : 0.15.0 Vendor: (none) Release : 3.el6 Build Date: Thu Nov 24 12:24:20 2011 Install Date: Fri Feb 24 08:56:13 2012 Build Host: osc-build.vm.griddynamics.net Group : Development/Tools Source RPM: qemu-0.15.0-3.el6.src.rpm Size : 0 License: GPLv2+ and LGPLv2+ and BSD Signature : RSA/SHA1, Thu Nov 24 12:25:34 2011, Key ID 499590cc596f1e88 URL : http://www.qemu.org/ Summary : QEMU metapackage for KVM support Description : This is a meta-package that provides a qemu-system- package for native architectures where kvm can be enabled. For example, in an x86 system, this will install qemu-system-x86 }}} And ah, /usr/libexec/qemu-kvm is just a symlink, to an executable which *is* from an RPM: {{{ [11:31:47] jbs@bbn-w1:/home/jbs +$ file /usr/libexec/qemu-kvm /usr/libexec/qemu-kvm: symbolic link to `/usr/bin/qemu-kvm' [11:32:17] jbs@bbn-w1:/home/jbs +$ rpm -qf /usr/bin/qemu-kvm qemu-system-x86-0.15.0-3.el6.x86_64 }}} Ok then. So, that's these RPMs: {{{ rpcbind-0.2.0-8.el6.x86_64 nfs-utils-1.2.3-15.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 xinetd-2.3.14-33.el6.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 ruby-1.8.7.352-4.el6_2.x86_64 puppet-2.6.14-1.el6.noarch qemu-system-x86-0.15.0-3.el6.x86_64 }}} Those all look like standard CentOS things: {{{ [11:39:19] jbs@bbn-w1:/home/jbs +$ yum -C list installed rpcbind-0.2.0-8.el6.x86_64 nfs-utils-1.2.3-15.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 xinetd-2.3.14-33.el6.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 ruby-1.8.7.352-4.el6_2.x86_64 puppet-2.6.14-1.el6.noarch qemu-system-x86-0.15.0-3.el6.x86_64 Loaded plugins: fastestmirror, priorities, refresh-packagekit, security 190 packages excluded due to repository priority protections Installed Packages nfs-utils.x86_64 1:1.2.3-15.el6 @anaconda-CentOS-201112091719.x86_64/6.2 openssh-server.x86_64 5.3p1-70.el6_2.2 @CentOS Updates/6.2 postfix.x86_64 2:2.6.6-2.2.el6_1 @anaconda-CentOS-201112091719.x86_64/6.2 puppet.noarch 2.6.14-1.el6 @epel qemu-system-x86.x86_64 2:0.15.0-3.el6 @ExoGENI/6.2 rpcbind.x86_64 0.2.0-8.el6 @anaconda-CentOS-201112091719.x86_64/6.2 ruby.x86_64 1.8.7.352-4.el6_2 @updates xinetd.x86_64 2:2.3.14-33.el6 @anaconda-CentOS-201112091719.x86_64/6.2 }}} ('yum -C' to use cached info only, because these bbn-w1 can't do DNS right now.) So, that identifies the source of each network-listening file, and since they're all standard CentOS RPMs, source RPMs for them can be found in standard CentOS places. I've created ExoGENI ticket [exoticket:104] for the DNS issue, but it's not a blocker for this test. == Step 3: verify VLANs on the rack management switch == === Overview of Step 3 === '''Using:''' * Establish a privileged login to the 8052 (management) switch * Obtain the list of all VLAN mappings for all interfaces * Determine which interfaces connect to worker nodes * Create a listing of worker node control interfaces to VLANs by interface type '''Verify:''' * All worker node IMM interfaces are access ports on VLAN 1006 * All worker node iSCSI interfaces are access ports on VLAN 1009 * Control interfaces for all workers in !OpenStack mode are access ports on VLAN 1007 * Control interfaces for all workers in bare metal mode are access ports on VLAN 1008 === Results of Step 3 from 2012-10-10 === Established a privileged login to the 8052 management switch: {{{ [lnevers@bbn-hn]$ ssh bbn-8052 Enter radius password: IBM Networking Operating System RackSwitch G8052. bbn-8052.bbn.xo> }}} Determined all VLAN mappings for all interfaces: {{{ bbn-8052.bbn.xo>show vlan VLAN Name Status Ports ---- -------------------------------- ------ ------------------------- 1 Default VLAN ena XGE3 20 VLAN 20 ena empty 829 VLAN 829 ena empty 1006 ExoGENI Management Private ena 1-20 XGE4 1007 OpenStack Worker Nodes ena 17-20 37-44 1008 xCAT (Bare Metal) ena 17-20 45 46 1009 iSCSI Worker ena 25-36 1010 Public Internet Connectivity ena 47-XGE2 1011 OpenStack VMs ena 17-20 37-44 1012 iSCSI Head Node ena 21-24 bbn-8052.bbn.xo> }}} Determined which interfaces connect to worker nodes, by displaying the ports associated with the OpenStack Worker nodes,and xCAT VLANs: {{{ bbn-8052.bbn.xo>show vlan 1006 VLAN Name Status Ports ---- -------------------------------- ------ ------------------------- 1006 ExoGENI Management Private ena 1-20 XGE4 Private-VLAN Type Mapped-To Status Ports ------------ --------- ------------------ ---------- -------------------- 1006 - - dis 1-20 XGE4 }}} ''Note: Worker node IMM interfaces are accessed ports on VLAN 1006'' {{{ bbn-8052.bbn.xo>show vlan 1007 VLAN Name Status Ports ---- -------------------------------- ------ ------------------------- 1007 OpenStack Worker Nodes ena 17-20 37-44 Private-VLAN Type Mapped-To Status Ports ------------ --------- ------------------ ---------- -------------------- 1007 - - dis 17-20 37-44 }}} ''Note: Control interfaces for all workers in !OpenStack mode are access ports on VLAN 1007'' {{{ bbn-8052.bbn.xo>show vlan 1008 VLAN Name Status Ports ---- -------------------------------- ------ ------------------------- 1008 xCAT (Bare Metal) ena 17-20 45 46 Private-VLAN Type Mapped-To Status Ports ------------ --------- ------------------ ---------- -------------------- 1008 - - dis 17-20 45 46 }}} '' Note: Worker node iSCSI interfaces are access ports on VLAN 1009'' {{{ VLAN Name Status Ports ---- -------------------------------- ------ ------------------------- 1009 iSCSI Worker ena 25-36 Private-VLAN Type Mapped-To Status Ports ------------ --------- ------------------ ---------- -------------------- 1009 - - dis 25-36 }}} '' Note: Control interfaces for all workers in bare metal mode are access ports on VLAN 1008'' Created a listing of worker node control interfaces to VLANs by interface type || Port || Worker Node || Interface type || VLAN || || || || 5 || bbn-w1 || IMM || 1006 || || 6 || bbn-w2 || IMM || 1006 || || 7 || bbn-w3 || IMM || 1006 || || 2 || bbn-w4 || IMM || 1006 || || 10 || bbn-w5 || IMM || 1006 || || 11 || bbn-w6 || IMM || 1006 || || 12 || bbn-w7 || IMM || 1006 || || 13 || bbn-w8 || IMM || 1006 || || 14 || bbn-w9 || IMM || 1006 || || 15 || bbn-w10 || IMM || 1006 || || 27 || bbn-w1 || iSCSI Worker || 1009 || || 28 || bbn-w2 || iSCSI Worker || 1009 || || 29 || bbn-w3 || iSCSI Worker || 1009 || || 30 || bbn-w4 || iSCSI Worker || 1009 || || 31 || bbn-w5 || iSCSI Worker || 1009 || || 32 || bbn-w6 || iSCSI Worker || 1009 || || 33 || bbn-w7 || iSCSI Worker || 1009 || || 34 || bbn-w8 || iSCSI Worker || 1009 || || 35 || bbn-w9 || iSCSI Worker || 1009 || || 36 || bbn-w10 || iSCSI Worker || 1009 || || 37 || bbn-w1 || !OpenStack Worker|| 1007 || || 38 || bbn-w2 || !OpenStack Worker|| 1007 || || 39 || bbn-w3 || !OpenStack Worker|| 1007 || || 40 || bbn-w4 || !OpenStack Worker|| 1007 || || 41 || bbn-w5 || !OpenStack Worker|| 1007 || || 42 || bbn-w6 || !OpenStack Worker|| 1007 || || 43 || bbn-w7 || !OpenStack Worker|| 1007 || || 44 || bbn-w8 || !OpenStack Worker|| 1007 || || 45 || bbn-w9 || (xCAT)Bare Metal|| 1008 || || 46 || bbn-w10 || (xCAT)Bare Metal|| 1008 || == Step 4: verify MAC addresses on the rack management switch == === Overview of Step 4 === '''Using:''' * Establish a privileged login to the 8052 (management) switch * Obtain a list of the full MAC address table of the switch * Use interface listings on hosts and devices to determine the identities of all MAC addresses '''Verify:''' * It is possible to identify and classify every MAC address visible on the switch === Results of Step 4 from 2012-10-10 === Established a privileged login to the 8052 management switch: {{{ [lnevers@bbn-hn]$ ssh bbn-8052 Enter radius password: IBM Networking Operating System RackSwitch G8052. bbn-8052.bbn.xo> }}} Determined full list of MAC address table for switch: {{{ bbn-8052.bbn.xo>show mac-address-table Mac address Aging Time: 300 Total number of FDB entries : 46 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:10:18:a8:73:d2 1012 53 TRK 00:10:18:a8:73:d4 1007 54 TRK 00:10:18:a8:73:d6 1007 54 TRK 00:80:e5:1d:f7:13 1012 23 FWD 00:80:e5:25:2e:a3 1012 24 FWD 00:80:e5:2d:01:64 1006 3 FWD 00:80:e5:2d:03:ac 1006 4 FWD 02:16:3e:2a:57:7d 1011 38 FWD 02:16:3e:2f:cd:83 1011 42 FWD 02:16:3e:3d:17:c2 1011 44 FWD 02:16:3e:4d:6a:83 1011 37 FWD 02:16:3e:73:13:54 1011 43 FWD 02:16:3e:76:da:63 1011 41 FWD 02:16:3e:7e:7d:d2 1011 54 TRK 08:17:f4:b5:2a:00 1006 XGE4 FWD 08:17:f4:b5:2a:3f 1006 XGE4 FWD 5c:f3:fc:6b:10:a8 1006 54 TRK 5c:f3:fc:6b:10:a8 1007 54 TRK 5c:f3:fc:6b:10:a8 1008 54 TRK 5c:f3:fc:6b:10:aa 1007 54 TRK 5c:f3:fc:ba:13:34 1010 47 FWD 5c:f3:fc:ba:13:36 1012 53 TRK 5c:f3:fc:ba:4e:78 1007 38 FWD 5c:f3:fc:ba:51:d4 1007 40 FWD 5c:f3:fc:ba:56:68 1007 37 FWD 5c:f3:fc:ba:65:c0 1007 39 FWD 5c:f3:fc:c0:14:c5 1006 1 FWD 5c:f3:fc:c0:50:09 1006 6 FWD 5c:f3:fc:c0:53:65 1006 2 FWD 5c:f3:fc:c0:57:f9 1006 5 FWD 5c:f3:fc:c0:67:51 1006 7 FWD 6c:ae:8b:1b:e7:8a 1008 46 FWD 6c:ae:8b:1b:e7:8e 1006 15 FWD 6c:ae:8b:1b:e7:f2 1007 42 FWD 6c:ae:8b:1b:e7:f6 1006 11 FWD 6c:ae:8b:1b:e9:d2 1008 45 FWD 6c:ae:8b:1b:e9:d6 1006 14 FWD 6c:ae:8b:1b:ee:fa 1007 41 FWD 6c:ae:8b:1b:ee:fe 1006 10 FWD 6c:ae:8b:1b:ef:52 1007 43 FWD 6c:ae:8b:1b:ef:56 1006 12 FWD 6c:ae:8b:1b:f0:aa 1007 44 FWD 6c:ae:8b:1b:f0:ae 1006 13 FWD cc:ef:48:7a:7a:a9 1010 48 FWD f8:c0:01:bb:f8:c0 1010 48 FWD f8:c0:01:bb:f8:cb 1006 9 FWD }}} Used interface listings on hosts and devices to determine the identities of all MAC addresses on the switch. {{{ bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d2 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:10:18:a8:73:d2 1012 53 TRK bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d4 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:10:18:a8:73:d4 1007 54 TRK bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d6 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:10:18:a8:73:d6 1007 54 TRK bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:1d:f7:13 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:80:e5:1d:f7:13 1012 23 FWD bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:25:2e:a3 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:80:e5:25:2e:a3 1012 24 FWD bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:2d:01:64 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:80:e5:2d:01:64 1006 3 FWD bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:2d:03:ac MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 00:80:e5:2d:03:ac 1006 4 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:2a:57:7d MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:2a:57:7d 1011 38 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:2f:cd:83 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:2f:cd:83 1011 42 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:3d:17:c2 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:3d:17:c2 1011 44 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:4d:6a:83 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:4d:6a:83 1011 37 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:73:13:54 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:73:13:54 1011 43 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:76:da:63 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:76:da:63 1011 41 FWD bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:7e:7d:d2 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 02:16:3e:7e:7d:d2 1011 54 TRK bbn-8052.bbn.xo>show mac-address-table address 08:17:f4:b5:2a:00 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 08:17:f4:b5:2a:00 1006 XGE4 FWD bbn-8052.bbn.xo>show mac-address-table address 08:17:f4:b5:2a:3f MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 08:17:f4:b5:2a:3f 1006 XGE4 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:6b:10:a8 1007 54 TRK 5c:f3:fc:6b:10:a8 1006 54 TRK 5c:f3:fc:6b:10:a8 1008 54 TRK bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:6b:10:a8 1007 54 TRK 5c:f3:fc:6b:10:a8 1006 54 TRK 5c:f3:fc:6b:10:a8 1008 54 TRK bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:6b:10:a8 1007 54 TRK 5c:f3:fc:6b:10:a8 1006 54 TRK 5c:f3:fc:6b:10:a8 1008 54 TRK bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:aa MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:6b:10:aa 1007 54 TRK bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:13:34 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:13:34 1010 47 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:13:36 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:13:36 1012 53 TRK bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:4e:78 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:4e:78 1007 38 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:51:d4 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:51:d4 1007 40 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:56:68 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:56:68 1007 37 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:65:c0 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:ba:65:c0 1007 39 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:14:c5 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:c0:14:c5 1006 1 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:50:09 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:c0:50:09 1006 6 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:53:65 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:c0:53:65 1006 2 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:57:f9 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:c0:57:f9 1006 5 FWD bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:67:51 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 5c:f3:fc:c0:67:51 1006 7 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:8a MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e7:8a 1008 46 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:8e MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e7:8e 1006 15 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:f2 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e7:f2 1007 42 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:f6 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e7:f6 1006 11 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e9:d2 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e9:d2 1008 45 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e9:d6 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:e9:d6 1006 14 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ee:fa MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:ee:fa 1007 41 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ee:fe MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:ee:fe 1006 10 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ef:52 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:ef:52 1007 43 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ef:56 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:ef:56 1006 12 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:f0:aa MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:f0:aa 1007 44 FWD bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:f0:ae MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- 6c:ae:8b:1b:f0:ae 1006 13 FWD bbn-8052.bbn.xo>show mac-address-table address cc:ef:48:7a:7a:a9 MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- cc:ef:48:7a:7a:a9 1010 48 FWD bbn-8052.bbn.xo>show mac-address-table address f8:c0:01:bb:f8:c0 MAC f8:c0:01:bb:f8:c0 not in FDB. bbn-8052.bbn.xo>show mac-address-table address f8:c0:01:bb:f8:cb MAC address VLAN Port Trnk State Permanent ----------------- -------- ------- ---- ----- --------- f8:c0:01:bb:f8:cb 1006 9 FWD }}} Following is the mapping of the MAC address listed in ''show mac-address-table'' command ouput above. The mapping is shown for "FWD" ports to hosts: || MAC Address || VLAN || PORT || PORT STATE || Hostname || || || 00:80:e5:1d:f7:13 || 1012 || 23 || FWD || head-target1.iscsi || || 00:80:e5:25:2e:a3 || 1012 || 24 || FWD || head-target2.iscsi || || 00:80:e5:2d:01:64 || 1006 || 3 || FWD || || || 00:80:e5:2d:03:ac || 1006 || 4 || FWD || || || 02:16:3e:2a:57:7d || 1011 || 38 || FWD || || || 02:16:3e:2f:cd:83 || 1011 || 42 || FWD || || || 02:16:3e:3d:17:c2 || 1011 || 44 || FWD || || || 02:16:3e:4d:6a:83 || 1011 || 37 || FWD || || || 02:16:3e:73:13:54 || 1011 || 43 || FWD || || || 02:16:3e:76:da:63 || 1011 || 41 || FWD || || || 08:17:f4:b5:2a:00 || 1006 || XGE4 || FWD || bbn-8264.bbn.xo || || 08:17:f4:b5:2a:3f || 1006 || XGE4 || FWD || || || 5c:f3:fc:ba:13:34 || 1010 || 47 || FWD || || || 5c:f3:fc:ba:4e:78 || 1007 || 38 || FWD || bbn-w2.local || || 5c:f3:fc:ba:51:d4 || 1007 || 40 || FWD || bbn-w4.local || || 5c:f3:fc:ba:56:68 || 1007 || 37 || FWD || bbn-w1.local || || 5c:f3:fc:ba:65:c0 || 1007 || 39 || FWD || bbn-w3.local || || 5c:f3:fc:c0:14:c5 || 1006 || 1 || FWD || bbn-hn.ipmi || || 5c:f3:fc:c0:50:09 || 1006 || 6 || FWD || bbn-w2.ipmi || || 5c:f3:fc:c0:53:65 || 1006 || 2 || FWD || bbn-w4.ipmi || || 5c:f3:fc:c0:57:f9 || 1006 || 5 || FWD || bbn-w1.ipmi || || 5c:f3:fc:c0:67:51 || 1006 || 7 || FWD || bbn-w3.ipmi || || 6c:ae:8b:1b:e7:8a || 1008 || 46 || FWD || bbn-w10.local || || 6c:ae:8b:1b:e7:8e || 1006 || 15 || FWD || bbn-w10.ipmi || || 6c:ae:8b:1b:e7:f2 || 1007 || 42 || FWD || bbn-w6.local || || 6c:ae:8b:1b:e7:f6 || 1006 || 11 || FWD || || || 6c:ae:8b:1b:e9:d2 || 1008 || 45 || FWD || bbn-w9.local || || 6c:ae:8b:1b:e9:d6 || 1006 || 14 || FWD || bbn-w9.ipmi || || 6c:ae:8b:1b:ee:fa || 1007 || 41 || FWD || bbn-w5.local || || 6c:ae:8b:1b:ee:fe || 1006 || 10 || FWD || bbn-w5.ipmi || || 6c:ae:8b:1b:ef:52 || 1007 || 43 || FWD || bbn-w7.local || || 6c:ae:8b:1b:ef:56 || 1006 || 12 || FWD || bbn-w7.ipmi || || 6c:ae:8b:1b:f0:aa || 1007 || 44 || FWD || bbn-w8.local || || 6c:ae:8b:1b:f0:ae || 1006 || 13 || FWD || bbn-w8.ipmi || || cc:ef:48:7a:7a:a9 || 1010 || 48 || FWD || ctrl-gw.exogeni.gpolab.bbn.com || || f8:c0:01:bb:f8:cb || 1006 || 9 || FWD || bbn-ssg.bbn.xo ||