wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus/EG-MON-1

Version 19 (modified by lnevers@bbn.com, 12 years ago) (diff)

--

Detailed test plan for EG-MON-1: Control Network Software and VLAN Inspection Test

This page is GPO's working page for performing EG-MON-1. It is public for informational purposes, but it is not an official status report. See GENIRacksHome/ExogeniRacks/AcceptanceTestStatus for the current status of ExoGENI acceptance tests.

Last update: 2012-10-10

Page format

  • The status chart summarizes the state of this test
  • The high-level description from test plan contains text copied exactly from the public test plan and acceptance criteria pages.
  • The steps contain things i will actually do/verify:
    • Steps may be composed of related substeps where i find this useful for clarity
    • Each step is either a preparatory step (identified by "(prep)") or a verification step (the default):
      • Preparatory steps are just things we have to do. They're not tests of the rack, but are prerequisites for subsequent verification steps
      • Verification steps are steps in which we will actually look at rack output and make sure it is as expected. They contain a Using: block, which lists the steps to run the verification, and an Expect: block which lists what outcome is expected for the test to pass.

Status of test

Meaning of states:

Step State Date completed Open Tickets Closed Tickets/Comments
1 Color(orange,Blocked)? exoticket:93
exoticket:96
exoticket:98
complete except for issues in open tickets
(exoticket:95)
2 Color(lightgreen,Pass)? 2012-08-15
3 Color(#63B8FF,In Progress)? (exoticket:10)
4 Color(lightgreen,Pass)? 2012-10-10 (exoticket:10)

High-level description from test plan

This test inspects the state of the rack control network, infrastructure nodes, and system software.

Procedure

  • A site administrator enumerates processes on each of the head node and an OpenStack worker node which listen for network connections from other nodes, identifies what version of what software package is in use for each, and verifies that we know the source of each piece of software and could get access to its source code.
  • A site administrator reviews the configuration of the rack management switch and verifies that each worker node's control interfaces are on the expected VLANs for that worker node's function (OpenStack or bare metal).
  • A site administrator reviews the MAC address table on the management switch, and verifies that all entries are identifiable and expected.

Criteria to verify as part of this test

  • VI.09. A public document explains how to identify the software versions and system file configurations running on the rack, and how to get information about recent changes to the rack software and configuration. (F.5)
  • VI.11. A public document describes the GENI software running on the rack, and explains how to get access to the source code of each piece of GENI software. (F.6)
  • VII.03. Site administrators can understand the expected control and dataplane network behavior of their rack. (F.2)
  • VII.04. Site administrators can view and investigate current system and network activity on their rack. (F.2)
  • VII.06. A site administrator can verify the control software and configurations on the rack at some point in time. (F.5)
  • VII.08. A site administrator can get access to source code for the version of each piece of GENI code installed on their site rack at some point in time. (F.6)
  • VII.09. A site administrator can determine the MAC addresses of all physical host interfaces, all network device interfaces, all active experimental VMs, and all recently-terminated experimental VMs. (C.3.f)
  • VII.10. A site administrator can locate current and recent CPU and memory utilization for each rack network device, and can find recent changes or errors in a log. (D.6.a)
  • VII.12. For each infrastructure and experimental host, a site administrator can locate current and recent uptime, CPU, disk, and memory utilization, interface traffic counters, process counts, and active user counts. (D.6.b)
  • VII.13. A site administrator can locate recent syslogs for all infrastructure and experimental hosts. (D.6.b)

Step 1: identify network-listening software on the head node

Overview of Step 1

Using:

  • Using netstat, enumerate processes on bbn-hn which listen for network connections from outside the node
  • For each process found:
    • Use the command-line or /proc to determine what executable file is running
    • Use RPM tools to determine whether the executable file is part of an RPM
    • Otherwise, use documentation or iterate with the ExoGENI team to determine the origin of the software
  • For each RPM found, identify a location from which a source RPM for that package can be obtained
  • For each non-RPM software source found, identify a location from which the source code for that version can be obtained.

Verify:

  • The source of each network-listening file can be identified
  • RPM source packages can be found for each RPM-sourced package
  • The source code and identifiable version (e.g. a git tag) can be found for each non-RPM software source

Results of Step 1 from 2012-08-03

A list of processes that are listening:

[11:11:51] jbs@bbn-hn:/home/jbs
+$ sudo netstat -anp | egrep ':\*.+LISTEN' | egrep -v '(127.0.0.1|192.168.103.10|10.100.0.1|10.103.0.1)' > netstat-raw.txt

[11:17:13] jbs@bbn-hn:/home/jbs
+$ cat netstat-raw.txt 
tcp        0      0 0.0.0.0:8811                0.0.0.0:*                   LISTEN      29140/python        
tcp        0      0 192.1.242.3:8139            0.0.0.0:*                   LISTEN      724/ruby            
tcp        0      0 0.0.0.0:36843               0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:875                 0.0.0.0:*                   LISTEN      3258/rpc.rquotad    
tcp        0      0 0.0.0.0:8812                0.0.0.0:*                   LISTEN      29162/python        
tcp        0      0 0.0.0.0:9292                0.0.0.0:*                   LISTEN      28780/python        
tcp        0      0 0.0.0.0:47308               0.0.0.0:*                   LISTEN      2806/rpc.statd      
tcp        0      0 0.0.0.0:8813                0.0.0.0:*                   LISTEN      29185/python        
tcp        0      0 0.0.0.0:8781                0.0.0.0:*                   LISTEN      28606/python        
tcp        0      0 0.0.0.0:8814                0.0.0.0:*                   LISTEN      29205/python        
tcp        0      0 0.0.0.0:8782                0.0.0.0:*                   LISTEN      28701/python        
tcp        0      0 0.0.0.0:8815                0.0.0.0:*                   LISTEN      29205/python        
tcp        0      0 0.0.0.0:8783                0.0.0.0:*                   LISTEN      28814/python        
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      2780/rpcbind        
tcp        0      0 0.0.0.0:8816                0.0.0.0:*                   LISTEN      29185/python        
tcp        0      0 0.0.0.0:8784                0.0.0.0:*                   LISTEN      28878/python        
tcp        0      0 0.0.0.0:14000               0.0.0.0:*                   LISTEN      19259/java          
tcp        0      0 0.0.0.0:8817                0.0.0.0:*                   LISTEN      29162/python        
tcp        0      0 0.0.0.0:8785                0.0.0.0:*                   LISTEN      28878/python        
tcp        0      0 0.0.0.0:4369                0.0.0.0:*                   LISTEN      4123/epmd           
tcp        0      0 0.0.0.0:10514               0.0.0.0:*                   LISTEN      2717/rsyslogd       
tcp        0      0 0.0.0.0:8818                0.0.0.0:*                   LISTEN      29140/python        
tcp        0      0 0.0.0.0:8786                0.0.0.0:*                   LISTEN      28814/python        
tcp        0      0 0.0.0.0:8787                0.0.0.0:*                   LISTEN      28701/python        
tcp        0      0 0.0.0.0:8788                0.0.0.0:*                   LISTEN      28606/python        
tcp        0      0 0.0.0.0:54004               0.0.0.0:*                   LISTEN      3274/rpc.mountd     
tcp        0      0 0.0.0.0:51413               0.0.0.0:*                   LISTEN      2378/java           
tcp        0      0 0.0.0.0:21                  0.0.0.0:*                   LISTEN      23122/vsftpd        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      3634/sshd           
tcp        0      0 0.0.0.0:8791                0.0.0.0:*                   LISTEN      28903/python        
tcp        0      0 0.0.0.0:11000               0.0.0.0:*                   LISTEN      29224/java          
tcp        0      0 0.0.0.0:8792                0.0.0.0:*                   LISTEN      28923/python        
tcp        0      0 0.0.0.0:49784               0.0.0.0:*                   LISTEN      3274/rpc.mountd     
tcp        0      0 0.0.0.0:8793                0.0.0.0:*                   LISTEN      28943/python        
tcp        0      0 0.0.0.0:3001                0.0.0.0:*                   LISTEN      23124/xcatd         
tcp        0      0 0.0.0.0:8794                0.0.0.0:*                   LISTEN      28965/python        
tcp        0      0 0.0.0.0:3002                0.0.0.0:*                   LISTEN      23127/xcatd         
tcp        0      0 0.0.0.0:8795                0.0.0.0:*                   LISTEN      28965/python        
tcp        0      0 0.0.0.0:8796                0.0.0.0:*                   LISTEN      28943/python        
tcp        0      0 192.1.242.3:636             0.0.0.0:*                   LISTEN      3199/slapd          
tcp        0      0 0.0.0.0:8797                0.0.0.0:*                   LISTEN      28923/python        
tcp        0      0 0.0.0.0:8798                0.0.0.0:*                   LISTEN      28903/python        
tcp        0      0 0.0.0.0:3551                0.0.0.0:*                   LISTEN      3660/apcupsd        
tcp        0      0 0.0.0.0:12000               0.0.0.0:*                   LISTEN      19470/java          
tcp        0      0 0.0.0.0:53504               0.0.0.0:*                   LISTEN      3274/rpc.mountd     
tcp        0      0 0.0.0.0:8801                0.0.0.0:*                   LISTEN      28987/python        
tcp        0      0 0.0.0.0:2049                0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:8802                0.0.0.0:*                   LISTEN      29016/python        
tcp        0      0 0.0.0.0:8803                0.0.0.0:*                   LISTEN      29042/python        
tcp        0      0 0.0.0.0:8804                0.0.0.0:*                   LISTEN      29084/python        
tcp        0      0 0.0.0.0:43972               0.0.0.0:*                   LISTEN      4163/beam.smp       
tcp        0      0 0.0.0.0:8805                0.0.0.0:*                   LISTEN      29084/python        
tcp        0      0 192.1.242.3:3333            0.0.0.0:*                   LISTEN      28639/python        
tcp        0      0 0.0.0.0:8773                0.0.0.0:*                   LISTEN      27286/nginx         
tcp        0      0 192.1.242.3:389             0.0.0.0:*                   LISTEN      3199/slapd          
tcp        0      0 0.0.0.0:8806                0.0.0.0:*                   LISTEN      29042/python        
tcp        0      0 0.0.0.0:8807                0.0.0.0:*                   LISTEN      29016/python        
tcp        0      0 0.0.0.0:9191                0.0.0.0:*                   LISTEN      28826/python        
tcp        0      0 0.0.0.0:7911                0.0.0.0:*                   LISTEN      27903/dhcpd         
tcp        0      0 0.0.0.0:8808                0.0.0.0:*                   LISTEN      28987/python        
tcp        0      0 0.0.0.0:3626                0.0.0.0:*                   LISTEN      27286/nginx         
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      3948/mysqld         
tcp        0      0 :::14443                    :::*                        LISTEN      19259/java          
tcp        0      0 :::51339                    :::*                        LISTEN      3274/rpc.mountd     
tcp        0      0 :::46254                    :::*                        LISTEN      -                   
tcp        0      0 :::111                      :::*                        LISTEN      2780/rpcbind        
tcp        0      0 :::50000                    :::*                        LISTEN      24491/java          
tcp        0      0 :::12080                    :::*                        LISTEN      19470/java          
tcp        0      0 :::8080                     :::*                        LISTEN      9114/java           
tcp        0      0 :::80                       :::*                        LISTEN      8477/httpd          
tcp        0      0 :::50001                    :::*                        LISTEN      1380/java           
tcp        0      0 :::13009                    :::*                        LISTEN      19625/java          
tcp        0      0 :::10514                    :::*                        LISTEN      2717/rsyslogd       
tcp        0      0 :::11443                    :::*                        LISTEN      29224/java          
tcp        0      0 :::50004                    :::*                        LISTEN      9566/java           
tcp        0      0 :::51413                    :::*                        LISTEN      2378/java           
tcp        0      0 ::1:53                      :::*                        LISTEN      13783/named         
tcp        0      0 :::22                       :::*                        LISTEN      3634/sshd           
tcp        0      0 :::45430                    :::*                        LISTEN      2806/rpc.statd      
tcp        0      0 :::55000                    :::*                        LISTEN      24491/java          
tcp        0      0 :::13080                    :::*                        LISTEN      19625/java          
tcp        0      0 :::55001                    :::*                        LISTEN      1380/java           
tcp        0      0 :::14009                    :::*                        LISTEN      19259/java          
tcp        0      0 ::1:25                      :::*                        LISTEN      4095/master         
tcp        0      0 ::1:6010                    :::*                        LISTEN      16835/sshd          
tcp        0      0 ::1:6011                    :::*                        LISTEN      32641/sshd          
tcp        0      0 :::12443                    :::*                        LISTEN      19470/java          
tcp        0      0 :::443                      :::*                        LISTEN      8477/httpd          
tcp        0      0 :::33595                    :::*                        LISTEN      3274/rpc.mountd     
tcp        0      0 :::55004                    :::*                        LISTEN      9566/java           
tcp        0      0 :::55740                    :::*                        LISTEN      9114/java           
tcp        0      0 :::6556                     :::*                        LISTEN      3642/xinetd         
tcp        0      0 :::6557                     :::*                        LISTEN      22989/xinetd        
tcp        0      0 :::14080                    :::*                        LISTEN      19259/java          
tcp        0      0 :::11009                    :::*                        LISTEN      29224/java          
tcp        0      0 :::2049                     :::*                        LISTEN      -                   
tcp        0      0 :::13443                    :::*                        LISTEN      19625/java          
tcp        0      0 :::58949                    :::*                        LISTEN      3274/rpc.mountd     
tcp        0      0 :::11080                    :::*                        LISTEN      29224/java          
tcp        0      0 :::5672                     :::*                        LISTEN      4163/beam.smp       
tcp        0      0 :::12009                    :::*                        LISTEN      19470/java          
tcp        0      0 :::11081                    :::*                        LISTEN      2378/java           
tcp        0      0 :::6633                     :::*                        LISTEN      9114/java     

That's 103 things, which seems like a lot, but I gather that this is normal. Ah, and there's some duplication in there.

Stripping that into a list of PIDs:

[11:22:25] jbs@bbn-hn:/home/jbs
+$ awk '{print $NF}' netstat-raw.txt | sed -r -e 's|/.*||' | sort -n | uniq > netstat-pids.txt 

[11:23:04] jbs@bbn-hn:/home/jbs
+$ cat netstat-pids.txt 
-
724
1380
2378
2717
2780
2806
3199
3258
3274
3634
3642
3660
3948
4095
4123
4163
8477
9114
9566
13783
16835
19259
19470
19625
22989
23122
23124
23127
24491
27286
27903
28606
28639
28701
28780
28814
28826
28878
28903
28923
28943
28965
28987
29016
29042
29084
29140
29162
29185
29205
29224
32641

For some of those, the PID is '-':

[11:25:15] jbs@bbn-hn:/home/jbs
+$ egrep -- - netstat-raw.txt 
tcp        0      0 0.0.0.0:36843               0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:2049                0.0.0.0:*                   LISTEN      -                   
tcp        0      0 :::46254                    :::*                        LISTEN      -                   
tcp        0      0 :::2049                     :::*                        LISTEN      -                   

lsof doesn't know what they are either:

[11:26:32] jbs@bbn-hn:/home/jbs
+$ sudo lsof -i | grep 2049

[11:27:15] jbs@bbn-hn:/home/jbs
+$ sudo lsof -i | grep 46254

[11:27:26] jbs@bbn-hn:/home/jbs
+$ sudo lsof -i | grep 36843

We think those are probably NFS-related things, which perhaps are listening from a kernel module rather than a process.

Leaving those aside, here are the others:

[11:40:08] jbs@bbn-hn:/home/jbs
+$ for pid in $(cat netstat-pids.txt) ; do ps -efwww | egrep "^[^ ]+ +$pid" ; done >& netstat-commands.txt 

[11:40:11] jbs@bbn-hn:/home/jbs
+$ cat netstat-commands.txt 
root       724     1  0 Jul03 ?        02:44:09 /usr/bin/ruby /usr/sbin/puppetd
2006      1380     1  0 Jul26 ?        00:06:48 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55001 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50001 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar
2006      2378  2372  0 Jul20 ?        00:08:06 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml
root      2717     1  0 Jun13 ?        00:09:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 4
rpc       2780     1  0 Jun13 ?        00:00:02 rpcbind
rpcuser   2806     1  0 Jun13 ?        00:00:00 rpc.statd
ldap      3199     1  0 Jun13 ?        00:10:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf
root      3258     1  0 Jun13 ?        00:00:00 rpc.rquotad
root      3274     1  0 Jun13 ?        00:00:00 rpc.mountd
root      3634     1  0 Jun13 ?        00:01:34 /usr/sbin/sshd
root      3642     1  0 Jun13 ?        00:00:18 xinetd -stayalive -pidfile /var/run/xinetd.pid
root      3660     1  0 Jun13 ?        00:04:13 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid
mysql     3948  3706  0 Jun13 ?        02:05:09 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=8192 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306
root      4095     1  0 Jun13 ?        00:00:28 /usr/libexec/postfix/master
rabbitmq  4123     1  0 Jun13 ?        00:00:16 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon
rabbitmq  4163  4162  0 Jun13 ?        00:18:00 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn"
root      8477     1  0 Jun22 ?        00:00:42 /usr/sbin/httpd
491       9114     1  3 Jun27 ?        1-10:41:10 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml
2006      9566     1  0 Aug01 ?        00:01:35 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-5- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55004 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50004 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar
named    13783     1  0 Jun22 ?        00:01:04 /usr/sbin/named -u named
jonmills 16835 16833  0 00:09 ?        00:00:00 sshd: jonmills@pts/2
2006     19259     1  0 Jul05 ?        00:24:49 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start
2006     19470     1  0 Jul20 ?        01:02:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start
2006     19625     1  0 Jul20 ?        00:29:16 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start
rack_bbn 22989     1  0 Jun18 ?        00:00:24 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf
root     23122     1  0 Jun22 ?        00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf
root     23124     1  0 Jun22 ?        00:00:06 xcatd: SSL listener
root     23127 23124  0 Jun22 ?        00:00:01 xcatd: install monitor
2006     24491     1  0 Aug01 ?        00:01:25 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55000 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50000 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar
root     27286     1  0 Jun18 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
dhcpd    27903     1  0 Jun22 ?        00:02:12 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006
nova     28606 28603  0 Jul06 ?        00:06:51 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8781.log --ec2_listen_port=8781 --osapi_listen_port=8788
nova     28639     1  0 Jul06 ?        00:00:07 /usr/bin/python /usr/bin/nova-objectstore --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-objectstore.log --pidfile=/var/run/nova/nova-objectstore.pid
nova     28701 28699  0 Jul06 ?        00:06:46 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8782.log --ec2_listen_port=8782 --osapi_listen_port=8787
glance   28780     1  0 Jul06 ?        00:00:54 /usr/bin/python /usr/bin/glance-api --config-file=/etc/glance/glance-api.conf --log-config=/etc/glance/logging-api.conf
nova     28814 28812  0 Jul06 ?        00:06:32 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8783.log --ec2_listen_port=8783 --osapi_listen_port=8786
glance   28826     1  0 Jul06 ?        00:00:50 /usr/bin/python /usr/bin/glance-registry --config-file=/etc/glance/glance-registry.conf -v --log-config=/etc/glance/logging-registry.conf
nova     28878 28876  0 Jul06 ?        00:06:09 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8784.log --ec2_listen_port=8784 --osapi_listen_port=8785
nova     28903 28901  0 Jul06 ?        00:06:12 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8791.log --ec2_listen_port=8791 --osapi_listen_port=8798
nova     28923 28921  0 Jul06 ?        00:06:49 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8792.log --ec2_listen_port=8792 --osapi_listen_port=8797
nova     28943 28941  0 Jul06 ?        00:06:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8793.log --ec2_listen_port=8793 --osapi_listen_port=8796
nova     28965 28963  0 Jul06 ?        00:07:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8794.log --ec2_listen_port=8794 --osapi_listen_port=8795
nova     28987 28985  0 Jul06 ?        00:06:01 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8801.log --ec2_listen_port=8801 --osapi_listen_port=8808
nova     29016 29014  0 Jul06 ?        00:06:33 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8802.log --ec2_listen_port=8802 --osapi_listen_port=8807
nova     29042 29040  0 Jul06 ?        00:09:05 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8803.log --ec2_listen_port=8803 --osapi_listen_port=8806
nova     29084 29080  0 Jul06 ?        00:06:34 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8804.log --ec2_listen_port=8804 --osapi_listen_port=8805
nova     29140 29138  0 Jul06 ?        00:06:11 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8811.log --ec2_listen_port=8811 --osapi_listen_port=8818
nova     29162 29160  0 Jul06 ?        00:06:29 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8812.log --ec2_listen_port=8812 --osapi_listen_port=8817
nova     29185 29183  0 Jul06 ?        00:06:42 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8813.log --ec2_listen_port=8813 --osapi_listen_port=8816
nova     29205 29203  0 Jul06 ?        00:06:47 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8814.log --ec2_listen_port=8814 --osapi_listen_port=8815
2006     29224     1  0 Jul20 ?        00:29:38 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start
jonmills 32641 32639  0 10:09 ?        00:00:00 sshd: jonmills@pts/1

Just to sanity check that we haven't missed anything:

[11:40:14] jbs@bbn-hn:/home/jbs
+$ wc -l netstat-*
   52 netstat-commands.txt
   53 netstat-pids.txt
  103 netstat-raw.txt
  208 total

The change from raw to pids is because of duplicates, and from pids to commands is because of the '-' PID dropping out.

So, let's start plowing through those.

This is from an RPM:

root       724     1  0 Jul03 ?        02:44:09 /usr/bin/ruby /usr/sbin/puppetd

[11:41:39] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/puppetd
puppet-2.7.18-1.el6.noarch

Lumping these Floodlight things together, Java is from an RPM, but floodlight.jar isn't:

2006      1380     1  0 Jul26 ?        00:06:48 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2601- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55001 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50001 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar
2006      9566     1  0 Aug01 ?        00:01:35 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-5- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55004 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50004 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar
2006     24491     1  0 Aug01 ?        00:01:25 java -server -d64 -Xmx2g -Xms2g -Xmn800m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:InlineSmallCode=8192 -XX:MaxInlineSize=8192 -XX:FreqInlineSize=8192 -XX:CompileThreshold=1500 -XX:PreBlockSpin=8 -Dnet.orca.vlan=vlan-2- -Dnet.floodlightcontroller.restserver.RestApiServer.port=55000 -Dnet.floodlightcontroller.core.FloodlightProvider.openflowport=50000 -Dfloodlight.modules=net.floodlightcontroller.learningswitch.LearningSwitch,net.floodlightcontroller.counter.NullCounterStore,net.floodlightcontroller.perfmon.NullPktInProcessingTime -jar /opt/floodlight/floodlight.jar

[15:56:51] jbs@bbn-hn:/home/jbs
+$ which java
java is /usr/java/default/bin/java

[15:56:52] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/java/default/bin/java
jdk-1.6.0_30-fcs.x86_64

[15:56:56] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/floodlight/floodlight.jar
file /opt/floodlight/floodlight.jar is not owned by any package

Java is from an RPM (see above), imageproxy isn't:

2006      2378  2372  0 Jul20 ?        00:08:06 java -classpath /opt/imageproxy/axis2:/opt/imageproxy/axis2/conf:/usr/java/latest/lib/tools.jar::/opt/imageproxy/axis2/lib/activation-1.1.jar:/opt/imageproxy/axis2/lib/antlr-2.7.7.jar:/opt/imageproxy/axis2/lib/apache-mime4j-core-0.7.2.jar:/opt/imageproxy/axis2/lib/axiom-api-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-dom-1.2.13.jar:/opt/imageproxy/axis2/lib/axiom-impl-1.2.13.jar:/opt/imageproxy/axis2/lib/axis2-adb-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-adb-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-ant-plugin-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-clustering-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-codegen-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-corba-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-fastinfoset-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-java2wsdl-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxbri-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jaxws-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-jibx-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-json-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-kernel-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-metadata-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-mtompolicy-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-saaj-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-soapmonitor-servlet-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-spring-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-http-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-transport-local-1.6.2.jar:/opt/imageproxy/axis2/lib/axis2-xmlbeans-1.6.2.jar:/opt/imageproxy/axis2/lib/bcel-5.1.jar:/opt/imageproxy/axis2/lib/commons-cli-1.2.jar:/opt/imageproxy/axis2/lib/commons-codec-1.3.jar:/opt/imageproxy/axis2/lib/commons-fileupload-1.2.jar:/opt/imageproxy/axis2/lib/commons-httpclient-3.1.jar:/opt/imageproxy/axis2/lib/commons-io-1.4.jar:/opt/imageproxy/axis2/lib/commons-logging-1.1.1.jar:/opt/imageproxy/axis2/lib/geronimo-annotation_1.0_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-jaxws_2.2_spec-1.0.jar:/opt/imageproxy/axis2/lib/geronimo-jta_1.1_spec-1.1.jar:/opt/imageproxy/axis2/lib/geronimo-saaj_1.3_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar:/opt/imageproxy/axis2/lib/geronimo-ws-metadata_2.0_spec-1.1.2.jar:/opt/imageproxy/axis2/lib/httpcore-4.0.jar:/opt/imageproxy/axis2/lib/jalopy-1.5rc3.jar:/opt/imageproxy/axis2/lib/jaxb-api-2.1.jar:/opt/imageproxy/axis2/lib/jaxb-impl-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxb-xjc-2.1.7.jar:/opt/imageproxy/axis2/lib/jaxen-1.1.1.jar:/opt/imageproxy/axis2/lib/jaxws-tools-2.1.3.jar:/opt/imageproxy/axis2/lib/jettison-1.0-RC2.jar:/opt/imageproxy/axis2/lib/jibx-bind-1.2.jar:/opt/imageproxy/axis2/lib/jibx-run-1.2.jar:/opt/imageproxy/axis2/lib/jsr311-api-1.0.jar:/opt/imageproxy/axis2/lib/juli-6.0.16.jar:/opt/imageproxy/axis2/lib/log4j-1.2.15.jar:/opt/imageproxy/axis2/lib/mail-1.4.jar:/opt/imageproxy/axis2/lib/mex-1.6.2-impl.jar:/opt/imageproxy/axis2/lib/neethi-3.0.2.jar:/opt/imageproxy/axis2/lib/regexp-1.2.jar:/opt/imageproxy/axis2/lib/tribes-6.0.16.jar:/opt/imageproxy/axis2/lib/woden-api-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-commons-1.0M9.jar:/opt/imageproxy/axis2/lib/woden-impl-dom-1.0M9.jar:/opt/imageproxy/axis2/lib/wsdl4j-1.6.2.jar:/opt/imageproxy/axis2/lib/wstx-asl-3.2.9.jar:/opt/imageproxy/axis2/lib/xalan-2.7.0.jar:/opt/imageproxy/axis2/lib/xmlbeans-2.3.0.jar:/opt/imageproxy/axis2/lib/xml-resolver-1.2.jar:/opt/imageproxy/axis2/lib/XmlSchema-1.4.7.jar: -Djava.endorsed.dirs=/opt/imageproxy/axis2/lib/endorsed:/usr/java/latest/jre/lib/endorsed:/usr/java/latest/lib/endorsed org.apache.axis2.transport.SimpleAxis2Server -repo /opt/imageproxy/axis2/repository -conf /opt/imageproxy/axis2/conf/axis2.xml

[15:57:00] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/imageproxy 
file /opt/imageproxy is not owned by any package

This is from an RPM:

root      2717     1  0 Jun13 ?        00:09:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 4

[15:59:03] jbs@bbn-hn:/home/jbs
+$ rpm -qf /sbin/rsyslogd
rsyslog-4.6.2-12.el6.x86_64

This is from an RPM:

rpc       2780     1  0 Jun13 ?        00:00:02 rpcbind

[16:00:13] jbs@bbn-hn:/home/jbs
+$ which rpcbind
rpcbind is /sbin/rpcbind

[16:00:28] jbs@bbn-hn:/home/jbs
+$ rpm -qf /sbin/rpcbind
rpcbind-0.2.0-8.el6.x86_64

This is from an RPM:

rpcuser   2806     1  0 Jun13 ?        00:00:00 rpc.statd

[16:00:32] jbs@bbn-hn:/home/jbs
+$ which rpc.statd
rpc.statd is /sbin/rpc.statd

[16:00:52] jbs@bbn-hn:/home/jbs
+$ rpm -qf /sbin/rpc.statd
nfs-utils-1.2.3-15.el6.x86_64

This is from an RPM:

ldap      3199     1  0 Jun13 ?        00:10:20 /usr/sbin/slapd -h ldap://192.1.242.3 ldaps://192.1.242.3 ldap://192.168.103.10 ldaps://192.168.103.10 ldap://10.100.0.1 ldaps://10.100.0.1 -u ldap -f /etc/openldap/slapd.conf

[16:00:58] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/slapd
openldap-servers-2.4.23-20.el6.x86_64

This is from an RPM:

root      3258     1  0 Jun13 ?        00:00:00 rpc.rquotad

[16:01:28] jbs@bbn-hn:/home/jbs
+$ which rpc.rquotad
rpc.rquotad is /usr/sbin/rpc.rquotad

[16:01:34] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/rpc.rquotad
quota-3.17-16.el6.x86_64

This is from an RPM:

root      3274     1  0 Jun13 ?        00:00:00 rpc.mountd

[16:01:40] jbs@bbn-hn:/home/jbs
+$ which rpc.mountd
rpc.mountd is /usr/sbin/rpc.mountd

[16:02:24] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/rpc.mountd
nfs-utils-1.2.3-15.el6.x86_64

This is from an RPM:

root      3634     1  0 Jun13 ?        00:01:34 /usr/sbin/sshd

[16:06:46] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/sshd 
openssh-server-5.3p1-70.el6_2.2.x86_64

This is from an RPM:

root      3642     1  0 Jun13 ?        00:00:18 xinetd -stayalive -pidfile /var/run/xinetd.pid

[16:07:15] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/xinetd
xinetd-2.3.14-33.el6.x86_64

This is from an RPM:

root      3660     1  0 Jun13 ?        00:04:13 /sbin/apcupsd -f /etc/apcupsd/ups06.conf -P /var/run/apcupsd-ups06.pid

[16:07:45] jbs@bbn-hn:/home/jbs
+$ rpm -qf /sbin/apcupsd 
apcupsd-3.14.8-4.el6.x86_64

This is from an RPM:

mysql     3948  3706  0 Jun13 ?        02:05:09 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --open-files-limit=8192 --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=3306

[16:08:22] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/libexec/mysqld
mysql-server-5.1.61-1.el6_2.1.x86_64

This is from an RPM:

root      4095     1  0 Jun13 ?        00:00:28 /usr/libexec/postfix/master

[16:08:28] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/libexec/postfix/master
postfix-2.6.6-2.2.el6_1.x86_64

This is from an RPM:

rabbitmq  4123     1  0 Jun13 ?        00:00:16 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon

[16:09:09] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/lib64/erlang/erts-5.8.5/bin/epmd
erlang-erts-R14B-04.1.el6.x86_64

This is from an RPM:

rabbitmq  4163  4162  0 Jun13 ?        00:18:00 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@bbn-hn -boot /var/lib/rabbitmq/mnesia/rabbit@bbn-hn-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@bbn-hn-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@bbn-hn"

[16:09:46] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/lib64/erlang/erts-5.8.5/bin/beam.smp
erlang-erts-R14B-04.1.el6.x86_64

[16:10:42] jbs@bbn-hn:/home/jbs
+$ rpm -qf /var/lib/rabbitmq 
rabbitmq-server-2.6.1-1.el6.noarch

(I don't really know what that is; I didn't check every path or file on that commandline, but I'm optimistic that they're all from RPMs.)

This is from an RPM:

root      8477     1  0 Jun22 ?        00:00:42 /usr/sbin/httpd

[16:13:19] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/httpd
httpd-2.2.15-15.el6.centos.1.x86_64

Java is from an RPM (see above), and so is FlowVisor:

491       9114     1  3 Jun27 ?        1-10:41:10 java -server -Xms256M -Xmx1500M -XX:OnError=flowvisor-crash-logger -XX:+UseConcMarkSweepGC -Dorg.flowvisor.config_dir=/opt/flowvisor/etc/flowvisor -Dorg.flowvisor.install_dir=/opt/flowvisor/libexec/flowvisor -Djavax.net.ssl.keyStore=/opt/flowvisor/etc/flowvisor/mySSLKeyStore -Djavax.net.ssl.keyStorePassword=CHANGEME_PASSWD -cp /opt/flowvisor/libexec/flowvisor/openflow.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-client-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-common-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/xmlrpc-server-3.1.3.jar:/opt/flowvisor/libexec/flowvisor/commons-logging-1.1.jar:/opt/flowvisor/libexec/flowvisor/ws-commons-util-1.0.2.jar:/opt/flowvisor/libexec/flowvisor/jsse.jar:/opt/flowvisor/libexec/flowvisor/asm-3.0.jar:/opt/flowvisor/libexec/flowvisor/cglib-2.2.jar:/opt/flowvisor/libexec/flowvisor/commons-codec-1.4.jar:/opt/flowvisor/libexec/flowvisor/gson-1.7.1.jar:/opt/flowvisor/libexec/flowvisor/jetty-continuation-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-http-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-io-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-security-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-server-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/jetty-util-7.0.2.v20100331.jar:/opt/flowvisor/libexec/flowvisor/servlet-api-2.5.jar:/opt/flowvisor/libexec/flowvisor/flowvisor.jar org.flowvisor.FlowVisor /opt/flowvisor/etc/flowvisor/config.xml

[16:14:02] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/flowvisor/libexec/flowvisor/flowvisor.jar
flowvisor-0.8.2-1.x86_64

This is from an RPM:

named    13783     1  0 Jun22 ?        00:01:04 /usr/sbin/named -u named

[16:15:45] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/named
bind-9.7.3-8.P3.el6_2.2.x86_64

This is from an RPM:

jonmills 16835 16833  0 00:09 ?        00:00:00 sshd: jonmills@pts/2

[16:16:07] jbs@bbn-hn:/home/jbs
+$ which sshd
sshd is /usr/sbin/sshd

[16:16:12] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/sshd
openssh-server-5.3p1-70.el6_2.2.x86_64

Java is from an RPM (see above), I don't think the other stuff here is:

2006     19259     1  0 Jul05 ?        00:24:49 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-aux/tomcat-7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:MaxPermSize=128m -Xmx1024m -Djavax.net.ssl.trustStore=/opt/orca-aux/tomcat-7/ssl/trusted.jks -Djavax.net.ssl.trustStorePassword=orcaorca -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=14000 -Djava.endorsed.dirs=/opt/orca-aux/tomcat-7/endorsed -classpath /opt/orca-aux/tomcat-7/bin/bootstrap.jar:/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-aux/tomcat-7 -Dcatalina.home=/opt/orca-aux/tomcat-7 -Djava.io.tmpdir=/opt/orca-aux/tomcat-7/temp org.apache.catalina.startup.Bootstrap start

[16:16:15] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar
file /opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package

[16:17:07] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-aux/tomcat-7/bin
file /opt/orca-aux/tomcat-7/bin is not owned by any package

[16:17:10] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-aux/tomcat-7
file /opt/orca-aux/tomcat-7 is not owned by any package

[16:17:11] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-aux
file /opt/orca-aux is not owned by any package

[16:18:40] jbs@bbn-hn:/home/jbs
+$ rpm -qa | grep -i orca || echo no match
no match

Java is from an RPM (see above), I don't think the other stuff here is:

2006     19470     1  0 Jul20 ?        01:02:40 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-12080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=12000 -Djava.endorsed.dirs=/opt/orca-12080/tomcat/endorsed -classpath /opt/orca-12080/tomcat/bin/bootstrap.jar:/opt/orca-12080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-12080/tomcat -Dcatalina.home=/opt/orca-12080/tomcat -Djava.io.tmpdir=/opt/orca-12080/tomcat/temp org.apache.catalina.startup.Bootstrap start

[16:19:43] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-12080/tomcat/bin/tomcat-juli.jar
file /opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package

[16:19:47] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-12080/tomcat/bin 
file /opt/orca-12080/tomcat/bin is not owned by any package

[16:19:49] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-12080 
file /opt/orca-12080 is not owned by any package

[16:19:51] jbs@bbn-hn:/home/jbs
+$ rpm -qa | grep -i orca || echo no match
no match

Java is from an RPM (see above), I don't think the other stuff here is:

2006     19625     1  0 Jul20 ?        00:29:16 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-13080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/orca-13080/tomcat/endorsed -classpath /opt/orca-13080/tomcat/bin/bootstrap.jar:/opt/orca-13080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-13080/tomcat -Dcatalina.home=/opt/orca-13080/tomcat -Djava.io.tmpdir=/opt/orca-13080/tomcat/temp org.apache.catalina.startup.Bootstrap start

[16:21:25] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-13080/tomcat/bin/tomcat-juli.jar
file /opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package

[16:21:38] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-13080/tomcat/bin 
file /opt/orca-13080/tomcat/bin is not owned by any package

[16:21:40] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-13080/tomcat 
file /opt/orca-13080/tomcat is not owned by any package

[16:21:42] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-13080 
file /opt/orca-13080 is not owned by any package

This is not from an RPM:

rack_bbn 22989     1  0 Jun18 ?        00:00:24 /omd/sites/rack_bbn/var/tmp/xinetd -pidfile /omd/sites/rack_bbn/tmp/run/xinetd.pid -filelog /omd/sites/rack_bbn/var/log/xinetd.log -f /omd/sites/rack_bbn/etc/xinetd.conf

[16:21:43] jbs@bbn-hn:/home/jbs
+$ rpm -qf /omd/sites/rack_bbn/var/tmp/xinetd
file /omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package

This is from an RPM:

root     23122     1  0 Jun22 ?        00:00:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf

[16:23:13] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/vsftpd 
vsftpd-2.2.2-6.el6_2.1.x86_64

This is from an RPM:

root     23124     1  0 Jun22 ?        00:00:06 xcatd: SSL listener

[16:23:32] jbs@bbn-hn:/home/jbs
+$ which xcatd
xcatd is /opt/xcat/sbin/xcatd

[16:23:35] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/xcat/sbin/xcatd
xCAT-server-2.6.9-snap201111090305.noarch

This is from an RPM:

root     23127 23124  0 Jun22 ?        00:00:01 xcatd: install monitor

[16:23:32] jbs@bbn-hn:/home/jbs
+$ which xcatd
xcatd is /opt/xcat/sbin/xcatd

[16:23:35] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/xcat/sbin/xcatd
xCAT-server-2.6.9-snap201111090305.noarch

This is from an RPM:

root     27286     1  0 Jun18 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf

[16:23:40] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/nginx 
nginx-0.8.54-1.el6.x86_64

This is from an RPM:

dhcpd    27903     1  0 Jun22 ?        00:02:12 /usr/sbin/dhcpd -user dhcpd -group dhcpd bond2.1007 bond2.1006

[16:24:50] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/dhcpd 
dhcp-4.1.1-25.P1.el6_2.1.x86_64

Lumping these nova things together, Python is from an RPM, as are nova-api and nova-objectstore:

nova     28606 28603  0 Jul06 ?        00:06:51 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8781.log --ec2_listen_port=8781 --osapi_listen_port=8788
nova     28639     1  0 Jul06 ?        00:00:07 /usr/bin/python /usr/bin/nova-objectstore --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-objectstore.log --pidfile=/var/run/nova/nova-objectstore.pid
nova     28701 28699  0 Jul06 ?        00:06:46 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8782.log --ec2_listen_port=8782 --osapi_listen_port=8787
nova     28814 28812  0 Jul06 ?        00:06:32 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8783.log --ec2_listen_port=8783 --osapi_listen_port=8786
nova     28878 28876  0 Jul06 ?        00:06:09 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8784.log --ec2_listen_port=8784 --osapi_listen_port=8785
nova     28903 28901  0 Jul06 ?        00:06:12 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8791.log --ec2_listen_port=8791 --osapi_listen_port=8798
nova     28923 28921  0 Jul06 ?        00:06:49 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8792.log --ec2_listen_port=8792 --osapi_listen_port=8797
nova     28943 28941  0 Jul06 ?        00:06:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8793.log --ec2_listen_port=8793 --osapi_listen_port=8796
nova     28965 28963  0 Jul06 ?        00:07:28 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8794.log --ec2_listen_port=8794 --osapi_listen_port=8795
nova     28987 28985  0 Jul06 ?        00:06:01 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8801.log --ec2_listen_port=8801 --osapi_listen_port=8808
nova     29016 29014  0 Jul06 ?        00:06:33 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8802.log --ec2_listen_port=8802 --osapi_listen_port=8807
nova     29042 29040  0 Jul06 ?        00:09:05 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8803.log --ec2_listen_port=8803 --osapi_listen_port=8806
nova     29084 29080  0 Jul06 ?        00:06:34 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8804.log --ec2_listen_port=8804 --osapi_listen_port=8805
nova     29140 29138  0 Jul06 ?        00:06:11 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8811.log --ec2_listen_port=8811 --osapi_listen_port=8818
nova     29162 29160  0 Jul06 ?        00:06:29 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8812.log --ec2_listen_port=8812 --osapi_listen_port=8817
nova     29185 29183  0 Jul06 ?        00:06:42 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8813.log --ec2_listen_port=8813 --osapi_listen_port=8816
nova     29205 29203  0 Jul06 ?        00:06:47 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf --logfile=/var/log/nova/nova-api-8814.log --ec2_listen_port=8814 --osapi_listen_port=8815

[16:27:28] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/python
python-2.6.6-29.el6.x86_64

[16:27:31] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/nova-api
openstack-nova-api-2011.3-b1281neuca2.noarch

[16:27:34] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/nova-objectstore 
openstack-nova-objectstore-2011.3-b1281neuca2.noarch

Lumping these glance things together, all of these are from RPMs:

glance   28780     1  0 Jul06 ?        00:00:54 /usr/bin/python /usr/bin/glance-api --config-file=/etc/glance/glance-api.conf --log-config=/etc/glance/logging-api.conf
glance   28826     1  0 Jul06 ?        00:00:50 /usr/bin/python /usr/bin/glance-registry --config-file=/etc/glance/glance-registry.conf -v --log-config=/etc/glance/logging-registry.conf

[16:28:17] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/python
python-2.6.6-29.el6.x86_64

[16:28:18] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/glance-api
openstack-glance-2011.3-b1277.noarch

[16:28:20] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/bin/glance-registry
openstack-glance-2011.3-b1277.noarch

Java is from an RPM (see above), I don't think the other stuff here is:

2006     29224     1  0 Jul20 ?        00:29:38 /usr/java/latest/bin/java -Djava.util.logging.config.file=/opt/orca-11080/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=11000 -Djava.endorsed.dirs=/opt/orca-11080/tomcat/endorsed -classpath /opt/orca-11080/tomcat/bin/bootstrap.jar:/opt/orca-11080/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/orca-11080/tomcat -Dcatalina.home=/opt/orca-11080/tomcat -Djava.io.tmpdir=/opt/orca-11080/tomcat/temp org.apache.catalina.startup.Bootstrap start

[16:29:14] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-11080/tomcat/bin/tomcat-juli.jar
file /opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package

[16:29:15] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-11080/tomcat/bin 
file /opt/orca-11080/tomcat/bin is not owned by any package

[16:29:17] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-11080/tomcat 
file /opt/orca-11080/tomcat is not owned by any package

[16:29:18] jbs@bbn-hn:/home/jbs
+$ rpm -qf /opt/orca-11080 
file /opt/orca-11080 is not owned by any package

[16:29:19] jbs@bbn-hn:/home/jbs
+$ rpm -qa | grep -i orca || echo no match
no match

This is from an RPM:

jonmills 32641 32639  0 10:09 ?        00:00:00 sshd: jonmills@pts/1

[16:29:51] jbs@bbn-hn:/home/jbs
+$ which sshd
sshd is /usr/sbin/sshd

[16:29:52] jbs@bbn-hn:/home/jbs
+$ rpm -qf /usr/sbin/sshd
openssh-server-5.3p1-70.el6_2.2.x86_64

So, that's these RPMs:

puppet-2.7.18-1.el6.noarch
jdk-1.6.0_30-fcs.x86_64
rsyslog-4.6.2-12.el6.x86_64
rpcbind-0.2.0-8.el6.x86_64
nfs-utils-1.2.3-15.el6.x86_64
openldap-servers-2.4.23-20.el6.x86_64
quota-3.17-16.el6.x86_64
nfs-utils-1.2.3-15.el6.x86_64
openssh-server-5.3p1-70.el6_2.2.x86_64
xinetd-2.3.14-33.el6.x86_64
apcupsd-3.14.8-4.el6.x86_64
mysql-server-5.1.61-1.el6_2.1.x86_64
postfix-2.6.6-2.2.el6_1.x86_64
erlang-erts-R14B-04.1.el6.x86_64
erlang-erts-R14B-04.1.el6.x86_64
rabbitmq-server-2.6.1-1.el6.noarch
httpd-2.2.15-15.el6.centos.1.x86_64
flowvisor-0.8.2-1.x86_64
bind-9.7.3-8.P3.el6_2.2.x86_64
openssh-server-5.3p1-70.el6_2.2.x86_64
vsftpd-2.2.2-6.el6_2.1.x86_64
xCAT-server-2.6.9-snap201111090305.noarch
xCAT-server-2.6.9-snap201111090305.noarch
nginx-0.8.54-1.el6.x86_64
dhcp-4.1.1-25.P1.el6_2.1.x86_64
python-2.6.6-29.el6.x86_64
openstack-nova-api-2011.3-b1281neuca2.noarch
openstack-nova-objectstore-2011.3-b1281neuca2.noarch
python-2.6.6-29.el6.x86_64
openstack-glance-2011.3-b1277.noarch
openstack-glance-2011.3-b1277.noarch
openssh-server-5.3p1-70.el6_2.2.x86_64

Sorted and uniq-ed:

apcupsd-3.14.8-4.el6.x86_64
bind-9.7.3-8.P3.el6_2.2.x86_64
dhcp-4.1.1-25.P1.el6_2.1.x86_64
erlang-erts-R14B-04.1.el6.x86_64
flowvisor-0.8.2-1.x86_64
httpd-2.2.15-15.el6.centos.1.x86_64
jdk-1.6.0_30-fcs.x86_64
mysql-server-5.1.61-1.el6_2.1.x86_64
nfs-utils-1.2.3-15.el6.x86_64
nginx-0.8.54-1.el6.x86_64
openldap-servers-2.4.23-20.el6.x86_64
openssh-server-5.3p1-70.el6_2.2.x86_64
openstack-glance-2011.3-b1277.noarch
openstack-nova-api-2011.3-b1281neuca2.noarch
openstack-nova-objectstore-2011.3-b1281neuca2.noarch
postfix-2.6.6-2.2.el6_1.x86_64
puppet-2.7.18-1.el6.noarch
python-2.6.6-29.el6.x86_64
quota-3.17-16.el6.x86_64
rabbitmq-server-2.6.1-1.el6.noarch
rpcbind-0.2.0-8.el6.x86_64
rsyslog-4.6.2-12.el6.x86_64
vsftpd-2.2.2-6.el6_2.1.x86_64
xCAT-server-2.6.9-snap201111090305.noarch
xinetd-2.3.14-33.el6.x86_64

And these things which aren't from RPMs:

/opt/floodlight/floodlight.jar is not owned by any package

/opt/imageproxy is not owned by any package

/opt/orca-aux/tomcat-7/bin/tomcat-juli.jar is not owned by any package
/opt/orca-aux/tomcat-7/bin is not owned by any package
/opt/orca-aux/tomcat-7 is not owned by any package
/opt/orca-aux is not owned by any package

/opt/orca-12080/tomcat/bin/tomcat-juli.jar is not owned by any package
/opt/orca-12080/tomcat/bin is not owned by any package
/opt/orca-12080 is not owned by any package

/opt/orca-13080/tomcat/bin/tomcat-juli.jar is not owned by any package
/opt/orca-13080/tomcat/bin is not owned by any package
/opt/orca-13080/tomcat is not owned by any package
/opt/orca-13080 is not owned by any package

/omd/sites/rack_bbn/var/tmp/xinetd is not owned by any package

/opt/orca-11080/tomcat/bin/tomcat-juli.jar is not owned by any package
/opt/orca-11080/tomcat/bin is not owned by any package
/opt/orca-11080/tomcat is not owned by any package
/opt/orca-11080 is not owned by any package

The xinetd in /omd is a copy of the stock xinetd:

[12:27:40] jbs@bbn-hn:/home/jbs
+$ md5sum /omd/sites/rack_bbn/var/tmp/xinetd /usr/sbin/xinetd 
1c0ef402cd3ad81e01cda637a6c02104  /omd/sites/rack_bbn/var/tmp/xinetd
1c0ef402cd3ad81e01cda637a6c02104  /usr/sbin/xinetd

So that essentialy is from an RPM. ExoGENI ticket 93 is tracking the task of identifying the versions and sources of the others.

As to the RPMs, these look like standard CentOS:

[11:58:34] jbs@bbn-hn:/home/jbs
+$ yum list installed apcupsd-3.14.8-4.el6.x86_64 bind-9.7.3-8.P3.el6_2.2.x86_64 dhcp-4.1.1-25.P1.el6_2.1.x86_64 erlang-erts-R14B-04.1.el6.x86_64 httpd-2.2.15-15.el6.centos.1.x86_64 mysql-server-5.1.61-1.el6_2.1.x86_64 nfs-utils-1.2.3-15.el6.x86_64 nginx-0.8.54-1.el6.x86_64 openldap-servers-2.4.23-20.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 python-2.6.6-29.el6.x86_64 quota-3.17-16.el6.x86_64 rabbitmq-server-2.6.1-1.el6.noarch rpcbind-0.2.0-8.el6.x86_64 rsyslog-4.6.2-12.el6.x86_64 vsftpd-2.2.2-6.el6_2.1.x86_64 xinetd-2.3.14-33.el6.x86_64
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: mirror.dattobackup.com
 * epel: mirror.metrocast.net
 * extras: mirror.dattobackup.com
 * updates: mirror.metrocast.net
Installed Packages
apcupsd.x86_64        3.14.8-4.el6      @epel                                   
dhcp.x86_64           12:4.1.1-25.P1.el6_2.1
                                        @CentOS Updates/6.2                     
erlang-erts.x86_64    R14B-04.1.el6     @UIedited_1/6.2                         
httpd.x86_64          2.2.15-15.el6.centos.1
                                        @CentOS Updates/6.2                     
mysql-server.x86_64   5.1.61-1.el6_2.1  @CentOS Updates/6.2                     
nfs-utils.x86_64      1:1.2.3-15.el6    @anaconda-CentOS-201112091719.x86_64/6.2
nginx.x86_64          0.8.54-1.el6      @UIedited_1/6.2                         
openldap-servers.x86_64
                      2.4.23-20.el6     @base                                   
openssh-server.x86_64 5.3p1-70.el6_2.2  @CentOS Updates/6.2                     
postfix.x86_64        2:2.6.6-2.2.el6_1 @anaconda-CentOS-201112091719.x86_64/6.2
python.x86_64         2.6.6-29.el6      @anaconda-CentOS-201112091719.x86_64/6.2
quota.x86_64          1:3.17-16.el6     @anaconda-CentOS-201112091719.x86_64/6.2
rabbitmq-server.noarch
                      2.6.1-1.el6       @UIedited_1/6.2                         
rpcbind.x86_64        0.2.0-8.el6       @anaconda-CentOS-201112091719.x86_64/6.2
rsyslog.x86_64        4.6.2-12.el6      @anaconda-CentOS-201112091719.x86_64/6.2
vsftpd.x86_64         2.2.2-6.el6_2.1   @CentOS Updates/6.2                     
xinetd.x86_64         2:2.3.14-33.el6   @anaconda-CentOS-201112091719.x86_64/6.2

These look like not:

[11:58:38] jbs@bbn-hn:/home/jbs
+$ yum list installed flowvisor-0.8.2-1.x86_64 jdk-1.6.0_30-fcs.x86_64 openstack-glance-2011.3-b1277.noarch openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch puppet-2.7.18-1.el6.noarch xCAT-server-2.6.9-snap201111090305.noarch
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: mirror.dattobackup.com
 * epel: mirror.metrocast.net
 * extras: mirror.dattobackup.com
 * updates: mirror.metrocast.net
Installed Packages
flowvisor.x86_64                  0.8.2-1                   @exogeni            
jdk.x86_64                        2000:1.6.0_30-fcs         @ExoGENI/6.2        
openstack-glance.noarch           1:2011.3-b1277            @ExoGENI-noarch/6.2 
openstack-nova-api.noarch         1:2011.3-b1281neuca2      @exogeni-noarch     
openstack-nova-objectstore.noarch 1:2011.3-b1281neuca2      @exogeni-noarch     
puppet.noarch                     2.7.18-1.el6              @puppetlabs-products
xCAT-server.noarch                4:2.6.9-snap201111090305  @xCAT/6.2           

I can find the name of a source RPM for each of those:

[12:12:44] jbs@bbn-hn:/home/jbs
+$ for package in flowvisor-0.8.2-1.x86_64 jdk-1.6.0_30-fcs.x86_64 openstack-glance-2011.3-b1277.noarch openstack-nova-api-2011.3-b1281neuca2.noarch openstack-nova-objectstore-2011.3-b1281neuca2.noarch puppet-2.7.18-1.el6.noarch xCAT-server-2.6.9-snap201111090305.noarch ; do rpm -q $package --qf 'Name       : %{NAME}\nSource RPM : %{SOURCERPM}\n\n' ; done
Name       : flowvisor
Source RPM : flowvisor-0.8.2-1.src.rpm

Name       : jdk
Source RPM : jdk-1.6.0_30-fcs.src.rpm

Name       : openstack-glance
Source RPM : openstack-glance-2011.3-b1277.src.rpm

Name       : openstack-nova-api
Source RPM : openstack-nova-2011.3-b1281neuca2.src.rpm

Name       : openstack-nova-objectstore
Source RPM : openstack-nova-2011.3-b1281neuca2.src.rpm

Name       : puppet
Source RPM : puppet-2.7.18-1.el6.src.rpm

Name       : xCAT-server
Source RPM : xCAT-server-2.6.9-snap201111090305.src.rpm

So, now to connect that with the repo URLs... That's a somewhat manual process, as you need to fish the repo URL out of the relevant file in /etc/yum.repos.d, but those seem to be:

[14:19:44] jbs@bbn-hn:/home/jbs
+$ grep baseurl /etc/yum.repos.d/exogeni.repo 
baseurl=http://software.exogeni.net/repo/exogeni/6/current/$basearch
baseurl=http://software.exogeni.net/repo/exogeni/6/current/noarch

[14:19:55] jbs@bbn-hn:/home/jbs
+$ grep baseurl /etc/yum.repos.d/puppetlabs.repo 
baseurl=http://yum.puppetlabs.com/el/6/products/$basearch
baseurl=http://yum.puppetlabs.com/el/6/dependencies/$basearch
baseurl=http://yum.puppetlabs.com/el/6/products/SRPMS
baseurl=http://yum.puppetlabs.com/el/6/dependencies/SRPMS

Hmm, I don't see a repo for xCAT anywhere:

[14:20:23] jbs@bbn-hn:/home/jbs
+$ grep -i xcat /etc/yum.repos.d/* || echo no match
no match

ExoGENI ticket 98 is tracking the task of making sure that repo exists.

Meanwhile, http://software.exogeni.net/repo/exogeni doesn't seem to have an obvious SRPM location, so I've created ExoGENI ticket 96 for that.

Puppet does have SRPMs, as the baseurl listings above imply, and http://yum.puppetlabs.com/el/6/products/SRPMS/puppet-2.7.18-1.el6.src.rpm has one for Puppet 2.7.18, so that seems good.

Aside from the open tickets (and the one question to exogeni-design), I think this step is now complete.

Step 2: identify network-listening software on an OpenStack worker node

Overview of Step 2

Using:

  • Using netstat, enumerate processes on bbn-w1 which listen for network connections from outside the node
  • For each process found:
    • Use the command-line or /proc to determine what executable file is running
    • Use RPM tools to determine whether the executable file is part of an RPM
    • Otherwise, use documentation or iterate with the ExoGENI team to determine the origin of the software
  • For each RPM found, identify a location from which a source RPM for that package can be obtained
  • For each non-RPM software source found, identify a location from which the source code for that version can be obtained.

Verify:

  • The source of each network-listening file can be identified
  • RPM source packages can be found for each RPM-sourced package
  • The source code and identifiable version (e.g. a git tag) can be found for each non-RPM software source

Results of Step 2 from 2012-08-15

A list of processes that are listening:

[11:20:57] jbs@bbn-w1:/home/jbs
+$ sudo netstat -anp | egrep ':\*.+LISTEN' | egrep -v '(127.0.0.1|192.168.103.10|10.100.0.1|10.103.0.1)' > netstat-raw.txt

[11:21:22] jbs@bbn-w1:/home/jbs
+$ cat netstat-raw.txt 
tcp        0      0 0.0.0.0:8139                0.0.0.0:*                   LISTEN      2572/ruby           
tcp        0      0 0.0.0.0:5900                0.0.0.0:*                   LISTEN      18815/qemu-kvm      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      1983/rpcbind        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      2272/sshd           
tcp        0      0 0.0.0.0:33762               0.0.0.0:*                   LISTEN      2001/rpc.statd      
tcp        0      0 :::111                      :::*                        LISTEN      1983/rpcbind        
tcp        0      0 :::22                       :::*                        LISTEN      2272/sshd           
tcp        0      0 ::1:25                      :::*                        LISTEN      2374/master         
tcp        0      0 :::6556                     :::*                        LISTEN      2280/xinetd         
tcp        0      0 :::48004                    :::*                        LISTEN      2001/rpc.statd      

Stripping that into a list of PIDs:

[11:21:36] jbs@bbn-w1:/home/jbs
+$ awk '{print $NF}' netstat-raw.txt | sed -r -e 's|/.*||' | sort -n | uniq > netstat-pids.txt 

[11:22:03] jbs@bbn-w1:/home/jbs
+$ cat netstat-pids.txt 
1983
2001
2272
2280
2374
2572
18815

Here are the commands:

[11:22:38] jbs@bbn-w1:/home/jbs
+$ for pid in $(cat netstat-pids.txt) ; do ps -efwww | egrep "^[^ ]+ +$pid" ; done >& netstat-commands.txt 

[11:23:47] jbs@bbn-w1:/home/jbs
+$ cat netstat-commands.txt 
rpc       1983     1  0 Jul17 ?        00:00:01 rpcbind
rpcuser   2001     1  0 Jul17 ?        00:00:00 rpc.statd
root      2272     1  0 Jul17 ?        00:00:00 /usr/sbin/sshd
root      2280     1  0 Jul17 ?        00:00:05 xinetd -stayalive -pidfile /var/run/xinetd.pid
root      2374     1  0 Jul17 ?        00:00:10 /usr/libexec/postfix/master
root      2572     1  0 Jul17 ?        01:55:23 /usr/bin/ruby /usr/sbin/puppetd
qemu     18815     1  0 Aug14 ?        00:01:20 /usr/libexec/qemu-kvm -S -M pc-0.14 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name instance-00000d78 -uuid efb7e543-0ed8-9dac-76dd-12345b4be3a9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000d78.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -kernel /var/lib/nova/instances/instance-00000d78/kernel -initrd /var/lib/nova/instances/instance-00000d78/ramdisk -append root=/dev/vda console=ttyS0 selinux=0 -drive file=/var/lib/nova/instances/instance-00000d78/disk,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/nova/instances/instance-00000d78/disk.local,if=none,id=drive-virtio-disk1,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=22,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:16:3e:69:b0:db,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:e6:5e:1a,bus=pci.0,addr=0x4 -chardev file,id=charserial0,path=/var/lib/nova/instances/instance-00000d78/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

Just to sanity check that we haven't missed anything:

[11:23:51] jbs@bbn-w1:/home/jbs
+$ wc -l netstat-*
   7 netstat-commands.txt
   7 netstat-pids.txt
  10 netstat-raw.txt
  24 total

The change from raw to pids is because of duplicates: Two for 1983, two for 2001, and two for 2272.

Let's plow through each of those.

This is from an RPM:

rpc       1983     1  0 Jul17 ?        00:00:01 rpcbind

[11:27:39] jbs@bbn-w1:/home/jbs
+$ which rpcbind
rpcbind is /sbin/rpcbind

[11:27:41] jbs@bbn-w1:/home/jbs
+$ rpm -qf /sbin/rpcbind
rpcbind-0.2.0-8.el6.x86_64

This is from an RPM:

rpcuser   2001     1  0 Jul17 ?        00:00:00 rpc.statd

[11:27:45] jbs@bbn-w1:/home/jbs
+$ which rpc.statd
rpc.statd is /sbin/rpc.statd

[11:28:02] jbs@bbn-w1:/home/jbs
+$ rpm -qf /sbin/rpc.statd
nfs-utils-1.2.3-15.el6.x86_64

This is from an RPM:

root      2272     1  0 Jul17 ?        00:00:00 /usr/sbin/sshd

[11:28:35] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/sbin/sshd
openssh-server-5.3p1-70.el6_2.2.x86_64

This is from an RPM:

root      2280     1  0 Jul17 ?        00:00:05 xinetd -stayalive -pidfile /var/run/xinetd.pid

[11:28:50] jbs@bbn-w1:/home/jbs
+$ which xinetd
xinetd is /usr/sbin/xinetd

[11:28:52] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/sbin/xinetd
xinetd-2.3.14-33.el6.x86_64

This is from an RPM:

root      2374     1  0 Jul17 ?        00:00:10 /usr/libexec/postfix/master

[11:28:56] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/libexec/postfix/master
postfix-2.6.6-2.2.el6_1.x86_64

These are from RPMs:

root      2572     1  0 Jul17 ?        01:55:23 /usr/bin/ruby /usr/sbin/puppetd

[11:29:12] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/bin/ruby
ruby-1.8.7.352-4.el6_2.x86_64

[11:29:29] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/sbin/puppetd
puppet-2.6.14-1.el6.noarch

This is not from an RPM; but there seems to be a qemu-kvm RPM; but it doesn't contain any files:

qemu     18815     1  0 Aug14 ?        00:01:20 /usr/libexec/qemu-kvm -S -M pc-0.14 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name instance-00000d78 -uuid efb7e543-0ed8-9dac-76dd-12345b4be3a9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000d78.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -kernel /var/lib/nova/instances/instance-00000d78/kernel -initrd /var/lib/nova/instances/instance-00000d78/ramdisk -append root=/dev/vda console=ttyS0 selinux=0 -drive file=/var/lib/nova/instances/instance-00000d78/disk,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/nova/instances/instance-00000d78/disk.local,if=none,id=drive-virtio-disk1,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=22,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:16:3e:69:b0:db,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:e6:5e:1a,bus=pci.0,addr=0x4 -chardev file,id=charserial0,path=/var/lib/nova/instances/instance-00000d78/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

[11:29:34] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/libexec/qemu-kvm
file /usr/libexec/qemu-kvm is not owned by any package

[11:30:58] jbs@bbn-w1:/home/jbs
+$ rpm -q qemu-kvm
qemu-kvm-0.15.0-3.el6.x86_64

[11:31:15] jbs@bbn-w1:/home/jbs
+$ rpm -ql qemu-kvm
(contains no files)

Ah:

[11:31:17] jbs@bbn-w1:/home/jbs
+$ rpm -qi qemu-kvm
Name        : qemu-kvm                     Relocations: (not relocatable)
Version     : 0.15.0                            Vendor: (none)
Release     : 3.el6                         Build Date: Thu Nov 24 12:24:20 2011
Install Date: Fri Feb 24 08:56:13 2012         Build Host: osc-build.vm.griddynamics.net
Group       : Development/Tools             Source RPM: qemu-0.15.0-3.el6.src.rpm
Size        : 0                                License: GPLv2+ and LGPLv2+ and BSD
Signature   : RSA/SHA1, Thu Nov 24 12:25:34 2011, Key ID 499590cc596f1e88
URL         : http://www.qemu.org/
Summary     : QEMU metapackage for KVM support
Description :
This is a meta-package that provides a qemu-system-<arch> package for native
architectures where kvm can be enabled. For example, in an x86 system, this
will install qemu-system-x86

And ah, /usr/libexec/qemu-kvm is just a symlink, to an executable which *is* from an RPM:

[11:31:47] jbs@bbn-w1:/home/jbs
+$ file /usr/libexec/qemu-kvm
/usr/libexec/qemu-kvm: symbolic link to `/usr/bin/qemu-kvm'

[11:32:17] jbs@bbn-w1:/home/jbs
+$ rpm -qf /usr/bin/qemu-kvm
qemu-system-x86-0.15.0-3.el6.x86_64

Ok then.

So, that's these RPMs:

rpcbind-0.2.0-8.el6.x86_64
nfs-utils-1.2.3-15.el6.x86_64
openssh-server-5.3p1-70.el6_2.2.x86_64
xinetd-2.3.14-33.el6.x86_64
postfix-2.6.6-2.2.el6_1.x86_64
ruby-1.8.7.352-4.el6_2.x86_64
puppet-2.6.14-1.el6.noarch
qemu-system-x86-0.15.0-3.el6.x86_64

Those all look like standard CentOS things:

[11:39:19] jbs@bbn-w1:/home/jbs
+$ yum -C list installed rpcbind-0.2.0-8.el6.x86_64 nfs-utils-1.2.3-15.el6.x86_64 openssh-server-5.3p1-70.el6_2.2.x86_64 xinetd-2.3.14-33.el6.x86_64 postfix-2.6.6-2.2.el6_1.x86_64 ruby-1.8.7.352-4.el6_2.x86_64 puppet-2.6.14-1.el6.noarch qemu-system-x86-0.15.0-3.el6.x86_64
Loaded plugins: fastestmirror, priorities, refresh-packagekit, security
190 packages excluded due to repository priority protections
Installed Packages
nfs-utils.x86_64      1:1.2.3-15.el6    @anaconda-CentOS-201112091719.x86_64/6.2
openssh-server.x86_64 5.3p1-70.el6_2.2  @CentOS Updates/6.2                     
postfix.x86_64        2:2.6.6-2.2.el6_1 @anaconda-CentOS-201112091719.x86_64/6.2
puppet.noarch         2.6.14-1.el6      @epel                                   
qemu-system-x86.x86_64
                      2:0.15.0-3.el6    @ExoGENI/6.2                            
rpcbind.x86_64        0.2.0-8.el6       @anaconda-CentOS-201112091719.x86_64/6.2
ruby.x86_64           1.8.7.352-4.el6_2 @updates                                
xinetd.x86_64         2:2.3.14-33.el6   @anaconda-CentOS-201112091719.x86_64/6.2

('yum -C' to use cached info only, because these bbn-w1 can't do DNS right now.)

So, that identifies the source of each network-listening file, and since they're all standard CentOS RPMs, source RPMs for them can be found in standard CentOS places.

I've created ExoGENI ticket 104 for the DNS issue, but it's not a blocker for this test.

Step 3: verify VLANs on the rack management switch

Overview of Step 3

Using:

  • Establish a privileged login to the 8052 (management) switch
  • Obtain the list of all VLAN mappings for all interfaces
  • Determine which interfaces connect to worker nodes
  • Create a listing of worker node control interfaces to VLANs by interface type

Verify:

  • All worker node IMM interfaces are access ports on VLAN 1006
  • All worker node iSCSI interfaces are access ports on VLAN 1009
  • Control interfaces for all workers in OpenStack mode are access ports on VLAN 1007
  • Control interfaces for all workers in bare metal mode are access ports on VLAN 1008

Results of Step 3 from 2012-10-10

Established a privileged login to the 8052 management switch:

[lnevers@bbn-hn]$ ssh bbn-8052
Enter radius password: 

IBM Networking Operating System RackSwitch G8052.

bbn-8052.bbn.xo>

Determined all VLAN mappings for all interfaces:

bbn-8052.bbn.xo>show vlan 
VLAN                Name                Status            Ports
----  --------------------------------  ------  -------------------------
1     Default VLAN                      ena     XGE3
20    VLAN 20                           ena     empty
829   VLAN 829                          ena     empty
1006  ExoGENI Management Private        ena     1-20 XGE4
1007  OpenStack Worker Nodes            ena     17-20 37-44
1008  xCAT (Bare Metal)                 ena     17-20 45 46
1009  iSCSI Worker                      ena     25-36
1010  Public Internet Connectivity      ena     47-XGE2
1011  OpenStack VMs                     ena     17-20 37-44
1012  iSCSI Head Node                   ena     21-24
bbn-8052.bbn.xo>

Determined which interfaces connect to worker nodes, by displaying the ports associated with the OpenStack Worker nodes,and xCAT VLANs:

bbn-8052.bbn.xo>show vlan 1006 
VLAN                Name                Status            Ports
----  --------------------------------  ------  -------------------------
1006  ExoGENI Management Private        ena     1-20 XGE4

Private-VLAN    Type      Mapped-To           Status       Ports
------------  ---------  ------------------  ---------- --------------------
1006          -          -                   dis        1-20 XGE4

Note: Worker node IMM interfaces are accessed ports on VLAN 1006

bbn-8052.bbn.xo>show vlan 1007 
VLAN                Name                Status            Ports
----  --------------------------------  ------  -------------------------
1007  OpenStack Worker Nodes            ena     17-20 37-44

Private-VLAN    Type      Mapped-To           Status       Ports
------------  ---------  ------------------  ---------- --------------------
1007          -          -                   dis        17-20 37-44

Note: Control interfaces for all workers in OpenStack mode are access ports on VLAN 1007

bbn-8052.bbn.xo>show vlan 1008
VLAN                Name                Status            Ports
----  --------------------------------  ------  -------------------------
1008  xCAT (Bare Metal)                 ena     17-20 45 46

Private-VLAN    Type      Mapped-To           Status       Ports
------------  ---------  ------------------  ---------- --------------------
1008          -          -                   dis        17-20 45 46

Note: Worker node iSCSI interfaces are access ports on VLAN 1009

VLAN                Name                Status            Ports
----  --------------------------------  ------  -------------------------
1009  iSCSI Worker                      ena     25-36

Private-VLAN    Type      Mapped-To           Status       Ports
------------  ---------  ------------------  ---------- --------------------
1009          -          -                   dis        25-36

Note: Control interfaces for all workers in bare metal mode are access ports on VLAN 1008

Created a listing of worker node control interfaces to VLANs by interface type

Port Worker Node Interface type VLAN
5 bbn-w1 IMM 1006
6 bbn-w2 IMM 1006
7 bbn-w3 IMM 1006
2 bbn-w4 IMM 1006
10 bbn-w5 IMM 1006
11 bbn-w6 IMM 1006
12 bbn-w7 IMM 1006
13 bbn-w8 IMM 1006
14 bbn-w9 IMM 1006
15 bbn-w10 IMM 1006
27 bbn-w1 iSCSI Worker 1009
28 bbn-w2 iSCSI Worker 1009
29 bbn-w3 iSCSI Worker 1009
30 bbn-w4 iSCSI Worker 1009
31 bbn-w5 iSCSI Worker 1009
32 bbn-w6 iSCSI Worker 1009
33 bbn-w7 iSCSI Worker 1009
34 bbn-w8 iSCSI Worker 1009
35 bbn-w9 iSCSI Worker 1009
36 bbn-w10 iSCSI Worker 1009
37 bbn-w1 OpenStack Worker 1007
38 bbn-w2 OpenStack Worker 1007
39 bbn-w3 OpenStack Worker 1007
40 bbn-w4 OpenStack Worker 1007
41 bbn-w5 OpenStack Worker 1007
42 bbn-w6 OpenStack Worker 1007
43 bbn-w7 OpenStack Worker 1007
44 bbn-w8 OpenStack Worker 1007
45 bbn-w9 (xCAT)Bare Metal 1008
46 bbn-w10 (xCAT)Bare Metal 1008

Step 4: verify MAC addresses on the rack management switch

Overview of Step 4

Using:

  • Establish a privileged login to the 8052 (management) switch
  • Obtain a list of the full MAC address table of the switch
  • Use interface listings on hosts and devices to determine the identities of all MAC addresses

Verify:

  • It is possible to identify and classify every MAC address visible on the switch

Results of Step 4 from 2012-10-10

Established a privileged login to the 8052 management switch:

[lnevers@bbn-hn]$ ssh bbn-8052
Enter radius password: 

IBM Networking Operating System RackSwitch G8052.

bbn-8052.bbn.xo>

Determined full list of MAC address table for switch:

bbn-8052.bbn.xo>show mac-address-table 
Mac address Aging Time: 300 

Total number of FDB entries : 61          
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:10:18:a8:73:d2    1012             53     TRK               
  00:10:18:a8:73:d4    1007             54     TRK               
  00:10:18:a8:73:d6    1007             54     TRK               
  00:80:e5:1d:f7:13    1012    23              FWD               
  00:80:e5:25:2e:a3    1012    24              FWD               
  00:80:e5:2d:01:64    1006    3               FWD               
  00:80:e5:2d:03:ac    1006    4               FWD               
  02:16:3e:02:f8:be    1011    44              FWD               
  02:16:3e:06:03:c8    1011    44              FWD               
  02:16:3e:07:2f:58    1011    41              FWD               
  02:16:3e:18:f3:fd    1011    37              FWD               
  02:16:3e:1a:e8:1e    1011    38              FWD               
  02:16:3e:1c:2b:7f    1011    38              FWD               
  02:16:3e:20:b9:e7    1011    40              FWD               
  02:16:3e:2c:38:33    1011             54     TRK               
  02:16:3e:2e:d9:ec    1011    43              FWD               
  02:16:3e:35:2d:c6    1011    40              FWD               
  02:16:3e:38:81:d4    1011    39              FWD               
  02:16:3e:3c:29:90    1011    42              FWD               
  02:16:3e:45:49:69    1011    41              FWD               
  02:16:3e:4d:67:aa    1011    43              FWD               
  02:16:3e:57:cc:be    1011    40              FWD               
  02:16:3e:63:a3:2e    1011    37              FWD               
  02:16:3e:64:87:e4    1011    42              FWD               
  02:16:3e:64:a3:40    1011    39              FWD               
  02:16:3e:66:40:a9    1011    41              FWD               
  02:16:3e:74:37:c6    1011    39              FWD               
  02:16:3e:7e:11:13    1011    37              FWD               
  02:16:3e:7f:36:22    1011    43              FWD               
  08:17:f4:b5:2a:00    1006    XGE4            FWD               
  08:17:f4:b5:2a:3f    1006    XGE4            FWD               
  5c:f3:fc:6b:10:a8    1006             54     TRK               
  5c:f3:fc:6b:10:a8    1007             54     TRK               
  5c:f3:fc:6b:10:a8    1008             54     TRK               
  5c:f3:fc:6b:10:aa    1007             54     TRK               
  5c:f3:fc:ba:13:34    1010    48              FWD               
  5c:f3:fc:ba:13:36    1012             53     TRK               
  5c:f3:fc:ba:4e:78    1007    38              FWD               
  5c:f3:fc:ba:51:d4    1007    40              FWD               
  5c:f3:fc:ba:56:68    1007    37              FWD               
  5c:f3:fc:ba:56:68    1011    37              FWD               
  5c:f3:fc:ba:65:c0    1007    39              FWD               
  5c:f3:fc:c0:14:c5    1006    1               FWD               
  5c:f3:fc:c0:50:09    1006    6               FWD               
  5c:f3:fc:c0:53:65    1006    2               FWD               
  5c:f3:fc:c0:57:f9    1006    5               FWD               
  5c:f3:fc:c0:67:51    1006    7               FWD               
  6c:ae:8b:1b:e7:8a    1008    46              FWD               
  6c:ae:8b:1b:e7:8e    1006    15              FWD               
  6c:ae:8b:1b:e7:f2    1007    42              FWD               
  6c:ae:8b:1b:e7:f6    1006    11              FWD               
  6c:ae:8b:1b:e9:d2    1008    45              FWD               
  6c:ae:8b:1b:e9:d6    1006    14              FWD               
  6c:ae:8b:1b:ee:fa    1007    41              FWD               
  6c:ae:8b:1b:ee:fe    1006    10              FWD               
  6c:ae:8b:1b:ef:52    1007    43              FWD               
  6c:ae:8b:1b:ef:56    1006    12              FWD               
  6c:ae:8b:1b:f0:aa    1007    44              FWD               
  6c:ae:8b:1b:f0:ae    1006    13              FWD               
  cc:ef:48:7a:7a:a9    1010    48              FWD               
  f8:c0:01:bb:f8:cb    1006    9               FWD 

Used interface listings on hosts and devices to determine the identities of all MAC addresses on the switch.

bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d2
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:10:18:a8:73:d2    1012             53     TRK               
bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d4
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:10:18:a8:73:d4    1007             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 00:10:18:a8:73:d6
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:10:18:a8:73:d6    1007             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:1d:f7:13
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:80:e5:1d:f7:13    1012    23              FWD               
bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:25:2e:a3
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:80:e5:25:2e:a3    1012    24              FWD               
bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:2d:01:64
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:80:e5:2d:01:64    1006    3               FWD               
bbn-8052.bbn.xo>show mac-address-table address 00:80:e5:2d:03:ac
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  00:80:e5:2d:03:ac    1006    4               FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:02:f8:be
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:02:f8:be    1011    44              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:06:03:c8
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:06:03:c8    1011    44              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:07:2f:58
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:07:2f:58    1011    41              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:18:f3:fd
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:18:f3:fd    1011    37              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:1a:e8:1e
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:1a:e8:1e    1011    38              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:1c:2b:7f
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:1c:2b:7f    1011    38              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:20:b9:e7
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:20:b9:e7    1011    40              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:2c:38:33
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:2c:38:33    1011             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:2e:d9:ec
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:2e:d9:ec    1011    43              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:35:2d:c6
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:35:2d:c6    1011    40              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:38:81:d4
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:38:81:d4    1011    39              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:3c:29:90
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:3c:29:90    1011    42              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:45:49:69
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:45:49:69    1011    41              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:4d:67:aa
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:4d:67:aa    1011    43              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:57:cc:be
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:57:cc:be    1011    40              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:63:a3:2e
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:63:a3:2e    1011    37              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:64:87:e4
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:64:87:e4    1011    42              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:64:a3:40
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:64:a3:40    1011    39              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:66:40:a9
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:66:40:a9    1011    41              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:74:37:c6
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:74:37:c6    1011    39              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:7e:11:13
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:7e:11:13    1011    37              FWD               
bbn-8052.bbn.xo>show mac-address-table address 02:16:3e:7f:36:22
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  02:16:3e:7f:36:22    1011    43              FWD               
bbn-8052.bbn.xo>show mac-address-table address 08:17:f4:b5:2a:00
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  08:17:f4:b5:2a:00    1006    XGE4            FWD               
bbn-8052.bbn.xo>show mac-address-table address 08:17:f4:b5:2a:3f
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  08:17:f4:b5:2a:3f    1006    XGE4            FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:6b:10:a8    1007             54     TRK               
  5c:f3:fc:6b:10:a8    1006             54     TRK               
  5c:f3:fc:6b:10:a8    1008             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:6b:10:a8    1007             54     TRK               
  5c:f3:fc:6b:10:a8    1006             54     TRK               
  5c:f3:fc:6b:10:a8    1008             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:a8
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:6b:10:a8    1007             54     TRK               
  5c:f3:fc:6b:10:a8    1006             54     TRK               
  5c:f3:fc:6b:10:a8    1008             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:6b:10:aa
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:6b:10:aa    1007             54     TRK               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:13:34
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:13:34    1010    48              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:13:36
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:13:36    1012             53     TRK               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:4e:78
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:4e:78    1007    38              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:51:d4
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:51:d4    1007    40              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:56:68
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:56:68    1011    37              FWD               
  5c:f3:fc:ba:56:68    1007    37              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:56:68
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:56:68    1011    37              FWD               
  5c:f3:fc:ba:56:68    1007    37              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:ba:65:c0
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:ba:65:c0    1007    39              FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:14:c5
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:c0:14:c5    1006    1               FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:50:09
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:c0:50:09    1006    6               FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:53:65
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:c0:53:65    1006    2               FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:57:f9
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:c0:57:f9    1006    5               FWD               
bbn-8052.bbn.xo>show mac-address-table address 5c:f3:fc:c0:67:51
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  5c:f3:fc:c0:67:51    1006    7               FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:8a
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e7:8a    1008    46              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:8e
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e7:8e    1006    15              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:f2
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e7:f2    1007    42              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e7:f6
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e7:f6    1006    11              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e9:d2
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e9:d2    1008    45              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:e9:d6
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:e9:d6    1006    14              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ee:fa
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:ee:fa    1007    41              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ee:fe
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:ee:fe    1006    10              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ef:52
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:ef:52    1007    43              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:ef:56
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:ef:56    1006    12              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:f0:aa
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:f0:aa    1007    44              FWD               
bbn-8052.bbn.xo>show mac-address-table address 6c:ae:8b:1b:f0:ae
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  6c:ae:8b:1b:f0:ae    1006    13              FWD               
bbn-8052.bbn.xo>show mac-address-table address cc:ef:48:7a:7a:a9
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  cc:ef:48:7a:7a:a9    1010    48              FWD               
bbn-8052.bbn.xo>show mac-address-table address f8:c0:01:bb:f8:cb
     MAC address       VLAN     Port    Trnk  State  Permanent  
  -----------------  --------  -------  ----  -----  ---------  
  f8:c0:01:bb:f8:cb    1006    9               FWD