Changes between Version 24 and Version 25 of GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-1


Ignore:
Timestamp:
03/12/13 14:35:02 (11 years ago)
Author:
chaos@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-1

    v24 v25  
    55''This page is GPO's working page for performing IG-MON-1.  It is public for informational purposes, but it is not an official status report.  See [wiki:GENIRacksHome/InstageniRacks/AcceptanceTestStatus] for the current status of InstaGENI acceptance tests.''
    66
    7 ''Last substantive edit of this page: 2013-02-28''
     7''Last substantive edit of this page: 2013-03-12''
    88
    99== Page format ==
     
    2020
    2121|| '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments'''                                                                        ||
    22 || 1          || [[Color(orange,Blocked)]]      ||                      ||                    || ([instaticket:24], [instaticket:80]) blocked on e-mail discussion of source/object tree availability ||
     22|| 1          || [[Color(green,Pass)]]          || 2013-03-12           ||                    || ([instaticket:24], [instaticket:80])                                                                ||
    2323|| 2          || [[Color(orange,Blocked)]]      ||                      ||                    || ([instaticket:24]) blocked on step 1                                                                 ||
    2424|| 3          || [[Color(green,Pass)]]          || 2012-11-21           ||                    ||                                                                                                      ||
     
    912912
    913913So, src definitely looks good, just asking Utah about obj as a sanity-check.
     914
     915=== Results of testing step 1: 2013-03-12 ===
     916
     917Here goes again --- again, easiest to redo from scratch.
     918
     919Testing on `boss.instageni.gpolab.bbn.com`:
     920
     921 * `sockstat -lL46` is my goto on FreeBSD.  Get a full list of all binaries of processes which are listening on IPv4 or IPv6 sockets from non-localhost addresses:
     922{{{
     923boss.instageni.gpolab.bbn.com,[~],14:24(0)$ for pid in $(sockstat -lL46 | awk '{print $3}' | grep -v PID | sort -u); do procstat -b $pid; done | awk '{print $3}' | sort -u
     924usage: procstat [-h] [-C] [-M core] [-N system] [-w interval]
     925                [-b | -c | -e | -f | -i | -j | -k | -l | -s | -t | -v | -x] [-a | pid ...]
     926/usr/libexec/sendmail/sendmail
     927/usr/local/bin/perl
     928/usr/local/bin/python2.7
     929/usr/local/libexec/pubsubd
     930/usr/local/libexec/tftpd
     931/usr/local/sbin/dhcpd
     932/usr/local/sbin/httpd
     933/usr/sbin/inetd
     934/usr/sbin/lwresd
     935/usr/sbin/mountd
     936/usr/sbin/nfsd
     937/usr/sbin/ntpd
     938/usr/sbin/rpcbind
     939/usr/sbin/sshd
     940/usr/sbin/syslogd
     941/usr/testbed/sbin/bootinfo
     942/usr/testbed/sbin/mfrisbeed
     943/usr/testbed/sbin/sdcollectd
     944/usr/testbed/sbin/tmcd
     945PATH
     946}}}
     947 * The errors were a couple of pids which get reported as `?`:
     948{{{
     949boss.instageni.gpolab.bbn.com,[~],07:57(0)$ sockstat -lL46
     950USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS     
     951...
     952?        ?          ?     ?  udp4   *:2049                *:*
     953?        ?          ?     ?  udp6   *:2049                *:*
     954}}}
     955 In the previous test, i concluded that this is normal nfsd weirdness and no cause for alarm.
     956 * So, the list of listeners is still:
     957{{{
     958/usr/libexec/sendmail/sendmail
     959/usr/local/bin/perl
     960/usr/local/bin/python2.7
     961/usr/local/libexec/pubsubd
     962/usr/local/libexec/tftpd
     963/usr/local/sbin/dhcpd
     964/usr/local/sbin/httpd
     965/usr/sbin/inetd
     966/usr/sbin/lwresd
     967/usr/sbin/mountd
     968/usr/sbin/nfsd
     969/usr/sbin/ntpd
     970/usr/sbin/rpcbind
     971/usr/sbin/sshd
     972/usr/sbin/syslogd
     973/usr/testbed/sbin/bootinfo
     974/usr/testbed/sbin/mfrisbeed
     975/usr/testbed/sbin/sdcollectd
     976/usr/testbed/sbin/tmcd
     977}}}
     978 * The following commands are sourced from freebsd packages:
     979{{{
     980boss.instageni.gpolab.bbn.com,[~],14:28(0)$ pkg_info -W /usr/local/bin/perl
     981/usr/local/bin/perl was installed by package perl-5.12.4_3
     982boss.instageni.gpolab.bbn.com,[~],14:29(0)$ pkg_info -W /usr/local/bin/python2.7/usr/local/bin/python2.7 was installed by package python27-2.7.2_4
     983boss.instageni.gpolab.bbn.com,[~],14:29(0)$ pkg_info -W /usr/local/libexec/pubsubd
     984/usr/local/libexec/pubsubd was installed by package pubsub-0.95
     985boss.instageni.gpolab.bbn.com,[~],14:29(0)$ pkg_info -W /usr/local/libexec/tftpd/usr/local/libexec/tftpd was installed by package emulab-tftp-hpa-0.48
     986boss.instageni.gpolab.bbn.com,[~],14:29(0)$ pkg_info -W /usr/local/sbin/dhcpd
     987/usr/local/sbin/dhcpd was installed by package isc-dhcp42-server-4.2.3_2
     988boss.instageni.gpolab.bbn.com,[~],14:29(0)$ pkg_info -W /usr/local/sbin/httpd
     989/usr/local/sbin/httpd was installed by package apache-2.2.22_5
     990}}}
     991   * So the summary of sourced packages here is:
     992{{{
     993apache-2.2.22_5
     994emulab-tftp-hpa-0.48
     995isc-dhcp42-server-4.2.3_2
     996perl-5.12.4_3
     997pubsub-0.95
     998python27-2.7.2_4
     999}}}
     1000   * Not bothering to verify the distinfo from /usr/ports again, because i did it before, and the versions haven't changed
     1001 * The following commands aren't part of packages (`pkg_info -W` reports nothing):
     1002{{{
     1003/usr/libexec/sendmail/sendmail
     1004/usr/sbin/inetd
     1005/usr/sbin/lwresd
     1006/usr/sbin/mountd
     1007/usr/sbin/nfsd
     1008/usr/sbin/ntpd
     1009/usr/sbin/rpcbind
     1010/usr/sbin/sshd
     1011/usr/sbin/syslogd
     1012/usr/testbed/sbin/bootinfo
     1013/usr/testbed/sbin/mfrisbeed
     1014/usr/testbed/sbin/sdcollectd
     1015/usr/testbed/sbin/tmcd
     1016}}}
     1017 * The assumption is that these are either part of the FreeBSD base system, or are part of Emulab.  How do we find out which?
     1018   * I can look in the canonical source of the emulab compile, which should be `/usr/testbed/obj/testbed`.  Here's a process for checking various things in `/usr/testbed/sbin`, on the suspicion that they are probably Emulab binaries:
     1019{{{
     1020cd /usr/testbed/obj/testbed
     1021shortname=bootinfo    # or whatever
     1022shortmd5=$(md5 /usr/testbed/sbin/$shortname | awk '{print $4}')
     1023for path in $(find . -type f -name $shortname 2> /dev/null); do md5 $path; done | grep $shortmd5
     1024}}}
     1025   * This time, i found matches for:
     1026{{{
     1027MD5 (./clientside/os/frisbee.redux/mfrisbeed) = 0d90a44d74e4c3520835de15a1743d01
     1028MD5 (./clientside/sensors/slothd/sdcollectd) = 11891c5aeb0734a88ce6c153186daaf4
     1029MD5 (./tmcd/tmcd) = ba4eba54de73ea90a1ab20be2bcd1ad5
     1030}}}
     1031   * Still not bootinfo, but that's an unsurprising consequence of the previous snowflakiness --- bootinfo hasn't been recompiled since the source was synced up properly:
     1032{{{
     1033-rwxr-xr-x  1 root  tbadmin  141046 Feb 26 11:10 /usr/testbed/sbin/bootinfo*
     1034}}}
     1035   So i don't expect this to be an issue for fresh rack installs.
     1036   * The src files on this boss now date from the approximate time of the last install:
     1037{{{
     1038boss.instageni.gpolab.bbn.com,[/usr/testbed/src/testbed],14:33(0)$ git log | head -3
     1039commit 7e584af01b968c18b7d87fe7e78a5dde3549c47f
     1040Author: Leigh B Stoller <stoller@flux.utah.edu>
     1041Date:   Fri Mar 8 12:22:38 2013 -0700
     1042
     1043boss.instageni.gpolab.bbn.com,[/usr/testbed/src/testbed],14:33(0)$ ls -lart /usr
     1044/testbed/sbin/ | tail
     1045-rwxr-xr-x   1 root  tbadmin    7658 Mar  8 19:39 load-descriptors*
     1046-rwxr-xr-x   1 root  tbadmin    4767 Mar  8 19:39 jabbersetup*
     1047lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 deljabberuser@ -> /usr/testbed/libexec/runsuid
     1048lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 deljabberchat@ -> /usr/testbed/libexec/runsuid
     1049lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 addjabberuser@ -> /usr/testbed/libexec/runsuid
     1050lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 addjabberchat@ -> /usr/testbed/libexec/runsuid
     1051drwxr-xr-x   2 root  tbadmin    1024 Mar  8 19:39 protogeni/
     1052lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 gencabundle@ -> /usr/testbed/libexec/runsuid
     1053lrwxr-xr-x   1 root  tbadmin      28 Mar  8 19:39 cleanupslice@ -> /usr/testbed/libexec/runsuid
     1054drwxrwxr-x   4 root  tbadmin    5120 Mar  8 19:39 ./
     1055}}}
    9141056
    9151057== Step 2: identify network-listening software on the ops node ==