Changes between Version 21 and Version 22 of GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-1


Ignore:
Timestamp:
02/28/13 08:12:24 (11 years ago)
Author:
chaos@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-1

    v21 v22  
    55''This page is GPO's working page for performing IG-MON-1.  It is public for informational purposes, but it is not an official status report.  See [wiki:GENIRacksHome/InstageniRacks/AcceptanceTestStatus] for the current status of InstaGENI acceptance tests.''
    66
    7 ''Last substantive edit of this page: 2013-01-07''
     7''Last substantive edit of this page: 2013-02-28''
    88
    99== Page format ==
     
    1919== Status of test ==
    2020
    21 || '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments'''                                                             ||
    22 || 1          || [[Color(orange,Blocked)]]      ||                      || [instaticket:80]   || ([instaticket:24]) blocked on availability of Emulab source/object tree on BBN boss      ||
    23 || 2          || [[Color(yellow,Complete)]]     ||                      ||                    || ([instaticket:24]) retest on BBN rack to verify availability of Emulab source/object tree ||
    24 || 3          || [[Color(green,Pass)]]          || 2012-11-21           ||                    ||                                                                                           ||
    25 || 4          || [[Color(yellow,Blocked-site)]] ||                      ||                    || blocked on receipt of the BBN rack                                                        ||
    26 || 5          || [[Color(green,Pass)]]          || 2012-05-17           ||                    ||                                                                                           ||
    27 || 6          || [[Color(green,Pass)]]          || 2013-01-07           ||                    || ([instaticket:25]) verified ability to find Emulab version within Emulab images at BBN    ||
    28 || 7          || [[Color(yellow,Blocked-site)]] ||                      ||                    || blocked on receipt of the BBN rack                                                        ||
     21|| '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments'''                                                                        ||
     22|| 1          || [[Color(orange,Blocked)]]      ||                      ||                    || ([instaticket:24], [instaticket:80]) blocked on e-mail discussion of source/object tree availability ||
     23|| 2          || [[Color(yellow,Complete)]]     ||                      ||                    || ([instaticket:24]) retest on BBN rack to verify availability of Emulab source/object tree            ||
     24|| 3          || [[Color(green,Pass)]]          || 2012-11-21           ||                    ||                                                                                                      ||
     25|| 4          || [[Color(yellow,Blocked-site)]] ||                      ||                    || blocked on receipt of the BBN rack                                                                   ||
     26|| 5          || [[Color(green,Pass)]]          || 2012-05-17           ||                    ||                                                                                                      ||
     27|| 6          || [[Color(green,Pass)]]          || 2013-01-07           ||                    || ([instaticket:25]) verified ability to find Emulab version within Emulab images at BBN               ||
     28|| 7          || [[Color(yellow,Blocked-site)]] ||                      ||                    || blocked on receipt of the BBN rack                                                                   ||
    2929
    3030== High-level description from test plan ==
     
    681681   * I'm not going to bother retesting the things from the FreeBSD base install --- as mentioned when i tested at Utah, the MD5 sums don't match, so i'd just be using the FreeBSD base install ISO to verify the identities of the binaries.  Since the listed binaries are the same this time as last time, i'm going to assume they are still reflected in the FreeBSD base distro.
    682682
    683 [instaticket:80] covers the question of the locations of the Emulab src/obj trees on the BBN boss node.
     683=== Results of testing step 1: 2013-02-28 ===
     684
     685[instaticket:80] covers the question of the locations of the Emulab src/obj trees on the BBN boss node: it turns out i was looking at the wrong boss node for much of the previous test, so i'm just going to redo it entirely.
     686
     687Testing on `boss.instageni.gpolab.bbn.com`:
     688
     689 * `sockstat -lL46` is my goto on FreeBSD.  Get a full list of all binaries of processes which are listening on IPv4 or IPv6 sockets from non-localhost addresses:
     690{{{
     691boss.instageni.gpolab.bbn.com,[~],07:57(0)$ for pid in $(sockstat -lL46 | awk '{print $3}' | grep -v PID | sort -u); do procstat -b $pid; done | awk '{print $3}' | sort -u
     692usage: procstat [-h] [-C] [-M core] [-N system] [-w interval]
     693                [-b | -c | -e | -f | -i | -j | -k | -l | -s | -t | -v | -x] [-a | pid ...]
     694/usr/libexec/sendmail/sendmail
     695/usr/local/bin/perl
     696/usr/local/bin/python2.7
     697/usr/local/libexec/pubsubd
     698/usr/local/libexec/tftpd
     699/usr/local/sbin/dhcpd
     700/usr/local/sbin/httpd
     701/usr/sbin/inetd
     702/usr/sbin/lwresd
     703/usr/sbin/mountd
     704/usr/sbin/nfsd
     705/usr/sbin/ntpd
     706/usr/sbin/rpcbind
     707/usr/sbin/sshd
     708/usr/sbin/syslogd
     709/usr/testbed/sbin/bootinfo
     710/usr/testbed/sbin/mfrisbeed
     711/usr/testbed/sbin/sdcollectd
     712/usr/testbed/sbin/tmcd
     713PATH
     714}}}
     715 * The errors were a couple of pids which get reported as `?`:
     716{{{
     717boss.instageni.gpolab.bbn.com,[~],07:57(0)$ sockstat -lL46
     718USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS     
     719...
     720?        ?          ?     ?  udp4   *:2049                *:*
     721?        ?          ?     ?  udp6   *:2049                *:*
     722}}}
     723 In the previous test, i concluded that this is normal nfsd weirdness and no cause for alarm.
     724 * So, the list of listeners is:
     725{{{
     726/usr/libexec/sendmail/sendmail
     727/usr/local/bin/perl
     728/usr/local/bin/python2.7
     729/usr/local/libexec/pubsubd
     730/usr/local/libexec/tftpd
     731/usr/local/sbin/dhcpd
     732/usr/local/sbin/httpd
     733/usr/sbin/inetd
     734/usr/sbin/lwresd
     735/usr/sbin/mountd
     736/usr/sbin/nfsd
     737/usr/sbin/ntpd
     738/usr/sbin/rpcbind
     739/usr/sbin/sshd
     740/usr/sbin/syslogd
     741/usr/testbed/sbin/bootinfo
     742/usr/testbed/sbin/mfrisbeed
     743/usr/testbed/sbin/sdcollectd
     744/usr/testbed/sbin/tmcd
     745}}}
     746 * The following commands are sourced from freebsd packages:
     747{{{
     748boss.instageni.gpolab.bbn.com,[~],08:00(0)$ pkg_info -W /usr/local/bin/perl
     749/usr/local/bin/perl was installed by package perl-5.12.4_3
     750
     751boss.instageni.gpolab.bbn.com,[~],08:00(0)$ pkg_info -W /usr/local/bin/python2.7
     752/usr/local/bin/python2.7 was installed by package python27-2.7.2_4
     753
     754boss.instageni.gpolab.bbn.com,[~],08:00(0)$ pkg_info -W /usr/local/libexec/pubsubd
     755/usr/local/libexec/pubsubd was installed by package pubsub-0.95
     756
     757boss.instageni.gpolab.bbn.com,[~],08:01(0)$ pkg_info -W /usr/local/libexec/tftpd
     758/usr/local/libexec/tftpd was installed by package emulab-tftp-hpa-0.48
     759
     760boss.instageni.gpolab.bbn.com,[~],08:01(0)$ pkg_info -W /usr/local/sbin/dhcpd
     761/usr/local/sbin/dhcpd was installed by package isc-dhcp42-server-4.2.3_2
     762
     763boss.instageni.gpolab.bbn.com,[~],08:01(0)$ pkg_info -W /usr/local/sbin/httpd
     764/usr/local/sbin/httpd was installed by package apache-2.2.22_5
     765}}}
     766   * So the summary of sourced packages here is:
     767{{{
     768apache-2.2.22_5
     769emulab-tftp-hpa-0.48
     770isc-dhcp42-server-4.2.3_2
     771perl-5.12.4_3
     772pubsub-0.95
     773python27-2.7.2_4
     774}}}
     775   * Not bothering to verify the distinfo from /usr/ports again, because i did it before, and the versions haven't changed
     776 * The following commands aren't part of packages (`pkg_info -W` reports nothing):
     777{{{
     778/usr/libexec/sendmail/sendmail
     779/usr/sbin/inetd
     780/usr/sbin/lwresd
     781/usr/sbin/mountd
     782/usr/sbin/nfsd
     783/usr/sbin/ntpd
     784/usr/sbin/rpcbind
     785/usr/sbin/sshd
     786/usr/sbin/syslogd
     787/usr/testbed/sbin/bootinfo
     788/usr/testbed/sbin/mfrisbeed
     789/usr/testbed/sbin/sdcollectd
     790/usr/testbed/sbin/tmcd
     791}}}
     792 * The assumption is that these are either part of the FreeBSD base system, or are part of Emulab.  How do we find out which?
     793   * I can look in the canonical source of the emulab compile, which should be `/usr/testbed/obj/testbed`.  Here's a process for checking various things in `/usr/testbed/sbin`, on the suspicion that they are probably Emulab binaries:
     794{{{
     795cd /usr/testbed/obj/testbed
     796shortname=bootinfo    # or whatever
     797shortmd5=$(md5 /usr/testbed/sbin/$shortname | awk '{print $4}')
     798for path in $(find . -type f -name $shortname 2> /dev/null); do md5 $path; done | grep $shortmd5
     799}}}
     800   * Hmm, the first thing i tried that on, bootinfo, did not match, and i'm sure i'm on the right boss now:
     801{{{
     802boss.instageni.gpolab.bbn.com,[~],08:07(0)$ md5 /usr/testbed/sbin/bootinfo /usr/testbed/obj/testbed/pxe/bootinfo
     803MD5 (/usr/testbed/sbin/bootinfo) = 880f20f3320ccdeed81b685756cfb8cd
     804MD5 (/usr/testbed/obj/testbed/pxe/bootinfo) = 1c651f32132f80888b5ef8896b042e12
     805
     806boss.instageni.gpolab.bbn.com,[~],08:08(0)$ ls -l /usr/testbed/sbin/bootinfo /usr/testbed/obj/testbed/pxe/bootinfo
     807-rwxr-xr-x  1 root  wheel    140471 Dec 11 15:57 /usr/testbed/obj/testbed/pxe/bootinfo*
     808-rwxr-xr-x  1 root  tbadmin  141046 Feb 26 11:10 /usr/testbed/sbin/bootinfo*
     809}}}
     810   * Ditto mfrisbeed:
     811{{{
     812boss.instageni.gpolab.bbn.com,[~],08:09(0)$ md5 /usr/testbed/sbin/mfrisbeed /usr
     813/testbed/obj/testbed/clientside/os/frisbee.redux/mfrisbeed
     814MD5 (/usr/testbed/sbin/mfrisbeed) = 0d90a44d74e4c3520835de15a1743d01
     815MD5 (/usr/testbed/obj/testbed/clientside/os/frisbee.redux/mfrisbeed) = 71125d6e351a78f8a930873cd203cf0b
     816
     817boss.instageni.gpolab.bbn.com,[~],08:09(0)$ ls -l /usr/testbed/sbin/mfrisbeed /usr/testbed/obj/testbed/clientside/os/frisbee.redux/mfrisbeed
     818-rwxr-xr-x  1 root  wheel    80952 Dec 11 15:57 /usr/testbed/obj/testbed/clientside/os/frisbee.redux/mfrisbeed*
     819-rwxr-xr-x  1 root  tbadmin  80952 Feb 15 18:01 /usr/testbed/sbin/mfrisbeed*
     820}}}
     821   * And the src files on this boss are old:
     822{{{
     823boss.instageni.gpolab.bbn.com,[~],08:10(0)$ ls -l /usr/testbed/src/
     824total 12
     825drwxr-xr-x  54 root  wheel  2048 Dec 11 15:50 emulab-devel/
     826drwxr-xr-x  53 root  wheel  2048 Dec 11 15:51 emulab-stable/
     827drwxr-xr-x  54 282   6000   2560 Dec 11 15:54 testbed/
     828}}}
     829[instaticket:24] claimed that, on a non-development rack, source and object files will be found in `/usr/testbed/{src,obj}/emulab-stable`.
     830
     831Picking this up via e-mail.
    684832
    685833== Step 2: identify network-listening software on the ops node ==