Changes between Version 1 and Version 2 of GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-5


Ignore:
Timestamp:
03/15/13 14:17:21 (7 years ago)
Author:
chaos@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENIRacksHome/InstageniRacks/AcceptanceTestStatus/IG-MON-5

    v1 v2  
    99== Status of test ==
    1010
    11 || '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments'''                                                               ||
    12 || 1A         || [[Color(#63B8FF,In Progress)]] ||                      || [gmocticket:164]   || hindered by slow download of GMOC data because of many interfaces; a workaround is possible ||
    13 || 1B         ||                                ||                      ||                    ||                                                                                             ||
    14 || 2A         ||                                ||                      ||                    ||                                                                                             ||
    15 || 2B         ||                                ||                      ||                    ||                                                                                             ||
    16 || 3          ||                                ||                      ||                    ||                                                                                             ||
    17 || 4          ||                                ||                      ||                    ||                                                                                             ||
    18 || 5          ||                                ||                      ||                    ||                                                                                             ||
    19 || 6          ||                                ||                      ||                    ||                                                                                             ||
    20 || 7          ||                                ||                      ||                    ||                                                                                             ||
     11|| '''Step''' || '''State'''                    || '''Date completed''' || '''Open Tickets''' || '''Closed Tickets/Comments''' ||
     12|| 1          || [[Color(#63B8FF,In Progress)]] ||                      ||                    ||                               ||
     13|| 2A         ||                                ||                      ||                    ||                               ||
     14|| 2B         ||                                ||                      ||                    ||                               ||
     15|| 3          ||                                ||                      ||                    ||                               ||
     16|| 4          ||                                ||                      ||                    ||                               ||
     17|| 5          ||                                ||                      ||                    ||                               ||
     18|| 6          ||                                ||                      ||                    ||                               ||
     19|| 7          ||                                ||                      ||                    ||                               ||
    2120
    2221== High-level description from test plan ==
     
    5049Verify that per-rack or per-aggregate summaries are collected of the count of distinct users who have been active on the rack, either by providing raw sliver data containing sliver users to GMOC, or by collecting data locally and producing trending summaries on demand.
    5150
    52 == Step 1: download all recent GMOC data for examination ==
    53 
    54 === Step 1A: download GMOC relational data objects for BBN and Utah racks ===
    55 
    56 ==== Overview of Step 1A ====
    57 
    58 Within a python shell on a node which has gmoc.py installed and configured with a valid submission username:
    59 {{{
     51== Step 1: test all relational data available for examination as of GEC16 ==
     52
     53This was an ad-hoc test used to check the results of relational data downloaded via gmoc.py from gmoc-db.  Because measurement data is not being submitted, and is not easy to map to relational data using the current gmoc.py, i checked relational data only.
     54
     55=== Results of testing: 2013-03-15 ===
     56
     57This test was performed on tikal (monitor.gpolab.bbn.com), which has an automated process which downloads all relational data from known GENI aggregates.
     58 * The procedure used to get the data was:
     59{{{
     60import pickle
    6061import sys
    6162sys.path.append('/usr/local/lib')
    62 import gmoc
    63 import gmoc_config
    64 
    65 config = gmoc_config.read_config_file()
    66 obsolete_url_suffix = 'xchange/webservice.pl'
    67 serviceurl = config['GMOC_REL_URL']
    68 if serviceurl.endswith(obsolete_url_suffix):
    69   serviceurl = serviceurl[:-1 * (len(obsolete_url_suffix))]
    70 
    71 f = open(config['PASSWD_FILE'])
    72 gmoc_passwd = f.readline().strip()
     63f = open('/var/lib/ganglia/gmoc-report/gmoc-relations.pck')
     64r = pickle.load(f)
    7365f.close()
    74 
    75 client = gmoc.GMOCClient(
    76            serviceURL = serviceurl,
    77            username = config['SITENAME'],
    78            password = gmoc_passwd,
    79          )
    80 
    81 bbnig = client.load(gmoc.Aggregate('instageni.gpolab.bbn.com:12369'))
    82 uthig = client.load(gmoc.Aggregate('utah.geniracks.net:12369'))
    83 bbnfoam = client.load(gmoc.Aggregate('foam.instageni.gpolab.bbn.com:3626'))
    84 uthfoam = client.load(gmoc.Aggregate('foam.utah.geniracks.net:3626'))
    85 }}}
    86 This should yield four gmoc.Aggregate objects.  Record the collection time of each object, in case the rest of the testing is done later.
    87 
    88 === Step 1B: download recent GMOC measurement data as XML ===
    89 
    90 ==== Overview of Step 1B ====
    91 
    92 Pretty-print the most recent XML measurement data download from GMOC:
    93 {{{
    94 import urllib
    95 from xml.dom import minidom
    96 usock = urllib.urlopen('http://gmoc-db.grnoc.iu.edu/web-services/gen_api.pl')
    97 dom = minidom.parse(usock)
    98 f = open('recent-gmoc.xml', 'w')
    99 f.write('%s\n' % dom.toprettyxml())
    100 f.close()
    101 }}}
    102 This should yield an XML file, `recent-gmoc.xml`.
     66}}}
     67 * The data was downloaded finishing at 12:57 on 2013-03-15:
     68{{{
     69tikal,[~],13:01(0)$ ls -l /var/lib/ganglia/gmoc-report/gmoc-relations.pck
     70-rw-r--r-- 2 ganglia ganglia 658573 Mar 15 12:57 /var/lib/ganglia/gmoc-report/gmoc-relations.pck
     71tikal,[~],13:01(0)$ date
     72Fri Mar 15 13:01:40 EDT 2013
     73}}}
     74 * Aggregates, resources, and slivers of interest for this test were pulled into variables:
     75{{{
     76bbnig = r['aggregate']['instageni.gpolab.bbn.com:12369']
     77uthig = r['aggregate']['utah.geniracks.net:12369']
     78nrtig = r['aggregate']['instageni.northwestern.edu:12369']
     79ukyig = r['aggregate']['lan.sdn.uky.edu:12369']
     80bbnfoam = r['aggregate']['foam.instageni.gpolab.bbn.com:3626']
     81uthfoam = r['aggregate']['foam.utah.geniracks.net:3626']
     82ig_ams = [bbnig, uthig, nrtig, ukyig]
     83slivers = []
     84resources = []
     85for am in ig_ams:
     86  resources.extend(am.resources)
     87  slivers.extend(am.slivers)
     88}}}
     89
     90Having gotten the data, i ran some tests on it.
     91
     92==== Check parameters of InstaGENI aggregate objects ====
     93
     94Do the reported parameters (id, type, version, operator, pop) of each aggregate look like we expect?  Yes:
     95{{{
     96>>> print "\n".join([a.id for a in ig_ams])
     97instageni.gpolab.bbn.com:12369
     98utah.geniracks.net:12369
     99instageni.northwestern.edu:12369
     100lan.sdn.uky.edu:12369
     101
     102>>> print "\n".join([a.type for a in ig_ams])
     103protogeni
     104protogeni
     105protogeni
     106protogeni
     107
     108>>> print "\n".join([a.version for a in ig_ams])
     1097e584af01b968c18b7d87fe7e78a5dde3549c47f
     1107e584af01b968c18b7d87fe7e78a5dde3549c47f
     1117e584af01b968c18b7d87fe7e78a5dde3549c47f
     1127e584af01b968c18b7d87fe7e78a5dde3549c47f
     113
     114>>> print "\n".join([a.operator.id for a in ig_ams])
     115urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     116urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     117urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     118urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     119
     120>>> print "\n".join([a.pop.id for a in ig_ams])
     121urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     122urn:publicid:IDN+gmoc.geni.net+pop+Utah
     123urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     124urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     125}}}
     126
     127==== Check parameters of InstaGENI resource objects ====
     128
     129Do the reported parameters (id, type, aggregate, operator, pop, state) of each aggregate look like we expect?:
     130{{{
     131>>> print "\n".join([r.id for r in resources])
     132urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc4
     133urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc5
     134urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1
     135urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2
     136urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc3
     137urn:publicid:IDN+utah.geniracks.net+node+pc2
     138urn:publicid:IDN+utah.geniracks.net+node+pc3
     139urn:publicid:IDN+utah.geniracks.net+node+pc1
     140urn:publicid:IDN+utah.geniracks.net+node+pc4
     141urn:publicid:IDN+utah.geniracks.net+node+pc5
     142urn:publicid:IDN+instageni.northwestern.edu+node+pc3
     143urn:publicid:IDN+instageni.northwestern.edu+node+pc2
     144urn:publicid:IDN+instageni.northwestern.edu+node+pc1
     145urn:publicid:IDN+instageni.northwestern.edu+node+pc5
     146urn:publicid:IDN+instageni.northwestern.edu+node+pc4
     147urn:publicid:IDN+lan.sdn.uky.edu+node+pc4
     148urn:publicid:IDN+lan.sdn.uky.edu+node+pc5
     149urn:publicid:IDN+lan.sdn.uky.edu+node+pc1
     150urn:publicid:IDN+lan.sdn.uky.edu+node+pc2
     151urn:publicid:IDN+lan.sdn.uky.edu+node+pc3
     152
     153>>> print "\n".join([r.type for r in resources])
     154host
     155host
     156vmserver
     157vmserver
     158host
     159host
     160vmserver
     161vmserver
     162host
     163vmserver
     164host
     165host
     166vmserver
     167host
     168host
     169host
     170host
     171vmserver
     172vmserver
     173host
     174
     175>>> print "\n".join([r.operator.id for r in resources])
     176urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     177urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     178urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     179urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     180urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     181urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     182urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     183urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     184urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     185urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     186urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     187urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     188urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     189urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     190urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     191urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     192urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     193urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     194urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     195urn:publicid:IDN+gmoc.geni.net+organization+InstaGENI
     196
     197>>> print "\n".join([r.pop.id for r in resources])
     198urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     199urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     200urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     201urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     202urn:publicid:IDN+gmoc.geni.net+pop+gpolab
     203urn:publicid:IDN+gmoc.geni.net+pop+Utah
     204urn:publicid:IDN+gmoc.geni.net+pop+Utah
     205urn:publicid:IDN+gmoc.geni.net+pop+Utah
     206urn:publicid:IDN+gmoc.geni.net+pop+Utah
     207urn:publicid:IDN+gmoc.geni.net+pop+Utah
     208urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     209urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     210urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     211urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     212urn:publicid:IDN+gmoc.geni.net+pop+Northwestern
     213urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     214urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     215urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     216urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     217urn:publicid:IDN+gmoc.geni.net+pop+Kentucky
     218
     219>>> print "\n".join([r.state for r in resources])
     220Unknown
     221Unknown
     222Unknown
     223Unknown
     224Unknown
     225Unknown
     226Unknown
     227Unknown
     228Unknown
     229Unknown
     230Unknown
     231Unknown
     232Unknown
     233Unknown
     234Unknown
     235Unknown
     236Unknown
     237Unknown
     238Unknown
     239Unknown
     240}}}
     241
     242==== Check parameters of InstaGENI resource interface objects ====
     243
     244Collect interface objects:
     245{{{
     246interfaces = []
     247for resource in resources:
     248  interfaces.extend(resource.interfaces)
     249}}}
     250
     251Do the reported parameters (id, IPv4 addresses, MAC addresses, VLANs) of each aggregate look like we expect?:
     252{{{
     253>>> print "\n".join([i.id for i in interfaces])
     254urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth99
     255urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth1
     256urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth3
     257urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth0
     258urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth2
     259urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth99
     260urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth1
     261urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth3
     262urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth0
     263urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth2
     264urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth99
     265urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1
     266urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth3
     267urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0
     268urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth2
     269urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth99
     270urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth1
     271urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth3
     272urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth0
     273urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth2
     274urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth99
     275urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth1
     276urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth3
     277urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth0
     278urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth2
     279urn:publicid:IDN+utah.geniracks.net+interface+pc2:eth1
     280urn:publicid:IDN+utah.geniracks.net+interface+pc2:eth3
     281urn:publicid:IDN+utah.geniracks.net+interface+pc2:eth0
     282urn:publicid:IDN+utah.geniracks.net+interface+pc2:eth2
     283urn:publicid:IDN+utah.geniracks.net+interface+pc2:ilo3
     284urn:publicid:IDN+utah.geniracks.net+interface+pc3:eth1
     285urn:publicid:IDN+utah.geniracks.net+interface+pc3:eth3
     286urn:publicid:IDN+utah.geniracks.net+interface+pc3:eth0
     287urn:publicid:IDN+utah.geniracks.net+interface+pc3:eth2
     288urn:publicid:IDN+utah.geniracks.net+interface+pc3:ilo3
     289urn:publicid:IDN+utah.geniracks.net+interface+pc1:eth0
     290urn:publicid:IDN+utah.geniracks.net+interface+pc1:eth1
     291urn:publicid:IDN+utah.geniracks.net+interface+pc1:eth3
     292urn:publicid:IDN+utah.geniracks.net+interface+pc1:eth2
     293urn:publicid:IDN+utah.geniracks.net+interface+pc1:ilo3
     294urn:publicid:IDN+utah.geniracks.net+interface+pc4:eth1
     295urn:publicid:IDN+utah.geniracks.net+interface+pc4:eth3
     296urn:publicid:IDN+utah.geniracks.net+interface+pc4:eth0
     297urn:publicid:IDN+utah.geniracks.net+interface+pc4:eth2
     298urn:publicid:IDN+utah.geniracks.net+interface+pc4:ilo3
     299urn:publicid:IDN+utah.geniracks.net+interface+pc5:eth1
     300urn:publicid:IDN+utah.geniracks.net+interface+pc5:eth3
     301urn:publicid:IDN+utah.geniracks.net+interface+pc5:eth0
     302urn:publicid:IDN+utah.geniracks.net+interface+pc5:eth2
     303urn:publicid:IDN+utah.geniracks.net+interface+pc5:ilo3
     304urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth99
     305urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth1
     306urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth3
     307urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth0
     308urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth2
     309urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth99
     310urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth1
     311urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth3
     312urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth0
     313urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth2
     314urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth99
     315urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth1
     316urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth3
     317urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth0
     318urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth2
     319urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth99
     320urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth1
     321urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth3
     322urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth0
     323urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth2
     324urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth99
     325urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth1
     326urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth3
     327urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth0
     328urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth2
     329urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth99
     330urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth1
     331urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth3
     332urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth0
     333urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth2
     334urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth99
     335urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth1
     336urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth3
     337urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth0
     338urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth2
     339urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth99
     340urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth1
     341urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth3
     342urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth0
     343urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth2
     344urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth99
     345urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth1
     346urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth3
     347urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth0
     348urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth2
     349urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth99
     350urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth1
     351urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth3
     352urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth0
     353urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth2
     354
     355>>> for i in interfaces:
     356...   if len(i.vlans) > 0:
     357...     print "%s: %s" % (i.id, " ".join(i.vlans))
     358...
     359
     360>>> for i in interfaces:
     361...   if len(i.addresses) > 0:
     362...     print "%s: %s" % (i.id, " ".join([ad.addr for ad in i.addresses]))
     363...
     364urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth0: 192.1.242.143
     365urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth0: 192.1.242.144
     366urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0: 192.1.242.140
     367urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth0: 192.1.242.141
     368urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc3:eth0: 192.1.242.142
     369urn:publicid:IDN+utah.geniracks.net+interface+pc2:eth0: 155.98.34.12
     370urn:publicid:IDN+utah.geniracks.net+interface+pc3:eth0: 155.98.34.13
     371urn:publicid:IDN+utah.geniracks.net+interface+pc1:eth0: 155.98.34.11
     372urn:publicid:IDN+utah.geniracks.net+interface+pc4:eth0: 155.98.34.14
     373urn:publicid:IDN+utah.geniracks.net+interface+pc5:eth0: 155.98.34.15
     374urn:publicid:IDN+instageni.northwestern.edu+interface+pc3:eth0: 165.124.51.142
     375urn:publicid:IDN+instageni.northwestern.edu+interface+pc2:eth0: 165.124.51.141
     376urn:publicid:IDN+instageni.northwestern.edu+interface+pc1:eth0: 165.124.51.140
     377urn:publicid:IDN+instageni.northwestern.edu+interface+pc5:eth0: 165.124.51.144
     378urn:publicid:IDN+instageni.northwestern.edu+interface+pc4:eth0: 165.124.51.143
     379urn:publicid:IDN+lan.sdn.uky.edu+interface+pc4:eth0: 128.163.232.19
     380urn:publicid:IDN+lan.sdn.uky.edu+interface+pc5:eth0: 128.163.232.20
     381urn:publicid:IDN+lan.sdn.uky.edu+interface+pc1:eth0: 128.163.232.16
     382urn:publicid:IDN+lan.sdn.uky.edu+interface+pc2:eth0: 128.163.232.17
     383urn:publicid:IDN+lan.sdn.uky.edu+interface+pc3:eth0: 128.163.232.18
     384}}}
     385
     386==== Check parameters of InstaGENI sliver objects ====
     387
     388Do the reported parameters (id, slice URN, sliver UUID, creator, created, expires, state) of each sliver look like we expect?:
     389{{{
     390>>> print "\n".join([s.id for s in slivers])
     391urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+211
     392urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2003
     393urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2204
     394urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2250
     395urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2336
     396urn:publicid:IDN+utah.geniracks.net+sliver+11075
     397urn:publicid:IDN+utah.geniracks.net+sliver+13957
     398urn:publicid:IDN+utah.geniracks.net+sliver+17299
     399urn:publicid:IDN+utah.geniracks.net+sliver+17303
     400urn:publicid:IDN+utah.geniracks.net+sliver+17319
     401urn:publicid:IDN+utah.geniracks.net+sliver+17471
     402urn:publicid:IDN+instageni.northwestern.edu+sliver+10
     403urn:publicid:IDN+instageni.northwestern.edu+sliver+109
     404urn:publicid:IDN+instageni.northwestern.edu+sliver+116
     405urn:publicid:IDN+instageni.northwestern.edu+sliver+183
     406urn:publicid:IDN+instageni.northwestern.edu+sliver+187
     407urn:publicid:IDN+lan.sdn.uky.edu+sliver+172
     408urn:publicid:IDN+lan.sdn.uky.edu+sliver+208
     409urn:publicid:IDN+lan.sdn.uky.edu+sliver+332
     410urn:publicid:IDN+lan.sdn.uky.edu+sliver+336
     411urn:publicid:IDN+lan.sdn.uky.edu+sliver+599
     412urn:publicid:IDN+lan.sdn.uky.edu+sliver+796
     413urn:publicid:IDN+lan.sdn.uky.edu+sliver+798
     414
     415>>> print "\n".join([s.slice.id for s in slivers])
     416urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon
     417urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16
     418urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15
     419urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptyl2pingdebug
     420urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+3sites-OF
     421urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755
     422urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon
     423urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16
     424urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15
     425urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+transgeo
     426urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptyl2pingdebug
     427urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755
     428urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon
     429urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+transgeo
     430urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15
     431urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16
     432urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755
     433urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon
     434urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs15
     435urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+jbs16
     436urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+magicwnd
     437urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-1439311
     438urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+acclne-1439312
     439
     440>>> print "\n".join([str(s.uuid) for s in slivers])
     44173a1c54c-56ae-11e2-bfa6-029e26f15299
     442c130641b-85ca-11e2-bfa6-029e26f15299
     443e701342e-876a-11e2-bfa6-029e26f15299
     4440e8ae133-8824-11e2-bfa6-029e26f15299
     445c9bd91c5-8ca8-11e2-921f-029e26f15299
     4463487385f-0981-11e2-8336-00009b6224df
     447f02cb7a8-1de1-11e2-8336-00009b6224df
     4487fe8c7f7-85c9-11e2-b374-00009b6224df
     44910dfc3e9-85ca-11e2-b374-00009b6224df
     4505a65fb87-85e9-11e2-b374-00009b6224df
     45139ce1f4c-8824-11e2-b374-00009b6224df
     45216b4a9ea-771a-11e2-bc42-02d6cd042885
     453f97c6dc4-7d12-11e2-bc43-02d6cd042885
     45474533ef6-7d19-11e2-bc43-02d6cd042885
     455cde625c7-8548-11e2-bc43-02d6cd042885
     456d0cbbbeb-854c-11e2-bc43-02d6cd042885
     4577aac9536-7d0a-11e2-9d0e-025e430ef5d2
     458a57d7b47-7d35-11e2-9d0e-025e430ef5d2
     459857bdc49-85c8-11e2-b51f-025e430ef5d2
     46018e38445-85c9-11e2-b51f-025e430ef5d2
     4617cb79944-8b5f-11e2-b51f-025e430ef5d2
     4623de2b862-8cb5-11e2-b51f-025e430ef5d2
     463560ae43f-8cb5-11e2-b51f-025e430ef5d2
     464
     465>>> print "\n".join([s.creator.id for s in slivers])
     466urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     467urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     468urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     469urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     470urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
     471urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     472urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     473urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     474urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     475urn:publicid:IDN+pgeni.gpolab.bbn.com+user+inki
     476urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     477urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     478urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     479urn:publicid:IDN+pgeni.gpolab.bbn.com+user+inki
     480urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     481urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     482urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     483urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty
     484urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     485urn:publicid:IDN+pgeni.gpolab.bbn.com+user+jbs
     486urn:publicid:IDN+pgeni.gpolab.bbn.com+user+inki
     487urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
     488urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
     489
     490>>> print "\n".join(["created: %s, expires: %s" % (s.created, s.expires) for s in slivers])
     491created: 2013-01-04 13:36:49, expires: 2013-03-22 12:50:41
     492created: 2013-03-05 12:27:49, expires: 2013-03-22 12:50:41
     493created: 2013-03-07 14:06:44, expires: 2013-03-22 12:50:41
     494created: 2013-03-08 12:12:07, expires: 2013-03-21 20:00:31
     495created: 2013-03-14 07:12:19, expires: 2013-03-22 12:50:41
     496created: 2012-09-28 09:28:58, expires: 2013-03-22 12:50:41
     497created: 2012-10-24 07:51:47, expires: 2013-03-22 12:50:41
     498created: 2013-03-05 12:18:51, expires: 2013-03-22 12:50:41
     499created: 2013-03-05 12:22:55, expires: 2013-03-22 12:50:41
     500created: 2013-03-05 16:06:54, expires: 2013-03-22 12:50:41
     501created: 2013-03-08 12:13:20, expires: 2013-03-21 20:00:31
     502created: 2013-02-14 19:47:56, expires: 2013-03-22 12:50:41
     503created: 2013-02-22 10:12:07, expires: 2013-03-22 12:50:41
     504created: 2013-02-22 10:58:32, expires: 2013-03-22 12:50:41
     505created: 2013-03-04 20:57:37, expires: 2013-03-22 12:50:41
     506created: 2013-03-04 21:26:19, expires: 2013-03-22 12:50:41
     507created: 2013-02-22 09:11:19, expires: 2013-03-22 12:50:41
     508created: 2013-02-22 14:20:19, expires: 2013-03-22 12:50:41
     509created: 2013-03-05 12:11:50, expires: 2013-03-22 12:50:41
     510created: 2013-03-05 12:15:58, expires: 2013-03-22 12:50:41
     511created: 2013-03-12 15:55:06, expires: 2013-03-22 12:50:41
     512created: 2013-03-14 08:41:28, expires: 2013-03-22 12:50:41
     513created: 2013-03-14 08:42:08, expires: 2013-03-22 12:50:41
     514
     515>>> print "\n".join([s.state for s in slivers])
     516Unknown
     517Unknown
     518Unknown
     519Unknown
     520Unknown
     521Unknown
     522Unknown
     523Unknown
     524Unknown
     525Unknown
     526Unknown
     527Unknown
     528Unknown
     529Unknown
     530Unknown
     531Unknown
     532Unknown
     533Unknown
     534Unknown
     535Unknown
     536Unknown
     537Unknown
     538Unknown
     539}}}
     540So, those expiration times look suspicious, and i checked with JBS about the eight slivers which are his, and he believes those reported times are in fact wrong.
     541
     542==== Check parameters of InstaGENI sliver/resource mapping objects ====
     543
     544Check whether each sliver has some resource_mappings (which we'd expect):
     545{{{
     546>>> print "\n".join([str(len(s.resource_mappings)) for s in slivers])
     5471
     5481
     5491
     5501
     5511
     5521
     5531
     5541
     5551
     5561
     5571
     5581
     5591
     5604
     5611
     5621
     5631
     5641
     5651
     5661
     5671
     5681
     5691
     570}}}
     571
     572Collect the resource_mapping objects so we can look at them:
     573{{{
     574mappings = []
     575for s in slivers:
     576  mappings.extend(s.resource_mappings)
     577}}}
     578
     579Do the reported parameters (type, sliver, resource) of each mapping look like we expect?:
     580{{{
     581>>> print "\n".join([str(m.type) for m in mappings])
     582None
     583None
     584None
     585None
     586None
     587None
     588None
     589None
     590None
     591None
     592None
     593None
     594None
     595None
     596None
     597None
     598None
     599None
     600None
     601None
     602None
     603None
     604None
     605None
     606None
     607None
     608
     609>>> print "\n".join([m.sliver.id for m in mappings])
     610urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+211
     611urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2003
     612urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2204
     613urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2250
     614urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2336
     615urn:publicid:IDN+utah.geniracks.net+sliver+11075
     616urn:publicid:IDN+utah.geniracks.net+sliver+13957
     617urn:publicid:IDN+utah.geniracks.net+sliver+17299
     618urn:publicid:IDN+utah.geniracks.net+sliver+17303
     619urn:publicid:IDN+utah.geniracks.net+sliver+17319
     620urn:publicid:IDN+utah.geniracks.net+sliver+17471
     621urn:publicid:IDN+instageni.northwestern.edu+sliver+10
     622urn:publicid:IDN+instageni.northwestern.edu+sliver+109
     623urn:publicid:IDN+instageni.northwestern.edu+sliver+116
     624urn:publicid:IDN+instageni.northwestern.edu+sliver+116
     625urn:publicid:IDN+instageni.northwestern.edu+sliver+116
     626urn:publicid:IDN+instageni.northwestern.edu+sliver+116
     627urn:publicid:IDN+instageni.northwestern.edu+sliver+183
     628urn:publicid:IDN+instageni.northwestern.edu+sliver+187
     629urn:publicid:IDN+lan.sdn.uky.edu+sliver+172
     630urn:publicid:IDN+lan.sdn.uky.edu+sliver+208
     631urn:publicid:IDN+lan.sdn.uky.edu+sliver+332
     632urn:publicid:IDN+lan.sdn.uky.edu+sliver+336
     633urn:publicid:IDN+lan.sdn.uky.edu+sliver+599
     634urn:publicid:IDN+lan.sdn.uky.edu+sliver+796
     635urn:publicid:IDN+lan.sdn.uky.edu+sliver+798
     636
     637>>> print "\n".join([m.resource.id for m in mappings])                         
     638urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1
     639urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1
     640urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2
     641urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2
     642urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1
     643urn:publicid:IDN+utah.geniracks.net+node+pc1
     644urn:publicid:IDN+utah.geniracks.net+node+pc1
     645urn:publicid:IDN+utah.geniracks.net+node+pc1
     646urn:publicid:IDN+utah.geniracks.net+node+pc5
     647urn:publicid:IDN+utah.geniracks.net+node+pc4
     648urn:publicid:IDN+utah.geniracks.net+node+pc5
     649urn:publicid:IDN+instageni.northwestern.edu+node+pc1
     650urn:publicid:IDN+instageni.northwestern.edu+node+pc1
     651urn:publicid:IDN+instageni.northwestern.edu+node+pc3
     652urn:publicid:IDN+instageni.northwestern.edu+node+pc2
     653urn:publicid:IDN+instageni.northwestern.edu+node+pc5
     654urn:publicid:IDN+instageni.northwestern.edu+node+pc4
     655urn:publicid:IDN+instageni.northwestern.edu+node+pc1
     656urn:publicid:IDN+instageni.northwestern.edu+node+pc1
     657urn:publicid:IDN+lan.sdn.uky.edu+node+pc1
     658urn:publicid:IDN+lan.sdn.uky.edu+node+pc2
     659urn:publicid:IDN+lan.sdn.uky.edu+node+pc1
     660urn:publicid:IDN+lan.sdn.uky.edu+node+pc2
     661urn:publicid:IDN+lan.sdn.uky.edu+node+pc5
     662urn:publicid:IDN+lan.sdn.uky.edu+node+pc1
     663urn:publicid:IDN+lan.sdn.uky.edu+node+pc1
     664}}}
     665
     666As a sanity-check, i'm interested in checking the type of each node which has more than one sliver mapped to it --- i expect those would all be vmservers:
     667 * First make sure sliver->resources match up well with resource->slivers:
     668{{{
     669resource_slivers = 0
     670sliver_resources = 0
     671for r in resources:
     672  resource_slivers += len(r.slivers)
     673
     674for s in slivers:
     675  sliver_resources += len(s.resources)
     676
     677>>> print resource_slivers
     67826
     679>>> print sliver_resources
     68026
     681}}}
     682 * Now do our sanity-check:
     683{{{
     684>>> for r in resources:
     685...   if len(r.slivers) > 1:
     686...     print r.type
     687...
     688vmserver
     689vmserver
     690vmserver
     691vmserver
     692vmserver
     693vmserver
     694vmserver
     695
     696>>> for r in resources:
     697...   if len(r.slivers) == 1:
     698...     print r.type
     699...
     700host
     701host
     702host
     703host
     704host
     705host
     706}}}
     707
     708==== Summary of issues detected on 2013-03-15 ====
     709
     710Things to discuss:
     711 * Need resource->state be "Unknown"?
     712 * Need sliver->state be "Unknown"?
     713 * Can we usefully report any VLAN information about interfaces?
     714 * Can we usefully report any non-control IPv4 address information about interfaces?
     715 * Can we report MAC address information about interfaces?
     716 * Should we have the emulab.net SA report slices, so that we can see IG slivers from emulab slices?
     717 * The reported sliver expiration times look suspicious and are probably wrong.
     718 * Resource mapping type should be something other than "None"
    103719
    104720== Step 2: verify AM reachability data ==