Changes between Version 4 and Version 5 of PlasticSlices/Outages


Ignore:
Timestamp:
06/15/11 17:05:37 (13 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • PlasticSlices/Outages

    v4 v5  
    2222|| Ticket #2399 || 05/27/11 16:22:37 || jbs@bbn.com: Stanford's FlowVisor has been down all day. ||
    2323|| Ticket #2399|| 05/31/11 18:10:47  || jbs@bbn.com:a bunch of stuff seems to have gone down about 30 seconds after I started running plastic-103||
     24
    2425== Baseline 4 Outages ==
    2526|| '''source'''|| '''time started'''  ||'''time resolved'''||  reason ||
    … …  
    2829== Baseline 5 Outages ==
    2930|| '''source'''|| '''time started'''  ||'''time resolved'''||  reason ||
    30 || Ticket #2373 ||
     31|| Ticket #2373 ||06/09/11 11:02:18 || jbs@bbn.com: plastic-102 Rutgers not working also Stanford -> Washington not working||
     32|| Ticket #2373 ||06/09/11 11:02:18 || jbs@bbn.com: plastic-103 Rutgers -> Stanford died again ||
     33|| Ticket #2373 ||06/09/11 11:02:18 || jbs@bbn.com: plastic-104 Indiana -> BBN died again ||
     34|| Ticket #2373 ||06/09/11 11:02:18 || jbs@bbn.com: plastic-108: Stanford -> Rutgers continues to fail ||
     35|| Ticket #2373 ||06/09/11 11:02:18 || jbs@bbn.com: plastic-110: Stanford -> Rutgers continues to not work at all||
    3136
    3237== Baseline 6 Outages ==
    3338|| '''source'''|| '''time started'''  ||'''time resolved'''||  reason ||
    34 || Ticket #2401||
     39|| Ticket #2401|| 06/10/11 18:17:51 || jbs@bbn.com: plastic-101 Indiana -> Rutgers is down. (general Internet2 outage ) (20:22:26 both back up) ||
     40|| Ticket #2401|| 06/10/11 18:17:51 || jbs@bbn.com: plastic-105 Wisconsin -> Clemson was stuck, after client& server restart ||
     41|| Ticket #2401|| 06/10/11 21:02:04 || jbs@bbn.com: plastic-103 orbitplc2.orbit-lab.org -> of-planet3.stanford.edu, but that hasn't worked in a while. ||
     42|| Ticket #2401|| 06/10/11 21:02:04 || jbs@bbn.com: plastic-105 Things that I hadn't stopped, were stuck, so I restarted them. ||
     43|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-103 Washington -> BBN was stuck; restarted the client. Stanford -> Washington. Rutgers -> Stanford still doesn't work ||
     44|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-105 BBN -> GT had died somewhere in here; I started up the client, and it's running fine now. ||
     45|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-107 Clemson -> Indiana was stuck; ||
     46|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-108 Rutgers -> Indiana had been retrying and was then stuck ||
     47|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-109 Every client was stuck ||
     48|| Ticket #2401|| 06/11/11 00:11:09 || jbs@bbn.com: plastic-110 Every client was stuck ||
     49|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-101 Indiana -> Rutgers, no pings flowing. At Rutgers, orbitplc2 rebooted + Indiana FV outage. ||
     50|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-103 Stanford -> BBN was stuck + Stanford -> Washington was stuck + Rutgers -> Stanford was down ||
     51|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-104 BBN -> Rutgers was down ||
     52|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-105 GT -> Wisconsin was down, the client at GT had died||
     53|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-106 Indiana -> GT isn't working ||
     54|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-108 Wisconsin -> Stanford was stuck; Stanford -> Rutgers was stuck; Rutgers -> Indiana is down; Indiana -> Wisconsin was stuck||
     55|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-109 Every connection was stuck ||
     56|| Ticket #2401|| 06/12/11 13:38:39 || jbs@bbn.com: plastic-110 Stanford -> Rutgers down; Stanford -> Rutgers down; Indiana -> GT down; Restart !FlowVisor on naxos. ||
     57
     58|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-101 Indiana -> Rutgers still has no pings flowing, because orbitplc2 still has no static ARP. ||
     59|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-102 Rutgers -> Stanford still has no pings flowing, because orbitplc2 still has no static ARP. ||
     60|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-103 Stanford -> BBN down; Stanford -> Washington down; Rutgers -> Stanford down ||
     61|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-105 GT -> Wisconsin down ||
     62|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-108 Wisconsin -> Stanford was stuck; Rutgers -> Indiana is down; Indiana -> Wisconsin was stuck ||
     63|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-109 Everyone *except* Wisconsin -> BBN was stuck ||
     64|| Ticket #2401|| 06/13/11 12:36:01  || jbs@bbn.com: plastic-110 Everyone was stuck or dead.||
     65|| Ticket #2401|| 06/13/11 20:01:41  || jbs@bbn.com: plastic-103 Wisconsin -> Stanford was stuck ||
     66|| Ticket #2401|| 06/13/11 20:01:41  || jbs@bbn.com: plastic-110 Washington -> Stanford was stuck||
     67|| Ticket #2401|| 06/14/11 11:39:26  || jbs@bbn.com: No traffic to or from Stanford  ||
     68|| Ticket #2401|| 06/14/11 12:01:20  || jbs@bbn.com: missed one of the NOX controllers, which caused the FV on 33106 to crash ||
     69|| Ticket #2401|| 06/14/11 14:03:01 || jbs@bbn.com: plastic-103 Everything was down ||
     70|| Ticket #2401|| 06/14/11 14:03:01 || jbs@bbn.com: plastic-105 Everything was down ||
     71|| Ticket #2401|| 06/14/11 14:03:01 || jbs@bbn.com: plastic-108 Wisconsin -> Stanford was down; Rutgers -> Indiana was down;||
     72|| Ticket #2401|| 06/14/11 14:03:01 || jbs@bbn.com: plastic-109 Everything was down except Wisconsin -> BBN; ||
     73|| Ticket #2401|| 06/14/11 14:03:01 || jbs@bbn.com: plastic-110 Everything was down except GT -> BBN; ||
     74|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: derailed by a problem with the NLR core and then the I2 core ||
     75|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-101 Rutgers -> Stanford and Stanford -> Washington are both down. ||
     76|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-102 Rutgers -> Stanford and Stanford -> Washington are both down. ||
     77|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-103 Stanford -> BBN was down; BBN -> Rutgers was down; Stanford -> Washington was down; and Rutgers -> Stanford ;||
     78|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-108 Wisconsin -> Stanford was stuck; Stanford -> Rutgers couldn't connect||
     79|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-109 Clemson -> Stanford was stuck; Stanford -> Wisconsin was stuck;||
     80|| Ticket #2401|| 06/15/11 15:20:05 || jbs@bbn.com: plastic-110 Washington -> Stanford was stuck; Stanford -> Rutgers was stuck;  Rutgers -> Indiana was stuck; Indiana -> GT was stuck; ||
     81
     82
    3583
    3684== Baseline 7 Outages ==