15 | | Internet2 won't change their schedule, but you should notify any conflicting events about the maintenance and work with them to avoid any impact as much as possible. |
16 | | |
17 | | If the consolidation event goes longer than the scheduled tickets, be sure to email updates to the GMOC when you know that will happen, and every 2 hours thereafter. If the event will continues to the next day, say so in your last ticket update, and tell them when you'll check in again the next day. (You don't have to update in between). |
18 | | |
19 | | If there are any significant problems during the event, be sure to escalate to Heidi Dempsey (hdempsey@bbn.com) while you work on them (in addition to noting them in the ticket). |
| 16 | Note that GMOC should check the GENI calendar for any conflicting events that are scheduled to overlap, and send followup email if they find any. Rather than wait for this to happen,check the existing GENI tickets or calendar yourself for conflicts and notify any affected event coordinator via email directly. |
| 17 | |
| 18 | === Check the test SCS for affected sites and generate warning emails === |
| 19 | |
| 20 | The GMOC may not have records of GENI connections for some nodes that are only supported on the test SCS (e.g. CloudLab). If the scheduled maintenance will affect a test SCS node (rack, switch etc.), email the contact for that node directly and cc: gpo-infra, informing them of the scheduled outage, and asking them to be available to test connectivity after the update. Add the test SCS node owner contact to any status update emails you send during the outage. (Sometimes test SCS nodes are no longer in use, so the owner may indicate they can be retired instead of revised. Include retiring resources as part of the outage work.) |
| 21 | |
| 22 | === Changing the Schedule and Escalation === |
| 23 | |
| 24 | Internet2 won't change their schedule, but we can work with affected sites and event contacts to try to priortize the work to minimize the outage impact for priority sites if needed. |
| 25 | |
| 26 | If the consolidation event goes longer than the scheduled outage ticket lasts, be sure to email updates to the GMOC and to anyone who was contacted via email (from the test SCS or event lists) as soon as you know an extension is needed. Update the same list if you need to extend more than once. If the event will continue to the next day, indicate when work will start again on the next day in your update. You should send updates to the GMOC ticket for any significant events or changes that happen during the maintenance (version updates, bad hardware etc.). Do not send these type of updates only to an ops email list, because that info won't get out to resource owners. |
| 27 | |
| 28 | If there are any significant problems during the event, escalate to Heidi Dempsey (hdempsey@bbn.com) while you work on them (in addition to noting them in the ticket). |