Context Navigation

← Previous Version
View Latest Version
Next Version →

Version 1 (modified by lnevers@bbn.com, 8 years ago) (diff)
--

GENI Switch Consolidation Procedure
1. 0. Generate Tickets and check for conflicts with any upcoming ticketed …
2. 1. Find Current Stitching Configuration

GENI Switch Consolidation Procedure

This page defines the steps required to update stitching to handle PoP device consolidation that is taking place in Internet2 AL2S. This consolidation will replace existing AL2S Brocade devices with Juniper devices, and will converge the two distinct devices that currently provide L2 and L3 services into a single converged Juniper device in locations where AL2S services exist. These steps outline the actions required at the GENI rack, AL2S AM, and SCS servers to incorporate URN changes (due to port changes) resulting from the consolidation.

The steps show examples based on details from previous switch consolidation and its effect on GENI stitching sites connected to this switch.

0. Generate Tickets and check for conflicts with any upcoming ticketed GENI events

Make sure tickets are opened at GMOC for the events listing all affected GENI resources. Also make sure that GMOC generates corresponding requests to Internet2 Engineering (GRNOC). Tickets should notify operators and experimenters. Adam Williams will coordinate efforts for GMOC, but initial requests should go to the usual GMOC email for ticket requests. (This has already been done for NYC maintenance.)

Note that Internet2 schedules both an IP and an AL2S outage (usually on different days) for each PoP consolidation. The IP event has no related GENI URN work needed, and will simply result in the GENI resources being unreachable (because the entire device is disconnected). The GMOC should create tickets for both events, since they both have GENI impact, and the rack admins should see the tickets if they read their GENI operators email.

Internet2 won't change their schedule, but you should notify any conflicting events about the maintenance and work with them to avoid any impact as much as possible.

If the event goes longer than the scheduled tickets, be sure to email updates to the GMOC when you know that will happen, and every 2 hours thereafter. If the event will continues to the next day, say so in your last ticket update, and tell them when you'll check in again the next day. (You don't have to update in between).

If there are any significant problems during the event, be sure to escalate to Heidi Dempsey (hdempsey@bbn.com) while you work on them (in addition to noting them in the ticket).

1. Find Current Stitching Configuration

The aggregate advertisement includes a stitching section which defines how VLANs are to be connected and which VLANs are associated with that stitching endpoint. To determine the impact of a consolidation on stitching you must start by collecting the the AL2S advertisement:

   omni -a al2s listresources -o

Review the content of the stitching section in the output file rspec-al2s-internet2-edu.xml and see that there any site affected.

For example there were several stitching endpoints for sdn-sw.newy32aoa.net.internet2.edu. Here is the list from an AL2S Advertisement:

 <stitch:node id="urn:publicid:IDN+al2s.internet2.edu+node+sdn-sw.newy32aoa.net.internet2.edu">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth1/1">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth1/1:iminds">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth5/2">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-og">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-eg">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-ig">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:host-gpolab">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:umass-eg">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth7/2">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth7/2:nysernet-ig">

From the above we will only request the "<stitch:link id" transition, the "<stitch:port id" transitions are implicit. From the above list there are 6 aggregates (2 InstaGENI, 2 ExoGENI, 1 OpenGENI and 1 iMinds) and 1 fixed endpoint (host-gpolab.

Note: In Stitching a fixed endpoint is a resource that is not a GENI aggregate but still supports stitching. Fixed endpoint are statically configured in the SCS servers to capture stitching information and are generally set up for specific demonstrations. Some example fixed-endpoints are:

[[PageOutline(1-3)]]

= GENI Consolidation - New York =

This page defines the steps required to update stitching to handle PoP device consolidation that is taking place in Internet2 AL2S. This consolidation will replace existing AL2S Brocade devices with Juniper devices, and will converge the two distinct devices that currently provide L2 and L3 services into a single converged Juniper device in locations where AL2S services exist.  These steps outline the actions required at the GENI rack, AL2S AM, and SCS servers to incorporate URN changes (due to port changes) resulting from the consolidation.

The steps shown below detail how the AL2S sdn-sw.newy32aoa.net transition will affect the GENI stitching sites connected to this switch.

== 0. Generate Tickets and check for conflicts with any upcoming ticketed GENI events ==

Make sure tickets are opened at GMOC for the events listing all affected GENI resources.  Also make sure that GMOC generates corresponding requests to Internet2 Engineering (GRNOC).  Tickets should notify operators and experimenters.  Adam Williams will coordinate efforts for GMOC, but initial requests should go to the usual GMOC email for ticket requests.  (This has already been done for NYC maintenance.)

Note that Internet2 schedules both an IP and an AL2S outage (usually on different days) for each PoP consolidation.  The IP event has no related GENI URN work needed, and will simply result in the GENI resources being unreachable (because the entire device is disconnected).  The GMOC should create tickets for both events, since they both have GENI impact, and the rack admins should see the tickets if they read their GENI operators email.

Internet2 won't change their schedule, but you should notify any conflicting events about the maintenance and work with them to avoid any impact as much as possible.

If the event goes longer than the scheduled tickets, be sure to email updates to the GMOC when you know that will happen, and every 2 hours thereafter.  If the event will continues to the next day, say so in your last ticket update, and tell them when you'll check in again the next day.  (You don't have to update in between).

If there are any significant problems during the event, be sure to escalate to Heidi while you work on them (in addition to noting them in the ticket).

== 1. Find Current Stitching Configuration ==

The aggregate advertisement includes a ''stitching'' section which defines how VLANs are to be connected and which VLANs are associated with that stitching endpoint.

Get the AL2S advertisement:
{{{
   omni -a al2s listresources -o
}}} 
Review the content of the stitching section in the output file rspec-al2s-internet2-edu.xml and see that there are several stitching endpoints for sdn-sw.newy32aoa.net.internet2.edu.  Here is the list:
{{{
 <stitch:node id="urn:publicid:IDN+al2s.internet2.edu+node+sdn-sw.newy32aoa.net.internet2.edu">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth1/1">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth1/1:iminds">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth5/2">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-og">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-eg">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-ig">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:host-gpolab">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth5/2:umass-eg">
 <stitch:port id="urn:publicid:IDN+al2s.internet2.edu+stitchport+sdn-sw.newy32aoa.net.internet2.edu:eth7/2">
 <stitch:link id="urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.newy32aoa.net.internet2.edu:eth7/2:nysernet-ig">
}}}
Note, from the above we will only request the "<stitch:link id" transition, the "<stitch:port id" transitions are implicit.

NB. If a site is listed (such as "sdn-sw.tuls.net.internet2.edu") but there no racks attached and the new URN is different from the old (In this particular case, the new URN "sdn-sw.tuls.net.internet2.edu" was the same as the old, so no changes were necessary), only the following is necessary:

 a. Ask the GMOC to update the URN at the AL2S and the production SCS.
 b. Ask Xi to update the URN at the test SCS.

== 2. Define Stitching Configuration Changes ==

These are the consolidations to update AL2S NEWY, according to email from Eric Boyd of Internet2:

{{{
 Old Hostname: sdn-sw.newy32aoa.net.internet2.edu
 New Hostname: rtsw.newy32aoa.net.internet2.edu
        'Old Interface'                       'New Interface'
 100GigabitEthernet1/1   100GE                   et-3/1/0.0
 100GigabitEthernet1/2   100GE                   et-3/3/0.0
 100GigabitEthernet3/1   100GE                   et-7/1/0.0
 100GigabitEthernet5/2   100GE                   et-7/2/0.0 (INCORRECT)
 100GigabitEthernet5/2   100GE                   et-7/3/0.0 (according to AJ Ragusa)
 100GigabitEthernet7/1   100GE                   et-4/1/0.0
 100GigabitEthernet7/2   100GE                   et-4/3/0.0
 10GigabitEthernet15/1   10GE                    xe-3/0/0.0
 10GigabitEthernet15/4   10GE                    xe-3/0/1.0
 10GigabitEthernet15/5   10GE                    xe-3/0/2.0
 10GigabitEthernet15/7   10GE                    xe-3/0/3.0
}}}
Hence, there are seven sites impacted by this URN transition. The table below highlights each of the transitions:
||'''Old URN                                             || ''' New URN ''' ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth1/1:iminds      || rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-og      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-eg      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-ig      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:host-gpolab || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:host-gpolab ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:umass-eg    || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:umass-eg ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth7/2:nysernet-ig || rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig ||


== 3. Request Stitching Changes from GENI Aggregate Operations Teams ==

This URN transition will require co-ordination with the following teams: 
  * InstaGENI: (geni-ops@googlegroups.com) Request will be handled by Hussam Nasir (nasir@netlab.uky.edu)
  * ExoGENI: (exogeni-ops@renci.org) Request will be handled by Mert Cevik (mcevik@renci.org) or Ilya Baldin (ilbadin@renci.org)
  * OpenGENI: Marshall Brinn (Mbrinn@bbn.com) and Regina Rosales-Hain (rrhain@bbn.com)
  * AL2S Aggregate and Production SCS: GMOC requests(gmoc@grnoc.iu.edu). Updates handled by Luke Fowler (luke@grnoc.iu.edu) or AJ Ragusa (aragusa@grnoc.iu.edu), cc: both on initial request.
  * Test SCS - Xi Yang (xyang@maxgigapop.net) 
  * iMinds - Brecht Vermeulen (brecht.vermeulen@iminds.be>

=== 3a. Define Change Request Details ===

GPO IG and NYSERNet URNs Changes Request to InstaGENI Team:
{{{
Link ID:          urnpublicid:IDN+instageni.gpolab.bbn.com+interface+procurve2:5.24.al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig
VLAN Range:       3596-3600,3706-3732,3746-3749

Link ID:          urnpublicid:IDN+instageni.nysernet.org+interface+procurve2:1.19.al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig
VLAN Range:       1700-1719
}}}

GPO EG URNs Change Request for ExoGENI Team:
{{{
Link ID:          urnpublicid:IDN+exogeni.net:bbnNet+interface+BbnNet:IBM:G8052:GigabitEthernet:1:2:ethernet
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg
VLAN Range:       3741,3736-3739
}}}

GPO OG URNs Change Request for OpenGENI Team:
{{{
Link ID:          urnpublicid:IDN+bbn-cam-ctrl-1.gpolab.bbn.com+interface+force10:3:al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og
VLAN Range:       2611-2630
}}}

Wall2 iMinds URN Change Request to Imind Team:
{{{
Link ID:          urnpublicid:IDN+wall2.ilabt.iminds.be+interface+c300b:0.12
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds
VLAN Range:       1125-1164
}}}

AL2S Aggregate URN Change Request for GMOC/GRNOC:
{{{
Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds
Remote Link ID:   urnpublicid:IDN+wall2.ilabt.iminds.be+interface+c300b:0.12
VLAN Range:       1125-1164

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og
Remote Link ID:   urnpublicid:IDN+bbn-cam-ctrl-1.gpolab.bbn.com+interface+force10:3:al2s
VLAN Range:       2611-2630

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg
Remote Link ID:   urnpublicid:IDN+exogeni.net:bbnNet+interface+BbnNet:IBM:G8052:GigabitEthernet:1:2:ethernet
VLAN Range:       3741,3736-3739

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig
Remote Link ID:   urnpublicid:IDN+instageni.gpolab.bbn.com+interface+procurve2:5.24.al2s
VLAN Range:       3596-3600,3706-3732,3746-3749

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig
Remote Link ID:   urnpublicid:IDN+instageni.nysernet.org+interface+procurve2:1.19.al2s
VLAN Range:       1700-1719

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:host-gpolab
Remote Link ID:   urnpublicid:IDN+gpolab.bbn.com+interface+switch:port:al2s
VLAN Range:       2646

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:umass-eg
Remote Link ID:   urnpublicid:IDN+exogeni.net:umassNet+interface+umassNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet
VLAN Range:       3581-3595

}}}

=== 3b. Make Requests for Aggregate Updates ===

Send Email to each of the teams to request the above changes:
  - IG 
  - EG
  - OG
  - iMinds
  - Internet2

As a courtesy, copy the rack admin contact(s) or email list from the operators page on these requests.  They don't have to take any action, but they may want to know that their racks will be potentially unable to stitch for a period of time during the scheduled outage.

Also copy kathryn.wong1@uky.edu, cody@uky.edu and caylin@uky.edu, With the exception of the ATLA consolidation, this work should not require any immediate action for monitoring, but the folks at UKY may want to note the "retired" URNs in their database, and to pay extra attention to their monitoring site during these transitions.
 

''' ''Once the requested changes are completed, verify that the requested changes appear in the rack aggregate advertisement.''' ''
{{{
$ for i in gpo-ig gpo-og gpo-eg nysernet-ig al2s wall2 ; do stitcher listresources -a $i -o; done
}}}


You can now review all output files to verify that the correct URN is in place for each advertisement.

==== InstaGENI Update Alternatives ====

InstaGENI may be slow to respond, so you may do any of the following:
  1. Ask Hussam (nasir@netlab.uky.edu) to run the commands below on the boss node.
  2. Or contact the site admin and have them log into boss node and run these commands.
  3. You can create an admin account on the boss node (via the web UI for the site, e.g. http://instageni.gpolab.bbn.com/ for gpo-ig) and once it is approved, you can run the commands.
   
Below is an old example for the commands issued on the uwashington-ig boss node:
{{{
 mysql tbdb -e 'update external_networks set external_interface="urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.seat.net.internet2.edu:et-4/3/0.0:uwashington-ig" where network_id="al2s"'

 mysql tbdb -e 'update external_networks set external_wire="urn:publicid:IDN+al2s.internet2.edu+link+rtsw.seat.net.internet2.edu:et-4/3/0.0:uwashington-ig" where network_id="al2s"'

}}}

''Note: Be aware of potential line wrapping pitfalls.''
   

== 4. Request SCS Update ==

In order for GENI Network Stitching to pick up these path configuration changes, an SCS Update must be run.  There are two SCS systems:
  - Test SCS maintained by Xi Yang (xyang@maxgigapop.net) 
  - Production SCS mainted by the Internet2 (gmoc@grnoc.iu.edu). Updates handled by Luke Fowler (luke@grnoc.iu.edu) or AJ Ragusa (aragusa@grnoc.iu.edu).

To find out which SCS knows about which aggregates, issue the following:

For the Production SCS:
{{{
$ python ~/gcf/src/gcf/omnilib/stitch/scs.py --listaggregates --scs_url http://geni-scs.net.internet2.edu:8081/geni/xmlrpc >scs-prod
}}}
The above shows that the GPO IG, GPO EG, and NYSERnet IG sites are known to the Production SCS.

For the Test SCS:
{{{
python gcf/gcf-current/src/gcf/omnilib/stitch/scs.py --listaggregates --scs_url http://nutshell.maxgigapop.net:8081/geni/xmlrpc > scs-test
}}}
The above shows that the GPO IG, GPO EG, GPO OG, NYSERnet IG, iMinds, and Umass site are known to the Production SCS.


Both Production and Test SCS are impacted by the updates. Send a request to:
 - the GMOC to the Production SCS
 - to Xi to update the Test SCS.


== 5. Validate Updated Stitching ==

When the updates are completed for all Aggregates and for SCS servers, ask Luisa to conduct stitching tests.

== 6. Update and close all tickets ==

Assuming all tests are successful, update and close all tickets by emailing the GMOC.  If there are outstanding issues that are significant, leave the ticket open until they are resolved.  If there are smaller outstanding issues, close the maintenance tickets, and open new tickets with the appropriate owners to track and resolve, ideally before the next maintenance.




Note:  If a site is listed (such as "sdn-sw.tuls.net.internet2.edu") but there no racks attached and the new URN is different from the old (In this particular case, the new URN "sdn-sw.tuls.net.internet2.edu" was the same as the old, so no changes were necessary), only the following is necessary:

 a. Ask the GMOC to update the URN at the AL2S and the production SCS.
 b. Ask Xi to update the URN at the test SCS.

== 2. Define Stitching Configuration Changes ==

These are the consolidations to update AL2S NEWY, according to email from Eric Boyd of Internet2:

{{{
 Old Hostname: sdn-sw.newy32aoa.net.internet2.edu
 New Hostname: rtsw.newy32aoa.net.internet2.edu
        'Old Interface'                       'New Interface'
 100GigabitEthernet1/1   100GE                   et-3/1/0.0
 100GigabitEthernet1/2   100GE                   et-3/3/0.0
 100GigabitEthernet3/1   100GE                   et-7/1/0.0
 100GigabitEthernet5/2   100GE                   et-7/2/0.0 (INCORRECT)
 100GigabitEthernet5/2   100GE                   et-7/3/0.0 (according to AJ Ragusa)
 100GigabitEthernet7/1   100GE                   et-4/1/0.0
 100GigabitEthernet7/2   100GE                   et-4/3/0.0
 10GigabitEthernet15/1   10GE                    xe-3/0/0.0
 10GigabitEthernet15/4   10GE                    xe-3/0/1.0
 10GigabitEthernet15/5   10GE                    xe-3/0/2.0
 10GigabitEthernet15/7   10GE                    xe-3/0/3.0
}}}
Hence, there are seven sites impacted by this URN transition. The table below highlights each of the transitions:
||'''Old URN                                             || ''' New URN ''' ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth1/1:iminds      || rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-og      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-eg      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:gpo-ig      || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:host-gpolab || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:host-gpolab ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth5/2:umass-eg    || rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:umass-eg ||
|| sdn-sw.newy32aoa.net.internet2.edu:eth7/2:nysernet-ig || rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig ||


== 3. Request Stitching Changes from GENI Aggregate Operations Teams ==

This URN transition will require co-ordination with the following teams: 
  * InstaGENI: (geni-ops@googlegroups.com) Request will be handled by Hussam Nasir (nasir@netlab.uky.edu)
  * ExoGENI: (exogeni-ops@renci.org) Request will be handled by Mert Cevik (mcevik@renci.org) or Ilya Baldin (ilbadin@renci.org)
  * OpenGENI: Marshall Brinn (Mbrinn@bbn.com) and Regina Rosales-Hain (rrhain@bbn.com)
  * AL2S Aggregate and Production SCS: GMOC requests(gmoc@grnoc.iu.edu). Updates handled by Luke Fowler (luke@grnoc.iu.edu) or AJ Ragusa (aragusa@grnoc.iu.edu), cc: both on initial request.
  * Test SCS - Xi Yang (xyang@maxgigapop.net) 
  * iMinds - Brecht Vermeulen (brecht.vermeulen@iminds.be>

=== 3a. Define Change Request Details ===

GPO IG and NYSERNet URNs Changes Request to InstaGENI Team:
{{{
Link ID:          urnpublicid:IDN+instageni.gpolab.bbn.com+interface+procurve2:5.24.al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig
VLAN Range:       3596-3600,3706-3732,3746-3749

Link ID:          urnpublicid:IDN+instageni.nysernet.org+interface+procurve2:1.19.al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig
VLAN Range:       1700-1719
}}}

GPO EG URNs Change Request for ExoGENI Team:
{{{
Link ID:          urnpublicid:IDN+exogeni.net:bbnNet+interface+BbnNet:IBM:G8052:GigabitEthernet:1:2:ethernet
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg
VLAN Range:       3741,3736-3739
}}}

GPO OG URNs Change Request for OpenGENI Team:
{{{
Link ID:          urnpublicid:IDN+bbn-cam-ctrl-1.gpolab.bbn.com+interface+force10:3:al2s
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og
VLAN Range:       2611-2630
}}}

Wall2 iMinds URN Change Request to Imind Team:
{{{
Link ID:          urnpublicid:IDN+wall2.ilabt.iminds.be+interface+c300b:0.12
Remote Link ID:   urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds
VLAN Range:       1125-1164
}}}

AL2S Aggregate URN Change Request for GMOC/GRNOC:
{{{
Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-3/1/0.0:iminds
Remote Link ID:   urnpublicid:IDN+wall2.ilabt.iminds.be+interface+c300b:0.12
VLAN Range:       1125-1164

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-og
Remote Link ID:   urnpublicid:IDN+bbn-cam-ctrl-1.gpolab.bbn.com+interface+force10:3:al2s
VLAN Range:       2611-2630

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-eg
Remote Link ID:   urnpublicid:IDN+exogeni.net:bbnNet+interface+BbnNet:IBM:G8052:GigabitEthernet:1:2:ethernet
VLAN Range:       3741,3736-3739

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:gpo-ig
Remote Link ID:   urnpublicid:IDN+instageni.gpolab.bbn.com+interface+procurve2:5.24.al2s
VLAN Range:       3596-3600,3706-3732,3746-3749

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-4/3/0.0:nysernet-ig
Remote Link ID:   urnpublicid:IDN+instageni.nysernet.org+interface+procurve2:1.19.al2s
VLAN Range:       1700-1719

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:host-gpolab
Remote Link ID:   urnpublicid:IDN+gpolab.bbn.com+interface+switch:port:al2s
VLAN Range:       2646

Link ID:          urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.newy32aoa.net.internet2.edu:et-7/3/0.0:umass-eg
Remote Link ID:   urnpublicid:IDN+exogeni.net:umassNet+interface+umassNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet
VLAN Range:       3581-3595

}}}

=== 3b. Make Requests for Aggregate Updates ===

Send Email to each of the teams to request the above changes:
  - IG 
  - EG
  - OG
  - iMinds
  - Internet2

As a courtesy, copy the rack admin contact(s) or email list from the operators page on these requests.  They don't have to take any action, but they may want to know that their racks will be potentially unable to stitch for a period of time during the scheduled outage.

Also copy kathryn.wong1@uky.edu, cody@uky.edu and caylin@uky.edu, With the exception of the ATLA consolidation, this work should not require any immediate action for monitoring, but the folks at UKY may want to note the "retired" URNs in their database, and to pay extra attention to their monitoring site during these transitions.
 

''' ''Once the requested changes are completed, verify that the requested changes appear in the rack aggregate advertisement.''' ''
{{{
$ for i in gpo-ig gpo-og gpo-eg nysernet-ig al2s wall2 ; do stitcher listresources -a $i -o; done
}}}


You can now review all output files to verify that the correct URN is in place for each advertisement.

==== InstaGENI Update Alternatives ====

InstaGENI may be slow to respond, so you may do any of the following:
  1. Ask Hussam (nasir@netlab.uky.edu) to run the commands below on the boss node.
  2. Or contact the site admin and have them log into boss node and run these commands.
  3. You can create an admin account on the boss node (via the web UI for the site, e.g. http://instageni.gpolab.bbn.com/ for gpo-ig) and once it is approved, you can run the commands.
   
Below is an old example for the commands issued on the uwashington-ig boss node:
{{{
 mysql tbdb -e 'update external_networks set external_interface="urn:publicid:IDN+al2s.internet2.edu+interface+rtsw.seat.net.internet2.edu:et-4/3/0.0:uwashington-ig" where network_id="al2s"'

 mysql tbdb -e 'update external_networks set external_wire="urn:publicid:IDN+al2s.internet2.edu+link+rtsw.seat.net.internet2.edu:et-4/3/0.0:uwashington-ig" where network_id="al2s"'

}}}

''Note: Be aware of potential line wrapping pitfalls.''
   

== 4. Request SCS Update ==

In order for GENI Network Stitching to pick up these path configuration changes, an SCS Update must be run.  There are two SCS systems:
  - Test SCS maintained by Xi Yang (xyang@maxgigapop.net) 
  - Production SCS mainted by the Internet2 (gmoc@grnoc.iu.edu). Updates handled by Luke Fowler (luke@grnoc.iu.edu) or AJ Ragusa (aragusa@grnoc.iu.edu).

To find out which SCS knows about which aggregates, issue the following:

For the Production SCS:
{{{
$ python ~/gcf/src/gcf/omnilib/stitch/scs.py --listaggregates --scs_url http://geni-scs.net.internet2.edu:8081/geni/xmlrpc >scs-prod
}}}
The above shows that the GPO IG, GPO EG, and NYSERnet IG sites are known to the Production SCS.

For the Test SCS:
{{{
python gcf/gcf-current/src/gcf/omnilib/stitch/scs.py --listaggregates --scs_url http://nutshell.maxgigapop.net:8081/geni/xmlrpc > scs-test
}}}
The above shows that the GPO IG, GPO EG, GPO OG, NYSERnet IG, iMinds, and Umass site are known to the Production SCS.


Both Production and Test SCS are impacted by the updates. Send a request to:
 - the GMOC to the Production SCS
 - to Xi to update the Test SCS.


== 5. Validate Updated Stitching ==

When the updates are completed for all Aggregates and for SCS servers, ask Luisa to conduct stitching tests.

== 6. Update and close all tickets ==

Assuming all tests are successful, update and close all tickets by emailing the GMOC.  If there are outstanding issues that are significant, leave the ticket open until they are resolved.  If there are smaller outstanding issues, close the maintenance tickets, and open new tickets with the appropriate owners to track and resolve, ideally before the next maintenance.

Download in other formats:

Plain Text