[[PageOutline(1-2)]] = CHK-004 GENI Long Running Slice Check = The GENI Long Running Slice uses 3 stitching sites in a `triangle` topology. The topology includes 1 compute resource at each of these sites: cenic-ig, uchicago-ig and nysernet-ig. All nodes are exchanging traffic continuously. This check verifies: - Slivers from all AL2S and Rack Aggregates - AL2S sliver VLAN and traffic information - Rack Aggregates traffic information = 1.0 GENI Long Running Slice - Slivers Check = == 1.1 Goals of the Long Running - Slivers Check == This sections shows how to verify the sliver allocation for the triangle slice. == 1.2 Steps for the Long Running Slice - Slivers Check == 1. Login to the [http://genimon.uky.edu/ GENI Monitoring System]. 2. Type `"triangle"` in the top left hand Search box. This will show results in the main panel. 3. In the `Slices` listing select `"ch.geni.net_ln-test_slice_triangle"`. This will show results in the main panel. 4. Select the `Slivers` tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates). == 1.3 Long Running Slice - Slivers Check - Pass Criteria == Step 4 should list these 4 slivers: [[Image(triangle-slivers.jpg)]] If the slivers are listed and the `"Last Updated"` field is less than 30 minutes this check passes. == 1.4 Long Running Slice - Slivers Check - Fail Criteria and Escalation == If step 4 fails to list 4 slivers than escalation depends on the sliver missing: '''__Escalation:__''' If site sliver missing - Report to InstaGENI team at geni-ops@googlegroups.com [[BR]] '''__Escalation:__''' If AL2S sliver missing - Report to GMOC team - gmoc@grnoc.iu.edu [[BR]] '''__Escalation:__''' If `"Last Updated"` field for any sliver is greater than 1 hour - Report to UKY Monitoring team at geni-ops@googlegroups.com [[BR]] = 2.0 GENI Long Running Slice - AL2S Check = == 2.1 Goals of the Long Running Slice - AL2S Check == This sections shows how to verify the resources allocated to the triangle sliver in AL2S. == 2.2 Steps for the Long Running Slice - AL2S Check == Assuming you are still logged into the [http://genimon.uky.edu/ GENI Monitoring System]: 1. Type `"triangle"` in the top left hand Search box. This will show results in the main panel. 2. Select the sliver reported by the aggregate `al2s.internet2.edu`. This will show results in the main panel for all links associated to the sliver. 3. The list of links should include 3 circuits: `I2-LOSA-NEWY32AOA-VLAN-51145`, `I2-LOSA-STAR-VLAN-51146`, `I2-NEWY32AOA-STAR-VLAN-51144` 4. Select `"I2-LOSA-NEWY32AOA-VLAN-51145"`. This will update the main panel to show ''two endpoints'' for the circuit: `sdn-sw.losa.net.internet2.edu:eth5/1:1550` and `sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714`. 5. Select `"sdn-sw.losa.net.internet2.edu:eth5/1:1550"`. This will update the main panel to show traffic statistics for the VLAN 1550 at the Los Angeles router interface "eth5/1" where cenic-ig is sending: [[Image(cenic-transmit.jpg)]] 6. Navigate `Back` to the list of interfaces and select `sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714`. This will update the main panel to show traffic statistics for the VLAN 1714 at the New York router interface "eth5/1" and is being received by the cenic-ig router: [[Image(cenic-receive.jpg)]] '''Blocking''': Periodically the GENI Monitoring interface stops showing one of the circuit endpoints (see ticket [http://trac.gpolab.bbn.com/ops-monitoring/ticket/301 301]) == 2.3 Long Running Slice - AL2S Check - Pass Criteria == If step 5 and 6 show the expected level of traffic, then this procedures passes. == 2.4 Long Running Slice - AL2S Check - Fail Criteria and Escalation == There are several steps where this procedure can fail to meet the expected criteria. If step 3 fails to list the 3 expected circuits: '''__Escalation:__''' If 1 or more circuits are missing - Report to GMOC team - gmoc@grnoc.iu.edu [[BR]] If step 4 fails to list one of the two expected endpoints: '''__Escalation:__''' - Report to GMOC team - gmoc@grnoc.iu.edu [[BR]] If step 5 or step 6 are missing traffic statistics: '''__Escalation:__''' If traffic statistics are missing - Report to GENI Operations team at geni-ops@googlegroups.com [[BR]] = 3.0 GENI Long Running Slice - Sites Check = == 3.1 Goals of the Long Running Slice - Site Check == This sections shows how to verify the resources allocated to the triangle sliver in each of the 3 site aggregates: cenic-ig, uchicago-ig and nysernet-ig.. == 3.2 Steps for the Long Running Slice - Sites Check == Assuming you are still logged into the [http://genimon.uky.edu/ GENI Monitoring System]: 1. Type `"triangle"` in the top left hand Search box. This will show results in the main panel. 2. In the `Slices` listing select `"ch.geni.net_ln-test_slice_triangle"`. This will show results in the main panel. 3. Select the `Slivers` tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates). From this list you can see that te slice has 3 nodes (1 at each site) and 3 links at AL2S, as shown below: [[Image(triangle-sliver-details.jpg, 80%)]] 4. You can select the each of the site slivers to determine which VM server or raw device the resource is allocated on. For example, selecting: - `"geni.uchicago.edu_sliver_5003"` shows that compute resource is on `"geni.uchicago.edu_node_pcvm3-1"` - `"instageni.cenic.net_sliver_1266"` shows that compute resource is on `"instageni.cenic.net_node_pcvm2-2"` - `"instageni.nysernet.org_sliver_24968"` shows that compute resource is on `"instageni.nysernet.org_node_pcvm3-4"` 5. You can determine site usage statistics by using the VLANs IDs allocated to the slice. VLANs can be determined from the AL2S sliver details: - Selected AL2S sliver `"urn:publicid:IDN+ch.geni.net:ln-test+slice+triangle:1bc9e85e-146c-4991-a4e9-96c5f3a5dee9"` - Select each of the links that are part of the sliver to find VLANs allocated: * I2-LOSA-NEWY32AOA-VLAN-51145 includes `sdn-sw.losa.net.internet2.edu:eth5/1:1550` and `sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714` * I2-LOSA-STAR-VLAN-51146 includes `sdn-sw.losa.net.internet2.edu:eth5/1:1565` and `sdn-sw.star.net.internet2.edu:eth7/1:3092` * I2-NEWY32AOA-STAR-VLAN-51144 includes `sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1715` and `sdn-sw.star.net.internet2.edu:eth7/1:3076` NOTE: Steps 3, 4 and 5 provide information to determine the topology for the slice. Following is a diagram of the details reported from step 3, 4 and 5: [[Image(Triangle-details.jpg,60%)]] 6. Assuming we are interested in the NYSERNet connections (sdn-sw.newy32aoa.net.internet2.edu), you can see from the previous step that there are two VLANs associated with the site (1714 and 1715). 7. Select '''Dashborad->VLANs''', this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID `1715` in the panel search to narrow the results to you sliver: [[Image(nysernet-1715.jpg)]] 8. Select the ID `instageni.nysernet.org_interface_pc3:eth1:1715` to see usage statistics: [[Image(nysernet-1715-usage.jpg, 80%)]] 9. For the seconds NYSERNet connection, select '''Dashborad->VLANs''', this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID `1714` in the panel search to narrow the results to you sliver: [[Image(nysernet-1714.jpg, 80%)]] 10. Select the ID `instageni.nysernet.org_interface_pc3:eth1:1714` to see usage statistics: [[Image(nysernet-1714-usage.jpg, 80%)]] == 3.3 Long Running Slice - Sites Check - Pass Criteria == If step 8 and 10 show the expected level of traffic, then this procedures passes. == 3.4 Long Running Slice - Sites Check - Fail Criteria and Escalation == There are several steps where this procedure can fail to meet the expected criteria. If step 3 fails to list all 4 slivers '''__Escalation:__''' - Report to InstaGENI team at geni-ops@googlegroups.com [[BR]] If step 5 fails to list the 3 expected links '''__Escalation:__''' - Report to InstaGENI team at geni-ops@googlegroups.com [[BR]] If steps 8 and 10 fail to show usage statistics: '''__Escalation:__''' - Report to InstaGENI team at geni-ops@googlegroups.com [[BR]]