wiki:GENIOperationsTrial/GENIExpSliceCheck

Version 24 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

CHK-004 GENI Long Running Slice Check

The GENI Long Running Slice uses 3 stitching sites in a triangle topology. The topology includes 1 compute resource at each of these sites: cenic-ig, uchicago-ig and nysernet-ig. All nodes are exchanging traffic continuously. This check verifies:

  • Slivers from all AL2S and Rack Aggregates
  • AL2S sliver VLAN and traffic information
  • Rack Aggregates traffic information

1.0 GENI Long Running Slice - Slivers Check

1.1 Goals of the Long Running - Slivers Check

This sections shows how to verify the sliver allocation for the triangle slice.

1.2 Steps for the Long Running Slice - Slivers Check

  1. Login to the GENI Monitoring System.
  2. Type "triangle" in the top left hand Search box. This will show results in the main panel.
  3. In the Slices listing select "ch.geni.net_ln-test_slice_triangle". This will show results in the main panel.
  4. Select the Slivers tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates).

1.3 Long Running Slice - Slivers Check - Pass Criteria

Step 4 should list these 4 slivers:

If the slivers are listed and the "Last Updated" field is less than 30 minutes this check passes.

1.4 Long Running Slice - Slivers Check - Fail Criteria and Escalation

If step 4 fails to list 4 slivers than escalation depends on the sliver missing:

Escalation: If site sliver missing - Report to InstaGENI team at geni-ops@googlegroups.com
Escalation: If AL2S sliver missing - Report to GMOC team - gmoc@grnoc.iu.edu
Escalation: If "Last Updated" field for any sliver is greater than 1 hour - Report to UKY Monitoring team at geni-ops@googlegroups.com

2.0 GENI Long Running Slice - AL2S Check

2.1 Goals of the Long Running Slice - AL2S Check

This sections shows how to verify the resources allocated to the triangle sliver in AL2S.

2.2 Steps for the Long Running Slice - AL2S Check

Assuming you are still logged into the GENI Monitoring System:

  1. Type "triangle" in the top left hand Search box. This will show results in the main panel.
  2. Select the sliver reported by the aggregate al2s.internet2.edu. This will show results in the main panel for all links associated to the sliver.
  3. The list of links should include 3 circuits: I2-LOSA-NEWY32AOA-VLAN-51145, I2-LOSA-STAR-VLAN-51146, I2-NEWY32AOA-STAR-VLAN-51144
  4. Select "I2-LOSA-NEWY32AOA-VLAN-51145". This will update the main panel to show two endpoints for the circuit: sdn-sw.losa.net.internet2.edu:eth5/1:1550 and sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714.
  5. Select "sdn-sw.losa.net.internet2.edu:eth5/1:1550". This will update the main panel to show traffic statistics for the VLAN 1550 at the Los Angeles router interface "eth5/1" where cenic-ig is sending:

  1. Navigate Back to the list of interfaces and select sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714. This will update the main panel to show traffic statistics for the VLAN 1714 at the New York router interface "eth5/1" and is being received by the cenic-ig router:

Blocking: Periodically the GENI Monitoring interface stops showing one of the circuit endpoints (see ticket 301)

2.3 Long Running Slice - AL2S Check - Pass Criteria

If step 5 and 6 show the expected level of traffic, then this procedures passes.

2.4 Long Running Slice - AL2S Check - Fail Criteria and Escalation

There are several steps where this procedure can fail to meet the expected criteria.

If step 3 fails to list the 3 expected circuits:

Escalation: If 1 or more circuits are missing - Report to GMOC team - gmoc@grnoc.iu.edu

If step 4 fails to list one of the two expected endpoints:

Escalation: - Report to GMOC team - gmoc@grnoc.iu.edu

If step 5 or step 6 are missing traffic statistics:

Escalation: If traffic statistics are missing - Report to GENI Operations team at geni-ops@googlegroups.com

3.0 GENI Long Running Slice - Sites Check

3.1 Goals of the Long Running Slice - Site Check

This sections shows how to verify the resources allocated to the triangle sliver in each of the 3 site aggregates: cenic-ig, uchicago-ig and nysernet-ig..

3.2 Steps for the Long Running Slice - Sites Check

Assuming you are still logged into the GENI Monitoring System:

  1. Type "triangle" in the top left hand Search box. This will show results in the main panel.
  2. In the Slices listing select "ch.geni.net_ln-test_slice_triangle". This will show results in the main panel.
  3. Select the Slivers tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates). From this list you can see that te slice has 3 nodes (1 at each site) and 3 links at AL2S, as shown below:

  1. You can select the each of the site slivers to determine which VM server or raw device the resource is allocated on. For example, selecting:
    • "geni.uchicago.edu_sliver_5003" shows that compute resource is on "geni.uchicago.edu_node_pcvm3-1"
    • "instageni.cenic.net_sliver_1266" shows that compute resource is on "instageni.cenic.net_node_pcvm2-2"
    • "instageni.nysernet.org_sliver_24968" shows that compute resource is on "instageni.nysernet.org_node_pcvm3-4"
  2. You can determine site usage statistics by using the VLANs IDs allocated to the slice. VLANs can be determined from the AL2S sliver details:
    • Selected AL2S sliver "urn:publicid:IDN+ch.geni.net:ln-test+slice+triangle:1bc9e85e-146c-4991-a4e9-96c5f3a5dee9"
    • Select each of the links that are part of the sliver to find VLANs allocated:
      • I2-LOSA-NEWY32AOA-VLAN-51145 includes sdn-sw.losa.net.internet2.edu:eth5/1:1550 and sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714
      • I2-LOSA-STAR-VLAN-51146 includes sdn-sw.losa.net.internet2.edu:eth5/1:1565 and sdn-sw.star.net.internet2.edu:eth7/1:3092
      • I2-NEWY32AOA-STAR-VLAN-51144 includes sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1715 and sdn-sw.star.net.internet2.edu:eth7/1:3076
  3. Assuming we are interested in the NYSERNet connections (sdn-sw.newy32aoa.net.internet2.edu), you can see from the previous step that there are two VLANs associated with the site (1714 and 1715).
  4. Select Dashborad->VLANs, this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID 1715 in the panel search to narrow the results to you sliver:

  1. Select the ID instageni.nysernet.org_interface_pc3:eth1:1715 to see usage statistics:

  1. For the seconds NYSERNet connection, select Dashborad->VLANs, this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID 1714 in the panel search to narrow the results to you sliver:

  1. Select the ID instageni.nysernet.org_interface_pc3:eth1:1714 to see usage statistics:

3.3 Long Running Slice - Sites Check - Pass Criteria

If step 8 and 10 show the expected level of traffic, then this procedures passes.

3.4 Long Running Slice - Sites Check - Fail Criteria and Escalation

There are several steps where this procedure can fail to meet the expected criteria.

If step 3 fails to list all 4 slivers

Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com

If step 5 fails to list the 3 expected links

Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com

If steps 8 and 10 fail to show usage statistics:

Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com

Attachments (9)