Version 23 (modified by 8 years ago) (diff) | ,
---|
CHK-004 GENI Long Running Slice Check
The GENI Long Running Slice uses 3 stitching sites in a triangle
topology. The topology includes 1 compute resource at each of these sites: cenic-ig, uchicago-ig and nysernet-ig. All nodes are exchanging traffic continuously. This check verifies:
- Slivers from all AL2S and Rack Aggregates
- AL2S sliver VLAN and traffic information
- Rack Aggregates traffic information
1.0 GENI Long Running Slice - Slivers Check
1.1 Goals of the Long Running - Slivers Check
This sections shows how to verify the sliver allocation for the triangle slice.
1.2 Steps for the Long Running Slice - Slivers Check
- Login to the GENI Monitoring System.
- Type
"triangle"
in the top left hand Search box. This will show results in the main panel. - In the
Slices
listing select"ch.geni.net_ln-test_slice_triangle"
. This will show results in the main panel. - Select the
Slivers
tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates).
1.3 Long Running Slice - Slivers Check - Pass Criteria
Step 4 should list these 4 slivers:
If the slivers are listed and the "Last Updated"
field is less than 30 minutes this check passes.
1.4 Long Running Slice - Slivers Check - Fail Criteria and Escalation
If step 4 fails to list 4 slivers than escalation depends on the sliver missing:
Escalation: If site sliver missing - Report to InstaGENI team at geni-ops@googlegroups.com
Escalation: If AL2S sliver missing - Report to GMOC team - gmoc@grnoc.iu.edu
Escalation: If"Last Updated"
field for any sliver is greater than 1 hour - Report to UKY Monitoring team at geni-ops@googlegroups.com
2.0 GENI Long Running Slice - AL2S Check
2.1 Goals of the Long Running Slice - AL2S Check
This sections shows how to verify the resources allocated to the triangle sliver in AL2S.
2.2 Steps for the Long Running Slice - AL2S Check
Assuming you are still logged into the GENI Monitoring System:
- Type
"triangle"
in the top left hand Search box. This will show results in the main panel. - Select the sliver reported by the aggregate
al2s.internet2.edu
. This will show results in the main panel for all links associated to the sliver. - The list of links should include 3 circuits:
I2-LOSA-NEWY32AOA-VLAN-51145
,I2-LOSA-STAR-VLAN-51146
,I2-NEWY32AOA-STAR-VLAN-51144
- Select
"I2-LOSA-NEWY32AOA-VLAN-51145"
. This will update the main panel to show two endpoints for the circuit:sdn-sw.losa.net.internet2.edu:eth5/1:1550
andsdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714
. - Select
"sdn-sw.losa.net.internet2.edu:eth5/1:1550"
. This will update the main panel to show traffic statistics for the VLAN 1550 at the Los Angeles router interface "eth5/1" where cenic-ig is sending:
- Navigate
Back
to the list of interfaces and selectsdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714
. This will update the main panel to show traffic statistics for the VLAN 1714 at the New York router interface "eth5/1" and is being received by the cenic-ig router:
Blocking: Periodically the GENI Monitoring interface stops showing one of the circuit endpoints (see ticket 301)
2.3 Long Running Slice - AL2S Check - Pass Criteria
If step 5 and 6 show the expected level of traffic, then this procedures passes.
2.4 Long Running Slice - AL2S Check - Fail Criteria and Escalation
There are several steps where this procedure can fail to meet the expected criteria.
If step 3 fails to list the 3 expected circuits:
Escalation: If 1 or more circuits are missing - Report to GMOC team - gmoc@grnoc.iu.edu
If step 4 fails to list one of the two expected endpoints:
Escalation: - Report to GMOC team - gmoc@grnoc.iu.edu
If step 5 or step 6 are missing traffic statistics:
Escalation: If traffic statistics are missing - Report to GENI Operations team at geni-ops@googlegroups.com
3.0 GENI Long Running Slice - Sites Check
3.1 Goals of the Long Running Slice - Site Check
This sections shows how to verify the resources allocated to the triangle sliver in each of the 3 site aggregates: cenic-ig, uchicago-ig and nysernet-ig..
3.2 Steps for the Long Running Slice - Sites Check
Assuming you are still logged into the GENI Monitoring System:
- Type
"triangle"
in the top left hand Search box. This will show results in the main panel. - In the
Slices
listing select"ch.geni.net_ln-test_slice_triangle"
. This will show results in the main panel. - Select the
Slivers
tab. This will results in a lists of 4 slivers (1 from AL2S and 3 from Rack Aggregates). From this list you can see that te slice has 3 nodes (1 at each site) and 3 links at AL2S, as shown below:
- You can select the each of the site slivers to determine which VM server or raw device the resource is allocated on. For example, selecting:
"geni.uchicago.edu_sliver_5003"
shows that compute resource is on"geni.uchicago.edu_node_pcvm3-1"
"instageni.cenic.net_sliver_1266"
shows that compute resource is on"instageni.cenic.net_node_pcvm2-2"
"instageni.nysernet.org_sliver_24968"
shows that compute resource is on"instageni.nysernet.org_node_pcvm3-4"
- You can determine site usage statistics by using the VLANs IDs allocated to the slice. VLANs can be determined from the AL2S sliver details:
- Selected AL2S sliver
"urn:publicid:IDN+ch.geni.net:ln-test+slice+triangle:1bc9e85e-146c-4991-a4e9-96c5f3a5dee9"
- Select each of the links that are part of the sliver to find VLANs allocated:
- I2-LOSA-NEWY32AOA-VLAN-51145 includes
sdn-sw.losa.net.internet2.edu:eth5/1:1550
andsdn-sw.newy32aoa.net.internet2.edu:eth7/2:1714
- I2-LOSA-STAR-VLAN-51146 includes
sdn-sw.losa.net.internet2.edu:eth5/1:1565
andsdn-sw.star.net.internet2.edu:eth7/1:3092
- I2-NEWY32AOA-STAR-VLAN-51144 includes
sdn-sw.newy32aoa.net.internet2.edu:eth7/2:1715
andsdn-sw.star.net.internet2.edu:eth7/1:3076
- I2-LOSA-NEWY32AOA-VLAN-51145 includes
- Selected AL2S sliver
- Assuming we are interested in the NYSERNet connections (sdn-sw.newy32aoa.net.internet2.edu), you can see from the previous step that there are two VLANs associated with the site (1714 and 1715).
- Select Dashborad->VLANs, this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID
1715
in the panel search to narrow the results to you sliver:
- Select the ID
instageni.nysernet.org_interface_pc3:eth1:1715
to see usage statistics:
- For the seconds NYSERNet connection, select Dashborad->VLANs, this will show the full list of VLAN in use in GENI. In the resulting VLAN listing type the VLAN ID
1714
in the panel search to narrow the results to you sliver:
- Select the ID
instageni.nysernet.org_interface_pc3:eth1:1714
to see usage statistics:
3.3 Long Running Slice - Sites Check - Pass Criteria
If step 8 and 10 show the expected level of traffic, then this procedures passes.
3.4 Long Running Slice - Sites Check - Fail Criteria and Escalation
There are several steps where this procedure can fail to meet the expected criteria.
If step X fails to list ???:
Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com
If step Y fails to list ???:
Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com
If step Z fails to list ???:
Escalation: - Report to InstaGENI team at geni-ops@googlegroups.com
Blocking: IG sliver details are not handled properly by the GENI Monitoring system (see ticket 300).
Attachments (9)
- triangle-slivers.jpg (104.3 KB) - added by 9 years ago.
- cenic-receive.jpg (359.6 KB) - added by 9 years ago.
- cenic-transmit.jpg (359.9 KB) - added by 9 years ago.
- triangle-sliver-details.jpg (160.1 KB) - added by 8 years ago.
- nysernet-1715.jpg (139.1 KB) - added by 8 years ago.
- nysernet-1714.jpg (131.8 KB) - added by 8 years ago.
- nysernet-1715-usage.jpg (809.9 KB) - added by 8 years ago.
- nysernet-1714-usage.jpg (826.9 KB) - added by 8 years ago.
- Triangle-details.jpg (80.4 KB) - added by 8 years ago.