wiki:GeniNetworkStitchingConfirmationTestStatus/PSC

Version 8 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

Pittsburgh Super Computing Center Stitching Confirmation Tests

This pages capture the detailed test logs for each test defined in the New Site Stitching Confirmation Tests page. For the sites status see the New Site Stitching Confirmation Tests Status page.

EG-ST-1 New Site to GPO IG topology

The PSC site advertises the following stitching details for the connection to AL2S.

    <ns2:stitching lastUpdateTime="Mon Apr 03 10:15:51 EDT 2017">
        <ns2:aggregate id="urn:publicid:IDN+exogeni.net:pscNet+authority+am" url="https://psc-hn.exogeni.net:11443/orca/xmlrpc">
            <ns2:aggregatetype>orca</ns2:aggregatetype>
            <ns2:stitchingmode>tree</ns2:stitchingmode>
            <ns2:scheduledservices>true</ns2:scheduledservices>
            <ns2:negotiatedservices>false</ns2:negotiatedservices>
            <ns2:node id="urn:publicid:IDN+exogeni.net:pscNet+node+orca-transit-net-vlan">
                <ns2:port id="urn:publicid:IDN+exogeni.net:pscNet+stitchport+PSCNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet">
                    <ns2:capacity>1000000kbps</ns2:capacity>
                    <ns2:maximumReservableCapacity>1000000kbps</ns2:maximumReservableCapacity>
                    <ns2:link id="urn:publicid:IDN+exogeni.net:pscNet+interface+PSCNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet">
                        <ns2:remoteLinkId>urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.pitt.net.internet2.edu:et-4/0/0.0:psc-eg</ns2:remoteLinkId>
                        <ns2:trafficEngineeringMetric>10</ns2:trafficEngineeringMetric>
                        <ns2:capacity>1000000kbps</ns2:capacity>
                        <ns2:maximumReservableCapacity>1000000kbps</ns2:maximumReservableCapacity>
                        <ns2:switchingCapabilityDescriptor>
                            <ns2:switchingcapType>l2sc</ns2:switchingcapType>
                            <ns2:encodingType>ethernet</ns2:encodingType>
                            <ns2:switchingCapabilitySpecificInfo>
                                <ns2:switchingCapabilitySpecificInfo_L2sc>
                                    <ns2:capability>consumer</ns2:capability>
                                    <ns2:vlanRangeAvailability>3976-3990</ns2:vlanRangeAvailability>
                                    <ns2:vlanTranslation>false</ns2:vlanTranslation>
                                </ns2:switchingCapabilitySpecificInfo_L2sc>
                            </ns2:switchingCapabilitySpecificInfo>
                        </ns2:switchingCapabilityDescriptor>
                    </ns2:link>
                </ns2:port>
            </ns2:node>
        </ns2:aggregate>
    </ns2:stitching>

Experimenter may not need any of this data, but it is helpful to reference when trying to determine how many VLANs are delegated for stitching at the site or how much bandwidth can be requested.

Create a slice and then create the stitched slivers with the RSpec EG-ST-1.rspec:

$ stitcher createsliver EG-ST-1a -a gwu-eg EG-ST-1.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
09:20:11 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
09:20:11 INFO    : Reading slice EG-ST-1a credential...
09:20:12 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-1a expires on 2017-04-10 13:20:11 UTC
09:20:12 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
09:20:12 INFO    : Calling SCS...
09:20:12 INFO    : Multi-AM reservation will include resources from these aggregates:
09:20:12 INFO    : 	<Aggregate psc-eg>
09:20:12 INFO    : 	<Aggregate gpo-ig>
09:20:12 INFO    : 	<Aggregate al2s>
09:20:12 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
09:20:18 INFO    : ... Allocation at <Aggregate psc-eg> complete.
09:20:19 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
09:20:37 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
09:20:37 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
09:20:44 INFO    : ... Allocation at <Aggregate al2s> complete.
09:20:44 INFO    : All aggregates are complete.
09:20:44 INFO    : Your resources expire at 2017-04-09T13:10:12 (UTC). 
09:20:44 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-1a-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-1a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each PSC EG and GPO IG aggregate:

$ readyToLogin.py EG-ST-1a --useSliceAggregates
  ....
eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 32314  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc4.instageni.gpolab.bbn.com

Measurements

Iperf InstaGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.3 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.1 sec   590 MBytes  82.4 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   153 MBytes  21.4 Mbits/sec
[  7]  0.0-60.1 sec   137 MBytes  19.1 Mbits/sec
[  4]  0.0-60.1 sec   114 MBytes  16.0 Mbits/sec
[  6]  0.0-60.1 sec   141 MBytes  19.7 Mbits/sec
[  3]  0.0-60.1 sec   134 MBytes  18.7 Mbits/sec
[SUM]  0.0-60.1 sec   679 MBytes  94.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-60.1 sec  64.0 MBytes  8.93 Mbits/sec
[  3]  0.0-60.1 sec  66.0 MBytes  9.21 Mbits/sec
[  8]  0.0-60.1 sec  74.5 MBytes  10.4 Mbits/sec
[  9]  0.0-60.2 sec  69.9 MBytes  9.74 Mbits/sec
[ 10]  0.0-60.2 sec  70.4 MBytes  9.81 Mbits/sec
[  5]  0.0-60.2 sec  73.0 MBytes  10.2 Mbits/sec
[ 12]  0.0-60.2 sec  66.0 MBytes  9.19 Mbits/sec
[  6]  0.0-60.3 sec  76.9 MBytes  10.7 Mbits/sec
[  4]  0.0-60.3 sec  62.6 MBytes  8.71 Mbits/sec
[ 11]  0.0-60.3 sec  73.5 MBytes  10.2 Mbits/sec
[SUM]  0.0-60.3 sec   697 MBytes  96.9 Mbits/sec

Iperf InstaGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 224 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   687 MBytes  96.0 Mbits/sec   0.095 ms 23017/512793 (4.5%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from InstaGENI GPO VM to the ExoGENI PSC VM

60 packets transmitted, 60 received, 0% packet loss, time 59090ms
rtt min/avg/max/mdev = 21.982/22.323/23.660/0.327 ms

Iperf ExoGENI PSC VM to GPO InstaGENI VM (TCP) - TCP window size: 85.3 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   700 MBytes  97.9 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-60.0 sec   118 MBytes  16.5 Mbits/sec
[  6]  0.0-60.0 sec   159 MBytes  22.2 Mbits/sec
[  3]  0.0-60.1 sec   114 MBytes  15.9 Mbits/sec
[  5]  0.0-60.1 sec   183 MBytes  25.5 Mbits/sec
[  4]  0.0-60.3 sec   126 MBytes  17.6 Mbits/sec
[SUM]  0.0-60.3 sec   699 MBytes  97.2 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  9]  0.0-59.3 sec  57.9 MBytes  8.18 Mbits/sec
[  5]  0.0-59.4 sec  65.0 MBytes  9.18 Mbits/sec
[  3]  0.0-59.5 sec  75.0 MBytes  10.6 Mbits/sec
[  4]  0.0-59.5 sec  72.0 MBytes  10.1 Mbits/sec
[  8]  0.0-59.5 sec  67.8 MBytes  9.54 Mbits/sec
[  7]  0.0-59.6 sec  49.5 MBytes  6.97 Mbits/sec
[ 11]  0.0-59.6 sec  68.2 MBytes  9.61 Mbits/sec
[ 12]  0.0-59.6 sec  75.0 MBytes  10.6 Mbits/sec
[  6]  0.0-59.7 sec  48.5 MBytes  6.81 Mbits/sec
[ 10]  0.0-60.3 sec   118 MBytes  16.5 Mbits/sec
[SUM]  0.0-60.3 sec   697 MBytes  96.9 Mbits/sec

Iperf ExoGENI PSC VM to GPO InstaGENI VM (UDP) - UDP buffer size: 224 KByte (default)

[  3] Server Report:
[  3]  0.0-60.1 sec   685 MBytes  95.7 Mbits/sec   0.102 ms 24349/512820 (4.7%)
[  3]  0.0-60.1 sec  1 datagrams received out-of-order

Ping from ExoGENI PSC VM to GPO InstaGENI VM

6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 22.168/22.328/22.499/0.203 ms

EG-ST-4 New Site to GPO EG interoperability

Create a slice and then create the stitched slivers with the RSpec EG-ST-4.rspec, which specifies a 100 Mbps link capacity. This was done to get around the mismatch in link capacity units that exists between IG and EG.

$ stitcher createsliver EG-ST-4a -a gwu-eg EG-ST-4.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc --noExoSM
12:25:36 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
12:25:36 INFO    : Requested to use GENI stitching instead of ExoGENI stitching
12:25:36 INFO    : Reading slice EG-ST-4a credential...
12:25:36 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a expires on 2017-04-10 15:32:38 UTC
12:25:36 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
12:25:36 INFO    : Calling SCS...
12:25:37 INFO    : Multi-AM reservation will include resources from these aggregates:
12:25:37 INFO    : 	<Aggregate psc-eg>
12:25:37 INFO    : 	<Aggregate al2s>
12:25:37 INFO    : 	<Aggregate gpo-eg>
12:25:37 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:25:43 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:25:43 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:25:52 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:25:52 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:25:54 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}}
12:25:54 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.))
12:25:54 INFO    : Deleting existing reservations...
12:25:54 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:25:55 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:25:55 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:25:58 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:25:58 INFO    : Calling SCS for the 2nd time...
12:25:59 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:26:29 INFO    : Multi-AM reservation will include resources from these aggregates:
12:26:29 INFO    : 	<Aggregate psc-eg>
12:26:29 INFO    : 	<Aggregate al2s>
12:26:29 INFO    : 	<Aggregate gpo-eg>
12:26:29 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:26:33 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/4eb0e935-c956-43b0-9fb4-72fda9cf8514#/4399a4db-8954-4040-b853-e15c60e43a2f.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:26:33 INFO    : Got an error reserving resources in EG-ST-4a at <Aggregate psc-eg>
12:26:35 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices.
12:26:35 INFO    : Will put <Aggregate psc-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate psc-eg> 2nd time with new suggested VLANs
12:26:35 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:27:05 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:27:11 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:27:11 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:27:19 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:27:19 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:27:21 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}}
12:27:21 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.))
12:27:21 INFO    : Deleting existing reservations...
12:27:21 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:27:23 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:27:23 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:27:26 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:27:26 INFO    : Calling SCS for the 3rd time...
12:27:26 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:27:56 INFO    : Multi-AM reservation will include resources from these aggregates:
12:27:56 INFO    : 	<Aggregate psc-eg>
12:27:56 INFO    : 	<Aggregate al2s>
12:27:56 INFO    : 	<Aggregate gpo-eg>
12:27:56 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:28:01 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/e80f5e4e-89b3-4e90-8b74-f5fad2c6489f#/a77b441f-071c-4044-a328-24bb9a00eecf.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:28:01 INFO    : Got an error reserving resources in EG-ST-4a at <Aggregate psc-eg>
12:28:02 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices.
12:28:02 INFO    : Will put <Aggregate psc-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate psc-eg> 2nd time with new suggested VLANs
12:28:02 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:28:32 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:28:38 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:28:38 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:28:47 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:28:47 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:28:54 INFO    : ... Allocation at <Aggregate al2s> complete.
12:28:54 INFO    : All aggregates are complete.
12:28:54 INFO    : Your resources expire at 2017-04-09T16:18:32 (UTC). 
12:28:54 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-4a-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-4a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each PSC and GPO aggregate:

$ readyToLogin.py EG-ST-4a --useSliceAggregates

eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4

eg-gpo's geni_status is: ready 
User lnevers logs in to eg-gpo using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.12

Measurements

Iperf ExoGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.3 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   632 MBytes  88.3 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   153 MBytes  21.4 Mbits/sec
[  3]  0.0-60.0 sec   120 MBytes  16.8 Mbits/sec
[  4]  0.0-60.1 sec   110 MBytes  15.4 Mbits/sec
[  6]  0.0-60.1 sec   149 MBytes  20.8 Mbits/sec
[  7]  0.0-60.1 sec   160 MBytes  22.4 Mbits/sec
[SUM]  0.0-60.1 sec   694 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  8]  0.0-60.0 sec  74.5 MBytes  10.4 Mbits/sec
[  6]  0.0-60.0 sec  86.4 MBytes  12.1 Mbits/sec
[ 12]  0.0-60.1 sec  61.5 MBytes  8.58 Mbits/sec
[  4]  0.0-60.2 sec  66.8 MBytes  9.30 Mbits/sec
[  9]  0.0-60.2 sec  72.5 MBytes  10.1 Mbits/sec
[  3]  0.0-60.2 sec  58.6 MBytes  8.17 Mbits/sec
[  7]  0.0-60.5 sec  80.8 MBytes  11.2 Mbits/sec
[  5]  0.0-60.7 sec  50.6 MBytes  6.99 Mbits/sec
[ 10]  0.0-60.9 sec  68.1 MBytes  9.38 Mbits/sec
[ 11]  0.0-61.0 sec  76.4 MBytes  10.5 Mbits/sec
[SUM]  0.0-61.0 sec   696 MBytes  95.7 Mbits/sec

Iperf ExoGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 224 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.039 ms 12218/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI GPO VM to the ExoGENI PSC VM

60 packets transmitted, 60 received, 0% packet loss, time 59081ms
rtt min/avg/max/mdev = 21.876/22.283/23.353/0.311 ms

Iperf ExoGENI PSC VM to ExoGENI GPO VM (TCP) - TCP window size: 85.3 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   692 MBytes  96.7 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.0 sec   155 MBytes  21.6 Mbits/sec
[  5]  0.0-60.0 sec   122 MBytes  17.0 Mbits/sec
[  7]  0.0-60.1 sec   115 MBytes  16.1 Mbits/sec
[  6]  0.0-60.1 sec   157 MBytes  21.9 Mbits/sec
[  3]  0.0-60.1 sec   145 MBytes  20.2 Mbits/sec
[SUM]  0.0-60.1 sec   694 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.0 sec  93.1 MBytes  13.0 Mbits/sec
[  9]  0.0-60.0 sec  52.0 MBytes  7.27 Mbits/sec
[  3]  0.0-60.0 sec  91.4 MBytes  12.8 Mbits/sec
[ 12]  0.0-60.0 sec  49.8 MBytes  6.95 Mbits/sec
[  7]  0.0-60.1 sec  63.4 MBytes  8.85 Mbits/sec
[ 11]  0.0-60.1 sec  75.0 MBytes  10.5 Mbits/sec
[  8]  0.0-60.2 sec  70.6 MBytes  9.84 Mbits/sec
[  5]  0.0-60.4 sec  61.5 MBytes  8.54 Mbits/sec
[  6]  0.0-60.8 sec  68.4 MBytes  9.43 Mbits/sec
[ 10]  0.0-60.8 sec  70.6 MBytes  9.74 Mbits/sec
[SUM]  0.0-60.8 sec   696 MBytes  95.9 Mbits/sec

Iperf ExoGENI PSC VM to ExoGENI GPO VM (UDP) - UDP buffer size: 224 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.033 ms 12270/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI PSC VM to ExoGENI GPO VM

60 packets transmitted, 60 received, 0% packet loss, time 59052ms
rtt min/avg/max/mdev = 21.973/22.281/24.151/0.369 ms