wiki:GeniNetworkStitchingConfirmationTestStatus/PSC

Version 15 (modified by lnevers@bbn.com, 7 years ago) (diff)

--

Pittsburgh Supercomputing Center Stitching Confirmation Tests

This pages capture the detailed test logs for each test defined in the New Site Stitching Confirmation Tests page. For the sites status see the New Site Stitching Confirmation Tests Status page.

EG-ST-1 New Site to GPO IG topology

The PSC site advertises the following stitching details for the connection to AL2S.

    <ns2:stitching lastUpdateTime="Mon Apr 03 10:15:51 EDT 2017">
        <ns2:aggregate id="urn:publicid:IDN+exogeni.net:pscNet+authority+am" url="https://psc-hn.exogeni.net:11443/orca/xmlrpc">
            <ns2:aggregatetype>orca</ns2:aggregatetype>
            <ns2:stitchingmode>tree</ns2:stitchingmode>
            <ns2:scheduledservices>true</ns2:scheduledservices>
            <ns2:negotiatedservices>false</ns2:negotiatedservices>
            <ns2:node id="urn:publicid:IDN+exogeni.net:pscNet+node+orca-transit-net-vlan">
                <ns2:port id="urn:publicid:IDN+exogeni.net:pscNet+stitchport+PSCNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet">
                    <ns2:capacity>1000000kbps</ns2:capacity>
                    <ns2:maximumReservableCapacity>1000000kbps</ns2:maximumReservableCapacity>
                    <ns2:link id="urn:publicid:IDN+exogeni.net:pscNet+interface+PSCNet:IBM:G8264:TenGigabitEthernet:1:1:ethernet">
                        <ns2:remoteLinkId>urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.pitt.net.internet2.edu:et-4/0/0.0:psc-eg</ns2:remoteLinkId>
                        <ns2:trafficEngineeringMetric>10</ns2:trafficEngineeringMetric>
                        <ns2:capacity>1000000kbps</ns2:capacity>
                        <ns2:maximumReservableCapacity>1000000kbps</ns2:maximumReservableCapacity>
                        <ns2:switchingCapabilityDescriptor>
                            <ns2:switchingcapType>l2sc</ns2:switchingcapType>
                            <ns2:encodingType>ethernet</ns2:encodingType>
                            <ns2:switchingCapabilitySpecificInfo>
                                <ns2:switchingCapabilitySpecificInfo_L2sc>
                                    <ns2:capability>consumer</ns2:capability>
                                    <ns2:vlanRangeAvailability>3976-3990</ns2:vlanRangeAvailability>
                                    <ns2:vlanTranslation>false</ns2:vlanTranslation>
                                </ns2:switchingCapabilitySpecificInfo_L2sc>
                            </ns2:switchingCapabilitySpecificInfo>
                        </ns2:switchingCapabilityDescriptor>
                    </ns2:link>
                </ns2:port>
            </ns2:node>
        </ns2:aggregate>
    </ns2:stitching>

Experimenter may not need any of this data, but it is helpful to reference when trying to determine how many VLANs are delegated for stitching at the site or how much bandwidth can be requested.

Create a slice and then create the stitched slivers with the RSpec EG-ST-1.rspec:

$ stitcher createsliver EG-ST-1a -a gwu-eg EG-ST-1.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
09:20:11 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
09:20:11 INFO    : Reading slice EG-ST-1a credential...
09:20:12 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-1a expires on 2017-04-10 13:20:11 UTC
09:20:12 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
09:20:12 INFO    : Calling SCS...
09:20:12 INFO    : Multi-AM reservation will include resources from these aggregates:
09:20:12 INFO    : 	<Aggregate psc-eg>
09:20:12 INFO    : 	<Aggregate gpo-ig>
09:20:12 INFO    : 	<Aggregate al2s>
09:20:12 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
09:20:18 INFO    : ... Allocation at <Aggregate psc-eg> complete.
09:20:19 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
09:20:37 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
09:20:37 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
09:20:44 INFO    : ... Allocation at <Aggregate al2s> complete.
09:20:44 INFO    : All aggregates are complete.
09:20:44 INFO    : Your resources expire at 2017-04-09T13:10:12 (UTC). 
09:20:44 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-1a-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-1a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each PSC EG and GPO IG aggregate:

$ readyToLogin.py EG-ST-1a --useSliceAggregates
  ....
eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 32314  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc4.instageni.gpolab.bbn.com

Measurements

Iperf InstaGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.1 sec   590 MBytes  82.4 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   153 MBytes  21.4 Mbits/sec
[  7]  0.0-60.1 sec   137 MBytes  19.1 Mbits/sec
[  4]  0.0-60.1 sec   114 MBytes  16.0 Mbits/sec
[  6]  0.0-60.1 sec   141 MBytes  19.7 Mbits/sec
[  3]  0.0-60.1 sec   134 MBytes  18.7 Mbits/sec
[SUM]  0.0-60.1 sec   679 MBytes  94.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-60.1 sec  64.0 MBytes  8.93 Mbits/sec
[  3]  0.0-60.1 sec  66.0 MBytes  9.21 Mbits/sec
[  8]  0.0-60.1 sec  74.5 MBytes  10.4 Mbits/sec
[  9]  0.0-60.2 sec  69.9 MBytes  9.74 Mbits/sec
[ 10]  0.0-60.2 sec  70.4 MBytes  9.81 Mbits/sec
[  5]  0.0-60.2 sec  73.0 MBytes  10.2 Mbits/sec
[ 12]  0.0-60.2 sec  66.0 MBytes  9.19 Mbits/sec
[  6]  0.0-60.3 sec  76.9 MBytes  10.7 Mbits/sec
[  4]  0.0-60.3 sec  62.6 MBytes  8.71 Mbits/sec
[ 11]  0.0-60.3 sec  73.5 MBytes  10.2 Mbits/sec
[SUM]  0.0-60.3 sec   697 MBytes  96.9 Mbits/sec

Iperf InstaGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   687 MBytes  96.0 Mbits/sec   0.095 ms 23017/512793 (4.5%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from InstaGENI GPO VM to the ExoGENI PSC VM

60 packets transmitted, 60 received, 0% packet loss, time 59090ms
rtt min/avg/max/mdev = 21.982/22.323/23.660/0.327 ms

Iperf ExoGENI PSC VM to GPO InstaGENI VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   700 MBytes  97.9 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-60.0 sec   118 MBytes  16.5 Mbits/sec
[  6]  0.0-60.0 sec   159 MBytes  22.2 Mbits/sec
[  3]  0.0-60.1 sec   114 MBytes  15.9 Mbits/sec
[  5]  0.0-60.1 sec   183 MBytes  25.5 Mbits/sec
[  4]  0.0-60.3 sec   126 MBytes  17.6 Mbits/sec
[SUM]  0.0-60.3 sec   699 MBytes  97.2 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  9]  0.0-59.3 sec  57.9 MBytes  8.18 Mbits/sec
[  5]  0.0-59.4 sec  65.0 MBytes  9.18 Mbits/sec
[  3]  0.0-59.5 sec  75.0 MBytes  10.6 Mbits/sec
[  4]  0.0-59.5 sec  72.0 MBytes  10.1 Mbits/sec
[  8]  0.0-59.5 sec  67.8 MBytes  9.54 Mbits/sec
[  7]  0.0-59.6 sec  49.5 MBytes  6.97 Mbits/sec
[ 11]  0.0-59.6 sec  68.2 MBytes  9.61 Mbits/sec
[ 12]  0.0-59.6 sec  75.0 MBytes  10.6 Mbits/sec
[  6]  0.0-59.7 sec  48.5 MBytes  6.81 Mbits/sec
[ 10]  0.0-60.3 sec   118 MBytes  16.5 Mbits/sec
[SUM]  0.0-60.3 sec   697 MBytes  96.9 Mbits/sec

Iperf ExoGENI PSC VM to GPO InstaGENI VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.1 sec   685 MBytes  95.7 Mbits/sec   0.102 ms 24349/512820 (4.7%)
[  3]  0.0-60.1 sec  1 datagrams received out-of-order

Ping from ExoGENI PSC VM to GPO InstaGENI VM

6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 22.168/22.328/22.499/0.203 ms

EG-ST-4 New Site to GPO EG interoperability

Create a slice and then create the stitched slivers with the RSpec EG-ST-4.rspec, which specifies a 100 Mbps link capacity and also use option "--noExoSM" to request GENI Stitching between two ExoGENI sites.

$ stitcher createsliver EG-ST-4a -a gwu-eg EG-ST-4.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc --noExoSM
12:25:36 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
12:25:36 INFO    : Requested to use GENI stitching instead of ExoGENI stitching
12:25:36 INFO    : Reading slice EG-ST-4a credential...
12:25:36 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a expires on 2017-04-10 15:32:38 UTC
12:25:36 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
12:25:36 INFO    : Calling SCS...
12:25:37 INFO    : Multi-AM reservation will include resources from these aggregates:
12:25:37 INFO    : 	<Aggregate psc-eg>
12:25:37 INFO    : 	<Aggregate al2s>
12:25:37 INFO    : 	<Aggregate gpo-eg>
12:25:37 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:25:43 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:25:43 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:25:52 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:25:52 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:25:54 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}}
12:25:54 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.))
12:25:54 INFO    : Deleting existing reservations...
12:25:54 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:25:55 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:25:55 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:25:58 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:25:58 INFO    : Calling SCS for the 2nd time...
12:25:59 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:26:29 INFO    : Multi-AM reservation will include resources from these aggregates:
12:26:29 INFO    : 	<Aggregate psc-eg>
12:26:29 INFO    : 	<Aggregate al2s>
12:26:29 INFO    : 	<Aggregate gpo-eg>
12:26:29 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:26:33 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/4eb0e935-c956-43b0-9fb4-72fda9cf8514#/4399a4db-8954-4040-b853-e15c60e43a2f.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:26:33 INFO    : Got an error reserving resources in EG-ST-4a at <Aggregate psc-eg>
12:26:35 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices.
12:26:35 INFO    : Will put <Aggregate psc-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate psc-eg> 2nd time with new suggested VLANs
12:26:35 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:27:05 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:27:11 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:27:11 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:27:19 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:27:19 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:27:21 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}}
12:27:21 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.))
12:27:21 INFO    : Deleting existing reservations...
12:27:21 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:27:23 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:27:23 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:27:26 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:27:26 INFO    : Calling SCS for the 3rd time...
12:27:26 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:27:56 INFO    : Multi-AM reservation will include resources from these aggregates:
12:27:56 INFO    : 	<Aggregate psc-eg>
12:27:56 INFO    : 	<Aggregate al2s>
12:27:56 INFO    : 	<Aggregate gpo-eg>
12:27:56 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:28:01 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/e80f5e4e-89b3-4e90-8b74-f5fad2c6489f#/a77b441f-071c-4044-a328-24bb9a00eecf.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:28:01 INFO    : Got an error reserving resources in EG-ST-4a at <Aggregate psc-eg>
12:28:02 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices.
12:28:02 INFO    : Will put <Aggregate psc-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate psc-eg> 2nd time with new suggested VLANs
12:28:02 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:28:32 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:28:38 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:28:38 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:28:47 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:28:47 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:28:54 INFO    : ... Allocation at <Aggregate al2s> complete.
12:28:54 INFO    : All aggregates are complete.
12:28:54 INFO    : Your resources expire at 2017-04-09T16:18:32 (UTC). 
12:28:54 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-4a-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-4a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each PSC and GPO aggregate:

$ readyToLogin.py EG-ST-4a --useSliceAggregates

eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4

eg-gpo's geni_status is: ready 
User lnevers logs in to eg-gpo using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.12

Measurements

Iperf ExoGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   632 MBytes  88.3 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   153 MBytes  21.4 Mbits/sec
[  3]  0.0-60.0 sec   120 MBytes  16.8 Mbits/sec
[  4]  0.0-60.1 sec   110 MBytes  15.4 Mbits/sec
[  6]  0.0-60.1 sec   149 MBytes  20.8 Mbits/sec
[  7]  0.0-60.1 sec   160 MBytes  22.4 Mbits/sec
[SUM]  0.0-60.1 sec   694 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  8]  0.0-60.0 sec  74.5 MBytes  10.4 Mbits/sec
[  6]  0.0-60.0 sec  86.4 MBytes  12.1 Mbits/sec
[ 12]  0.0-60.1 sec  61.5 MBytes  8.58 Mbits/sec
[  4]  0.0-60.2 sec  66.8 MBytes  9.30 Mbits/sec
[  9]  0.0-60.2 sec  72.5 MBytes  10.1 Mbits/sec
[  3]  0.0-60.2 sec  58.6 MBytes  8.17 Mbits/sec
[  7]  0.0-60.5 sec  80.8 MBytes  11.2 Mbits/sec
[  5]  0.0-60.7 sec  50.6 MBytes  6.99 Mbits/sec
[ 10]  0.0-60.9 sec  68.1 MBytes  9.38 Mbits/sec
[ 11]  0.0-61.0 sec  76.4 MBytes  10.5 Mbits/sec
[SUM]  0.0-61.0 sec   696 MBytes  95.7 Mbits/sec

Iperf ExoGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.039 ms 12218/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI GPO VM to the ExoGENI PSC VM

60 packets transmitted, 60 received, 0% packet loss, time 59081ms
rtt min/avg/max/mdev = 21.876/22.283/23.353/0.311 ms

Iperf ExoGENI PSC VM to ExoGENI GPO VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   692 MBytes  96.7 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.0 sec   155 MBytes  21.6 Mbits/sec
[  5]  0.0-60.0 sec   122 MBytes  17.0 Mbits/sec
[  7]  0.0-60.1 sec   115 MBytes  16.1 Mbits/sec
[  6]  0.0-60.1 sec   157 MBytes  21.9 Mbits/sec
[  3]  0.0-60.1 sec   145 MBytes  20.2 Mbits/sec
[SUM]  0.0-60.1 sec   694 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.0 sec  93.1 MBytes  13.0 Mbits/sec
[  9]  0.0-60.0 sec  52.0 MBytes  7.27 Mbits/sec
[  3]  0.0-60.0 sec  91.4 MBytes  12.8 Mbits/sec
[ 12]  0.0-60.0 sec  49.8 MBytes  6.95 Mbits/sec
[  7]  0.0-60.1 sec  63.4 MBytes  8.85 Mbits/sec
[ 11]  0.0-60.1 sec  75.0 MBytes  10.5 Mbits/sec
[  8]  0.0-60.2 sec  70.6 MBytes  9.84 Mbits/sec
[  5]  0.0-60.4 sec  61.5 MBytes  8.54 Mbits/sec
[  6]  0.0-60.8 sec  68.4 MBytes  9.43 Mbits/sec
[ 10]  0.0-60.8 sec  70.6 MBytes  9.74 Mbits/sec
[SUM]  0.0-60.8 sec   696 MBytes  95.9 Mbits/sec

Iperf ExoGENI PSC VM to ExoGENI GPO VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.033 ms 12270/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI PSC VM to ExoGENI GPO VM

60 packets transmitted, 60 received, 0% packet loss, time 59052ms
rtt min/avg/max/mdev = 21.973/22.281/24.151/0.369 ms

EG-ST-2 New Site to GPO IG Loop topology

Create a slice and then create the stitched slivers with the RSpec EG-ST-2.rspec:

$ stitcher createsliver EG-ST-2b ./EG-ST-2.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:06:04 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
11:06:04 INFO    : Reading slice EG-ST-2b credential...
11:06:04 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-2b expires on 2017-05-29 15:06:04 UTC
11:06:04 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:06:04 INFO    : Calling SCS...
11:06:06 INFO    : Multi-AM reservation will include resources from these aggregates:
11:06:06 INFO    : 	<Aggregate psc-eg>
11:06:06 INFO    : 	<Aggregate gpo-ig>
11:06:06 INFO    : 	<Aggregate al2s>
11:06:06 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
11:06:13 INFO    : ... Allocation at <Aggregate psc-eg> complete.
11:06:13 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
11:06:35 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
11:06:35 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
11:06:37 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3718', 'code': {'geni_code': 24}}
11:06:37 INFO    : Deleting some reservations to retry, avoiding failed VLAN...
11:06:37 INFO    : Doing deletesliver at <Aggregate gpo-ig>...
11:08:08 ERROR   :  {'output': 'start/restart in progress; try again later', 'code': {'protogeni_error_log': 'urn:publicid:IDN+instageni.gpolab.bbn.com+log+787546a1d3d481e59dc0a3047d0b8882', 'am_type': 'protogeni', 'geni_code': 14, 'am_code': 14, 'protogeni_error_url': 'https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=787546a1d3d481e59dc0a3047d0b8882'}, 'value': 0}
11:08:18 INFO    :  ... aggregate was busy, will retry ...
11:09:25 INFO    :  ... done.
11:09:25 INFO    : Will put <Aggregate al2s> back in the pool to allocate. Got: Retrying reservations at earlier AMs to avoid unavailable VLAN tag at <Aggregate al2s>....
11:09:25 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


11:09:55 INFO    : Checking that prior reservation at <Aggregate gpo-ig> has been cleared up....
11:09:57 ERROR   :  {'output': 'No slice or aggregate here', 'code': {'protogeni_error_log': 'urn:publicid:IDN+instageni.gpolab.bbn.com+log+d4a49bfa1180cbe09ee4f92b8a849dac', 'am_type': 'protogeni', 'geni_code': 12, 'am_code': 12, 'protogeni_error_url': 'https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=d4a49bfa1180cbe09ee4f92b8a849dac'}, 'value': 0}
11:09:57 INFO    : ... it is, so can try a new reservation.
11:09:57 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
11:10:36 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
11:10:36 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
11:10:50 INFO    : ... Allocation at <Aggregate al2s> complete.
11:10:50 INFO    : All aggregates are complete.
11:10:50 INFO    : Your resources expire at 2017-05-28T14:56:06 (UTC). 
11:10:50 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-2b-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-2b at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s).

Determined login information at each PSC and GPO aggregate:

$ readyToLogin.py EG-ST-2b --useSliceAggregates
  ....
eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.7

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 25170  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com

Login to GPO host and ping the remote on each of the two interfaces. Below is the ping output for the GPO site:

lnevers@ig-gpo:~$ ping 10.10.4.2 -c 60 -q
PING 10.10.4.2 (10.10.4.2) 56(84) bytes of data.

--- 10.10.4.2 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59041ms
rtt min/avg/max/mdev = 21.989/22.563/45.379/2.978 ms
lnevers@ig-gpo:~$ ping 192.168.4.2 -c 60 -q
PING 192.168.4.2 (192.168.4.2) 56(84) bytes of data.

--- 192.168.4.2 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59077ms
rtt min/avg/max/mdev = 21.885/22.582/45.334/2.997 ms
lnevers@ig-gpo:~$ 

EG-ST-3 EG-ST-3 New Site 3 node linear topology

Create a slice and then create the stitched slivers with the RSpec EG-ST-3.rspec:

$ stitcher.py createsliver EG-ST-3d EG-ST-3.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:23:28 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
11:23:28 INFO    : Reading slice EG-ST-3d credential...
11:23:28 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-3d expires on 2017-05-29 15:23:27 UTC
11:23:28 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:23:28 INFO    : Calling SCS...
11:23:29 INFO    : Multi-AM reservation will include resources from these aggregates:
11:23:29 INFO    : 	<Aggregate sox-ig>
11:23:29 INFO    : 	<Aggregate gpo-ig>
11:23:29 INFO    : 	<Aggregate al2s>
11:23:29 INFO    : 	<Aggregate psc-eg>
11:23:29 INFO    : Stitcher doing createsliver at <Aggregate sox-ig>...
11:23:50 INFO    : ... Allocation at <Aggregate sox-ig> complete.
11:23:50 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
11:24:08 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
11:24:08 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
11:24:14 INFO    : ... Allocation at <Aggregate psc-eg> complete.
11:24:14 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
11:24:28 INFO    : ... Allocation at <Aggregate al2s> complete.
11:24:28 INFO    : All aggregates are complete.
11:24:28 INFO    : Your resources expire at 2017-05-28T15:13:29 (UTC). 
11:24:28 INFO    : Saved combined reservation RSpec at 4 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-3d-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-3d at 4 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s).

Determined login information the PSC host:

$ readyToLogin.py EG-ST-3d --useSliceAggregates
  ....
eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.8

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 25138  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc5.instageni.gpolab.bbn.com

ig-sox's geni_status is: ready 
User lnevers logs in to ig-sox using:
	ssh -p 25042  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.sox.net

Login to the PSC host and ping each remote:

lnevers@eg-psc:~$ ping 192.168.2.1 -c 60 -q
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.

--- 192.168.2.1 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59073ms
rtt min/avg/max/mdev = 21.970/22.702/49.063/3.469 ms
lnevers@eg-psc:~$ # Utah IG
lnevers@eg-psc:~$ ping 192.168.4.1 -c 60 -q
PING 192.168.4.1 (192.168.4.1) 56(84) bytes of data.

--- 192.168.4.1 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59083ms
rtt min/avg/max/mdev = 25.165/25.782/51.683/3.373 ms
lnevers@eg-psc:~$ 

EG-ST-4 New Site to GPO EG interoperability

Create a slice and then create the stitched slivers with the RSpec EG-ST-4.rspec, which specifies a 100 Mbps link capacity. This was done to get around the mismatch in link capacity units that exists between IG and EG.

$ stitcher.py createsliver EG-ST-4b EG-ST-4.rspec -o --noExoSM --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
12:44:24 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
12:44:24 INFO    : Requested to use GENI stitching instead of ExoGENI stitching
12:44:24 INFO    : Reading slice EG-ST-4b credential...
12:44:24 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b expires on 2017-05-29 16:44:23 UTC
12:44:24 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
12:44:24 INFO    : Calling SCS...
12:44:25 INFO    : Multi-AM reservation will include resources from these aggregates:
12:44:25 INFO    : 	<Aggregate psc-eg>
12:44:25 INFO    : 	<Aggregate al2s>
12:44:25 INFO    : 	<Aggregate gpo-eg>
12:44:25 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:44:30 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:44:30 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:44:39 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:44:39 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:44:41 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}}
12:44:41 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.))
12:44:41 INFO    : Deleting existing reservations...
12:44:41 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:44:42 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:44:42 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:44:48 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:44:48 INFO    : Calling SCS for the 2nd time...
12:44:49 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:45:19 INFO    : Multi-AM reservation will include resources from these aggregates:
12:45:19 INFO    : 	<Aggregate psc-eg>
12:45:19 INFO    : 	<Aggregate al2s>
12:45:19 INFO    : 	<Aggregate gpo-eg>
12:45:19 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:45:25 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:45:25 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:45:34 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:45:34 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:45:36 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3741', 'code': {'geni_code': 24}}
12:45:36 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3741.))
12:45:36 INFO    : Deleting existing reservations...
12:45:36 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:45:38 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:45:38 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:45:40 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:45:40 INFO    : Calling SCS for the 3rd time...
12:45:41 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:46:11 INFO    : Multi-AM reservation will include resources from these aggregates:
12:46:11 INFO    : 	<Aggregate psc-eg>
12:46:11 INFO    : 	<Aggregate al2s>
12:46:11 INFO    : 	<Aggregate gpo-eg>
12:46:11 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:46:17 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:46:17 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:46:27 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:46:27 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:46:28 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}}
12:46:28 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.))
12:46:28 INFO    : Deleting existing reservations...
12:46:28 INFO    : Doing deletesliver at <Aggregate psc-eg>...
12:46:31 INFO    : Deleted reservation at <Aggregate psc-eg>.
12:46:31 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:46:34 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:46:34 INFO    : Calling SCS for the 4th time...
12:46:34 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:47:04 INFO    : Multi-AM reservation will include resources from these aggregates:
12:47:04 INFO    : 	<Aggregate psc-eg>
12:47:04 INFO    : 	<Aggregate al2s>
12:47:04 INFO    : 	<Aggregate gpo-eg>
12:47:04 INFO    : Stitcher doing createsliver at <Aggregate psc-eg>...
12:47:09 INFO    : ... Allocation at <Aggregate psc-eg> complete.
12:47:09 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:47:24 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/11da9516-29d5-475e-a736-799f99cf441c#/98cbfced-a524-480e-a926-d2ba6385ac74.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:47:24 INFO    : Got an error reserving resources in EG-ST-4b at <Aggregate gpo-eg>
12:47:26 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b at gpo-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b among active slices.
12:47:26 INFO    : Will put <Aggregate gpo-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate gpo-eg> 2nd time with new suggested VLANs
12:47:26 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:47:56 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:48:06 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:48:06 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:48:14 INFO    : ... Allocation at <Aggregate al2s> complete.
12:48:14 INFO    : All aggregates are complete.
12:48:14 INFO    : Your resources expire at 2017-05-28T16:37:04 (UTC). 
12:48:14 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-4b-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-4b at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each PSC and GPO aggregate:

$ readyToLogin.py EG-ST-4b --useSliceAggregates
eg-psc's geni_status is: ready 
User lnevers logs in to eg-psc using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.7

eg-gpo's geni_status is: ready 
User lnevers logs in to eg-gpo using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.8

Measurements

Iperf ExoGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-05-22

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   683 MBytes  95.5 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   130 MBytes  18.2 Mbits/sec
[  7]  0.0-60.0 sec   122 MBytes  17.0 Mbits/sec
[  4]  0.0-60.1 sec   132 MBytes  18.4 Mbits/sec
[  6]  0.0-60.1 sec   172 MBytes  24.0 Mbits/sec
[  3]  0.0-60.2 sec   139 MBytes  19.3 Mbits/sec
[SUM]  0.0-60.2 sec   694 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec  76.1 MBytes  10.6 Mbits/sec
[  8]  0.0-60.1 sec  54.2 MBytes  7.58 Mbits/sec
[  3]  0.0-60.1 sec  61.5 MBytes  8.58 Mbits/sec
[  7]  0.0-60.3 sec  79.1 MBytes  11.0 Mbits/sec
[  6]  0.0-60.3 sec  74.6 MBytes  10.4 Mbits/sec
[ 11]  0.0-60.3 sec  76.1 MBytes  10.6 Mbits/sec
[ 10]  0.0-60.3 sec  62.9 MBytes  8.74 Mbits/sec
[  4]  0.0-60.5 sec  64.8 MBytes  8.99 Mbits/sec
[ 12]  0.0-60.5 sec  64.1 MBytes  8.89 Mbits/sec
[  9]  0.0-60.6 sec  82.8 MBytes  11.4 Mbits/sec
[SUM]  0.0-60.6 sec   696 MBytes  96.3 Mbits/sec

Iperf ExoGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default)

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   719 MBytes   101 Mbits/sec
[  3] Sent 512821 datagrams
[  3] Server Report:
[  3]  0.0-60.3 sec   701 MBytes  97.6 Mbits/sec  15.721 ms 12897/512820 (2.5%)
[  3]  0.0-60.3 sec  1 datagrams received out-of-order

Ping from ExoGENI GPO VM to the ExoGENI PSC VM

60 packets transmitted, 60 received, 0% packet loss, time 59079ms
rtt min/avg/max/mdev = 21.896/22.208/23.522/0.310 ms

Iperf ExoGENI PSC VM to ExoGENI GPO VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-05-22

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   693 MBytes  96.8 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-60.0 sec   153 MBytes  21.4 Mbits/sec
[  7]  0.0-60.2 sec   131 MBytes  18.3 Mbits/sec
[  5]  0.0-60.3 sec   178 MBytes  24.8 Mbits/sec
[  3]  0.0-60.5 sec   136 MBytes  18.9 Mbits/sec
[  4]  0.0-60.8 sec  93.9 MBytes  12.9 Mbits/sec
[SUM]  0.0-60.8 sec   693 MBytes  95.5 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.0 sec  71.6 MBytes  10.0 Mbits/sec
[ 12]  0.0-60.0 sec  62.1 MBytes  8.68 Mbits/sec
[ 10]  0.0-60.0 sec  59.2 MBytes  8.28 Mbits/sec
[  8]  0.0-60.1 sec   106 MBytes  14.7 Mbits/sec
[  3]  0.0-60.1 sec  78.5 MBytes  11.0 Mbits/sec
[  5]  0.0-60.2 sec  87.8 MBytes  12.2 Mbits/sec
[ 11]  0.0-60.2 sec  47.1 MBytes  6.56 Mbits/sec
[  9]  0.0-60.3 sec  67.0 MBytes  9.32 Mbits/sec
[  6]  0.0-60.6 sec  64.8 MBytes  8.96 Mbits/sec
[  7]  0.0-60.6 sec  53.4 MBytes  7.39 Mbits/sec
[SUM]  0.0-60.6 sec   697 MBytes  96.5 Mbits/sec

Iperf ExoGENI PSC VM to ExoGENI GPO VM (UDP) - UDP buffer size: 208 KByte (default)

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   719 MBytes   101 Mbits/sec
[  3] Sent 512821 datagrams
[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.064 ms 12336/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI PSC VM to ExoGENI GPO VM

60 packets transmitted, 60 received, 0% packet loss, time 59074ms
rtt min/avg/max/mdev = 21.923/22.152/23.657/0.291 ms