[[PageOutline]] = Pittsburgh Supercomputing Center Stitching Confirmation Tests = This pages capture the detailed test logs for each test defined in the [wiki:GeniNetworkStitchingConfirmationTests New Site Stitching Confirmation Tests] page. For the sites status see the [wiki:GeniNetworkStitchingConfirmationTestStatus New Site Stitching Confirmation Tests Status] page. == EG-ST-1 New Site to GPO IG topology == The PSC site advertises the following stitching details for the connection to AL2S. {{{ orca tree true false 1000000kbps 1000000kbps urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.pitt.net.internet2.edu:et-4/0/0.0:psc-eg 10 1000000kbps 1000000kbps l2sc ethernet consumer 3976-3990 false }}} Experimenter may not need any of this data, but it is helpful to reference when trying to determine how many VLANs are delegated for stitching at the site or how much bandwidth can be requested. Create a slice and then create the stitched slivers with the RSpec [http://groups.geni.net/geni/browser/trunk/stitch-examples/PSC/EG-ST-1.rspec EG-ST-1.rspec]: {{{ $ stitcher createsliver EG-ST-1a -a gwu-eg EG-ST-1.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc 09:20:11 INFO : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf 09:20:11 INFO : Reading slice EG-ST-1a credential... 09:20:12 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-1a expires on 2017-04-10 13:20:11 UTC 09:20:12 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 09:20:12 INFO : Calling SCS... 09:20:12 INFO : Multi-AM reservation will include resources from these aggregates: 09:20:12 INFO : 09:20:12 INFO : 09:20:12 INFO : 09:20:12 INFO : Stitcher doing createsliver at ... 09:20:18 INFO : ... Allocation at complete. 09:20:19 INFO : Stitcher doing createsliver at ... 09:20:37 INFO : ... Allocation at complete. 09:20:37 INFO : Stitcher doing createsliver at ... 09:20:44 INFO : ... Allocation at complete. 09:20:44 INFO : All aggregates are complete. 09:20:44 INFO : Your resources expire at 2017-04-09T13:10:12 (UTC). 09:20:44 INFO : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-1a-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice EG-ST-1a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s). }}} Determined login information at each PSC EG and GPO IG aggregate: {{{ $ readyToLogin.py EG-ST-1a --useSliceAggregates .... eg-psc's geni_status is: ready User lnevers logs in to eg-psc using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4 ig-gpo's geni_status is: ready User lnevers logs in to ig-gpo using: ssh -p 32314 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc4.instageni.gpolab.bbn.com }}} === Measurements === '''Iperf InstaGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-04-03'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.1 sec 590 MBytes 82.4 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 153 MBytes 21.4 Mbits/sec [ 7] 0.0-60.1 sec 137 MBytes 19.1 Mbits/sec [ 4] 0.0-60.1 sec 114 MBytes 16.0 Mbits/sec [ 6] 0.0-60.1 sec 141 MBytes 19.7 Mbits/sec [ 3] 0.0-60.1 sec 134 MBytes 18.7 Mbits/sec [SUM] 0.0-60.1 sec 679 MBytes 94.8 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 7] 0.0-60.1 sec 64.0 MBytes 8.93 Mbits/sec [ 3] 0.0-60.1 sec 66.0 MBytes 9.21 Mbits/sec [ 8] 0.0-60.1 sec 74.5 MBytes 10.4 Mbits/sec [ 9] 0.0-60.2 sec 69.9 MBytes 9.74 Mbits/sec [ 10] 0.0-60.2 sec 70.4 MBytes 9.81 Mbits/sec [ 5] 0.0-60.2 sec 73.0 MBytes 10.2 Mbits/sec [ 12] 0.0-60.2 sec 66.0 MBytes 9.19 Mbits/sec [ 6] 0.0-60.3 sec 76.9 MBytes 10.7 Mbits/sec [ 4] 0.0-60.3 sec 62.6 MBytes 8.71 Mbits/sec [ 11] 0.0-60.3 sec 73.5 MBytes 10.2 Mbits/sec [SUM] 0.0-60.3 sec 697 MBytes 96.9 Mbits/sec }}} '''Iperf InstaGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ 3] Server Report: [ 3] 0.0-60.0 sec 687 MBytes 96.0 Mbits/sec 0.095 ms 23017/512793 (4.5%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from InstaGENI GPO VM to the ExoGENI PSC VM ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59090ms rtt min/avg/max/mdev = 21.982/22.323/23.660/0.327 ms }}} '''Iperf ExoGENI PSC VM to GPO InstaGENI VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-04-03'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 700 MBytes 97.9 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 7] 0.0-60.0 sec 118 MBytes 16.5 Mbits/sec [ 6] 0.0-60.0 sec 159 MBytes 22.2 Mbits/sec [ 3] 0.0-60.1 sec 114 MBytes 15.9 Mbits/sec [ 5] 0.0-60.1 sec 183 MBytes 25.5 Mbits/sec [ 4] 0.0-60.3 sec 126 MBytes 17.6 Mbits/sec [SUM] 0.0-60.3 sec 699 MBytes 97.2 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 9] 0.0-59.3 sec 57.9 MBytes 8.18 Mbits/sec [ 5] 0.0-59.4 sec 65.0 MBytes 9.18 Mbits/sec [ 3] 0.0-59.5 sec 75.0 MBytes 10.6 Mbits/sec [ 4] 0.0-59.5 sec 72.0 MBytes 10.1 Mbits/sec [ 8] 0.0-59.5 sec 67.8 MBytes 9.54 Mbits/sec [ 7] 0.0-59.6 sec 49.5 MBytes 6.97 Mbits/sec [ 11] 0.0-59.6 sec 68.2 MBytes 9.61 Mbits/sec [ 12] 0.0-59.6 sec 75.0 MBytes 10.6 Mbits/sec [ 6] 0.0-59.7 sec 48.5 MBytes 6.81 Mbits/sec [ 10] 0.0-60.3 sec 118 MBytes 16.5 Mbits/sec [SUM] 0.0-60.3 sec 697 MBytes 96.9 Mbits/sec }}} '''Iperf ExoGENI PSC VM to GPO InstaGENI VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ 3] Server Report: [ 3] 0.0-60.1 sec 685 MBytes 95.7 Mbits/sec 0.102 ms 24349/512820 (4.7%) [ 3] 0.0-60.1 sec 1 datagrams received out-of-order }}} '''Ping from ExoGENI PSC VM to GPO InstaGENI VM ''' {{{ 6 packets transmitted, 6 received, 0% packet loss, time 5007ms rtt min/avg/max/mdev = 22.168/22.328/22.499/0.203 ms }}} == EG-ST-4 New Site to GPO EG interoperability == Create a slice and then create the stitched slivers with the RSpec [http://groups.geni.net/geni/browser/trunk/stitch-examples/PSC/EG-ST-4.rspec EG-ST-4.rspec], which specifies a 100 Mbps link capacity and also use option "--noExoSM" to request GENI Stitching between two ExoGENI sites. {{{ $ stitcher createsliver EG-ST-4a -a gwu-eg EG-ST-4.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc --noExoSM 12:25:36 INFO : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf 12:25:36 INFO : Requested to use GENI stitching instead of ExoGENI stitching 12:25:36 INFO : Reading slice EG-ST-4a credential... 12:25:36 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a expires on 2017-04-10 15:32:38 UTC 12:25:36 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 12:25:36 INFO : Calling SCS... 12:25:37 INFO : Multi-AM reservation will include resources from these aggregates: 12:25:37 INFO : 12:25:37 INFO : 12:25:37 INFO : 12:25:37 INFO : Stitcher doing createsliver at ... 12:25:43 INFO : ... Allocation at complete. 12:25:43 INFO : Stitcher doing createsliver at ... 12:25:52 INFO : ... Allocation at complete. 12:25:52 INFO : Stitcher doing createsliver at ... 12:25:54 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}} 12:25:54 WARNING : Stitching failed but will retry: Circuit reservation failed at . Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.)) 12:25:54 INFO : Deleting existing reservations... 12:25:54 INFO : Doing deletesliver at ... 12:25:55 INFO : Deleted reservation at . 12:25:55 INFO : Doing deletesliver at ... 12:25:58 INFO : Deleted reservation at . 12:25:58 INFO : Calling SCS for the 2nd time... 12:25:59 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:26:29 INFO : Multi-AM reservation will include resources from these aggregates: 12:26:29 INFO : 12:26:29 INFO : 12:26:29 INFO : 12:26:29 INFO : Stitcher doing createsliver at ... 12:26:33 ERROR : {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/4eb0e935-c956-43b0-9fb4-72fda9cf8514#/4399a4db-8954-4040-b853-e15c60e43a2f.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}} 12:26:33 INFO : Got an error reserving resources in EG-ST-4a at 12:26:35 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices. 12:26:35 INFO : Will put back in the pool to allocate. Got: VLAN was unavailable. Retry 2nd time with new suggested VLANs 12:26:35 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:27:05 INFO : Stitcher doing createsliver at ... 12:27:11 INFO : ... Allocation at complete. 12:27:11 INFO : Stitcher doing createsliver at ... 12:27:19 INFO : ... Allocation at complete. 12:27:19 INFO : Stitcher doing createsliver at ... 12:27:21 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}} 12:27:21 WARNING : Stitching failed but will retry: Circuit reservation failed at . Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.)) 12:27:21 INFO : Deleting existing reservations... 12:27:21 INFO : Doing deletesliver at ... 12:27:23 INFO : Deleted reservation at . 12:27:23 INFO : Doing deletesliver at ... 12:27:26 INFO : Deleted reservation at . 12:27:26 INFO : Calling SCS for the 3rd time... 12:27:26 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:27:56 INFO : Multi-AM reservation will include resources from these aggregates: 12:27:56 INFO : 12:27:56 INFO : 12:27:56 INFO : 12:27:56 INFO : Stitcher doing createsliver at ... 12:28:01 ERROR : {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/e80f5e4e-89b3-4e90-8b74-f5fad2c6489f#/a77b441f-071c-4044-a328-24bb9a00eecf.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}} 12:28:01 INFO : Got an error reserving resources in EG-ST-4a at 12:28:02 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a at psc-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4a among active slices. 12:28:02 INFO : Will put back in the pool to allocate. Got: VLAN was unavailable. Retry 2nd time with new suggested VLANs 12:28:02 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:28:32 INFO : Stitcher doing createsliver at ... 12:28:38 INFO : ... Allocation at complete. 12:28:38 INFO : Stitcher doing createsliver at ... 12:28:47 INFO : ... Allocation at complete. 12:28:47 INFO : Stitcher doing createsliver at ... 12:28:54 INFO : ... Allocation at complete. 12:28:54 INFO : All aggregates are complete. 12:28:54 INFO : Your resources expire at 2017-04-09T16:18:32 (UTC). 12:28:54 INFO : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-4a-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice EG-ST-4a at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s). }}} Determined login information at each PSC and GPO aggregate: {{{ $ readyToLogin.py EG-ST-4a --useSliceAggregates eg-psc's geni_status is: ready User lnevers logs in to eg-psc using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.4 eg-gpo's geni_status is: ready User lnevers logs in to eg-gpo using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.12 }}} === Measurements === '''Iperf ExoGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-04-03'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 632 MBytes 88.3 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 153 MBytes 21.4 Mbits/sec [ 3] 0.0-60.0 sec 120 MBytes 16.8 Mbits/sec [ 4] 0.0-60.1 sec 110 MBytes 15.4 Mbits/sec [ 6] 0.0-60.1 sec 149 MBytes 20.8 Mbits/sec [ 7] 0.0-60.1 sec 160 MBytes 22.4 Mbits/sec [SUM] 0.0-60.1 sec 694 MBytes 96.8 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 8] 0.0-60.0 sec 74.5 MBytes 10.4 Mbits/sec [ 6] 0.0-60.0 sec 86.4 MBytes 12.1 Mbits/sec [ 12] 0.0-60.1 sec 61.5 MBytes 8.58 Mbits/sec [ 4] 0.0-60.2 sec 66.8 MBytes 9.30 Mbits/sec [ 9] 0.0-60.2 sec 72.5 MBytes 10.1 Mbits/sec [ 3] 0.0-60.2 sec 58.6 MBytes 8.17 Mbits/sec [ 7] 0.0-60.5 sec 80.8 MBytes 11.2 Mbits/sec [ 5] 0.0-60.7 sec 50.6 MBytes 6.99 Mbits/sec [ 10] 0.0-60.9 sec 68.1 MBytes 9.38 Mbits/sec [ 11] 0.0-61.0 sec 76.4 MBytes 10.5 Mbits/sec [SUM] 0.0-61.0 sec 696 MBytes 95.7 Mbits/sec }}} '''Iperf ExoGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ 3] Server Report: [ 3] 0.0-60.0 sec 702 MBytes 98.1 Mbits/sec 0.039 ms 12218/512820 (2.4%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from ExoGENI GPO VM to the ExoGENI PSC VM ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59081ms rtt min/avg/max/mdev = 21.876/22.283/23.353/0.311 ms }}} '''Iperf ExoGENI PSC VM to ExoGENI GPO VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-04-03'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 692 MBytes 96.7 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 155 MBytes 21.6 Mbits/sec [ 5] 0.0-60.0 sec 122 MBytes 17.0 Mbits/sec [ 7] 0.0-60.1 sec 115 MBytes 16.1 Mbits/sec [ 6] 0.0-60.1 sec 157 MBytes 21.9 Mbits/sec [ 3] 0.0-60.1 sec 145 MBytes 20.2 Mbits/sec [SUM] 0.0-60.1 sec 694 MBytes 96.8 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 93.1 MBytes 13.0 Mbits/sec [ 9] 0.0-60.0 sec 52.0 MBytes 7.27 Mbits/sec [ 3] 0.0-60.0 sec 91.4 MBytes 12.8 Mbits/sec [ 12] 0.0-60.0 sec 49.8 MBytes 6.95 Mbits/sec [ 7] 0.0-60.1 sec 63.4 MBytes 8.85 Mbits/sec [ 11] 0.0-60.1 sec 75.0 MBytes 10.5 Mbits/sec [ 8] 0.0-60.2 sec 70.6 MBytes 9.84 Mbits/sec [ 5] 0.0-60.4 sec 61.5 MBytes 8.54 Mbits/sec [ 6] 0.0-60.8 sec 68.4 MBytes 9.43 Mbits/sec [ 10] 0.0-60.8 sec 70.6 MBytes 9.74 Mbits/sec [SUM] 0.0-60.8 sec 696 MBytes 95.9 Mbits/sec }}} '''Iperf ExoGENI PSC VM to ExoGENI GPO VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ 3] Server Report: [ 3] 0.0-60.0 sec 702 MBytes 98.1 Mbits/sec 0.033 ms 12270/512820 (2.4%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from ExoGENI PSC VM to ExoGENI GPO VM ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59052ms rtt min/avg/max/mdev = 21.973/22.281/24.151/0.369 ms }}} == EG-ST-2 New Site to GPO IG Loop topology == Create a slice and then create the stitched slivers with the RSpec [http://groups.geni.net/geni/browser/trunk/stitch-examples/PSC/EG-ST-2.rspec EG-ST-2.rspec]: {{{ $ stitcher createsliver EG-ST-2b ./EG-ST-2.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc 11:06:04 INFO : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf 11:06:04 INFO : Reading slice EG-ST-2b credential... 11:06:04 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-2b expires on 2017-05-29 15:06:04 UTC 11:06:04 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 11:06:04 INFO : Calling SCS... 11:06:06 INFO : Multi-AM reservation will include resources from these aggregates: 11:06:06 INFO : 11:06:06 INFO : 11:06:06 INFO : 11:06:06 INFO : Stitcher doing createsliver at ... 11:06:13 INFO : ... Allocation at complete. 11:06:13 INFO : Stitcher doing createsliver at ... 11:06:35 INFO : ... Allocation at complete. 11:06:35 INFO : Stitcher doing createsliver at ... 11:06:37 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3718', 'code': {'geni_code': 24}} 11:06:37 INFO : Deleting some reservations to retry, avoiding failed VLAN... 11:06:37 INFO : Doing deletesliver at ... 11:08:08 ERROR : {'output': 'start/restart in progress; try again later', 'code': {'protogeni_error_log': 'urn:publicid:IDN+instageni.gpolab.bbn.com+log+787546a1d3d481e59dc0a3047d0b8882', 'am_type': 'protogeni', 'geni_code': 14, 'am_code': 14, 'protogeni_error_url': 'https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=787546a1d3d481e59dc0a3047d0b8882'}, 'value': 0} 11:08:18 INFO : ... aggregate was busy, will retry ... 11:09:25 INFO : ... done. 11:09:25 INFO : Will put back in the pool to allocate. Got: Retrying reservations at earlier AMs to avoid unavailable VLAN tag at .... 11:09:25 INFO : Pausing for 30 seconds for Aggregates to free up resources... 11:09:55 INFO : Checking that prior reservation at has been cleared up.... 11:09:57 ERROR : {'output': 'No slice or aggregate here', 'code': {'protogeni_error_log': 'urn:publicid:IDN+instageni.gpolab.bbn.com+log+d4a49bfa1180cbe09ee4f92b8a849dac', 'am_type': 'protogeni', 'geni_code': 12, 'am_code': 12, 'protogeni_error_url': 'https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=d4a49bfa1180cbe09ee4f92b8a849dac'}, 'value': 0} 11:09:57 INFO : ... it is, so can try a new reservation. 11:09:57 INFO : Stitcher doing createsliver at ... 11:10:36 INFO : ... Allocation at complete. 11:10:36 INFO : Stitcher doing createsliver at ... 11:10:50 INFO : ... Allocation at complete. 11:10:50 INFO : All aggregates are complete. 11:10:50 INFO : Your resources expire at 2017-05-28T14:56:06 (UTC). 11:10:50 INFO : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-2b-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice EG-ST-2b at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s). }}} Determined login information at each PSC and GPO aggregate: {{{ $ readyToLogin.py EG-ST-2b --useSliceAggregates .... eg-psc's geni_status is: ready User lnevers logs in to eg-psc using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.7 ig-gpo's geni_status is: ready User lnevers logs in to ig-gpo using: ssh -p 25170 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com }}} Login to GPO host and ping the remote on each of the two interfaces. Below is the ping output for the GPO site: {{{ lnevers@ig-gpo:~$ ping 10.10.4.2 -c 60 -q PING 10.10.4.2 (10.10.4.2) 56(84) bytes of data. --- 10.10.4.2 ping statistics --- 60 packets transmitted, 60 received, 0% packet loss, time 59041ms rtt min/avg/max/mdev = 21.989/22.563/45.379/2.978 ms lnevers@ig-gpo:~$ ping 192.168.4.2 -c 60 -q PING 192.168.4.2 (192.168.4.2) 56(84) bytes of data. --- 192.168.4.2 ping statistics --- 60 packets transmitted, 60 received, 0% packet loss, time 59077ms rtt min/avg/max/mdev = 21.885/22.582/45.334/2.997 ms lnevers@ig-gpo:~$ }}} == EG-ST-3 EG-ST-3 New Site 3 node linear topology == Create a slice and then create the stitched slivers with the RSpec [http://groups.geni.net/geni/browser/trunk/stitch-examples/PSC/EG-ST-3.rspec EG-ST-3.rspec]: {{{ $ stitcher.py createsliver EG-ST-3d EG-ST-3.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc 11:23:28 INFO : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf 11:23:28 INFO : Reading slice EG-ST-3d credential... 11:23:28 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-3d expires on 2017-05-29 15:23:27 UTC 11:23:28 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 11:23:28 INFO : Calling SCS... 11:23:29 INFO : Multi-AM reservation will include resources from these aggregates: 11:23:29 INFO : 11:23:29 INFO : 11:23:29 INFO : 11:23:29 INFO : 11:23:29 INFO : Stitcher doing createsliver at ... 11:23:50 INFO : ... Allocation at complete. 11:23:50 INFO : Stitcher doing createsliver at ... 11:24:08 INFO : ... Allocation at complete. 11:24:08 INFO : Stitcher doing createsliver at ... 11:24:14 INFO : ... Allocation at complete. 11:24:14 INFO : Stitcher doing createsliver at ... 11:24:28 INFO : ... Allocation at complete. 11:24:28 INFO : All aggregates are complete. 11:24:28 INFO : Your resources expire at 2017-05-28T15:13:29 (UTC). 11:24:28 INFO : Saved combined reservation RSpec at 4 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-3d-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice EG-ST-3d at 4 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s). }}} Determined login information the PSC host: {{{ $ readyToLogin.py EG-ST-3d --useSliceAggregates .... eg-psc's geni_status is: ready User lnevers logs in to eg-psc using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.8 ig-gpo's geni_status is: ready User lnevers logs in to ig-gpo using: ssh -p 25138 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc5.instageni.gpolab.bbn.com ig-sox's geni_status is: ready User lnevers logs in to ig-sox using: ssh -p 25042 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.sox.net }}} Login to the PSC host and ping each remote: {{{ lnevers@eg-psc:~$ ping 192.168.2.1 -c 60 -q PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data. --- 192.168.2.1 ping statistics --- 60 packets transmitted, 60 received, 0% packet loss, time 59073ms rtt min/avg/max/mdev = 21.970/22.702/49.063/3.469 ms lnevers@eg-psc:~$ # Sox IG lnevers@eg-psc:~$ ping 192.168.4.1 -c 60 -q PING 192.168.4.1 (192.168.4.1) 56(84) bytes of data. --- 192.168.4.1 ping statistics --- 60 packets transmitted, 60 received, 0% packet loss, time 59083ms rtt min/avg/max/mdev = 25.165/25.782/51.683/3.373 ms lnevers@eg-psc:~$ }}} == EG-ST-4 New Site to GPO EG interoperability == Create a slice and then create the stitched slivers with the RSpec [http://groups.geni.net/geni/browser/trunk/stitch-examples/PSC/EG-ST-4.rspec EG-ST-4.rspec], which specifies a 100 Mbps link capacity. This was done to get around the mismatch in link capacity units that exists between IG and EG. {{{ $ stitcher.py createsliver EG-ST-4b EG-ST-4.rspec -o --noExoSM --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc 12:44:24 INFO : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf 12:44:24 INFO : Requested to use GENI stitching instead of ExoGENI stitching 12:44:24 INFO : Reading slice EG-ST-4b credential... 12:44:24 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b expires on 2017-05-29 16:44:23 UTC 12:44:24 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 12:44:24 INFO : Calling SCS... 12:44:25 INFO : Multi-AM reservation will include resources from these aggregates: 12:44:25 INFO : 12:44:25 INFO : 12:44:25 INFO : 12:44:25 INFO : Stitcher doing createsliver at ... 12:44:30 INFO : ... Allocation at complete. 12:44:30 INFO : Stitcher doing createsliver at ... 12:44:39 INFO : ... Allocation at complete. 12:44:39 INFO : Stitcher doing createsliver at ... 12:44:41 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}} 12:44:41 WARNING : Stitching failed but will retry: Circuit reservation failed at . Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.)) 12:44:41 INFO : Deleting existing reservations... 12:44:41 INFO : Doing deletesliver at ... 12:44:42 INFO : Deleted reservation at . 12:44:42 INFO : Doing deletesliver at ... 12:44:48 INFO : Deleted reservation at . 12:44:48 INFO : Calling SCS for the 2nd time... 12:44:49 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:45:19 INFO : Multi-AM reservation will include resources from these aggregates: 12:45:19 INFO : 12:45:19 INFO : 12:45:19 INFO : 12:45:19 INFO : Stitcher doing createsliver at ... 12:45:25 INFO : ... Allocation at complete. 12:45:25 INFO : Stitcher doing createsliver at ... 12:45:34 INFO : ... Allocation at complete. 12:45:34 INFO : Stitcher doing createsliver at ... 12:45:36 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3741', 'code': {'geni_code': 24}} 12:45:36 WARNING : Stitching failed but will retry: Circuit reservation failed at . Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3741.)) 12:45:36 INFO : Deleting existing reservations... 12:45:36 INFO : Doing deletesliver at ... 12:45:38 INFO : Deleted reservation at . 12:45:38 INFO : Doing deletesliver at ... 12:45:40 INFO : Deleted reservation at . 12:45:40 INFO : Calling SCS for the 3rd time... 12:45:41 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:46:11 INFO : Multi-AM reservation will include resources from these aggregates: 12:46:11 INFO : 12:46:11 INFO : 12:46:11 INFO : 12:46:11 INFO : Stitcher doing createsliver at ... 12:46:17 INFO : ... Allocation at complete. 12:46:17 INFO : Stitcher doing createsliver at ... 12:46:27 INFO : ... Allocation at complete. 12:46:27 INFO : Stitcher doing createsliver at ... 12:46:28 ERROR : {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}} 12:46:28 WARNING : Stitching failed but will retry: Circuit reservation failed at . Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.)) 12:46:28 INFO : Deleting existing reservations... 12:46:28 INFO : Doing deletesliver at ... 12:46:31 INFO : Deleted reservation at . 12:46:31 INFO : Doing deletesliver at ... 12:46:34 INFO : Deleted reservation at . 12:46:34 INFO : Calling SCS for the 4th time... 12:46:34 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:47:04 INFO : Multi-AM reservation will include resources from these aggregates: 12:47:04 INFO : 12:47:04 INFO : 12:47:04 INFO : 12:47:04 INFO : Stitcher doing createsliver at ... 12:47:09 INFO : ... Allocation at complete. 12:47:09 INFO : Stitcher doing createsliver at ... 12:47:24 ERROR : {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/11da9516-29d5-475e-a736-799f99cf441c#/98cbfced-a524-480e-a926-d2ba6385ac74.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}} 12:47:24 INFO : Got an error reserving resources in EG-ST-4b at 12:47:26 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b at gpo-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4b among active slices. 12:47:26 INFO : Will put back in the pool to allocate. Got: VLAN was unavailable. Retry 2nd time with new suggested VLANs 12:47:26 INFO : Pausing for 30 seconds for Aggregates to free up resources... 12:47:56 INFO : Stitcher doing createsliver at ... 12:48:06 INFO : ... Allocation at complete. 12:48:06 INFO : Stitcher doing createsliver at ... 12:48:14 INFO : ... Allocation at complete. 12:48:14 INFO : All aggregates are complete. 12:48:14 INFO : Your resources expire at 2017-05-28T16:37:04 (UTC). 12:48:14 INFO : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/PSC/EG-ST-4b-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice EG-ST-4b at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s). }}} Determined login information at each PSC and GPO aggregate: {{{ $ readyToLogin.py EG-ST-4b --useSliceAggregates eg-psc's geni_status is: ready User lnevers logs in to eg-psc using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@147.72.248.7 eg-gpo's geni_status is: ready User lnevers logs in to eg-gpo using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.8 }}} === Measurements === '''Iperf ExoGENI GPO VM to ExoGENI PSC VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-05-22'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 683 MBytes 95.5 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 130 MBytes 18.2 Mbits/sec [ 7] 0.0-60.0 sec 122 MBytes 17.0 Mbits/sec [ 4] 0.0-60.1 sec 132 MBytes 18.4 Mbits/sec [ 6] 0.0-60.1 sec 172 MBytes 24.0 Mbits/sec [ 3] 0.0-60.2 sec 139 MBytes 19.3 Mbits/sec [SUM] 0.0-60.2 sec 694 MBytes 96.8 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 76.1 MBytes 10.6 Mbits/sec [ 8] 0.0-60.1 sec 54.2 MBytes 7.58 Mbits/sec [ 3] 0.0-60.1 sec 61.5 MBytes 8.58 Mbits/sec [ 7] 0.0-60.3 sec 79.1 MBytes 11.0 Mbits/sec [ 6] 0.0-60.3 sec 74.6 MBytes 10.4 Mbits/sec [ 11] 0.0-60.3 sec 76.1 MBytes 10.6 Mbits/sec [ 10] 0.0-60.3 sec 62.9 MBytes 8.74 Mbits/sec [ 4] 0.0-60.5 sec 64.8 MBytes 8.99 Mbits/sec [ 12] 0.0-60.5 sec 64.1 MBytes 8.89 Mbits/sec [ 9] 0.0-60.6 sec 82.8 MBytes 11.4 Mbits/sec [SUM] 0.0-60.6 sec 696 MBytes 96.3 Mbits/sec }}} '''Iperf ExoGENI GPO VM to ExoGENI PSC VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512821 datagrams [ 3] Server Report: [ 3] 0.0-60.3 sec 701 MBytes 97.6 Mbits/sec 15.721 ms 12897/512820 (2.5%) [ 3] 0.0-60.3 sec 1 datagrams received out-of-order }}} '''Ping from ExoGENI GPO VM to the ExoGENI PSC VM ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59079ms rtt min/avg/max/mdev = 21.896/22.208/23.522/0.310 ms }}} '''Iperf ExoGENI PSC VM to ExoGENI GPO VM (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2017-05-22'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 693 MBytes 96.8 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 153 MBytes 21.4 Mbits/sec [ 7] 0.0-60.2 sec 131 MBytes 18.3 Mbits/sec [ 5] 0.0-60.3 sec 178 MBytes 24.8 Mbits/sec [ 3] 0.0-60.5 sec 136 MBytes 18.9 Mbits/sec [ 4] 0.0-60.8 sec 93.9 MBytes 12.9 Mbits/sec [SUM] 0.0-60.8 sec 693 MBytes 95.5 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 71.6 MBytes 10.0 Mbits/sec [ 12] 0.0-60.0 sec 62.1 MBytes 8.68 Mbits/sec [ 10] 0.0-60.0 sec 59.2 MBytes 8.28 Mbits/sec [ 8] 0.0-60.1 sec 106 MBytes 14.7 Mbits/sec [ 3] 0.0-60.1 sec 78.5 MBytes 11.0 Mbits/sec [ 5] 0.0-60.2 sec 87.8 MBytes 12.2 Mbits/sec [ 11] 0.0-60.2 sec 47.1 MBytes 6.56 Mbits/sec [ 9] 0.0-60.3 sec 67.0 MBytes 9.32 Mbits/sec [ 6] 0.0-60.6 sec 64.8 MBytes 8.96 Mbits/sec [ 7] 0.0-60.6 sec 53.4 MBytes 7.39 Mbits/sec [SUM] 0.0-60.6 sec 697 MBytes 96.5 Mbits/sec }}} '''Iperf ExoGENI PSC VM to ExoGENI GPO VM (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512821 datagrams [ 3] Server Report: [ 3] 0.0-60.0 sec 702 MBytes 98.1 Mbits/sec 0.064 ms 12336/512820 (2.4%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from ExoGENI PSC VM to ExoGENI GPO VM ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59074ms rtt min/avg/max/mdev = 21.923/22.152/23.657/0.291 ms }}} == EG-ST-5 Site Information == Various pages include stitching information for a site. Each of the following were verified for this site: 1. Verified that stitching VLANs and Device URN information exists in the [http://groups.geni.net/geni/wiki/GeniAggregate/PSCExoGENI#GENINetworkStitching PSC Aggregate] page. 2. To BE Verified that Delegated GENI Stitching VLANs for site are documented at the [https://wiki.maxgigapop.net/twiki/bin/view/GENI/StaticNetworksView Delegated GENI Stitching VLANs]. 3. No longer possible for Stitching Computation Service logs to be reviewed. 4. To BE Added site to the list of [wiki:GeniNetworkStitchingSites GENI Network Stitching Sites].