[[PageOutline]] = CG-EXP-4: GENI Stitching Support = This test allocated resources in both NCSU2 and NCSU Cisco racks. Note the following Omni nick names are used: {{{ ncsu2-eg=urn:publicid:IDN+exogeni.net:ncsu2vmsite+authority+am,https://ncsu2-hn.exogeni.net:11443/orca/xmlrpc ncsu-eg=urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am,https://ncsu-hn.exogeni.net:11443/orca/xmlrpc }}} == Test Status == This section captures the status for each step in the acceptance test plan. || '''Stitching topologies''' ||'''State''' || ''' !Tickets/Comments''' || ||NCSU2 to NCSU2 || N/A ||Not planned at this time || || ||NCSU2 to GPO EG || N/A || This scenario is ExoGENI stitching; not available at this time and covered in [http://groups.geni.net/geni/wiki/GENIRacksHome/CiscoGENIRacks/AcceptanceTestStatus/CG-EXP-2 CG-EXP-2] || ||NCSU2 to IG ION site ||[[Color(green,Pass)]]||~~Stitching setup; remoteLinkId definition~~|| ||NCSU2 to IG AL2S site ||[[Color(yellow,Complete)]] || Throughput low?? || ||NYSERNet IGNCSU2<-AL2S->Stanford-IG||[[Color(red,Fail)]]|| Know ExoGENI issue http://groups.geni.net/exogeni/ticket/193 || [[BR]] || '''State Legend''' || '''Description''' || || [[Color(green,Pass)]] || Test completed and met all criteria || || [[Color(#98FB98,Pass: most criteria)]] || Test completed and met most criteria. Exceptions documented || || [[Color(red,Fail)]] || Test completed and failed to meet criteria. || || [[Color(yellow,Complete)]] || Test completed but will require re-execution due to expected changes || || [[Color(orange,Blocked)]] || Blocked by ticketed issue(s). || || [[Color(#63B8FF,In Progress)]] || Currently under test. || [[BR]] == Test Details == Testing scenario cover both ExoGENI and GENI stitching. A mix of test topologies are run with the GENI Stitching feature. === NCSU2 to GPO EG === This is ExoGENI Stitching, which is covered in [http://groups.geni.net/geni/wiki/GENIRacksHome/CiscoGENIRacks/AcceptanceTestStatus/CG-EXP-2 CG-EXP-2]. Using the RSpec [http://groups.geni.net/geni/browser/trunk/GENIRacks/CiscoGENI/RSpecs/AcceptanceTests/cisco-ncsu2-eg-gpo.rspec cisco-ncsu2-eg-gpo.rspec] tried to create an ExoGENI stitched sliver via the ExoSM: {{{ $ omni createsliver -a eg-sm CG-CT-4a ./cisco-ncsu2-eg-gpo.rspec 14:25:14 INFO : Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache' 14:25:14 INFO : Loading config file '/home/lnevers/.gcf/omni_config' 14:25:14 INFO : Using control framework portal 14:25:14 INFO : Member Authority is https://ch.geni.net/MA (from config) 14:25:14 INFO : Slice Authority is https://ch.geni.net/SA (from config) 14:25:14 INFO : Substituting AM nickname eg-sm with URL https://geni.renci.org:11443/orca/xmlrpc, URN urn:publicid:IDN+exogeni.net+authority+am 14:25:14 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4a expires on 2015-01-16 19:24:40 UTC 14:25:14 INFO : Creating sliver(s) from rspec file ./cisco-ncsu2-eg-gpo.rspec for slice urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4a 14:25:20 WARNING : Failed CreateSliver for slice CG-CT-4a at eg-sm. Error from Aggregate: code 2: Embedding workflow ERROR: 4:No Edge Domain Exist:http://geni-orca.renci.org/owl/ec743afb-fa45-4031-ab4c-23b0f5df7806#eg-gpo:http://geni-orca.renci.org/owl/ec743afb-fa45-4031-ab4c-23b0f5df7806#cg-ncsu2. Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions. 14:25:20 INFO : ------------------------------------------------------ 14:25:20 INFO : Completed createsliver: Options as run: aggregate: ['eg-sm'] framework: portal project: ln-test Args: createsliver CG-CT-4a ./cisco-ncsu2-eg-gpo.rspec Result Summary: Failed CreateSliver for slice CG-CT-4a at eg-sm. Error from Aggregate: code 2: Embedding workflow ERROR: 4:No Edge Domain Exist:http://geni-orca.renci.org/owl/ec743afb-fa45-4031-ab4c-23b0f5df7806#eg-gpo:http://geni-orca.renci.org/owl/ec743afb-fa45-4031-ab4c-23b0f5df7806#cg-ncsu2. Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions. }}} === NCSU2 to IG ION site === Using the RSpec [http://groups.geni.net/geni/browser/trunk/GENIRacks/CiscoGENI/RSpecs/AcceptanceTests/stitch-eg-ncsu2-ig-gpo.rspec stitch-eg-ncsu2-ig-gpo.rspec] the following topology was created: GPO IG <-ION-><-AL2S->NCSU2 The stitcher was used to create the sliver: {{{ $ stitcher createsliver CG-CT-4 ./stitch-eg-ncsu2-ig-gpo.rspec 14:16:02 INFO : Configured logging from file /home/lnevers/gcf-2.7/src/gcf/stitcher_logging.conf 14:16:02 INFO : Reading slice CG-CT-4 credential... 14:16:03 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4 expires on 2015-01-16 19:15:12 UTC 14:16:03 INFO : Using SCS at http://nutshell.maxgigapop.net:8081/geni/xmlrpc 14:16:03 INFO : Calling SCS... 14:16:04 INFO : Multi-AM reservation will include resources from these aggregates: 14:16:04 INFO : 14:16:04 INFO : 14:16:04 INFO : 14:16:04 INFO : 14:16:04 INFO : Stitcher doing createsliver at ... 14:16:09 INFO : ... Allocation at complete. 14:16:09 INFO : Stitcher doing createsliver at ... 14:16:24 INFO : ... Allocation at complete. 14:16:24 INFO : Stitcher doing createsliver at ... 14:16:33 INFO : ... Allocation at complete. 14:16:33 INFO : Stitcher doing createsliver at ... 14:17:19 INFO : DCN AM : must wait for status ready.... 14:17:19 INFO : Pausing 30 seconds to let circuit become ready... 14:17:56 INFO : DCN circuit 146681 is ready at 14:18:04 INFO : ... Allocation at complete. 14:18:04 INFO : All aggregates are complete. 14:18:04 INFO : Your resources expire at 2015-01-15T19:06:04 (UTC). 14:18:04 INFO : Saved combined reservation RSpec at 4 AMs to file '/home/lnevers/gcf-test/cisco/CG-CT-4-manifest-rspec-multiam-combined.xml' Success: Reserved resources in slice CG-CT-4 at 4 Aggregates (including 2 intermediate aggregate(s) not in the original request), creating 1 stitched link(s). }}} Determine login for NCSU2 host: {{{ $ readyToLogin CG-CT-4 -a ncsu2-eg ... eg-ncsu2's geni_status is: ready User lnevers logs in to eg-ncsu2 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@152.48.13.191 }}} Determine login for GPO IG host: {{{ $ readyToLogin CG-CT-4 -a gpo-ig ... ig-gpo's geni_status is: ready User lnevers logs in to ig-gpo using: ssh -p 33338 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc5.instageni.gpolab.bbn.com }}} ==== Measurements ==== '''Iperf GPO IG VM to NCSU2 CG (TCP) - TCP window size: 23.5 KByte (default)''' ''Collected: 2015-01-09'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 715 MBytes 100 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 119 MBytes 16.6 Mbits/sec [ 4] 0.0-60.1 sec 124 MBytes 17.4 Mbits/sec [ 5] 0.0-60.2 sec 161 MBytes 22.4 Mbits/sec [ 7] 0.0-60.2 sec 122 MBytes 16.9 Mbits/sec [ 3] 0.0-60.2 sec 172 MBytes 24.0 Mbits/sec [SUM] 0.0-60.2 sec 698 MBytes 97.3 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 59.2 MBytes 8.28 Mbits/sec [ 10] 0.0-60.0 sec 87.6 MBytes 12.2 Mbits/sec [ 11] 0.0-60.1 sec 75.9 MBytes 10.6 Mbits/sec [ 3] 0.0-60.1 sec 62.5 MBytes 8.73 Mbits/sec [ 12] 0.0-60.1 sec 66.0 MBytes 9.21 Mbits/sec [ 7] 0.0-60.3 sec 71.1 MBytes 9.89 Mbits/sec [ 5] 0.0-60.4 sec 47.2 MBytes 6.56 Mbits/sec [ 4] 0.0-60.5 sec 60.0 MBytes 8.31 Mbits/sec [ 9] 0.0-60.7 sec 85.8 MBytes 11.9 Mbits/sec [ 8] 0.0-61.0 sec 91.2 MBytes 12.6 Mbits/sec [SUM] 0.0-61.0 sec 707 MBytes 97.2 Mbits/sec }}} '''Iperf GPO IG VM to NCSU2 CG (UDP) - UDP buffer size: 224 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512809 datagrams [ 3] Server Report: [ 3] 0.0-60.0 sec 694 MBytes 97.0 Mbits/sec 0.121 ms 17641/512808 (3.4%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from GPO IG VM to the NCSU2 CG ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59099ms rtt min/avg/max/mdev = 17.163/17.569/18.046/0.234 ms }}} '''Iperf NCSU2 CG to GPO IG (TCP) - TCP window size: 85.5 KByte (default)''' ''Collected: 2015-01-09'' __One Client_ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.1 sec 35.0 MBytes 4.89 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 37.0 MBytes 5.17 Mbits/sec [ 3] 0.0-60.2 sec 38.0 MBytes 5.30 Mbits/sec [ 7] 0.0-60.3 sec 33.6 MBytes 4.68 Mbits/sec [ 4] 0.0-60.4 sec 36.8 MBytes 5.11 Mbits/sec [ 5] 0.0-60.7 sec 36.9 MBytes 5.10 Mbits/sec [SUM] 0.0-60.7 sec 182 MBytes 25.2 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.2 sec 38.6 MBytes 5.38 Mbits/sec [ 7] 0.0-60.2 sec 37.6 MBytes 5.24 Mbits/sec [ 11] 0.0-60.2 sec 38.2 MBytes 5.33 Mbits/sec [ 8] 0.0-60.2 sec 38.0 MBytes 5.29 Mbits/sec [ 12] 0.0-60.3 sec 37.8 MBytes 5.26 Mbits/sec [ 9] 0.0-60.3 sec 37.6 MBytes 5.24 Mbits/sec [ 4] 0.0-60.3 sec 38.1 MBytes 5.31 Mbits/sec [ 10] 0.0-60.3 sec 40.0 MBytes 5.56 Mbits/sec [ 5] 0.0-60.4 sec 37.6 MBytes 5.23 Mbits/sec [ 6] 0.0-60.6 sec 37.8 MBytes 5.23 Mbits/sec [SUM] 0.0-60.6 sec 381 MBytes 52.8 Mbits/sec }}} Note: If a larger TCP window size was specified, was able to get much better throughput. '''Iperf NCSU2 CG to GPO IG (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512821 datagrams [ 3] Server Report: [ 3] 0.0-60.1 sec 685 MBytes 95.6 Mbits/sec 0.067 ms 24133/512820 (4.7%) [ 3] 0.0-60.1 sec 9 datagrams received out-of-order }}} '''Ping from GPO IG VM to the NCSU2 CG ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59097ms rtt min/avg/max/mdev = 17.201/17.547/18.112/0.214 ms }}} === NCSU2 to IG AL2S site === Using the RSpec [http://groups.geni.net/geni/browser/trunk/GENIRacks/CiscoGENI/RSpecs/AcceptanceTests/stitch-eg-ncsu2-ig-chic.rspec stitch-eg-ncsu2-ig-chic.rspec] created a stitched sliver to a remote AL2S site: {{{ $ stitcher deletesliver CG-CT-4x;stitcher createsliver CG-CT-4x ./stitch-eg-ncsu2-ig-chic.rspec --defaultCapacity=100000 08:36:00 INFO : Configured logging from file /home/lnevers/gcf-2.8-pre1/src/gcf/stitcher_logging.conf 08:36:00 INFO : Reading slice CG-CT-4x aggregates from file /home/lnevers/.gcf/ch-geni-net-ln-test-CG-CT-4x-amlist.txt 08:36:00 INFO : Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache' 08:36:00 INFO : Loading config file '/home/lnevers/.gcf/omni_config' 08:36:00 INFO : Using control framework portal 08:36:00 INFO : Member Authority is https://ch.geni.net/MA (from config) 08:36:00 INFO : Slice Authority is https://ch.geni.net/SA (from config) 08:36:01 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4x expires on 2015-01-26 13:16:54 UTC 08:36:02 INFO : Deleted sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4x at ncsu2-eg 08:36:04 INFO : (PG log url - look here for details on any failures: https://www.geni.uchicago.edu/spewlogfile.php3?logfile=1516bd8dbdc510ae2e78bad62af8117f) 08:36:05 INFO : Deleted sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4x at chicago-ig 08:36:29 INFO : Deleted sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4x at oess Deleted slivers on 3 out of a possible 3 aggregates 08:36:29 INFO : Configured logging from file /home/lnevers/gcf-2.8-pre1/src/gcf/stitcher_logging.conf 08:36:30 INFO : Reading slice CG-CT-4x credential... 08:36:30 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+CG-CT-4x expires on 2015-01-26 13:16:54 UTC 08:36:30 INFO : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc 08:36:30 INFO : Calling SCS... 08:37:31 INFO : Multi-AM reservation will include resources from these aggregates: 08:37:31 INFO : 08:37:31 INFO : 08:37:31 INFO : 08:37:31 INFO : Stitcher doing createsliver at ... 08:37:36 INFO : ... Allocation at complete. 08:37:36 INFO : Stitcher doing createsliver at ... 08:37:56 INFO : ... Allocation at complete. 08:37:56 INFO : Stitcher doing createsliver at ... 08:38:01 INFO : ... Allocation at complete. 08:38:01 INFO : All aggregates are complete. 08:38:02 INFO : Your resources expire at 2015-01-25T13:27:31 (UTC). 08:38:02 INFO : Saved combined reservation RSpec at 3 AMs to file '/home/lnevers/gcf-test/cisco/CG-CT-4x-manifest-rspec-multiam-combined.xml' }}} Determine login: {{{ $ readyToLogin CG-CT-4x -a chicago-ig ... ig-chic's geni_status is: ready User lnevers logs in to ig-chic using: ssh -p 30266 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc3.geni.uchicago.edu $ readyToLogin CG-CT-4x -a ncsu2-eg ... eg-ncsu2's geni_status is: ready User lnevers logs in to eg-ncsu2 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@152.48.13.194 }}} ==== Measurements ==== '''Iperf Chicago IG VM to NCSU2 CG (TCP) - TCP window size: 23.5 KByte (default)''' ''Collected: 2015-01-19'' __One Client__ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 697 MBytes 97.4 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.1 sec 142 MBytes 19.9 Mbits/sec [ 6] 0.0-60.1 sec 113 MBytes 15.8 Mbits/sec [ 3] 0.0-60.1 sec 121 MBytes 16.9 Mbits/sec [ 4] 0.0-60.2 sec 133 MBytes 18.5 Mbits/sec [ 7] 0.0-60.3 sec 188 MBytes 26.1 Mbits/sec [SUM] 0.0-60.3 sec 698 MBytes 97.1 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 12] 0.0-60.0 sec 59.0 MBytes 8.25 Mbits/sec [ 7] 0.0-60.1 sec 46.5 MBytes 6.50 Mbits/sec [ 11] 0.0-60.1 sec 67.9 MBytes 9.48 Mbits/sec [ 6] 0.0-60.1 sec 62.6 MBytes 8.74 Mbits/sec [ 3] 0.0-60.1 sec 68.6 MBytes 9.58 Mbits/sec [ 8] 0.0-60.2 sec 94.4 MBytes 13.1 Mbits/sec [ 5] 0.0-60.3 sec 92.1 MBytes 12.8 Mbits/sec [ 10] 0.0-60.4 sec 75.9 MBytes 10.5 Mbits/sec [ 4] 0.0-60.5 sec 95.5 MBytes 13.2 Mbits/sec [ 9] 0.0-60.7 sec 52.9 MBytes 7.31 Mbits/sec [SUM] 0.0-60.7 sec 715 MBytes 98.9 Mbits/sec }}} '''Iperf Chicago IG VM to NCSU2 CG (UDP) - UDP buffer size: 224 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512778 datagrams [ 3] Server Report: [ 3] 0.0-60.0 sec 693 MBytes 96.9 Mbits/sec 0.138 ms 18523/512777 (3.6%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order }}} '''Ping from Chicago IG VM to the NCSU2 CG ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59091ms rtt min/avg/max/mdev = 25.204/25.391/25.798/0.228 ms }}} '''Iperf NCSU2 CG to Chicago IG (TCP) - TCP window size: 85.0 KByte (default)''' ''Collected: 2015-01-19'' __One Client__ {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.2 sec 66.0 MBytes 9.19 Mbits/sec }}} __Five Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 7] 0.0-60.1 sec 76.5 MBytes 10.7 Mbits/sec [ 4] 0.0-60.1 sec 77.5 MBytes 10.8 Mbits/sec [ 5] 0.0-60.2 sec 81.4 MBytes 11.3 Mbits/sec [ 6] 0.0-60.3 sec 67.4 MBytes 9.38 Mbits/sec [ 3] 0.0-60.3 sec 82.9 MBytes 11.5 Mbits/sec [SUM] 0.0-60.3 sec 386 MBytes 53.7 Mbits/sec }}} __Ten Clients__ {{{ [ ID] Interval Transfer Bandwidth [ 12] 0.0-60.2 sec 67.9 MBytes 9.47 Mbits/sec [ 13] 0.0-60.2 sec 59.0 MBytes 8.23 Mbits/sec [ 7] 0.0-60.2 sec 62.6 MBytes 8.73 Mbits/sec [ 8] 0.0-60.2 sec 46.5 MBytes 6.48 Mbits/sec [ 4] 0.0-60.3 sec 68.6 MBytes 9.54 Mbits/sec [ 9] 0.0-60.5 sec 94.4 MBytes 13.1 Mbits/sec [ 6] 0.0-60.5 sec 92.1 MBytes 12.8 Mbits/sec [ 5] 0.0-60.8 sec 95.5 MBytes 13.2 Mbits/sec [ 11] 0.0-60.8 sec 75.9 MBytes 10.5 Mbits/sec [ 10] 0.0-60.8 sec 52.9 MBytes 7.29 Mbits/sec [SUM] 0.0-60.8 sec 715 MBytes 98.6 Mbits/sec }}} '''Iperf NCSU2 CG to Chicago IG (UDP) - UDP buffer size: 208 KByte (default) ''' {{{ [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 719 MBytes 101 Mbits/sec [ 3] Sent 512821 datagrams [ 3] Server Report: [ 3] 0.0-60.1 sec 681 MBytes 95.0 Mbits/sec 0.072 ms 26950/512820 (5.3%) [ 3] 0.0-60.1 sec 1 datagrams received out-of-order }}} '''Ping from Chicago IG VM to the NCSU2 CG ''' {{{ 60 packets transmitted, 60 received, 0% packet loss, time 59091ms rtt min/avg/max/mdev = 25.283/25.482/25.966/0.147 ms }}} === NCSU2 Multiple Stitched links === Using the RSpec [http://groups.geni.net/geni/browser/trunk/GENIRacks/CiscoGENI/RSpecs/AcceptanceTests/stitch-3linear.rspec stitch-3linear.rspec] requested the following topology: NYSERNet IGNCSU2<-AL2S->Stanford-IG The create sliver: {{{ $ stitcher createsliver 3linear ./stitch-3linear.rspec 15:27:46 INFO : Configured logging from file /home/lnevers/gcf-2.7/src/gcf/stitcher_logging.conf 15:27:46 INFO : Reading slice 3linear credential... 15:27:46 INFO : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+3linear expires on 2015-01-12 20:21:43 UTC 15:27:46 INFO : Using SCS at http://nutshell.maxgigapop.net:8081/geni/xmlrpc 15:27:46 INFO : Calling SCS... 15:27:48 INFO : Multi-AM reservation will include resources from these aggregates: 15:27:48 INFO : 15:27:48 INFO : 15:27:48 INFO : 15:27:48 INFO : 15:27:48 INFO : 15:27:48 INFO : Stitcher doing createsliver at ... 15:27:51 ERROR : {'output': 'Invalid slice urn urn:publicid:IDN+ch.geni.net:ln-test+slice+3linear', 'code': {'geni_code': 2}} 15:27:51 INFO : Got an error reserving resources in 3linear at 15:27:52 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+3linear at ncsu2-eg (got result 'False'). (no reason given) 15:27:52 WARNING : Stitching failed but will retry: Circuit reservation failed at (AMAPIError: Error from Aggregate: code 2: Invalid slice urn urn:publicid:IDN+ch.geni.net:ln-test+slice+3linear.). Try again from the SCS }}}