wiki:GeniNetworkStitchingConfirmationTestStatus/UNF

Version 14 (modified by lnevers@bbn.com, 2 years ago) (diff)

--

University of Northern Florida Stitching Confirmation Tests

This pages capture the detailed test logs for each test defined in the New Site Stitching Confirmation Tests page. For the sites status see the New Site Stitching Confirmation Tests Status page.

EG-ST-1 New Site to GPO IG topology

The UNF site advertises the following stitching details for the connection to AL2S.

    <ns2:stitching lastUpdateTime="Mon Aug 15 14:41:36 EDT 2016">
        <ns2:aggregate id="urn:publicid:IDN+exogeni.net:unfNet+authority+am" url="https://unf-hn.exogeni.net:11443/orca/xmlrpc">
            <ns2:aggregatetype>orca</ns2:aggregatetype>
            <ns2:stitchingmode>tree</ns2:stitchingmode>
            <ns2:scheduledservices>true</ns2:scheduledservices>
            <ns2:negotiatedservices>false</ns2:negotiatedservices>
            <ns2:node id="urn:publicid:IDN+exogeni.net:unfNet+node+orca-transit-net-vlan">
                <ns2:port id="urn:publicid:IDN+exogeni.net:unfNet+stitchport+UNFNet:IBM:G8052:TenGigabitEthernet:1:1:ethernet">
                    <ns2:capacity>10000000kbps</ns2:capacity>
                    <ns2:maximumReservableCapacity>10000000kbps</ns2:maximumReservableCapacity>
                    <ns2:link id="urn:publicid:IDN+exogeni.net:unfNet+interface+UNFNet:IBM:G8052:TenGigabitEthernet:1:1:ethernet">
                        <ns2:remoteLinkId>urn:publicid:IDN+al2s.internet2.edu+interface+sdn-sw.jack.net.internet2.edu:eth1/2:unf-eg</ns2:remoteLinkId>
                        <ns2:trafficEngineeringMetric>10</ns2:trafficEngineeringMetric>
                        <ns2:capacity>10000000kbps</ns2:capacity>
                        <ns2:maximumReservableCapacity>10000000kbps</ns2:maximumReservableCapacity>
                        <ns2:switchingCapabilityDescriptor>
                            <ns2:switchingcapType>l2sc</ns2:switchingcapType>
                            <ns2:encodingType>ethernet</ns2:encodingType>
                            <ns2:switchingCapabilitySpecificInfo>
                                <ns2:switchingCapabilitySpecificInfo_L2sc>
                                    <ns2:capability>consumer</ns2:capability>
                                    <ns2:vlanRangeAvailability>1470-1484</ns2:vlanRangeAvailability>
                                    <ns2:vlanTranslation>false</ns2:vlanTranslation>
                                </ns2:switchingCapabilitySpecificInfo_L2sc>
                            </ns2:switchingCapabilitySpecificInfo>
                        </ns2:switchingCapabilityDescriptor>
                    </ns2:link>
                </ns2:port>
            </ns2:node>
        </ns2:aggregate>
    </ns2:stitching>

Experimenter may not need any of this data, but it is helpful to reference when trying to determine how many VLANs are delegated for stitching at the site or how much bandwidth can be requested.

Create a slice and then create the stitched slivers with the RSpec EG-ST-1-al2s.rspec:

$ stitcher.py createsliver EG-ST-1 EG-ST-1-al2s.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
14:43:23 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
14:43:23 INFO    : Reading slice EG-ST-1 credential...
14:43:23 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-1 expires on 2016-08-22 18:42:30 UTC
14:43:23 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
14:43:23 INFO    : Calling SCS...
14:43:25 INFO    : Multi-AM reservation will include resources from these aggregates:
14:43:25 INFO    : 	<Aggregate unf-eg>
14:43:25 INFO    : 	<Aggregate gpo-ig>
14:43:25 INFO    : 	<Aggregate al2s>
14:43:25 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
14:43:32 INFO    : ... Allocation at <Aggregate unf-eg> complete.
14:43:32 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
14:43:56 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
14:43:56 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
14:44:02 INFO    : ... Allocation at <Aggregate al2s> complete.
14:44:02 INFO    : All aggregates are complete.
14:44:02 INFO    : Your resources expire at 2016-08-21T18:33:25 (UTC). 
14:44:02 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/UNF/EG-ST-1-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-1 at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each UNF EG and GPO IG aggregate:

$ readyToLogin.py EG-ST-1 --useSliceAggregates
  ....
eg-unf's geni_status is: ready 
User lnevers logs in to eg-unf using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@139.62.242.25

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 32058  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc5.instageni.gpolab.bbn.com

Measurements

Iperf InstaGENI GPO VM to ExoGENI UNF VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2016-08-15

One Client

[  3] local 192.168.1.1 port 59464 connected with 192.168.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   710 MBytes  99.2 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-60.0 sec   130 MBytes  18.2 Mbits/sec
[  3]  0.0-60.0 sec   123 MBytes  17.1 Mbits/sec
[  5]  0.0-60.1 sec   159 MBytes  22.2 Mbits/sec
[  4]  0.0-60.1 sec   126 MBytes  17.5 Mbits/sec
[  6]  0.0-60.2 sec   157 MBytes  21.9 Mbits/sec
[SUM]  0.0-60.2 sec   695 MBytes  96.8 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[ 10]  0.0-60.1 sec  65.5 MBytes  9.14 Mbits/sec
[  4]  0.0-60.1 sec  75.1 MBytes  10.5 Mbits/sec
[  5]  0.0-60.1 sec  63.9 MBytes  8.91 Mbits/sec
[  9]  0.0-60.1 sec  72.8 MBytes  10.1 Mbits/sec
[  8]  0.0-60.2 sec  65.8 MBytes  9.17 Mbits/sec
[  3]  0.0-60.2 sec  62.4 MBytes  8.69 Mbits/sec
[ 12]  0.0-60.2 sec  73.4 MBytes  10.2 Mbits/sec
[  7]  0.0-60.2 sec  71.8 MBytes  10.0 Mbits/sec
[  6]  0.0-60.2 sec  69.4 MBytes  9.66 Mbits/sec
[ 11]  0.0-60.2 sec  78.1 MBytes  10.9 Mbits/sec
[SUM]  0.0-60.2 sec   698 MBytes  97.2 Mbits/sec

Iperf InstaGENI GPO VM to ExoGENI UNF VM (UDP) - UDP buffer size: 208 KByte (default)

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   719 MBytes   101 Mbits/sec
[  3] Sent 512783 datagrams
[  3] Server Report:
[  3]  0.0-60.0 sec   687 MBytes  96.0 Mbits/sec   0.153 ms 22980/512782 (4.5%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from InstaGENI GPO VM to the ExoGENI UNF VM

60 packets transmitted, 60 received, 0% packet loss, time 59099ms
rtt min/avg/max/mdev = 30.164/30.540/31.054/0.211 ms

Iperf ExoGENI UNF VM to GPO InstaGENI VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2016-08-15

One Client

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   707 MBytes  98.8 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   116 MBytes  16.2 Mbits/sec
[  3]  0.0-60.1 sec   173 MBytes  24.1 Mbits/sec
[  4]  0.0-60.2 sec   159 MBytes  22.2 Mbits/sec
[  7]  0.0-60.2 sec   166 MBytes  23.1 Mbits/sec
[  6]  0.0-60.4 sec  93.1 MBytes  12.9 Mbits/sec
[SUM]  0.0-60.4 sec   706 MBytes  98.1 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-60.1 sec  72.5 MBytes  10.1 Mbits/sec
[ 12]  0.0-60.2 sec  74.4 MBytes  10.4 Mbits/sec
[  7]  0.0-60.2 sec   105 MBytes  14.7 Mbits/sec
[  5]  0.0-60.2 sec  76.0 MBytes  10.6 Mbits/sec
[  9]  0.0-60.4 sec  71.4 MBytes  9.92 Mbits/sec
[  8]  0.0-60.4 sec  52.5 MBytes  7.30 Mbits/sec
[  3]  0.0-60.4 sec  72.4 MBytes  10.1 Mbits/sec
[  4]  0.0-60.4 sec  66.2 MBytes  9.20 Mbits/sec
[ 11]  0.0-60.5 sec  46.9 MBytes  6.50 Mbits/sec
[ 10]  0.0-60.5 sec  70.2 MBytes  9.74 Mbits/sec
[SUM]  0.0-60.5 sec   708 MBytes  98.2 Mbits/sec

Iperf ExoGENI UNF VM to GPO InstaGENI VM (UDP) - UDP buffer size: 208 KByte (default)

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   719 MBytes   101 Mbits/sec
[  3] Sent 512821 datagrams
[  3] Server Report:
[  3]  0.0-60.1 sec   686 MBytes  95.7 Mbits/sec   7.526 ms 23828/512820 (4.6%)
[  3]  0.0-60.1 sec  1 datagrams received out-of-order

Ping from ExoGENI UNF VM to GPO InstaGENI VM

60 packets transmitted, 60 received, 0% packet loss, time 59099ms
rtt min/avg/max/mdev = 30.259/30.617/31.066/0.265 ms

EG-ST-2 New Site to GPO IG Loop topology

Create a slice and then create the stitched slivers with the RSpec EG-ST-2.rspec:

$ stitcher.py createsliver EG-ST-2 EG-ST-2.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
10:20:10 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
10:20:10 INFO    : Reading slice EG-ST-2 credential...
10:20:10 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-2 expires on 2017-05-29 14:19:51 UTC
10:20:10 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
10:20:10 INFO    : Calling SCS...
10:20:11 INFO    : Multi-AM reservation will include resources from these aggregates:
10:20:11 INFO    : 	<Aggregate unf-eg>
10:20:11 INFO    : 	<Aggregate gpo-ig>
10:20:11 INFO    : 	<Aggregate al2s>
10:20:11 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
10:20:20 INFO    : ... Allocation at <Aggregate unf-eg> complete.
10:20:20 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
10:20:38 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
10:20:39 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
10:20:53 INFO    : ... Allocation at <Aggregate al2s> complete.
10:20:53 INFO    : All aggregates are complete.
10:20:53 INFO    : Your resources expire at 2017-05-28T14:10:11 (UTC). 
10:20:53 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/UNF/EG-ST-2-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-2 at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s).

Determined login information at each UNF and GPO aggregate:

$ readyToLogin.py EG-ST-2 --useSliceAggregates
  ....
eg-unf's geni_status is: ready 
User lnevers logs in to eg-unf using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@139.62.242.11

ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 25106  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com

Login to GPO host and ping the remote on each of the two interfaces. Below is the ping output for the GPO site:

lnevers@ig-gpo:~$ ping 10.10.4.2 -c 60 -q
PING 10.10.4.2 (10.10.4.2) 56(84) bytes of data.

--- 10.10.4.2 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59098ms
rtt min/avg/max/mdev = 29.869/29.968/30.571/0.168 ms
lnevers@ig-gpo:~$ ping 192.168.4.2 -c 60 -q
PING 192.168.4.2 (192.168.4.2) 56(84) bytes of data.

--- 192.168.4.2 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59082ms
rtt min/avg/max/mdev = 29.898/30.013/30.238/0.150 ms
lnevers@ig-gpo:~$ 

EG-ST-3 EG-ST-3 New Site 3 node linear topology

Create a slice and then create the stitched slivers with the RSpec EG-ST-3.rspec:

$ stitcher.py createsliver EG-ST-3a EG-ST-3.rspec -o --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:03:16 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
11:03:16 INFO    : Reading slice EG-ST-3 credential...
11:03:16 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-3 expires on 2017-05-29 14:38:34 UTC
11:03:16 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
11:03:16 INFO    : Calling SCS...
11:03:17 INFO    : Multi-AM reservation will include resources from these aggregates:
11:03:17 INFO    : 	<Aggregate unf-eg>
11:03:17 INFO    : 	<Aggregate sox-ig>
11:03:17 INFO    : 	<Aggregate gpo-ig>
11:03:17 INFO    : 	<Aggregate al2s>
11:03:17 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
11:03:26 INFO    : ... Allocation at <Aggregate unf-eg> complete.
11:03:26 INFO    : Stitcher doing createsliver at <Aggregate sox-ig>...
11:03:43 INFO    : ... Allocation at <Aggregate sox-ig> complete.
11:03:43 INFO    : Stitcher doing createsliver at <Aggregate gpo-ig>...
11:04:01 INFO    : ... Allocation at <Aggregate gpo-ig> complete.
11:04:01 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
11:04:15 INFO    : ... Allocation at <Aggregate al2s> complete.
11:04:15 INFO    : All aggregates are complete.
11:04:15 INFO    : Your resources expire at 2017-05-28T14:53:17 (UTC). 
11:04:15 INFO    : Saved combined reservation RSpec at 4 AM(s) to file '/home/lnevers/gcf-test/stitch-test/UNF/EG-ST-3-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-3 at 4 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 2 stitched link(s).

Determined login information the UNF host:

$ readyToLogin.py EG-ST-3 --useSliceAggregates
  ....
ig-gpo's geni_status is: ready 
User lnevers logs in to ig-gpo using:
	ssh -p 25138  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com

ig-unf's geni_status is: ready 
User lnevers logs in to ig-unf using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@139.62.242.10

ig-sox's geni_status is: ready 
User lnevers logs in to ig-sox using:
	ssh -p 25010  -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.sox.net

Login to the UNF host and ping each remote:

lnevers@ig-unf:~$ ping 192.168.2.1 -c 60 -q
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.

--- 192.168.2.1 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59079ms
rtt min/avg/max/mdev = 29.895/30.583/60.353/3.893 ms
lnevers@ig-unf:~$ # Utah IG
lnevers@ig-unf:~$ ping 192.168.4.1 -c 60 -q
PING 192.168.4.1 (192.168.4.1) 56(84) bytes of data.

--- 192.168.4.1 ping statistics ---
60 packets transmitted, 60 received, 0% packet loss, time 59102ms
rtt min/avg/max/mdev = 6.860/7.060/14.204/0.935 ms
lnevers@ig-unf:~$ }}}

EG-ST-4 New Site to GPO EG interoperability

Create a slice and then create the stitched slivers with the RSpec EG-ST-4.rspec, which specifies a 100 Mbps link capacity. This was done to get around the mismatch in link capacity units that exists between IG and EG.

$ stitcher createsliver EG-ST-4h EG-ST-4.rspec --scsURL https://nutshell.maxgigapop.net:8443/geni/xmlrpc --noExoSM
12:56:31 INFO    : Configured logging from file /home/lnevers/geni-tools-2.10/src/gcf/stitcher_logging.conf
12:56:31 INFO    : Requested to use GENI stitching instead of ExoGENI stitching
12:56:31 INFO    : Reading slice EG-ST-4h credential...
12:56:32 INFO    : Slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h expires on 2017-04-10 15:32:32 UTC
12:56:32 INFO    : Using SCS at https://nutshell.maxgigapop.net:8443/geni/xmlrpc
12:56:32 INFO    : Calling SCS...
12:56:33 INFO    : Multi-AM reservation will include resources from these aggregates:
12:56:33 INFO    : 	<Aggregate unf-eg>
12:56:33 INFO    : 	<Aggregate al2s>
12:56:33 INFO    : 	<Aggregate gpo-eg>
12:56:33 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
12:56:40 INFO    : ... Allocation at <Aggregate unf-eg> complete.
12:56:40 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:56:49 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:56:49 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:56:50 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739', 'code': {'geni_code': 24}}
12:56:50 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3739.))
12:56:50 INFO    : Deleting existing reservations...
12:56:50 INFO    : Doing deletesliver at <Aggregate unf-eg>...
12:56:53 INFO    : Deleted reservation at <Aggregate unf-eg>.
12:56:53 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:56:57 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:56:57 INFO    : Calling SCS for the 2nd time...
12:56:57 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:57:27 INFO    : Multi-AM reservation will include resources from these aggregates:
12:57:27 INFO    : 	<Aggregate unf-eg>
12:57:27 INFO    : 	<Aggregate al2s>
12:57:27 INFO    : 	<Aggregate gpo-eg>
12:57:27 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
12:57:34 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/84763fd6-1e16-4f72-bab8-fe5edff16b4e#/42c8044e-62b1-4eee-868f-5c8ccdd534c9.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:57:34 INFO    : Got an error reserving resources in EG-ST-4h at <Aggregate unf-eg>
12:57:35 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h at unf-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h among active slices.
12:57:35 INFO    : Will put <Aggregate unf-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate unf-eg> 2nd time with new suggested VLANs
12:57:35 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:58:05 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
12:58:13 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/0a1735bb-460e-4a8b-9d18-f26cbb2f6cfa#/e2a466f2-a6b5-4784-8863-c07e2e7d411b.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:58:13 INFO    : Got an error reserving resources in EG-ST-4h at <Aggregate unf-eg>
12:58:15 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h at unf-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h among active slices.
12:58:15 INFO    : Will put <Aggregate unf-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate unf-eg> 3rd time with new suggested VLANs
12:58:15 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:58:45 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
12:58:52 INFO    : ... Allocation at <Aggregate unf-eg> complete.
12:58:52 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:59:00 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
12:59:00 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
12:59:02 ERROR   :  {'output': 'Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738', 'code': {'geni_code': 24}}
12:59:02 WARNING : Stitching failed but will retry: Circuit reservation failed at <Aggregate al2s>. Try again from the SCS. (Error: Topology too complex for local redo - ask Stitching Service to find a VLAN tag (AMAPIError: Error from Aggregate: code 24: Exception: requested VLAN unavailable: rtsw.newy32aoa.net.internet2.edu,et-7/3/0.0 VLAN=3738.))
12:59:02 INFO    : Deleting existing reservations...
12:59:02 INFO    : Doing deletesliver at <Aggregate unf-eg>...
12:59:05 INFO    : Deleted reservation at <Aggregate unf-eg>.
12:59:05 INFO    : Doing deletesliver at <Aggregate gpo-eg>...
12:59:07 INFO    : Deleted reservation at <Aggregate gpo-eg>.
12:59:07 INFO    : Calling SCS for the 3rd time...
12:59:08 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


12:59:38 INFO    : Multi-AM reservation will include resources from these aggregates:
12:59:38 INFO    : 	<Aggregate unf-eg>
12:59:38 INFO    : 	<Aggregate al2s>
12:59:38 INFO    : 	<Aggregate gpo-eg>
12:59:38 INFO    : Stitcher doing createsliver at <Aggregate unf-eg>...
12:59:45 INFO    : ... Allocation at <Aggregate unf-eg> complete.
12:59:45 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
12:59:54 ERROR   :  {'output': 'ERROR: Embedding workflow ERROR: 1:Error in building the dependency tree, probably not available vlan path OR trying to reuse a stitching tag:http://geni-orca.renci.org/owl/bad33c4a-4e86-4fe8-b339-44125b8b5546#/23259e32-f0c8-4a14-82ad-1917ec22d127.\n Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.', 'code': {'geni_code': 2}}
12:59:54 INFO    : Got an error reserving resources in EG-ST-4h at <Aggregate gpo-eg>
12:59:56 WARNING : Failed to delete sliver urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h at gpo-eg (got result 'None'). Error from Aggregate: code 12: ERROR: unable to find slice urn:publicid:IDN+ch.geni.net:ln-test+slice+EG-ST-4h among active slices.
12:59:56 INFO    : Will put <Aggregate gpo-eg> back in the pool to allocate. Got: VLAN was unavailable. Retry <Aggregate gpo-eg> 2nd time with new suggested VLANs
12:59:56 INFO    : Pausing for 30 seconds for Aggregates to free up resources...


13:00:26 INFO    : Stitcher doing createsliver at <Aggregate gpo-eg>...
13:00:34 INFO    : ... Allocation at <Aggregate gpo-eg> complete.
13:00:34 INFO    : Stitcher doing createsliver at <Aggregate al2s>...
13:00:43 INFO    : ... Allocation at <Aggregate al2s> complete.
13:00:43 INFO    : All aggregates are complete.
13:00:43 INFO    : Your resources expire at 2017-04-09T16:49:38 (UTC). 
13:00:43 INFO    : Saved combined reservation RSpec at 3 AM(s) to file '/home/lnevers/gcf-test/stitch-test/UNF/EG-ST-4h-manifest-rspec-multiam-combined.xml'
Success: Reserved resources in slice EG-ST-4h at 3 Aggregates (including 1 intermediate aggregate(s) not in the original request), creating 1 stitched link(s).

Determined login information at each UNF and GPO aggregate:

$ readyToLogin.py EG-ST-4h --useSliceAggregates

eg-gpo's geni_status is: ready 
User lnevers logs in to eg-gpo using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@192.1.242.12

eg-unf's geni_status is: ready 
User lnevers logs in to eg-unf using:
	ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@139.62.242.15

Measurements

Iperf ExoGENI GPO VM to ExoGENI UNF VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   694 MBytes  97.0 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   122 MBytes  17.0 Mbits/sec
[  6]  0.0-60.0 sec   144 MBytes  20.1 Mbits/sec
[  5]  0.0-60.1 sec   119 MBytes  16.6 Mbits/sec
[  4]  0.0-60.3 sec   175 MBytes  24.4 Mbits/sec
[  7]  0.0-60.6 sec   134 MBytes  18.6 Mbits/sec
[SUM]  0.0-60.6 sec   694 MBytes  96.1 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  71.2 MBytes  9.96 Mbits/sec
[ 10]  0.0-60.0 sec  82.9 MBytes  11.6 Mbits/sec
[  5]  0.0-60.1 sec  93.9 MBytes  13.1 Mbits/sec
[ 12]  0.0-60.1 sec  62.4 MBytes  8.70 Mbits/sec
[  4]  0.0-60.1 sec  70.4 MBytes  9.82 Mbits/sec
[  9]  0.0-60.2 sec  53.6 MBytes  7.47 Mbits/sec
[  7]  0.0-60.2 sec  60.1 MBytes  8.37 Mbits/sec
[ 11]  0.0-60.3 sec  78.0 MBytes  10.9 Mbits/sec
[  6]  0.0-60.4 sec  60.4 MBytes  8.39 Mbits/sec
[  8]  0.0-60.9 sec  64.1 MBytes  8.84 Mbits/sec
[SUM]  0.0-60.9 sec   697 MBytes  96.1 Mbits/sec

Iperf ExoGENI GPO VM to ExoGENI UNF VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   702 MBytes  98.1 Mbits/sec   0.017 ms 12196/512820 (2.4%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI GPO VM to the ExoGENI UNF VM

60 packets transmitted, 60 received, 0% packet loss, time 59085ms
rtt min/avg/max/mdev = 29.843/30.088/31.062/0.339 ms

Iperf ExoGENI UNF VM to ExoGENI GPO VM (TCP) - TCP window size: 85.0 KByte (default)

Collected: 2017-04-03

One Client_

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   693 MBytes  96.9 Mbits/sec

Five Clients

[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec   173 MBytes  24.2 Mbits/sec
[  4]  0.0-60.0 sec   205 MBytes  28.6 Mbits/sec
[  3]  0.0-60.1 sec   106 MBytes  14.8 Mbits/sec
[  7]  0.0-60.1 sec   113 MBytes  15.8 Mbits/sec
[  6]  0.0-60.3 sec  96.1 MBytes  13.4 Mbits/sec
[SUM]  0.0-60.3 sec   694 MBytes  96.6 Mbits/sec

Ten Clients

[ ID] Interval       Transfer     Bandwidth
[ 11]  0.0-60.1 sec  54.5 MBytes  7.61 Mbits/sec
[  9]  0.0-60.1 sec  58.8 MBytes  8.20 Mbits/sec
[  4]  0.0-60.1 sec  57.2 MBytes  7.99 Mbits/sec
[  6]  0.0-60.1 sec  97.2 MBytes  13.6 Mbits/sec
[ 10]  0.0-60.2 sec  80.9 MBytes  11.3 Mbits/sec
[  3]  0.0-60.2 sec  66.4 MBytes  9.25 Mbits/sec
[ 12]  0.0-60.3 sec  55.9 MBytes  7.78 Mbits/sec
[  5]  0.0-60.3 sec  61.5 MBytes  8.56 Mbits/sec
[  8]  0.0-60.6 sec   100 MBytes  13.9 Mbits/sec
[  7]  0.0-60.7 sec  65.0 MBytes  8.98 Mbits/sec
[SUM]  0.0-60.7 sec   697 MBytes  96.4 Mbits/sec

Iperf ExoGENI UNF VM to ExoGENI GPO VM (UDP) - UDP buffer size: 208 KByte (default)

[  3] Server Report:
[  3]  0.0-60.0 sec   701 MBytes  98.0 Mbits/sec   0.049 ms 12657/512819 (2.5%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Ping from ExoGENI UNF VM to ExoGENI GPO VM

60 packets transmitted, 60 received, 0% packet loss, time 59082ms
rtt min/avg/max/mdev = 29.874/30.046/31.905/0.353 ms