Version 26 (modified by 11 years ago) (diff) | ,
---|
University of Houston ExoGENI Confirmation Tests
For details about the tests in this page, see the ExoGENI Confirmation Tests page.
For site status see the ExoGENI New Site Confirmation Tests Status page.
Several ExoGENI site aggregates are used on this page that are aliased with Omni nick_names:
uh-eg=,https://uh-hn.exogeni.net:11443/orca/xmlrpc gpo-eg=,https://bbn-hn.exogeni.net:11443/orca/xmlrpc sm-eg=,https://geni.renci.org:11443/orca/xmlrpc of-uh-eg=,https://uh-hn.exogeni.net:3626/foam/gapi/1 of-gpo-eg=,https://bbn-hn.exogeni.net:3626/foam/gapi/1
EG-CT-1 - Access to New Site VM resources
First getvesion to determine orca_version running at this aggregate:
$ omni.py -a eg-uh getversion INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Substituting AM nickname eg-uh with URL https://uh-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Substituting AM nickname eg-uh with URL https://uh-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:AM URN: unspecified_AM_URN (url: https://uh-hn.exogeni.net:11443/orca/xmlrpc) has version: INFO:omni:{ 'geni_ad_rspec_versions': [ { 'extensions': [ 'http://hpn.east.isi.edu/rspec/ext/stitch/0.1/stitch-schema.xsd', 'http://www.protogeni.net/resources/rspec/ext/emulab/1/ptop_extension.xsd'], 'namespace': 'http://www.geni.net/resources/rspec/3', 'schema': 'http://www.geni.net/resources/rspec/3/ad.xsd', 'type': 'GENI', 'version': '3'}], 'geni_am_type': 'orca', 'geni_api': 2, 'geni_api_versions': { '2': 'https://uh-hn.exogeni.net:11443/orca/xmlrpc/geni'}, 'geni_request_rspec_versions': [ { 'extensions': [ 'http://www.geni.net/resources/rspec/ext/shared-vlan/1', 'http://www.geni.net/resources/rspec/ext/postBootScript/1'], 'namespace': 'http://www.geni.net/resources/rspec/3', 'schema': 'http://www.geni.net/resources/rspec/3/request.xsd', 'type': 'GENI', 'version': '3'}], 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5495'} INFO:omni: ------------------------------------------------------------ INFO:omni: Completed getversion: Options as run: aggregate: ['eg-uh'] framework: portal noGetVersionCache: True project: ln-prj Args: getversion Result Summary: Got version for https://uh-hn.exogeni.net:11443/orca/xmlrpc INFO:omni: ============================================================
Compared with versions in other racks, the following version were found:
getversion-bbn-hn-exogeni-net-11443-orca.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5495'} getversion-fiu-hn-exogeni-net-11443-orca.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5495'} getversion-geni-renci-org-11443-orca.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5505'} getversion-rci-hn-exogeni-net-11443-orca.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5495'} getversion-uh-hn-exogeni-net-11443-orca.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-5495'}
Created a slice:
$ omni.py createslice EG-CT-1 INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Created slice with Name EG-CT-1, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-1, Expiration 2013-06-21 00:50:33 INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createslice: Options as run: framework: portal project: ln-prj Args: createslice EG-CT-1 Result Summary: Created slice with Name EG-CT-1, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-1, Expiration 2013-06-21 00:50:33 INFO:omni: ============================================================
Created a 4 VMs sliver using the RSpec EG-CT-1-uh.rspec
$ omni.py createsliver -a uh-eg EG-CT-1 EG-CT-1-uh.rspec INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Substituting AM nickname uh-eg with URL https://uh-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-1 expires within 1 day on 2013-06-21 00:50:33 UTC INFO:omni:Substituting AM nickname uh-eg with URL https://uh-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Substituting AM nickname uh-eg with URL https://uh-hn.exogeni.net:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Creating sliver(s) from rspec file EG-CT-1-uh.rspec for slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-1 INFO:omni:Got return from CreateSliver for slice EG-CT-1 at https://uh-hn.exogeni.net:11443/orca/xmlrpc: INFO:omni:<?xml version="1.0" encoding="UTF-8" standalone="yes"?> INFO:omni: <!-- Reserved resources for: Slice: EG-CT-1 at AM: URN: unspecified_AM_URN URL: https://uh-hn.exogeni.net:11443/orca/xmlrpc --> INFO:omni: <rspec type="manifest" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/manifest.xsd http://hpn.east.isi.edu/rspec/ext/stitch/0.1/ http://hpn.east.isi.edu/rspec/ext/stitch/0.1/stitch-schema.xsd http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1/slice_info.xsd?format=raw http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1/sliver_info.xsd?format=raw" xmlns:ns2="http://hpn.east.isi.edu/rspec/ext/stitch/0.1/" xmlns="http://www.geni.net/resources/rspec/3" xmlns:ns4="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1" xmlns:ns3="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+850d3d04-999b-4bfb-b032-5f3c354399ba:VM-1" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-vm-cloud" client_id="VM-1"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="m1.small"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:52:a9" client_id="VM-1:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.1"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-20T13:02:30.000-04:00" expiration_time="2013-06-20T20:50:33.000-04:00" creation_time="2013-06-20T13:02:30.000-04:00" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+850d3d04-999b-4bfb-b032-5f3c354399ba:VM-2" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-vm-cloud" client_id="VM-2"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="m1.large"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:5e:79" client_id="VM-2:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.2"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-20T13:02:30.000-04:00" expiration_time="2013-06-20T20:50:33.000-04:00" creation_time="2013-06-20T13:02:30.000-04:00" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+850d3d04-999b-4bfb-b032-5f3c354399ba:VM-3" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-vm-cloud" client_id="VM-3"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="m1.large"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:1d:ff" client_id="VM-3:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.3"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-20T13:02:30.000-04:00" expiration_time="2013-06-20T20:50:33.000-04:00" creation_time="2013-06-20T13:02:30.000-04:00" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+850d3d04-999b-4bfb-b032-5f3c354399ba:VM-4" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-vm-cloud" client_id="VM-4"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="m1.large"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:3d:9e" client_id="VM-4:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.4"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-20T13:02:30.000-04:00" expiration_time="2013-06-20T20:50:33.000-04:00" creation_time="2013-06-20T13:02:30.000-04:00" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <link vlantag="unknown" sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+850d3d04-999b-4bfb-b032-5f3c354399ba:lan0" client_id="lan0"> <interface_ref client_id="VM-4:if0"/> <interface_ref client_id="VM-3:if0"/> <interface_ref client_id="VM-2:if0"/> <interface_ref client_id="VM-1:if0"/> <ns4:geni_sliver_info state="configuring" start_time="2013-06-20T13:02:30.000-04:00" expiration_time="2013-06-20T20:50:33.000-04:00" creation_time="2013-06-20T13:02:30.000-04:00" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </link> <ns3:geni_slice_info state="configuring" uuid="0dc532f3-7408-4562-9e29-c0a32ab419df" urn="urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-1"/> </rspec> INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createsliver: Options as run: aggregate: ['uh-eg'] framework: portal project: ln-prj Args: createsliver EG-CT-1 EG-CT-1-uh.rspec Result Summary: Got Reserved resources RSpec from uh-hn-exogeni-net-11443-orca INFO:omni: ============================================================
When the sliver was ready determine login information:
$ readyToLogin.py -a uh-eg EG-CT-1 <....> ================================================================================ LOGIN INFO for AM: https://uh-hn.exogeni.net:11443/orca/xmlrpc ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' User root logs in to VM-1 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.5 User root logs in to VM-2 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.6 User root logs in to VM-3 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.4 User root logs in to VM-4 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.7
Measurements
Login to two of the nodes and collect iperf and ping statistics. All measurements are collected over 60 seconds:
Collected: 2013-06-20
Iperf ExoGENI VM-2 to VM-1 (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 24.0 GBytes 3.43 Gbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 12.3 GBytes 1.76 Gbits/sec [ 5] 0.0-60.0 sec 12.7 GBytes 1.82 Gbits/sec [ 6] 0.0-60.0 sec 12.7 GBytes 1.82 Gbits/sec [ 4] 0.0-60.0 sec 12.8 GBytes 1.83 Gbits/sec [ 7] 0.0-60.0 sec 12.7 GBytes 1.83 Gbits/sec [SUM] 0.0-60.0 sec 63.3 GBytes 9.06 Gbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 6.54 GBytes 936 Mbits/sec [ 3] 0.0-60.0 sec 6.55 GBytes 937 Mbits/sec [ 6] 0.0-60.0 sec 6.56 GBytes 939 Mbits/sec [ 9] 0.0-60.0 sec 6.59 GBytes 944 Mbits/sec [ 8] 0.0-60.0 sec 6.55 GBytes 938 Mbits/sec [ 7] 0.0-60.0 sec 6.55 GBytes 937 Mbits/sec [ 10] 0.0-60.0 sec 6.55 GBytes 938 Mbits/sec [ 11] 0.0-60.0 sec 6.57 GBytes 941 Mbits/sec [ 4] 0.0-60.0 sec 6.55 GBytes 937 Mbits/sec [ 12] 0.0-60.0 sec 6.56 GBytes 939 Mbits/sec [SUM] 0.0-60.0 sec 65.6 GBytes 9.39 Gbits/sec
Iperf ExoGENI VM-2 to the VM-1 (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 21.2 GBytes 3.03 Gbits/sec [ 3] Sent 15473710 datagrams [ 3] Server Report: [ 3] 0.0-60.2 sec 10.1 GBytes 1.43 Gbits/sec 15.728 ms 8130395/15473698 (53
Ping ExoGENI VM-2 to the VM-1
60 packets transmitted, 60 received, 0% packet loss, time 58999ms rtt min/avg/max/mdev = 0.142/0.308/0.568/0.086 ms
EG-CT-2 - Access to New Site bare metal and VM resources
Create a slice:
$ omni.py createslice EG-CT-2 INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Created slice with Name EG-CT-2, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-2, Expiration 2013-07-03 19:50:44 INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createslice: Options as run: framework: portal project: ln-prj Args: createslice EG-CT-2 Result Summary: Created slice with Name EG-CT-2, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-2, Expiration 2013-07-03 19:50:44 INFO:omni: ============================================================
Create a sliver with one VM and one bare metal node using RSpec EG-CT-2-uh.rspec. Note: Bare metal nodes are only available via ExoSM.
$ omni.py createsliver EG-CT-2 -a sm-eg EG-CT-2-uh.rspec INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-2 expires within 1 day on 2013-07-03 19:50:44 UTC INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Creating sliver(s) from rspec file EG-CT-2-uh.rspec for slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-2 INFO:omni:Got return from CreateSliver for slice EG-CT-2 at https://geni.renci.org:11443/orca/xmlrpc: INFO:omni:<?xml version="1.0" encoding="UTF-8" standalone="yes"?> INFO:omni: <!-- Reserved resources for: Slice: EG-CT-2 at AM: URN: unspecified_AM_URN URL: https://geni.renci.org:11443/orca/xmlrpc --> INFO:omni: <rspec type="manifest" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/manifest.xsd http://hpn.east.isi.edu/rspec/ext/stitch/0.1/ http://hpn.east.isi.edu/rspec/ext/stitch/0.1/stitch-schema.xsd http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1/slice_info.xsd?format=raw http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1/sliver_info.xsd?format=raw http://www.geni.net/resources/rspec/ext/postBootScript/1 http://www.geni.net/resources/rspec/ext/postBootScript/1/request.xsd" xmlns:ns2="http://hpn.east.isi.edu/rspec/ext/stitch/0.1/" xmlns="http://www.geni.net/resources/rspec/3" xmlns:ns4="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1" xmlns:ns3="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1" xmlns:ns5="http://www.geni.net/resources/rspec/ext/postBootScript/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+147847e6-460e-4bc0-9d2f-20968efc4c46:BM-1" exclusive="true" component_name="orca-xcat-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-xcat-cloud" client_id="BM-1"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="ExoGENI-M4"> <disk_image version="d1044d9162bd7851e3fc2c57a8251ad6b3641c0c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.8.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:48:5e" client_id="BM-1:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.1"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-07-03T11:51:17.000Z" expiration_time="2013-07-03T19:50:44.000Z" creation_time="2013-07-03T11:51:17.000Z" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <node sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+147847e6-460e-4bc0-9d2f-20968efc4c46:VM-1" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:uhvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:uhvmsite+node+orca-vm-cloud" client_id="VM-1"> <location latitude="29.72327" longitude="-95.34269" country="Unspecified"/> <sliver_type name="m1.small"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services> <ns5:services_post_boot_script type="velocity">#!/bin/bash # Automatically generated boot script execString=&quot;/bin/sh -c \&quot;sudo yum install iperf -y\&quot;&quot; eval $execString </ns5:services_post_boot_script> </services> <interface mac_address="fe:16:3e:00:63:83" client_id="VM-1:if0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.1.2"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-07-03T11:51:17.000Z" expiration_time="2013-07-03T19:50:44.000Z" creation_time="2013-07-03T11:51:17.000Z" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <link vlantag="unknown" sliver_id="urn:publicid:IDN+exogeni.net:uhvmsite+sliver+147847e6-460e-4bc0-9d2f-20968efc4c46:lan0" client_id="lan0"> <interface_ref client_id="BM-1:if0"/> <interface_ref client_id="VM-1:if0"/> <ns4:geni_sliver_info state="configuring" start_time="2013-07-03T11:51:17.000Z" expiration_time="2013-07-03T19:50:44.000Z" creation_time="2013-07-03T11:51:17.000Z" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </link> <ns3:geni_slice_info state="configuring" uuid="ff68c0d0-65a4-4596-a774-06bd47d57072" urn="urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-2"/> </rspec> INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createsliver: Options as run: aggregate: ['sm-eg'] framework: portal project: ln-prj Args: createsliver EG-CT-2 EG-CT-2-uh.rspec Result Summary: Got Reserved resources RSpec from geni-renci-org-11443-orca INFO:omni: ============================================================
When sliver is ready, check for login information:
$ readyToLogin.py EG-CT-2 -a sm-eg <...> User root logs in to BM-1 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.116 User root logs in to VM-1 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.4
Measurements
Collected: 2013-07-03
Iperf ExoGENI BM-1 to VM-1 (TCP)
One Client_
ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 22.7 GBytes 3.25 Gbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 7.05 GBytes 1.01 Gbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-60.0 sec 7.72 GBytes 1.10 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 7.17 GBytes 1.03 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 7.34 GBytes 1.05 Gbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 6.98 GBytes 999 Mbits/sec [SUM] 0.0-60.0 sec 36.3 GBytes 5.19 Gbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 7] 0.0-60.0 sec 3.63 GBytes 519 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 3.68 GBytes 527 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 3.91 GBytes 560 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 0.0-60.0 sec 3.75 GBytes 537 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-60.0 sec 3.74 GBytes 536 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-60.0 sec 3.79 GBytes 543 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-60.0 sec 3.68 GBytes 527 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 3.82 GBytes 547 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 0.0-60.0 sec 3.61 GBytes 516 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 3.78 GBytes 542 Mbits/sec [SUM] 0.0-60.0 sec 37.4 GBytes 5.35 Gbits/sec
Iperf ExoGENI BM-1 to the VM-1 (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0- 0.1 sec 76.1 KBytes 9.99 Mbits/sec [ 3] Sent 53 datagrams read failed: Connection refused [ 3] WARNING: did not receive ack of last datagram after 1 tries.
Ping from ExoGENI BM-1 to the VM-1
60 packets transmitted, 60 received, 0% packet loss, time 59002ms rtt min/avg/max/mdev = 0.195/0.395/3.993/0.486 ms
Collected: 2013-07-XX
Iperf ExoGENI VM-1 to BM-1 (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 47.7 GBytes 6.82 Gbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 12.4 GBytes 1.77 Gbits/sec [ 5] 0.0-60.0 sec 12.1 GBytes 1.73 Gbits/sec [ 6] 0.0-60.0 sec 12.0 GBytes 1.71 Gbits/sec [ 3] 0.0-60.0 sec 11.9 GBytes 1.70 Gbits/sec [ 7] 0.0-60.0 sec 12.3 GBytes 1.76 Gbits/sec [SUM] 0.0-60.0 sec 60.6 GBytes 8.67 Gbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 6.07 GBytes 869 Mbits/sec [ 3] 0.0-60.0 sec 6.06 GBytes 868 Mbits/sec [ 6] 0.0-60.0 sec 6.12 GBytes 876 Mbits/sec [ 7] 0.0-60.0 sec 6.14 GBytes 879 Mbits/sec [ 9] 0.0-60.0 sec 6.12 GBytes 876 Mbits/sec [ 8] 0.0-60.0 sec 6.13 GBytes 878 Mbits/sec [ 10] 0.0-60.0 sec 6.06 GBytes 868 Mbits/sec [ 11] 0.0-60.0 sec 5.99 GBytes 858 Mbits/sec [ 4] 0.0-60.0 sec 6.11 GBytes 875 Mbits/sec [ 12] 0.0-60.0 sec 6.11 GBytes 874 Mbits/sec [SUM] 0.0-60.0 sec 60.9 GBytes 8.72 Gbits/sec
Iperf ExoGENI VM-1 to the BM-1 (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.4 MBytes 9.99 Mbits/sec [ 3] Sent 51022 datagrams read failed: Connection refused [ 3] WARNING: did not receive ack of last datagram after 4 tries.
Ping from ExoGENI VM-1 to the BM-1
60 packets transmitted, 60 received, 0% packet loss, time 58998ms rtt min/avg/max/mdev = 0.198/0.372/4.393/0.527 ms
EG-CT-3 - Multiple sites experiment
The GPO and Houston racks are used in this experiment. First create a slice:
$ omni.py createslice EG-CT-3
Then create a sliver via the ExoSM that includes both GPO and Houston VMs using the RSpec the RSpec EG-CT-3-uh.rspec
$ omni.py createsliver -a sm-eg EG-CT-3 EG-CT-3-uh.rspec INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-3 expires within 1 day on 2013-06-21 19:43:16 UTC INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Substituting AM nickname sm-eg with URL https://geni.renci.org:11443/orca/xmlrpc, URN unspecified_AM_URN INFO:omni:Creating sliver(s) from rspec file EG-CT-3-uh.rspec for slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-3 INFO:omni:Got return from CreateSliver for slice EG-CT-3 at https://geni.renci.org:11443/orca/xmlrpc: INFO:omni:<?xml version="1.0" encoding="UTF-8" standalone="yes"?> INFO:omni: <!-- Reserved resources for: Slice: EG-CT-3 at AM: URN: unspecified_AM_URN URL: https://geni.renci.org:11443/orca/xmlrpc --> INFO:omni: <rspec type="manifest" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/manifest.xsd http://hpn.east.isi.edu/rspec/ext/stitch/0.1/ http://hpn.east.isi.edu/rspec/ext/stitch/0.1/stitch-schema.xsd http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1/slice_info.xsd?format=raw http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1 http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1/sliver_info.xsd?format=raw" xmlns:ns2="http://hpn.east.isi.edu/rspec/ext/stitch/0.1/" xmlns="http://www.geni.net/resources/rspec/3" xmlns:ns4="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/sliver-info/1" xmlns:ns3="http://groups.geni.net/exogeni/attachment/wiki/RspecExtensions/slice-info/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <node sliver_id="urn:publicid:IDN+exogeni.net:fiuvmsite+sliver+2ee8e1b2-876d-474c-a2f1-9ecbdf3fcef0:uh" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:fiuvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:fiuvmsite+node+orca-vm-cloud" client_id="uh"> <location latitude="25.757292" longitude="-80.37404" country="Unspecified"/> <sliver_type name="m1.large"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:66:c2" client_id="uh:0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.2.2"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-21T11:44:09.000Z" expiration_time="2013-06-21T19:43:16.000Z" creation_time="2013-06-21T11:44:09.000Z" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <node sliver_id="urn:publicid:IDN+exogeni.net:bbnvmsite+sliver+2ee8e1b2-876d-474c-a2f1-9ecbdf3fcef0:gpo" exclusive="false" component_name="orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:bbnvmsite+authority+am" component_id="urn:publicid:IDN+exogeni.net:bbnvmsite+node+orca-vm-cloud" client_id="gpo"> <location latitude="42.38974" longitude="-71.14751" country="Unspecified"/> <sliver_type name="m1.large"> <disk_image version="64ad567ce3b1c0dbaa15bad673bbf556a9593e1c" name="http://geni-images.renci.org/images/standard/debian/deb6-neuca-v1.0.6.xml"/> </sliver_type> <services/> <interface mac_address="fe:16:3e:00:47:f8" client_id="gpo:0"> <ip type="ipv4" netmask="255.255.255.0" address="172.16.2.1"/> </interface> <ns4:geni_sliver_info state="configuring" start_time="2013-06-21T11:44:09.000Z" expiration_time="2013-06-21T19:43:16.000Z" creation_time="2013-06-21T11:44:09.000Z" creator_urn="lnevers@bbn.com, urn:publicid:IDN+ch.geni.net+user+lnevers"/> </node> <link vlantag="unknown" sliver_id="urn:publicid:IDN+exogeni.net+sliver+2ee8e1b2-876d-474c-a2f1-9ecbdf3fcef0:center" client_id="center"> <interface_ref client_id="uh:0"/> <interface_ref client_id="gpo:0"/> </link> <ns2:stitching lastUpdateTime="Fri Jun 21 11:44:34 UTC 2013"> <ns2:path id="urn:publicid:IDN+exogeni.net+sliver+2ee8e1b2-876d-474c-a2f1-9ecbdf3fcef0:center"> <ns2:hop type="strict" id="hop1"> <ns2:link id="urn:publicid:IDN+exogeni.net:fiuNet+interface+fiuNet:Domain:vlan:61d5c4fb-5250-41a5-b208-d8d12634bac1:vlan"> <ns2:remoteLinkId>urn:publicid:IDN+exogeni.net:nlr+interface+nlr:Domain:vlan:a1fd5893-8e5b-4ff4-845e-f1b8d4120e97:vlan</ns2:remoteLinkId> <ns2:trafficEngineeringMetric>unknown</ns2:trafficEngineeringMetric> <ns2:capacity>0</ns2:capacity> <ns2:maximumReservableCapacity>0</ns2:maximumReservableCapacity> <ns2:minimumReservableCapacity>10000000</ns2:minimumReservableCapacity> <ns2:granularity>10000000</ns2:granularity> <ns2:unreservedCapacity>unknown</ns2:unreservedCapacity> <ns2:switchingCapabilityDescriptor> <ns2:switchingcapType>l2sc</ns2:switchingcapType> <ns2:encodingType>ethernet</ns2:encodingType> <ns2:switchingCapabilitySpecificInfo> <ns2:switchingCapabilitySpecificInfo_L2sc> <ns2:vlanRangeAvailability>unknown</ns2:vlanRangeAvailability> </ns2:switchingCapabilitySpecificInfo_L2sc> </ns2:switchingCapabilitySpecificInfo> </ns2:switchingCapabilityDescriptor> </ns2:link> <ns2:nextHop>hop2</ns2:nextHop> </ns2:hop> <ns2:hop type="strict" id="hop2"> <ns2:link id="urn:publicid:IDN+exogeni.net:nlr+interface+nlr:Domain:vlan:a1fd5893-8e5b-4ff4-845e-f1b8d4120e97:vlan"> <ns2:remoteLinkId>urn:publicid:IDN+exogeni.net:bbnNet+interface+bbnNet:Domain:vlan:1af82f62-458d-41c4-a3da-b6d079ac6f3b:vlan</ns2:remoteLinkId> <ns2:trafficEngineeringMetric>unknown</ns2:trafficEngineeringMetric> <ns2:capacity>0</ns2:capacity> <ns2:maximumReservableCapacity>0</ns2:maximumReservableCapacity> <ns2:minimumReservableCapacity>10000000</ns2:minimumReservableCapacity> <ns2:granularity>10000000</ns2:granularity> <ns2:unreservedCapacity>unknown</ns2:unreservedCapacity> <ns2:switchingCapabilityDescriptor> <ns2:switchingcapType>l2sc</ns2:switchingcapType> <ns2:encodingType>ethernet</ns2:encodingType> <ns2:switchingCapabilitySpecificInfo> <ns2:switchingCapabilitySpecificInfo_L2sc> <ns2:vlanRangeAvailability>unknown</ns2:vlanRangeAvailability> </ns2:switchingCapabilitySpecificInfo_L2sc> </ns2:switchingCapabilitySpecificInfo> </ns2:switchingCapabilityDescriptor> </ns2:link> <ns2:nextHop>hop3</ns2:nextHop> </ns2:hop> <ns2:hop type="strict" id="hop3"> <ns2:link id="urn:publicid:IDN+exogeni.net:bbnNet+interface+bbnNet:Domain:vlan:1af82f62-458d-41c4-a3da-b6d079ac6f3b:vlan"> <ns2:remoteLinkId>unknown</ns2:remoteLinkId> <ns2:trafficEngineeringMetric>unknown</ns2:trafficEngineeringMetric> <ns2:capacity>0</ns2:capacity> <ns2:maximumReservableCapacity>0</ns2:maximumReservableCapacity> <ns2:minimumReservableCapacity>10000000</ns2:minimumReservableCapacity> <ns2:granularity>10000000</ns2:granularity> <ns2:unreservedCapacity>unknown</ns2:unreservedCapacity> <ns2:switchingCapabilityDescriptor> <ns2:switchingcapType>l2sc</ns2:switchingcapType> <ns2:encodingType>ethernet</ns2:encodingType> <ns2:switchingCapabilitySpecificInfo> <ns2:switchingCapabilitySpecificInfo_L2sc> <ns2:vlanRangeAvailability>unknown</ns2:vlanRangeAvailability> </ns2:switchingCapabilitySpecificInfo_L2sc> </ns2:switchingCapabilitySpecificInfo> </ns2:switchingCapabilityDescriptor> </ns2:link> <ns2:nextHop>null</ns2:nextHop> </ns2:hop> </ns2:path> </ns2:stitching> <ns3:geni_slice_info state="configuring" uuid="e053f6e4-743e-4498-b550-1e0a7ae44a31" urn="urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-3"/> </rspec> INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createsliver: Options as run: aggregate: ['sm-eg'] framework: portal project: ln-prj Args: createsliver EG-CT-3 EG-CT-3-uh.rspec Result Summary: Got Reserved resources RSpec from geni-renci-org-11443-orca INFO:omni: ============================================================
Determine login information for allocated nodes:
$ readyToLogin.py -a sm-eg EG-CT-3 <...> ================================================================================ LOGIN INFO for AM: https://geni.renci.org:11443/orca/xmlrpc ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' User root logs in to uh using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@131.94.144.40 User root logs in to gpo using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@192.1.242.10
Measurements
Collected: 2013-06-21
Iperf ExoGENI GPO VM to Houston VM (TCP) - TCP window size: 16.0 KB
One Client_
Five Clients
Ten Clients
Iperf ExoGENI GPO VM to Houston VM (UDP) - 1470 byte datagrams & UDP buffer size: 136 KByte
Ping from GPO VM to Houston VM
EG-CT-4 - Multiple sites OpenFlow experiment and interoperability
First create a slice:
$ omni.py createslice EG-CT-4 INFO:omni:Loading config file /home/lnevers/.gcf/omni_config INFO:omni:Using control framework portal INFO:omni:Created slice with Name EG-CT-4, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-4, Expiration 2013-06-21 20:02:19 INFO:omni: ------------------------------------------------------------ INFO:omni: Completed createslice: Options as run: framework: portal project: ln-prj Args: createslice EG-CT-4 Result Summary: Created slice with Name EG-CT-4, URN urn:publicid:IDN+ch.geni.net:ln-prj+slice+EG-CT-4, Expiration 2013-06-21 20:02:19 INFO:omni: ============================================================
Then create slivers at all OpenFlow aggregates. This confirmation test creates a sliver at each of the following aggregates using the RSpec specified. RSpecs can be found [here
$ omni.py createsliver -a of-nlr EG-CT-4 EG-CT-4-openflow-nlr.rspec -V1 $ omni.py createsliver -a of-rutgers EG-CT-4 EG-CT-4-openflow-rutgers.rspec -V1 $ omni.py createsliver -a of-i2 EG-CT-4 EG-CT-4-openflow-i2.rspec -V1 $ omni.py createsliver -a of-gpo EG-CT-4 EG-CT-4-openflow-gpo.rspec -V1 $ omni.py createsliver -a of-gpo-eg EG-CT-4 EG-CT-4-openflow-eg-gpo.rspec -V1 $ omni.py createsliver -a of-uen EG-CT-4 EG-CT-4-openflow-uen.rspec -V1 $ omni.py createsliver -a of-utah-ig EG-CT-4 EG-CT-4-openflow-ig-utah.rspec -V1 $ omni.py createsliver -a of-uh-eg EG-CT-4 EG-CT-4-openflow-eg-uh.rspec -V1 $ omni.py createsliver -a of-gpo-ig EG-CT-4 EG-CT-4-openflow-ig-gpo.rspec -V1
Then create a sliver at each of the compute resource aggregates, RSpecs used can be found here
$ omni.py createsliver -a pg-utah EG-CT-4 EG-CT-4-rutgers-wapg-pg-utah.rspec $ omni.py createsliver -a utah-ig EG-CT-4 EG-CT-4-ig-utah.rspec $ omni.py createsliver -a gpo-ig EG-CT-4 EG-CT-4-ig-gpo.rspec $ omni.py createsliver -a gpo-eg EG-CT-4 EG-CT-4-eg-gpo.rspec $ omni.py createsliver -a uh-eg EG-CT-4 EG-CT-4-eg-uh.rspec
Once the slivers were ready, determined login for PG and Indiana hosts on shared VLAN 1750:
$ readyToLogin.py -a pg-utah EG-CT-4 <...> ================================================================================ LOGIN INFO for AM: https://www.emulab.net:12369/protogeni/xmlrpc/am/2.0 ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' utah-pg's geni_status is: ready (am_status:ready) User lnevers logs in to utah-pg using: ssh -p 34106 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc533.emulab.net User lnevers logs in to wapg-rutgers-pg51 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pg51.emulab.net
Determined login for Utah InstaGENI:
$ readyToLogin.py -a utah-ig EG-CT-4 <...> ================================================================================ LOGIN INFO for AM: https://utah.geniracks.net:12369/protogeni/xmlrpc/am/2.0 ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' utah-ig's geni_status is: ready (am_status:ready) User lnevers logs in to utah-ig using: ssh -p 30010 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc1.utah.geniracks.net User lnevers logs in to utah-ig2 using: ssh -p 30011 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc1.utah.geniracks.net
Determined login for GPO InstaGENI:
$ readyToLogin.py -a gpo-ig EG-CT-4 <...> ================================================================================ LOGIN INFO for AM: https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0 ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' gpo-ig's geni_status is: ready (am_status:ready) User lnevers logs in to gpo-ig using: ssh -p 31802 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com User lnevers logs in to gpo-ig2 using: ssh -p 31803 -i /home/lnevers/.ssh/geni_cert_portal_key lnevers@pc2.instageni.gpolab.bbn.com
Determined login for GPO ExoGENI:
$ readyToLogin.py -a gpo-eg EG-CT-4 <...> ================================================================================ LOGIN INFO for AM: https://bbn-hn.exogeni.net:11443/orca/xmlrpc ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' User root logs in to gpo-vm using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@192.1.242.11
Determined login for Houston ExoGENI:
$ readyToLogin.py -a uh-eg EG-CT-4 <...> ================================================================================ LOGIN INFO for AM: https://uh-hn.exogeni.net:11443/orca/xmlrpc ================================================================================ For more login info, see the section entitled: 'Providing a private key to ssh' in 'readyToLogin.py -h' User root logs in to uh-vm using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.5 User root logs in to uh-vm1 using: ssh -i /home/lnevers/.ssh/geni_cert_portal_key root@129.7.98.4
Measurements
This section captures measurements collected between the following endpoints:
- Houston ExoGENI VM to Houston ExoGENI VM
- Houston ExoGENI VM to GPO ExoGENI VM
- Houston ExoGENI VM to GPO InstaGENI VM
- Houston ExoGENI VM to Utah InstaGENI VM
- Houston ExoGENI VM to Utah PG VM
- Houston ExoGENI VM to Rutgers WAPG
- The measurements collected for Houston ExoGENI VM to Houston ExoGENI VM
Iperf Houston ExoGENI VM to Houston ExoGENI VM (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 29.2 GBytes 4.19 Gbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 12.0 GBytes 1.71 Gbits/sec [ 5] 0.0-60.0 sec 12.3 GBytes 1.76 Gbits/sec [ 6] 0.0-60.0 sec 12.4 GBytes 1.77 Gbits/sec [ 4] 0.0-60.0 sec 12.5 GBytes 1.79 Gbits/sec [ 7] 0.0-60.0 sec 12.4 GBytes 1.77 Gbits/sec [SUM] 0.0-60.0 sec 61.5 GBytes 8.81 Gbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 6.52 GBytes 933 Mbits/sec [ 6] 0.0-60.0 sec 6.46 GBytes 924 Mbits/sec [ 3] 0.0-60.0 sec 6.49 GBytes 929 Mbits/sec [ 7] 0.0-60.0 sec 6.49 GBytes 929 Mbits/sec [ 9] 0.0-60.0 sec 6.50 GBytes 931 Mbits/sec [ 8] 0.0-60.0 sec 6.49 GBytes 929 Mbits/sec [ 10] 0.0-60.0 sec 6.52 GBytes 933 Mbits/sec [ 11] 0.0-60.0 sec 6.50 GBytes 930 Mbits/sec [ 4] 0.0-60.0 sec 6.50 GBytes 931 Mbits/sec [ 12] 0.0-60.0 sec 6.51 GBytes 932 Mbits/sec [SUM] 0.0-60.0 sec 65.0 GBytes 9.30 Gbits/sec
Iperf Houston ExoGENI VM to Houston ExoGENI VM(UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-20.0 sec 5.41 GBytes 2.32 Gbits/sec [ 3] Sent 3953948 datagrams [ 3] Server Report: [ 3] 0.0-20.0 sec 5.40 GBytes 2.32 Gbits/sec 0.006 ms 11548/3953947 (0.29%) [ 3] 0.0-20.0 sec 10 datagrams received out-of-order
Ping from Houston ExoGENI VM to GPO ExoGENI VM
60 packets transmitted, 60 received, 0% packet loss, time 58998ms rtt min/avg/max/mdev = 0.045/0.310/0.516/0.102 ms
- The measurements collected for Houston ExoGENI VM to GPO ExoGENI VM:
Iperf Houston ExoGENI VM to GPO ExoGENI VM (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 84.8 MBytes 11.9 Mbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 83.2 MBytes 11.6 Mbits/sec [ 5] 0.0-60.0 sec 83.4 MBytes 11.7 Mbits/sec [ 6] 0.0-60.0 sec 85.1 MBytes 11.9 Mbits/sec [ 3] 0.0-60.0 sec 85.0 MBytes 11.9 Mbits/sec [ 7] 0.0-60.1 sec 85.9 MBytes 12.0 Mbits/sec [SUM] 0.0-60.1 sec 423 MBytes 59.1 Mbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 86.5 MBytes 12.1 Mbits/sec [ 12] 0.0-60.0 sec 84.5 MBytes 11.8 Mbits/sec [ 8] 0.0-60.0 sec 86.3 MBytes 12.1 Mbits/sec [ 10] 0.0-60.0 sec 84.7 MBytes 11.8 Mbits/sec [ 11] 0.0-60.0 sec 79.4 MBytes 11.1 Mbits/sec [ 7] 0.0-60.1 sec 78.8 MBytes 11.0 Mbits/sec [ 3] 0.0-60.1 sec 82.5 MBytes 11.5 Mbits/sec [ 5] 0.0-60.1 sec 86.0 MBytes 12.0 Mbits/sec [ 6] 0.0-60.1 sec 77.0 MBytes 10.8 Mbits/sec [ 9] 0.0-60.1 sec 86.6 MBytes 12.1 Mbits/sec [SUM] 0.0-60.1 sec 832 MBytes 116 Mbits/sec
Iperf Houston ExoGENI VM to GPO ExoGENI VM(UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 3] Sent 51022 datagrams [ 3] Server Report: [ 3] 0.0-59.3 sec 71.4 MBytes 10.1 Mbits/sec 0.014 ms 92/51021 (0.18%) [ 3] 0.0-59.3 sec 464 datagrams received out-of-order
Ping from Houston ExoGENI VM to GPO ExoGENI VM
60 packets transmitted, 60 received, 0% packet loss, time 59106ms rtt min/avg/max/mdev = 77.580/111.407/1575.823/202.119 ms, pipe 2
- The measurements collected for Houston ExoGENI VM and GPO InstaGENI VM:
Iperf Houston ExoGENI VM to GPO InstaGENI VM (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 114 MBytes 15.9 Mbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 7] 0.0-60.0 sec 97.0 MBytes 13.6 Mbits/sec [ 5] 0.0-60.0 sec 109 MBytes 15.2 Mbits/sec [ 4] 0.0-60.0 sec 92.6 MBytes 12.9 Mbits/sec [ 3] 0.0-60.1 sec 108 MBytes 15.2 Mbits/sec [ 6] 0.0-60.1 sec 89.0 MBytes 12.4 Mbits/sec [SUM] 0.0-60.1 sec 496 MBytes 69.2 Mbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 80.9 MBytes 11.3 Mbits/sec [ 5] 0.0-60.0 sec 91.4 MBytes 12.8 Mbits/sec [ 9] 0.0-60.0 sec 95.0 MBytes 13.3 Mbits/sec [ 8] 0.0-60.0 sec 72.5 MBytes 10.1 Mbits/sec [ 10] 0.0-60.0 sec 87.7 MBytes 12.2 Mbits/sec [ 3] 0.0-60.0 sec 71.8 MBytes 10.0 Mbits/sec [ 6] 0.0-60.0 sec 69.4 MBytes 9.69 Mbits/sec [ 12] 0.0-60.1 sec 101 MBytes 14.1 Mbits/sec [ 11] 0.0-60.1 sec 73.9 MBytes 10.3 Mbits/sec [ 7] 0.0-60.1 sec 85.2 MBytes 11.9 Mbits/sec [SUM] 0.0-60.1 sec 828 MBytes 116 Mbits/sec
Iperf Houston ExoGENI VM to GPO InstaGENI VM (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 3] Sent 51022 datagrams [ 3] Server Report: [ 3] 0.0-59.4 sec 71.3 MBytes 10.1 Mbits/sec 0.008 ms 146/51022 (0.29%) [ 3] 0.0-59.4 sec 362 datagrams received out-of-order
Ping from Houston ExoGENI VM to GPO InstaGENI VM
60 packets transmitted, 60 received, 0% packet loss, time 59086ms rtt min/avg/max/mdev = 77.346/104.167/1361.289/168.596 ms, pipe 2
- The measurements collected for Houston ExoGENI VM and Utah InstaGENI VM:
Iperf Houston ExoGENI VM to Utah InstaGENI VM (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.2 sec 19.0 MBytes 2.65 Mbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 6] 0.0-60.0 sec 22.9 MBytes 3.20 Mbits/sec [ 7] 0.0-60.0 sec 27.6 MBytes 3.86 Mbits/sec [ 4] 0.0-60.2 sec 27.9 MBytes 3.89 Mbits/sec [ 5] 0.0-60.3 sec 27.3 MBytes 3.79 Mbits/sec [ 3] 0.0-60.7 sec 16.8 MBytes 2.31 Mbits/sec [SUM] 0.0-60.7 sec 122 MBytes 16.9 Mbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 21.9 MBytes 3.05 Mbits/sec [ 10] 0.0-60.1 sec 26.5 MBytes 3.70 Mbits/sec [ 3] 0.0-60.1 sec 23.3 MBytes 3.26 Mbits/sec [ 12] 0.0-60.1 sec 20.8 MBytes 2.90 Mbits/sec [ 7] 0.0-60.1 sec 22.1 MBytes 3.09 Mbits/sec [ 8] 0.0-60.1 sec 15.1 MBytes 2.11 Mbits/sec [ 9] 0.0-60.1 sec 15.9 MBytes 2.22 Mbits/sec [ 5] 0.0-60.1 sec 25.7 MBytes 3.59 Mbits/sec [ 6] 0.0-60.2 sec 17.6 MBytes 2.45 Mbits/sec [ 11] 0.0-62.8 sec 22.9 MBytes 3.06 Mbits/sec [SUM] 0.0-62.8 sec 212 MBytes 28.3 Mbits/sec
Iperf Houston ExoGENI VM to Utah InstaGENI VM (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 3] Sent 51022 datagrams [ 3] Server Report: [ 3] 0.0-58.6 sec 70.1 MBytes 10.0 Mbits/sec 0.016 ms 1019/51021 (2%) [ 3] 0.0-58.6 sec 170 datagrams received out-of-order
Ping from Houston ExoGENI VM to Utah InstaGENI VM
60 packets transmitted, 60 received, 0% packet loss, time 59023ms rtt min/avg/max/mdev = 243.190/309.075/2549.395/340.654 ms, pipe 3
- The measurements collected for Houston ExoGENI VM and Utah PG VM:
Iperf Houston ExoGENI VM to Utah PG VM (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.1 sec 29.1 MBytes 4.06 Mbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 27.9 MBytes 3.90 Mbits/sec [ 6] 0.0-60.1 sec 26.1 MBytes 3.64 Mbits/sec [ 5] 0.0-60.1 sec 25.4 MBytes 3.55 Mbits/sec [ 7] 0.0-60.1 sec 24.4 MBytes 3.41 Mbits/sec [ 3] 0.0-60.2 sec 16.5 MBytes 2.30 Mbits/sec [SUM] 0.0-60.2 sec 120 MBytes 16.8 Mbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 9] 0.0-60.0 sec 19.1 MBytes 2.67 Mbits/sec [ 8] 0.0-60.0 sec 17.9 MBytes 2.50 Mbits/sec [ 4] 0.0-60.0 sec 18.7 MBytes 2.62 Mbits/sec [ 7] 0.0-60.1 sec 21.7 MBytes 3.02 Mbits/sec [ 5] 0.0-60.1 sec 23.6 MBytes 3.30 Mbits/sec [ 3] 0.0-60.2 sec 14.0 MBytes 1.95 Mbits/sec [ 6] 0.0-60.2 sec 21.1 MBytes 2.94 Mbits/sec [ 10] 0.0-60.4 sec 12.5 MBytes 1.73 Mbits/sec [ 12] 0.0-60.4 sec 18.2 MBytes 2.53 Mbits/sec [ 11] 0.0-60.4 sec 11.0 MBytes 1.53 Mbits/sec [SUM] 0.0-60.4 sec 178 MBytes 24.7 Mbits/sec
Iperf Houston ExoGENI VM to Utah PG VM (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 3] Sent 51022 datagrams [ 3] Server Report: [ 3] 0.0-58.7 sec 70.1 MBytes 10.0 Mbits/sec 0.014 ms 982/51021 (1.9%) [ 3] 0.0-58.7 sec 140 datagrams received out-of-order
Ping from Houston ExoGENI VM to Utah PG VM
60 packets transmitted, 60 received, 0% packet loss, time 59078ms rtt min/avg/max/mdev = 243.153/304.423/2415.427/315.445 ms, pipe 3
- The measurements collected for Houston ExoGENI VM to Rutgers WAPG:
Iperf Houston ExoGENI VM to the Rutgers WAPG (TCP)
One Client_
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.1 sec 62.2 MBytes 8.69 Mbits/sec
Five Clients
[ ID] Interval Transfer Bandwidth [ 5] 0.0-60.0 sec 59.3 MBytes 8.29 Mbits/sec [ 6] 0.0-60.0 sec 53.9 MBytes 7.52 Mbits/sec [ 3] 0.0-60.1 sec 52.3 MBytes 7.29 Mbits/sec [ 7] 0.0-60.1 sec 55.5 MBytes 7.74 Mbits/sec [ 4] 0.0-60.3 sec 56.9 MBytes 7.92 Mbits/sec [SUM] 0.0-60.3 sec 278 MBytes 38.7 Mbits/sec
Ten Clients
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 52.3 MBytes 7.31 Mbits/sec [ 8] 0.0-60.0 sec 47.6 MBytes 6.65 Mbits/sec [ 6] 0.0-60.1 sec 51.6 MBytes 7.21 Mbits/sec [ 11] 0.0-60.1 sec 45.3 MBytes 6.32 Mbits/sec [ 7] 0.0-60.1 sec 48.4 MBytes 6.75 Mbits/sec [ 9] 0.0-60.1 sec 52.5 MBytes 7.33 Mbits/sec [ 4] 0.0-60.1 sec 54.2 MBytes 7.56 Mbits/sec [ 5] 0.0-60.1 sec 49.3 MBytes 6.88 Mbits/sec [ 12] 0.0-60.1 sec 43.5 MBytes 6.07 Mbits/sec [ 10] 0.0-60.7 sec 47.0 MBytes 6.49 Mbits/sec [SUM] 0.0-60.7 sec 492 MBytes 68.0 Mbits/sec
Iperf Houston ExoGENI VM to the Rutgers WAPG (UDP)
[ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 3] Sent 51022 datagrams [ 3] Server Report: [ 3] 0.0-59.2 sec 71.1 MBytes 10.1 Mbits/sec 0.006 ms 283/51021 (0.55%) [ 3] 0.0-59.2 sec 408 datagrams received out-of-order
Ping from Houston ExoGENI VM to Rutgers WAPG
60 packets transmitted, 60 received, 0% packet loss, time 59093ms rtt min/avg/max/mdev = 157.507/201.241/1969.577/252.321 ms, pipe 2
EG-CT-5 - Experiment Monitoring
Reviewed content of the GMOC Monitoring page for aggregates and Houston ExoGENI compute aggregate was not found.
EG-CT-6 - Administrative Tests
Administrator accounts on an ExoGENI rack are documented at https://wiki.exogeni.net/doku.php?id=public:operators:start, with https://wiki.exogeni.net/doku.php?id=public:operators:start#authentication_authorization proving insight into accounts creation and usage. With requested account accessed rack head node and verified root access and group membership:
LNM:~$ ssh uh-hn.exogeni.net lnevers@uh-hn.exogeni.net's password: Creating home directory for lnevers. |-----------------------------------------------------------------| | ____ ____ ____ ____ ____ ____ ____ | | ||E |||x |||o |||G |||E |||N |||I || | | ||__|||__|||__|||__|||__|||__|||__|| | | |/__\|/__\|/__\|/__\|/__\|/__\|/__\| | | | |-----------------------------------------------------------------| [lnevers@uh-hn ~]$ sudo whoami [sudo] password for lnevers: root [lnevers@uh-hn ~]$
From head node verified login and administrative access to each of the worker nodes that supply VMs. For each worker node execute the following:
[lnevers@uh-hn ~]$ for i in 1 2 3 4 5 6 7 8 9 10; do sudo ssh root@uh-w$i "echo -n 'Executing on: ' ; hostname;whoami"; done [sudo] password for lnevers: Executing on: uh-w1 root Executing on: uh-w2 root Executing on: uh-w3 root Executing on: uh-w4 root Executing on: uh-w5 root Executing on: uh-w6 root Executing on: uh-w7 root Executing on: uh-w8 root Executing on: uh-w9 root Executing on: uh-w10 root [lnevers@uh-hn ~]$
Note: No longer need to specify identity as described in exoticket:171.
Note: Node uh-w9 and uh-10 are bare metal and were down because not in use, so had to create a sliver to get the nodes up and be able to login to them.
Connect to the management switch: Note: exoticket 189
[lnevers@uh-hn ~]$ ssh uh-8052.uh.xo >ena #show interface status
Connect to the OpenFlow switch: Note: exoticket 189
ssh lnevers@uh-8264.uh.xo >ena #show interface status #show version brief
Verify FOAM and FlowVisor configuration files ownership and paths:
[lnevers@uh-hn ~]$ ls -l /etc/foam.passwd /etc/flowvisor.passwd /etc/flowvisor/fvpasswd /opt/foam/etc/foampasswd lrwxrwxrwx 1 root flowvisor 21 Jan 31 03:45 /etc/flowvisor/fvpasswd -> /etc/flowvisor.passwd -r--r----- 1 flowvisor uhadmins 25 Jan 31 03:45 /etc/flowvisor.passwd -r--r----- 1 root uhadmins 25 Jan 31 03:45 /etc/foam.passwd lrwxrwxrwx 1 root root 16 Jan 31 03:45 /opt/foam/etc/foampasswd -> /etc/foam.passwd
Check FOAM version and FOAM configuration for site.admin.email, geni.site-tag, email.from settings on the Houston head node:
[lnevers@uh-hn ~]$ foamctl admin:get-version --passwd-file=/etc/foam.passwd { "version": "0.10.5" } [lnevers@uh-hn ~]$ foamctl config:get-value --key="site.admin.email" --passwd-file=/opt/foam/etc/foampasswd { "value": "exogeni-foam-admin@gpolab.bbn.com" } [lnevers@uh-hn ~]$ foamctl config:get-value --key="geni.site-tag" --passwd-file=/opt/foam/etc/foampasswd { "value": "uh-hn.exogeni.net" } [lnevers@uh-hn ~]$ foamctl config:get-value --key="email.from" --passwd-file=/opt/foam/etc/foampasswd { "value": "exogeni-foam-admin@gpolab.bbn.com" } [lnevers@uh-hn ~]$ foamctl config:get-value --key="geni.approval.approve-on-creation" --passwd-file=/opt/foam/etc/foampasswd { "value": 0 }
Show FOAM slivers details:
[lnevers@uh-hn ~]$ foamctl geni:list-slivers --passwd-file=/opt/foam/etc/foampasswd { "slivers": [ { "status": "approved", "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755:2dc6effd-a00e-4821-a039-aab8ed92cd2a", "creation": "2013-04-25 15:31:49.390627+00:00", "pend_reason": "Request has underspecified VLAN requests", "expiration": "2013-07-15 00:00:00+00:00", "deleted": "False", "user": "urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty", "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tupty1755", "enabled": true, "email": "tupty@bbn.com", "flowvisor_slice": "2dc6effd-a00e-4821-a039-aab8ed92cd2a", "desc": "tupty1755 OpenFlow resources at UH", "ref": null, "id": 1, "uuid": "2dc6effd-a00e-4821-a039-aab8ed92cd2a" }, { "status": "approved", "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon:1ab2f1fa-1936-4ebf-ac76-248de4e87fc6", "creation": "2013-06-04 13:38:21.853310+00:00", "pend_reason": "Request has underspecified VLAN requests", "expiration": "2013-07-15 00:00:00+00:00", "deleted": "False", "user": "urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty", "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptymon", "enabled": true, "email": "tupty@bbn.com", "flowvisor_slice": "1ab2f1fa-1936-4ebf-ac76-248de4e87fc6", "desc": "tuptymon OpenFlow resources at UH", "ref": null, "id": 2, "uuid": "1ab2f1fa-1936-4ebf-ac76-248de4e87fc6" }, { "status": "approved", "sliver_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptyexclusive:d3ceaacb-61b1-4917-aec8-5a8a92732db7", "creation": "2013-06-05 20:22:16.681591+00:00", "pend_reason": "Request has underspecified VLAN requests", "expiration": "2013-07-15 00:00:00+00:00", "deleted": "False", "user": "urn:publicid:IDN+pgeni.gpolab.bbn.com+user+tupty", "slice_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+tuptyexclusive", "enabled": true, "email": "tupty@bbn.com", "flowvisor_slice": "d3ceaacb-61b1-4917-aec8-5a8a92732db7", "desc": "tuptyexclusive OpenFlow resources at UH ExoGENI.", "ref": null, "id": 6, "uuid": "d3ceaacb-61b1-4917-aec8-5a8a92732db7" } ] } [lnevers@uh-hn ~]$
Check the FlowVisor version, list of devices, get details for a device, list of active slices, and details for one of the slices on the Houston Head node:
[lnevers@uh-hn ~]$ /opt/flowvisor/bin/fvctl --passwd-file=/etc/flowvisor/fvpasswd ping hello Got reply: PONG(fvadmin): FV version=flowvisor-0.8.1::hello [lnevers@uh-hn ~]$ /opt/flowvisor/bin/fvctl --passwd-file=/etc/flowvisor/fvpasswd listDevices Device 0: 00:01:74:99:75:12:fe:00 [lnevers@uh-hn ~]$ /opt/flowvisor/bin/fvctl --passwd-file=/etc/flowvisor/fvpasswd getDeviceInfo 00:01:74:99:75:12:fe:00 nPorts=21 portList=41,42,43,44,45,46,47,48,51,52,53,54,55,56,57,58,59,60,64,50,49 dpid=00:01:74:99:75:12:fe:00 remote=/192.168.106.10:6633-->/192.168.106.4:51714 portNames=41(41),42(42),43(43),44(44),45(45),46(46),47(47),48(48),51(51),52(52),53(53),54(54),55(55),56(56),57(57),58(58),59(59),60(60),64(64),50(50),49(49) [lnevers@uh-hn ~]$ /opt/flowvisor/bin/fvctl --passwd-file=/etc/flowvisor/fvpasswd listSlices Slice 0: orca-4 Slice 1: fvadmin Slice 2: orca-3 Slice 3: orca-2 Slice 4: d3ceaacb-61b1-4917-aec8-5a8a92732db7 Slice 5: 2dc6effd-a00e-4821-a039-aab8ed92cd2a Slice 6: orca-861 Slice 7: orca-863 Slice 8: orca-862 Slice 9: 1ab2f1fa-1936-4ebf-ac76-248de4e87fc6 [lnevers@uh-hn ~]$ /opt/flowvisor/bin/fvctl --passwd-file=/etc/flowvisor/fvpasswd getSliceInfo 1ab2f1fa-1936-4ebf-ac76-248de4e87fc6 Got reply: connection_1=00:01:74:99:75:12:fe:00-->/129.7.98.2:42604-->naxos.gpolab.bbn.com/192.1.249.133:31750 contact_email=tupty@bbn.com controller_hostname=naxos.gpolab.bbn.com controller_port=31750 creator=fvadmin [lnevers@uh-hn ~]$
Verify alerts for the compute resource Aggregate Manager and FOAM Aggregate Manager are being reported to the GPO Tango GENI Nagios monitoring and that all alerts have status OK.
Email help@geni.net for GENI support or email me with feedback on this page!
Attachments (11)
- Houston-nagios.jpg (744.9 KB) - added by 11 years ago.
- EG-CT-5-uh-aggregate-detail.jpg (158.4 KB) - added by 11 years ago.
- EG-CT-5-uh-aggregate.jpg (137.1 KB) - added by 11 years ago.
- EG-CT-5-uh-foam.jpg (137.9 KB) - added by 11 years ago.
- EG-CT-5-uh-measure.jpg (94.5 KB) - added by 11 years ago.
- EG-CT-5-uh-OFmeasure.jpg (128.2 KB) - added by 11 years ago.
- EG-CT-5-uh-OFmeasureSliver.jpg (580.7 KB) - added by 11 years ago.
- EG-CT-5-uh-OFresources.jpg (171.7 KB) - added by 11 years ago.
- EG-CT-5-uh-OFsliver.jpg (833.6 KB) - added by 11 years ago.
- EG-CT-5-uh-resources.jpg (100.7 KB) - added by 11 years ago.
- EG-CT-5-uh-sliver.jpg (92.4 KB) - added by 11 years ago.