wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus/EG-EXP-4

Version 37 (modified by lnevers@bbn.com, 11 years ago) (diff)

--

EG-EXP-4: ExoGENI Multi-site Acceptance Test

This page captures status for the test case EG-EXP-4, which verifies the ability to support basic operations of VMs and data flows between two racks. For overall status see the ExoGENI Acceptance Test Status page.

Last update: 10/17/12

Test Status

This section captures the status for each step in the acceptance test plan.

Step State Date completed Ticket Comments
Step 1 Color(yellow,Complete)? Run with ExoSM
Step 2 Color(yellow,Complete)?
Step 3 Color(yellow,Complete)?
Step 4 Color(yellow,Complete)?
Step 5 Color(yellow,Complete)?
Step 6 Color(yellow,Complete)?
Step 7 Color(yellow,Complete)? Run with ExoSM
Step 8 Color(yellow,Complete)?
Step 9 Color(yellow,Complete)?
Step 10 Color(yellow,Complete)?
Step 11 Color(yellow,Complete)?
Step 12 Color(yellow,Complete)?
Step 13 Color(#63B8FF,In Progress)? writing non-ip packet send
Step 14 Color(orange,Blocked)? monitoring does not support bare metal nodes, no VM data at this time
Step 15 Color(yellow,Complete)?
Step 16 Color(orange,Blocked)? monitoring does not support bare metal nodes
Step 17 Color(yellow,Complete)?


State Legend Description
Color(green,Pass)? Test completed and met all criteria
Color(#98FB98,Pass: most criteria)? Test completed and met most criteria. Exceptions documented
Color(red,Fail)? Test completed and failed to meet criteria.
Color(yellow,Complete)? Test completed but will require re-execution due to expected changes
Color(orange,Blocked)? Blocked by ticketed issue(s).
Color(#63B8FF,In Progress)? Currently under test.


Test Plan Steps

This test case is modified to use ExoSM to request resources across sites. The omni commands shown in this test use nicknames nick_names for aggregates, which are:

eg-sm=,https://geni.renci.org:11443/orca/xmlrpc

is used in place of individual site's aggregates:

eg-gpo=,https://bbn-hn.exogeni.net:11443/orca/xmlrpc
eg-renci=,https://rci-hn.exogeni.net:11443/orca/xmlrpc

Step 1. As Experimenter1, Request ListResources RENCI ExoGENI

Using the credentials lnevers@bbn.com, request listresources from ExoSM, to determine which resources can be requested for the first experiment:

$ omni.py -a eg-sm listresources -o

The above command generates a file named rspec-geni-renci-org-11443-orca.xml.

Step 2. Review ListResources output

Reviewed content of output file named rspec-geni-renci-org-11443-orca.xml and determined site information for VM request.

Step 3. Define a request RSpec

Define a request RSpec for a VM at BBN ExoGENI, a VM at RENCI ExoGENI and an unbound exclusive non-OpenFlow VLAN to connect the 2 endpoints. RSpec created for this experiment is EG-EXP-4-exp1.rspec

Step 4. Create the first slice.

Using the following command create a slice for the first experiment:

 $ omni.py createslice EG-EXP-4-exp1

Step 5. Create a sliver

Using the ExoSM and the RSpecs defined above create a sliver with one VM at BBN and one VM at RENCI:

$ omni.py createsliver -a eg-sm EG-EXP-4-exp1 EG-EXP-4-exp1.rspec

Verify that sliver status is ready:

$ omni.py sliverstatus -a eg-sm EG-EXP-4-exp1  

When sliverstatus reports geni_status as ready, you can collect a listresource for the sliver to determined which VMs are allocated to the sliver:

$  omni.py listresources -a eg-sm EG-EXP-4-exp1  -o
$  egrep hostname EG-EXP-4-exp1-rspec-geni-renci-org-11443-orca.xml 
   <login authentication="ssh-keys" hostname="152.54.14.34" port="22" username="root"/>      
   <login authentication="ssh-keys" hostname="192.1.242.12" port="22" username="root"/>

Also can determine login information as follows:

$ examples/readyToLogin.py -a eg-sm EG-EXP-4-exp1    
.....

XXXXX

== Step 6. Log in to each of the systems, and send traffic to the other system, leave traffic running ==

Connect to the RENCI VM and send traffic to the BBN VM:
{{{
$ ssh root@152.54.14.34
root@debian:~# ifconfig|egrep 172.16
          inet addr:172.16.7.2  Bcast:172.16.7.255  Mask:255.255.255.0
root@debian:~# ping 172.16.7.1
PING 172.16.7.1 (172.16.7.1) 56(84) bytes of data.
64 bytes from 172.16.7.1: icmp_req=1 ttl=64 time=48.2 ms
64 bytes from 172.16.7.1: icmp_req=2 ttl=64 time=18.6 ms
64 bytes from 172.16.7.1: icmp_req=3 ttl=64 time=17.9 ms
}}}

Connect to the BBN VM and send traffic to the RENCI VM:
{{{
$ ssh root@192.1.242.12
root@debian:~# ifconfig |egrep 172.16
          inet addr:172.16.7.1  Bcast:172.16.7.255  Mask:255.255.255.0
root@debian:~# ping 172.16.7.2 
PING 172.16.7.2 (172.16.7.2) 56(84) bytes of data.
64 bytes from 172.16.7.2: icmp_req=1 ttl=64 time=24.4 ms
64 bytes from 172.16.7.2: icmp_req=2 ttl=64 time=17.9 ms
64 bytes from 172.16.7.2: icmp_req=3 ttl=64 time=17.8 ms
}}}

== Step 7. As Experimenter2, Request !ListResources from RENCI ExoGENI ==

Using the credentials lnevers2@bbn.com, request listresources from ExoSM, to determine which resources can be requested:
{{{
$ omni.py -a eg-sm listresources -o
}}}

== Step 8. Define a request RSpec ==

Define a request RSpec for one VM and one bare metal node each with two interfaces in the BBN ExoGENI rack, two VMs at RENCI, and two VLANs to connect the BBN ExoGENI to the RENCI ExoGENI.

RSpec created for this topology is [http://groups.geni.net/geni/browser/trunk/GENIRacks/ExoGENI/Spiral4/Rspecs/AcceptanceTests/EG-EXP-4/EG-EXP-4-exp2.rspec EG-EXP-4-exp2.rspec]

and it defines the following mapping for network, interfaces, and network addresses:

 - BBN Bare Metal if0 172.16.2.1 connects to BBN VM 2 if0 172.16.2.2
 - BBN Bare Metal if1 172.16.3.1 connects to RCI VM 1 if0 172.16/.3.2
 - BBN VM2 if1 172.16.4.1 connects to RCI VM 2 if0 172.16.4.2

== Step 9. Create a second slice  ==

Using the following command create a slice for the second experiment:
{{{
$ omni.py createslice EG-EXP-4-exp2
}}}

== Step 10. In the second slice, create a sliver at the RENCI ExoGENI aggregate using the RSpecs defined above  ==

Using the ExoSM and the RSpecs defined in step 8 create a sliver:
{{{
$ omni.py createsliver -a eg-sm EG-EXP-4-exp2 EG-EXP-4-exp2.rspec
}}}

Verify that sliver status is ready:
{{{
$ omni.py sliverstatus -a eg-sm EG-EXP-4-exp2 
}}}
Determined which nodes (VMs and bare metal) are allocated to this sliver:
{{{
$ omni.py listresources -a eg-sm EG-EXP-4-exp2 -o
$ egrep hostname EG-EXP-4-exp2-rspec-geni-renci-org-11443-orca.xml 
<login authentication="ssh-keys" hostname="192.1.242.13" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.110" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="152.54.14.35" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="152.54.14.37" port="22" username="root"/>    
}}}

Also can determine login information as follows:
{{{
$ examples/readyToLogin.py -a eg-sm EG-EXP-4-exp2    
.....

XXXX

== Step 11. Log in to each of the end-point systems, and send traffic to the other end-point system which shares the same VLAN  ==

Logged in to BBN VM and pinged each of the remotes:
{{{
$ ssh root@192.1.242.13
root@debian:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:16:3e:0c:60:0f  
          inet addr:10.103.0.10  Bcast:10.103.0.255  Mask:255.255.255.0
          inet6 addr: fe80::16:3eff:fe0c:600f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:278 errors:0 dropped:0 overruns:0 frame:0
          TX packets:248 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:39892 (38.9 KiB)  TX bytes:35307 (34.4 KiB)

eth1      Link encap:Ethernet  HWaddr 52:54:00:bf:8e:ad  
          inet addr:172.16.4.1  Bcast:172.16.4.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:febf:8ead/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1005 (1005.0 B)  TX bytes:398 (398.0 B)

eth2      Link encap:Ethernet  HWaddr 52:54:00:d7:20:a4  
          inet addr:172.16.2.2  Bcast:172.16.2.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fed7:20a4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2060 (2.0 KiB)  TX bytes:398 (398.0 B)
root@debian:~# ping 172.16.4.2 -c 3
PING 172.16.4.2 (172.16.4.2) 56(84) bytes of data.
64 bytes from 172.16.4.2: icmp_req=1 ttl=64 time=58.9 ms
64 bytes from 172.16.4.2: icmp_req=2 ttl=64 time=18.0 ms
64 bytes from 172.16.4.2: icmp_req=3 ttl=64 time=17.9 ms

--- 172.16.4.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 17.916/31.659/58.970/19.311 ms
root@debian:~# ping 172.16.2.1 -c 3
PING 172.16.2.1 (172.16.2.1) 56(84) bytes of data.
64 bytes from 172.16.2.1: icmp_req=1 ttl=64 time=11.5 ms
64 bytes from 172.16.2.1: icmp_req=2 ttl=64 time=0.442 ms
64 bytes from 172.16.2.1: icmp_req=3 ttl=64 time=0.399 ms

--- 172.16.2.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.399/4.135/11.566/5.254 ms
}}}

Log in to BBN bare metal and pinged each of the remotes:

{{{
$ ssh root@192.1.242.110
[root@bbn-w4 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 5C:F3:FC:BA:51:D4  
          inet addr:10.100.0.14  Bcast:10.100.0.255  Mask:255.255.255.0
          inet6 addr: fe80::5ef3:fcff:feba:51d4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:211 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1595 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:34898 (34.0 KiB)  TX bytes:205446 (200.6 KiB)
          Interrupt:28 Memory:92000000-92012800 

eth2      Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
          inet6 addr: fe80::207:43ff:fe11:a460/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:285 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:22953 (22.4 KiB)  TX bytes:1774 (1.7 KiB)
          Interrupt:24 

eth2.24   Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
          inet addr:172.16.2.1  Bcast:172.16.2.255  Mask:255.255.255.0
          inet6 addr: fe80::207:43ff:fe11:a460/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:930 (930.0 b)  TX bytes:756 (756.0 b)

eth2.2603 Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
          inet addr:172.16.3.1  Bcast:172.16.3.255  Mask:255.255.255.0
          inet6 addr: fe80::207:43ff:fe11:a460/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:590 (590.0 b)  TX bytes:378 (378.0 b)

[root@bbn-w4 ~]# ping -c 3 172.16.3.2
PING 172.16.3.2 (172.16.3.2) 56(84) bytes of data.
64 bytes from 172.16.3.2: icmp_seq=1 ttl=64 time=46.8 ms
64 bytes from 172.16.3.2: icmp_seq=2 ttl=64 time=17.6 ms
64 bytes from 172.16.3.2: icmp_seq=3 ttl=64 time=17.6 ms

--- 172.16.3.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2021ms
rtt min/avg/max/mdev = 17.688/27.402/46.827/13.736 ms
[root@bbn-w4 ~]# ping -c 3 172.16.2.2
PING 172.16.2.2 (172.16.2.2) 56(84) bytes of data.
64 bytes from 172.16.2.2: icmp_seq=1 ttl=64 time=4.96 ms
64 bytes from 172.16.2.2: icmp_seq=2 ttl=64 time=0.444 ms
64 bytes from 172.16.2.2: icmp_seq=3 ttl=64 time=0.404 ms

--- 172.16.2.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.404/1.937/4.964/2.140 ms
[root@bbn-w4 ~]# 
}}}

== Step 12. Verify traffic handling per experiment, VM isolation, and MAC address assignment  ==


Verified MAC address assignment for the BBN VM:
{{{
lnevers@sendaria:~/gcf-1.6.2$ ssh root@192.1.242.13
root@debian:~# ifconfig|grep HWaddr
eth0      Link encap:Ethernet  HWaddr 02:16:3e:0c:60:0f  
eth1      Link encap:Ethernet  HWaddr 52:54:00:bf:8e:ad  
eth2      Link encap:Ethernet  HWaddr 52:54:00:d7:20:a4  
root@debian:~# 
}}}

Verified MAC address assignment for the BBN bare metal:
{{{
Connection to 192.1.242.13 closed.
lnevers@sendaria:~/gcf-1.6.2$ ssh root@192.1.242.110
[root@bbn-w4 ~]# ifconfig|grep HWaddr
eth0      Link encap:Ethernet  HWaddr 5C:F3:FC:BA:51:D4  
eth2      Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
eth2.24   Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
eth2.2603 Link encap:Ethernet  HWaddr 00:07:43:11:A4:60  
}}}

== Step 13. Construct and send a non-IP Ethernet packet over the data plane interface. ==

On two ExoGENI hosts in the BBN rack, installed and compiled the the GPO package PingPlus, which is available at the [http://software.geni.net/local-sw/ GPO Software Repository] to generate a non-IP Ethernet packet.

Determined interface information on each host.  Host1:
{{{
root@debian:~/pingPlus-0.2# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 52:54:00:9c:3f:8a  
          inet addr:172.16.1.1  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe9c:3f8a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1128 (1.1 KiB)  TX bytes:398 (398.0 B)
}}}
Host2:
{{{
eth1      Link encap:Ethernet  HWaddr 52:54:00:0e:e5:a1  
          inet addr:172.16.1.2  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe0e:e5a1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:916 (916.0 B)  TX bytes:398 (398.0 B)
}}}

Started the pingPlusListener on host1:
{{{
$ sudo ./pingPlusListener 10111
}}}

Started tcpdump on host1:
{{{{
$ sudo tcpdump ether host 52:54:00:0e:e5:a1
}}}}

On host 2 sent 10 packets with !pingPlus to host1:
{{{
$ sudo ./pingPlus 52:54:00:9c:3f:8a eth0 10111 12
}}}

Packets were received on host1:
{{{

}}}

TCPdump captured the content of the Ethernet packet:
{{{

}}}
== Step 14. Review baseline monitoring statistics ==

Current monitoring does not support bare metal nodes and no resource detail is available for the slice. Captures will be added when data is available. 

== Step 15. Run test for at least 4 hours  ==

Current test run is 1 hour, will increase in future test run.

== Step 16. Review baseline monitoring statistics ==

Current monitoring does not support bare metal nodes and no resource detail is available for the slice. Captures will be added when data is available. 

== Step 17. Stop traffic and delete slivers ==

As experimenter1, delete the sliver:
{{{
$ omni.py deletesliver -a exosm EG-EXP-4-exp1 
}}}

As experimenter2 delete the sliver:
{{{
$ omni.py deletesliver -a exosm EG-EXP-4-exp2    
}}}

}}}