wiki:GENIRacksHome/ExogeniRacks/AcceptanceTestStatus/EG-EXP-2

Version 25 (modified by lnevers@bbn.com, 9 years ago) (diff)

--

EG-EXP-2: ExoGENI Single Site Acceptance Test

This page captures status for the test case EG-EXP-2, which verifies basic operations of VMs and data flows within one rack. For overall status see the ExoGENI Acceptance Test Status page.

Test Status

This section captures the status for each step in the acceptance test plan.

Step State Ticket Comments
Step 1 Color(yellow,Complete)?
Step 2 Color(yellow,Complete)? Only one supported OS image is available, no image playpen
Step 3 Color(yellow,Complete)? listresources does not include list of user custom images
Step 4 Color(yellow,Complete)?
Step 5 Color(yellow,Complete)?
Step 6 Color(yellow,Complete)?
Step 7 Color(yellow,Complete)?
Step 8 Color(yellow,Complete)?
Step 9 Color(yellow,Complete)?
Step 10 Color(yellow,Complete)? NO custom images are available, no image playpen available
Step 11 Color(yellow,Complete)?
Step 12 Color(yellow,Complete)?
Step 13 Color(yellow,Complete)?
Step 14 Color(yellow,Complete)?
Step 15 Color(yellow,Complete)?
Step 16 Color(yellow,Complete)?
Step 17 Color(yellow,Complete)?
Step 18 Color(yellow,Complete)?


State Legend Description
Color(green,Pass)? Test completed and met all criteria
Color(#98FB98,Pass: most criteria)? Test completed and met most criteria. Exceptions documented
Color(red,Fail)? Test completed and failed to meet criteria.
Color(yellow,Complete)? Test completed but will require re-execution due to expected changes
Color(orange,Blocked)? Blocked by ticketed issue(s).
Color(#63B8FF,In Progress)? Currently under test.


Test Plan Steps

The first experiment in this plane requires that two GPO Ubuntu images have been tested in the ExoGENI image playpen environment and have been uploaded to the RENCI VM image repository using available ExoGENI documentation. Also required is that one Ubuntu image is for the VM and one Ubuntu image is for the bare metal node in this test. Based on the approach chosen for ExoGENI, an Image Playpen system is required to allow user-define OS support. The Image Playpen was originally planned to be a be available when x3650M4 nodes become available the week of 05/28/2012, but is yet to be available as of 07/26/2012

Both user account used for this test have the following nick_names defined in the omni_config:

exobbn=,https://bbn-hn.exogeni.net:11443/orca/xmlrpc
exorci=,https://rci-hn.exogeni.net:11443/orca/xmlrpc
exosm=,https://geni.renci.org:11443/orca/xmlrpc

Notes:

  1. Bare Metal nodes are available via ExoSM only.
  2. Bare Metal node only support one version at this time.
  3. Current test uses 1 Bare metal at BBN and 1 at RENCI.

Step 1. As Experimenter1, request ListResources from BBN ExoGENI

User account lnevers is used for this tests, and the listresources is requested as follows from the BBN ExoGENI resource aggregate:

$ ./src/omni.py -a exobbn listresources --available -o

Step 2. Review advertisement RSpec for a list of OS images which can be loaded, and identify available resources

Feature not available in ExoGENI listresources. Available image information is collected out-of-band, so when custom Images are tested they will not be part of list resources ouptuput. Verification will be based on custom image successful installation and usage in the ExoGENI rack.

Step 3. Verify that the GPO Ubuntu image is available

Same as step 2.

Step 4. Define a request RSpec for two VMs, each with a GPO Ubuntu image

Created the RSpec for the two VMs named EG-EXP-2-exp1.rspec

Step 5. Create the first slice

$ omni.py createslice EG-EXP-2-exp1

Step 6. Create a sliver in the first slice, using the RSpec defined in step 4.

$ omni.py  -a exobbn createsliver EG-EXP-2-exp1  EG-EXP-2-exp1.rspec

Verify that the sliver creation was completed, by looking for "'geni_status': 'ready'" in the output of the sliverstatus command:

$ omni.py -a exobbn sliverstatus EG-EXP-2-exp1        

Once sliver is ready, determine which hosts were assigned:

$ omni.py -a exobbn listresources EG-EXP-2-exp1 -o
$ egrep hostname EG-EXP-2-exp1-rspec-bbn-hn-exogeni-net-11443-orca.xml 
<login authentication="ssh-keys" hostname="192.1.242.12" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.13" port="22" username="root"/>  

Step 7. Log in to each of the systems, and send traffic to the other system sharing a VLAN

$ ssh root@192.1.242.12
root@debian:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:16:3e:5d:c7:38  
          inet addr:10.103.0.13  Bcast:10.103.0.255  Mask:255.255.255.0
          inet6 addr: fe80::16:3eff:fe5d:c738/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:124 errors:0 dropped:0 overruns:0 frame:0
          TX packets:129 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:20713 (20.2 KiB)  TX bytes:18945 (18.5 KiB)

eth1      Link encap:Ethernet  HWaddr 52:54:00:3c:86:9a  
          inet addr:172.16.1.1  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe3c:869a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:996 (996.0 B)  TX bytes:378 (378.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@debian:~# ping 172.16.1.2 -c 4
PING 172.16.1.2 (172.16.1.2) 56(84) bytes of data.
64 bytes from 172.16.1.2: icmp_req=1 ttl=64 time=10.1 ms
64 bytes from 172.16.1.2: icmp_req=2 ttl=64 time=0.772 ms
64 bytes from 172.16.1.2: icmp_req=3 ttl=64 time=1.78 ms
64 bytes from 172.16.1.2: icmp_req=4 ttl=64 time=0.618 ms

--- 172.16.1.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 0.618/3.322/10.116/3.947 ms

Step 8. Using root privileges on one of the VMs load a Kernel module.

Show current modules loaded, load a module, and then remove the module:

root@debian:~# lsmod
Module                  Size  Used by
acpiphp                15157  0 
pci_hotplug            21587  1 acpiphp
snd_pcm                60487  0 
snd_timer              15598  1 snd_pcm
snd                    46526  2 snd_pcm,snd_timer
soundcore               4598  1 snd
snd_page_alloc          6249  1 snd_pcm
i2c_piix4               8328  0 
psmouse                49937  0 
processor              29935  0 
button                  4650  0 
serio_raw               3752  0 
pcspkr                  1699  0 
virtio_balloon          2961  0 
evdev                   7352  0 
i2c_core               15819  1 i2c_piix4
ext3                  106710  1 
jbd                    37317  1 ext3
mbcache                 5050  1 ext3
dm_mod                 53946  0 
ata_generic             3239  0 
virtio_blk              4209  1 
ata_piix               21124  0 
virtio_net             10573  0 
uhci_hcd               18521  0 
ehci_hcd               32097  0 
libata                133776  2 ata_generic,ata_piix
floppy                 49087  0 
thermal                11674  0 
thermal_sys            11942  2 processor,thermal
virtio_pci              5511  0 
virtio_ring             3258  1 virtio_pci
virtio                  3309  4 virtio_balloon,virtio_blk,virtio_net,virtio_pci
usbcore               123122  3 uhci_hcd,ehci_hcd
nls_base                6377  1 usbcore
scsi_mod              126725  1 libata
root@debian:~# 
root@debian:~# modprobe dummy 
root@debian:~# lsmod |grep dummy
dummy                   1584  0 
root@debian:~# modprobe -r dummy
root@debian:~# lsmod |grep dummy
root@debian:~# 

Step 9. As Experimenter2, request ListResources from BBN ExoGENI.

User account lnevers1 will be used for this tests, and the listresources is requested as follows from the ExoSMto get a list of resources. Request is modified from original plan because bare metal node are only available fro the ExoSM at this time:

$ omni.py -a exosm listresources --available -o

Step 10. Define a request RSpec for two bare metal nodes, both using the uploaded GPO Ubuntu images.

Create the RSpec the 2 Bare metal nodes named EG-EXP-2-exp2.rspec. This test is currently modified to use the ExoSM rather than the BBN rack SM. The RSpec used has 1 bare metal node at BBN ExoGENI and one at RENCI ExoGENI.

Step 11. Create the second slice.

$ omni.py createslice  EG-EXP-2-exp2

Step 12. Create a sliver in the second slice, using the RSpec defined in step 10.

$ omni.py -a exosm createsliver EG-EXP-2-exp2 EG-EXP-2-exp1.rspec 

Step 13. Log in to each of the systems, and send traffic to the other system.

Verify that the sliver creation was completed, by looking for "'geni_status': 'ready'" in the output of the sliverstatus command:

$ omni.py -a exosm sliverstatus EG-EXP-2-exp2        

Once the sliver is ready, determine which hosts were assigned:

$ omni.py -a exosm listresources EG-EXP-2-exp2 -o
$ egrep hostname EG-EXP-2-exp2-rspec-geni-renci-org-11443-orca.xml
<login authentication="ssh-keys" hostname="192.1.242.14" port="22" username="root"/>      
<login authentication="ssh-keys" hostname="192.1.242.15" port="22" username="root"/>      

Step 14. Verify that experimenters 1 and 2 cannot use the control plane to access each others resources

Verify that attempt to login into other experiment nodes fails and verify that users are not able to mount each others experiment file system.

As experimenter1 (lnevers@bbn.com) attempt in to login to the second experiment EG-EXP-2-exp2:

lnevers@arendia:~$ ssh 192.1.242.14 -l root
root@192.1.242.14's password: 
Permission denied, please try again.
root@192.1.242.14's password: 
Permission denied, please try again.
root@192.1.242.14's password: 
Permission denied (publickey,password).
lnevers@arendia:~$ 

As experimenter1 (lnevers@bbn.com) login to own host and then attempt to get access to experimenter2 (lnevers1@bbn.com) host:

lnevers@arendia:~$ ssh root@192.1.242.12
root@debian:~# ssh 10.103.0.12
The authenticity of host '10.103.0.12 (10.103.0.12)' can't be established.
RSA key fingerprint is e7:1d:db:48:93:6c:9b:95:77:b8:46:b9:ea:9e:06:21.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.103.0.12' (RSA) to the list of known hosts.
root@10.103.0.12's password: 
Permission denied, please try again.
root@10.103.0.12's password: 
Permission denied, please try again.
root@10.103.0.12's password: 
Permission denied (publickey,password).
root@debian:~# 

Step 15. Review system statistics and VM isolation and network isolation on data plane.

Reviewed usage statistics on the nodes:

root@debian:~# top

top - 14:40:58 up 40 min,  2 users,  load average: 0.00, 0.00, 0.00
Tasks:  54 total,   1 running,  53 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 99.7%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    121796k total,   113948k used,     7848k free,     5288k buffers
Swap:        0k total,        0k used,        0k free,    85428k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                             
    1 root      20   0  8352  784  648 S  0.0  0.6   0:00.32 init                                 
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd                             
    3 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0                          
    4 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0                          
    5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0                           
    6 root      20   0     0    0    0 S  0.0  0.0   0:00.03 events/0                             
    7 root      20   0     0    0    0 S  0.0  0.0   0:00.00 cpuset                               
    8 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khelper                              
    9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 netns                                
   10 root      20   0     0    0    0 S  0.0  0.0   0:00.00 async/mgr                            
   11 root      20   0     0    0    0 S  0.0  0.0   0:00.00 pm                                   
   12 root      20   0     0    0    0 S  0.0  0.0   0:00.00 sync_supers                          
   13 root      20   0     0    0    0 S  0.0  0.0   0:00.00 bdi-default                          
   14 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kintegrityd/0                        
   15 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kblockd/0                            
   16 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpid                               
   17 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify                         
   18 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_hotplug                        
   19 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kseriod            

No issues were found.

Step 16. Verify that each VM has a distinct MAC address for that interface.

Issued ifconfig on each assigned host and verified that each host has unique MAC address.

The ifconfig output from the first host:

$ ssh 192.1.242.14 -l root
root@debian:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:16:3e:1d:a9:37  
          inet addr:10.103.0.12  Bcast:10.103.0.255  Mask:255.255.255.0
          inet6 addr: fe80::16:3eff:fe1d:a937/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:444 errors:0 dropped:0 overruns:0 frame:0
          TX packets:333 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:62367 (60.9 KiB)  TX bytes:52890 (51.6 KiB)

eth1      Link encap:Ethernet  HWaddr 52:54:00:3f:35:36  
          inet addr:172.16.1.1  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe3f:3536/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1664 (1.6 KiB)  TX bytes:398 (398.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

The ifconfig output from the second host:

$ ssh 192.1.242.15 -l root
root@debian:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:16:3e:60:e6:e4  
          inet addr:10.103.0.15  Bcast:10.103.0.255  Mask:255.255.255.0
          inet6 addr: fe80::16:3eff:fe60:e6e4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:229 errors:0 dropped:0 overruns:0 frame:0
          TX packets:203 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:37642 (36.7 KiB)  TX bytes:28810 (28.1 KiB)

eth1      Link encap:Ethernet  HWaddr 52:54:00:23:e9:16  
          inet addr:172.16.1.2  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe23:e916/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:882 (882.0 B)  TX bytes:398 (398.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B) 

Step 17. Verify that VMs' MAC addresses are learned on the data plane switch.

Ability to exchange traffic shows that the mac addresses are learned on the data plane switch. On the second host sent traffic to remote:

root@debian:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 52:54:00:23:e9:16  
          inet addr:172.16.1.2  Bcast:172.16.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe23:e916/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:928 (928.0 B)  TX bytes:398 (398.0 B)

root@debian:~# ping 172.16.1.1 -c 3
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_req=1 ttl=64 time=10.6 ms
64 bytes from 172.16.1.1: icmp_req=2 ttl=64 time=0.593 ms
64 bytes from 172.16.1.1: icmp_req=3 ttl=64 time=0.763 ms

--- 172.16.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.593/4.016/10.692/4.721 ms

Step 18. Stop traffic and delete slivers.

Stopped traffic and deleted sliver at the ExoSM:

$ omni.py -a exosm deletesliver EG-EXP-2-exp2