[[PageOutline]]

= IG-MON-3: GENI Active Experiment Inspection Test =

This page captures status for the test case IG-MON-3, which verifies the ability to start, stop, and terminate experiments by experimenters and administrators.   For overall status see the [wiki:GENIRacksHome/InstageniRacks/AcceptanceTestStatus InstaGENI Acceptance Test Status] page. 

''Last update: 2013-03-06''

== Test Status ==

|| '''Step''' || '''State'''          || '''Tickets''' ||'''Notes'''               ||
|| 1          || [[Color(green,Pass)]]||               ||         ||
|| 2          || [[Color(green,Pass)]]||               ||         ||
|| 3          || [[Color(green,Pass)]]||               ||         ||
|| 4          || [[Color(green,Pass)]]||               ||         ||
|| 5          || [[Color(green,Pass)]]||               ||         ||
|| 6          ||                           ||                 || blocked on availability of OpenFlow functionality  ||
|| 7          ||                           ||                 || ready to test non-OpenFlow functionality ||
|| 8          ||                           ||                 || ready to test non-OpenFlow functionality ||

[[BR]]
|| '''State Legend'''                     || '''Description'''                                                    ||
|| [[Color(green,Pass)]]                  || Test completed and met all criteria                                  ||
|| [[Color(#98FB98,Pass: most criteria)]] || Test completed and met most criteria. Exceptions documented          ||
|| [[Color(red,Fail)]]                    || Test completed and failed to meet criteria.                          ||
|| [[Color(yellow,Complete)]]             || Test completed but will require re-execution due to expected changes ||
|| [[Color(orange,Blocked)]]              || Blocked by ticketed issue(s).                                        ||
|| [[Color(#63B8FF,In Progress)]]         || Currently under test.                                                ||
[[BR]]

= Test Plan Steps =

In this test case, three experiments are started and various administrative features are used to determine experiment state and resources.

== Step 1. Start a VM experiment and leave it running ==

As experimenter create a 2 VMs experiment in the GPO InstaGENI rack. RSpec used:
{{{
<?xml version="1.0" encoding="UTF-8"?>
<!-- 
This example RSpec shows how to request 2 VMs at the GPO InstaGENI Rack. 
-->
<rspec type="request" 
	xmlns="http://www.geni.net/resources/rspec/3" 
	xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" 
	xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xsi:schemaLocation="http://www.geni.net/resources/rspec/3 
	http://www.geni.net/resources/rspec/3/request.xsd">  

  <node client_id="VM-1" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="VM-1:if0">
      <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  </node>
  <node client_id="VM-2" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="VM-2:if0">
      <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  </node>
  <link client_id="lan0">
    <component_manager name="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm"/>
    <interface_ref client_id="VM-1:if0"/>
    <interface_ref client_id="VM-2:if0"/>
    <link_type name="lan"/>
  </link>
</rspec>
}}}

Create a slice and sliver:
{{{

$ omni.py createslice EG-MON-3
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Created slice with Name EG-MON-3, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3, Expiration 2013-03-07 15:09:24+00:00
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createslice:
  Options as run:
		framework: pg

  Args: createslice EG-MON-3
  Result Summary: Created slice with Name EG-MON-3, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3, Expiration 2013-03-07 15:09:24+00:00
INFO:omni: ============================================================

$ omni.py createsliver -a ig-gpo EG-MON-3  ./instageni-2vm-at-gpo.rspec 
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3 expires on 2013-03-07 15:09:24 UTC
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./instageni-2vm-at-gpo.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3
INFO:omni: (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=388177e82b203e84aa3694dda7bf441d)
INFO:omni:Got return from CreateSliver for slice EG-MON-3 at https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0:
INFO:omni:<!-- Reserved resources for:
	Slice: EG-MON-3
	at AM:
	URN: unspecified_AM_URN
	URL: https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0
 -->
INFO:omni:<rspec xmlns="http://www.geni.net/resources/rspec/3" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="manifest" xsi:schemaLocation="http://www.geni.net/resources/rspec/3   http://www.geni.net/resources/rspec/3/manifest.xsd">  

<node client_id="VM-1" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false" 
    component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2048">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="VM-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:lo0" 
       sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2052" mac_address="02104639f347">
       <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"/>
    </interface>
   <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm2-1"/>
   <host name="VM-1.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/>
   <services>
   <login authentication="ssh-keys" hostname="pc2.instageni.gpolab.bbn.com" port="31290" username="lnevers"/>
   </services>
</node>

<node client_id="VM-2" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false" 
    component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2049">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="VM-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:lo0" 
      sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2053" mac_address="020063c74361">
       <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm2-6"/>
  <host name="VM-2.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/>
  <services>
  <login authentication="ssh-keys" hostname="pc2.instageni.gpolab.bbn.com" port="31291" username="lnevers"/>
  </services>
</node>

<link client_id="lan0" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2051">
    <component_manager name="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm"/>
    <interface_ref client_id="VM-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:lo0" 
      sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2052"/>
    <interface_ref client_id="VM-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:lo0" 
     sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2053"/>
    <link_type name="lan"/>
</link>
</rspec>

INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:
  Options as run:
		aggregate: ['ig-gpo']
		framework: pg
  Args: createsliver EG-MON-3 ./instageni-2vm-at-gpo.rspec
  Result Summary: Got Reserved resources RSpec from instageni-gpolab-bbn-com-protogeniv2 
INFO:omni: ============================================================

}}}

Determine login information for the VM resources assigned and that the VMs are "ready" for use:
{{{
$ readyToLogin.py -a ig-gpo EG-MON-3                               
<...>
VM-2's geni_status is: ready (am_status:ready) 
User lnevers logins to VM-2 using:
	xterm -e ssh -p 31291  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com &

VM-1's geni_status is: ready (am_status:ready) 
User lnevers logins to VM-1 using:
	xterm -e ssh -p 31290  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com &
}}}

Login to VM-1, verify hostname, address assignment and initial ARP table entries:
{{{
$ ssh -p 31290  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com 

[lnevers@VM-1 ~]$ /sbin/ifconfig mv1.12
mv1.12    Link encap:Ethernet  HWaddr 02:10:46:39:F3:47  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::10:46ff:fe39:f347/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:6 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:552 (552.0 b)  TX bytes:0 (0.0 b)

[lnevers@VM-1 ~]$ /sbin/arp -a
pc2.instageni.gpolab.bbn.com (192.1.242.141) at 10:60:4b:9b:82:14 [ether] on eth999
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth999

}}}
Login to VM-2, verify hostname, address assignment, MAC address for data plane interface, and initial ARP table entries:
{{{
$ ssh -p 31291  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com 
[lnevers@VM-2 ~]$ /sbin/ifconfig mv6.13
mv6.13    Link encap:Ethernet  HWaddr 02:00:63:C7:43:61  
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::63ff:fec7:4361/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:6 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[lnevers@VM-2 ~]$ /sbin/arp -a
pc2.instageni.gpolab.bbn.com (192.1.242.141) at 10:60:4b:9b:82:14 [ether] on eth999
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth999
}}}

Install iperf on both VMs and exchange TCP traffic from VM-1 client to VM-2 server over the data plane interface and look at ARP table entries:

{{{
[lnevers@VM-1 ~]$ /usr/bin/iperf -c 192.168.1.2 -t 60
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.1 port 55185 connected with 192.168.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   687 MBytes  96.0 Mbits/sec

[lnevers@VM-1 ~]$  /sbin/arp -a
pc2.instageni.gpolab.bbn.com (192.1.242.141) at 10:60:4b:9b:82:14 [ether] on eth999
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth999
VM-2-lan0 (192.168.1.2) at 02:00:63:c7:43:61 [ether] on mv1.12
}}}

Show ARP entries on VM2 (iperf server):
{{{
[lnevers@VM-2 ~]$ /sbin/arp -a
pc2.instageni.gpolab.bbn.com (192.1.242.141) at 10:60:4b:9b:82:14 [ether] on eth999
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth999
VM-1-lan0 (192.168.1.1) at 02:10:46:39:f3:47 [ether] on mv6.13

}}}

Leave experiment running.

== Step 2. Start a bare metal node experiment and leave it running ==

As experimenter create a 2 Raw PCs experiment in the GPO InstaGENI rack. RSpec used:
{{{
<?xml version="1.0" encoding="UTF-8"?>
<rspec type="request" 
	xmlns="http://www.geni.net/resources/rspec/3" 
	xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" 
	xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xsi:schemaLocation="http://www.geni.net/resources/rspec/3 
	http://www.geni.net/resources/rspec/3/request.xsd">  

  <node client_id="PC-1" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc5"          
        component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="true">
    <sliver_type name="raw-pc"/>
    <interface client_id="PC-1:if0">
      <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  </node>

  <node client_id="PC-2" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc4"  
        component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="true">
    <sliver_type name="raw-pc"/>
    <interface client_id="PC-2:if0">
      <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  </node>
  <link client_id="lan0">
    <component_manager name="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm"/>
    <interface_ref client_id="PC-1:if0"/>
    <interface_ref client_id="PC-2:if0"/>
    <link_type name="lan"/>
  </link>
</rspec>

}}}

Create a slice and sliver:

{{{
$ omni.py createslice IG-MON-3
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Created slice with Name IG-MON-3, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3, Expiration 2013-03-07 15:47:53+00:00
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createslice:
  Options as run:
		framework: pg

  Args: createslice IG-MON-3
  Result Summary: Created slice with Name IG-MON-3, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3, Expiration 2013-03-07 15:47:53+00:00
INFO:omni: ============================================================

$ omni.py createsliver -a ig-gpo IG-MON-3 ./instageni-2rawpc-at-gpo.rspec 
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3 expires on 2013-03-07 15:47:53 UTC
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./instageni-2rawpc-at-gpo.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3
INFO:omni: (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=deb52735b826a50320d5274e8e4c6c8e)
INFO:omni:Got return from CreateSliver for slice IG-MON-3 at https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0:
INFO:omni:<!-- Reserved resources for:
	Slice: IG-MON-3
	at AM:
	URN: unspecified_AM_URN
	URL: https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0
 -->
INFO:omni:<rspec xmlns="http://www.geni.net/resources/rspec/3" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="manifest" xsi:schemaLocation="http://www.geni.net/resources/rspec/3   http://www.geni.net/resources/rspec/3/manifest.xsd">  

  <node client_id="PC-1" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc5" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2042">
    <sliver_type name="raw-pc"/>
    <interface client_id="PC-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2046" mac_address="10604B9C476A">
      <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pc5"/><host name="PC-1.IG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/><services><login authentication="ssh-keys" hostname="pc5.instageni.gpolab.bbn.com" port="22" username="lnevers"/></services></node>

  <node client_id="PC-2" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc4" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2043">
    <sliver_type name="raw-pc"/>
    <interface client_id="PC-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2047" mac_address="10604B9600D6">
      <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pc4"/><host name="PC-2.IG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/><services><login authentication="ssh-keys" hostname="pc4.instageni.gpolab.bbn.com" port="22" username="lnevers"/></services></node>
  <link client_id="lan0" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2045" vlantag="258">
    <component_manager name="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm"/>
    <interface_ref client_id="PC-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc5:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2046"/>
    <interface_ref client_id="PC-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc4:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2047"/>
    <link_type name="lan"/>
  </link>
</rspec>
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:
  Options as run:
		aggregate: ['ig-gpo']
		framework: pg
  Args: createsliver IG-MON-3 ./instageni-2rawpc-at-gpo.rspec
  Result Summary: Got Reserved resources RSpec from instageni-gpolab-bbn-com-protogeniv2 
INFO:omni: ============================================================
}}}

Determine login information for the sliver resources:
{{{
$ readyToLogin.py -a ig-gpo IG-MON-3                   
<...>
PC-1's geni_status is: ready (am_status:ready) 
User lnevers logins to PC-1 using:
	xterm -e ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc5.instageni.gpolab.bbn.com &

PC-2's geni_status is: ready (am_status:ready) 
User lnevers logins to PC-2 using:
	xterm -e ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc4.instageni.gpolab.bbn.com &
}}}

Login to PC-1, verify hostname, address assignment and initial ARP table entries:
{{{
$ ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc5.instageni.gpolab.bbn.com
[lnevers@pc-1 ~]$ /sbin/ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 10:60:4B:9C:47:6A  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::1260:4bff:fe9c:476a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:492 (492.0 b)
          Interrupt:37 Memory:f2000000-f2012800 

[lnevers@pc-1 ~]$ /sbin/arp -a
ops.instageni.gpolab.bbn.com (192.1.242.133) at 02:07:2d:51:f4:6a [ether] on eth0
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth0
? (192.1.242.129) at cc:ef:48:7a:7a:a9 [ether] on eth0
}}}


Login to PC-2, verify hostname, address assignment and initial ARP table entries:
{{{
$ ssh -i /home/lnevers/.ssh/id_rsa lnevers@pc4.instageni.gpolab.bbn.com 
[lnevers@pc-2 ~]$ /sbin/ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 10:60:4B:96:00:D6  
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::1260:4bff:fe96:d6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:492 (492.0 b)
          Interrupt:37 Memory:f2000000-f2012800 

[lnevers@pc-2 ~]$ /sbin/arp -a
ops.instageni.gpolab.bbn.com (192.1.242.133) at 02:07:2d:51:f4:6a [ether] on eth0
boss.instageni.gpolab.bbn.com (192.1.242.132) at 02:9e:26:f1:52:99 [ether] on eth0
? (192.1.242.129) at cc:ef:48:7a:7a:a9 [ether] on eth0
}}}

Install iperf on both PCs and exchange TCP traffic from PC-1 client to PC-2 server over the data plane interface and look at ARP table entries:
{{{
[lnevers@pc-1 ~]$ /usr/bin/iperf -c 192.168.1.2 -i 1
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.1 port 58436 connected with 192.168.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   113 MBytes   947 Mbits/sec
[  3]  1.0- 2.0 sec   112 MBytes   940 Mbits/sec
[  3]  2.0- 3.0 sec   112 MBytes   943 Mbits/sec
[  3]  3.0- 4.0 sec   112 MBytes   942 Mbits/sec
[  3]  4.0- 5.0 sec   112 MBytes   940 Mbits/sec
}}}

Leave sliver running.


== Step 3. Start an OpenFlow experiment and leave it running == 

As experimenter create a 2 VMs !OpenFlow experiment in the GPO InstaGENI rack. RSpec used:
{{{
<?xml version="1.0" encoding="UTF-8"?>
<rspec xmlns="http://www.geni.net/resources/rspec/3"
       xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:sharedvlan="http://www.protogeni.net/resources/rspec/ext/shared-vlan/1"
       xs:schemaLocation="http://www.geni.net/resources/rspec/3
           http://www.geni.net/resources/rspec/3/request.xsd
           http://www.protogeni.net/resources/rspec/ext/shared-vlan/1
           http://www.protogeni.net/resources/rspec/ext/shared-vlan/1/request.xsd"
       type="request">

  <node client_id="gpo-ig" exclusive="false">
    <sliver_type name="emulab-openvz" />
    <interface client_id="gpo-ig:if0">
      <ip address="10.42.18.43" netmask="255.255.255.0" type="ipv4" />
    </interface>
  </node>

  <node client_id="gpo-ig2" exclusive="false">
    <sliver_type name="emulab-openvz" />
    <interface client_id="gpo-ig2:if0">
      <ip address="10.42.18.42" netmask="255.255.255.0" type="ipv4" />
    </interface>
  </node>

  <link client_id="openflow-mesoscale-0">
    <interface_ref client_id="gpo-ig:if0" />
    <sharedvlan:link_shared_vlan name="mesoscale-openflow" />
  </link>

  <link client_id="openflow-mesoscale-1">
    <interface_ref client_id="gpo-ig2:if0" />
    <sharedvlan:link_shared_vlan name="mesoscale-openflow" />
  </link>
</rspec>
}}}

Create a sliver and sliver for the 2 VMs on shared !OpenFlow VLAN 1750 in the GPO InstaGENI rack:
{{{
$ omni.py createslice IG-MON-3-OF
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Created slice with Name IG-MON-3-OF, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF, Expiration 2013-03-07 16:26:04+00:00
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createslice:
  Options as run:
		framework: pg

  Args: createslice IG-MON-3-OF
  Result Summary: Created slice with Name IG-MON-3-OF, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF, Expiration 2013-03-07 16:26:04+00:00
INFO:omni: ============================================================


$ omni.py createsliver -a ig-gpo IG-MON-3-OF ./instageni-2vm-vlan1750-at-gpo.rspec 
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF expires on 2013-03-07 16:26:04 UTC
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname ig-gpo with URL https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./instageni-2vm-vlan1750-at-gpo.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF
INFO:omni: (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=8af1230117ca9b383b6510da21d7abc7)
INFO:omni:Got return from CreateSliver for slice IG-MON-3-OF at https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0:
INFO:omni:<!-- Reserved resources for:
	Slice: IG-MON-3-OF
	at AM:
	URN: unspecified_AM_URN
	URL: https://instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0
 -->
INFO:omni:<rspec xmlns="http://www.geni.net/resources/rspec/3" xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xmlns:sharedvlan="http://www.protogeni.net/resources/rspec/ext/shared-vlan/1" xs:schemaLocation="http://www.geni.net/resources/rspec/3            http://www.geni.net/resources/rspec/3/manifest.xsd            http://www.protogeni.net/resources/rspec/ext/shared-vlan/1            http://www.protogeni.net/resources/rspec/ext/shared-vlan/1/request.xsd" type="manifest">

  <node client_id="gpo-ig" exclusive="false" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2065">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="gpo-ig:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2069" mac_address="02e922fceb01">
      <ip address="10.42.18.43" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm2-9"/><host name="gpo-ig.IG-MON-3-OF.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/><services><login authentication="ssh-keys" hostname="pc2.instageni.gpolab.bbn.com" port="32058" username="lnevers"/></services></node>

  <node client_id="gpo-ig2" exclusive="false" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2066">
    <sliver_type name="emulab-openvz"/>
    <interface client_id="gpo-ig2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth2" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2071" mac_address="0240e5291a6f">
      <ip address="10.42.18.42" netmask="255.255.255.0" type="ipv4"/>
    </interface>
  <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm2-10"/><host name="gpo-ig2.IG-MON-3-OF.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com"/><services><login authentication="ssh-keys" hostname="pc2.instageni.gpolab.bbn.com" port="32059" username="lnevers"/></services></node>

  <link xmlns:sharedvlan="http://www.protogeni.net/resources/rspec/ext/shared-vlan/1" client_id="openflow-mesoscale-0" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2068">
    <interface_ref client_id="gpo-ig:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2069"/>
    <sharedvlan:link_shared_vlan name="mesoscale-openflow"/>
  </link>

  <link xmlns:sharedvlan="http://www.protogeni.net/resources/rspec/ext/shared-vlan/1" client_id="openflow-mesoscale-1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2070">
    <interface_ref client_id="gpo-ig2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth2" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2071"/>
    <sharedvlan:link_shared_vlan name="mesoscale-openflow"/>
  </link>
</rspec>
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:
  Options as run:
		aggregate: ['ig-gpo']
		framework: pg
  Args: createsliver IG-MON-3-OF ./instageni-2vm-vlan1750-at-gpo.rspec
  Result Summary: Got Reserved resources RSpec from instageni-gpolab-bbn-com-protogeniv2 
INFO:omni: ============================================================
}}}

A FOAM sliver is needed to allow the traffic exchange. Using this FOAM RSpec:
{{{
<?xml version="1.0" encoding="UTF-8"?>
<rspec xmlns="http://www.geni.net/resources/rspec/3"
       xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:openflow="http://www.geni.net/resources/rspec/ext/openflow/3"
       xs:schemaLocation="http://www.geni.net/resources/rspec/3
           http://www.geni.net/resources/rspec/3/request.xsd
           http://www.geni.net/resources/rspec/ext/openflow/3
           http://www.geni.net/resources/rspec/ext/openflow/3/of-resv.xsd"
       type="request">

  <openflow:sliver description=" InstaGENI OpenFlow" email="lnevers@bbn.com">
    <openflow:controller url="tcp:mallorea.gpolab.bbn.com:33018" type="primary" />

    <openflow:group name="bbn-instageni-1750">
      <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:foam.instageni.gpolab.bbn.com+datapath+06:d6:84:34:97:c6:c9:00" 
	component_manager_id="urn:publicid:IDN+openflow:foam:foam.instageni.gpolab.bbn.com+authority+am"  />
    </openflow:group>

    <openflow:match>

      <openflow:use-group name="bbn-instageni-1750" />

      <openflow:packet>
        <openflow:dl_type value="0x800,0x806"/>
        <openflow:nw_dst value="10.42.18.0/24"/>
        <openflow:nw_src value="10.42.18.0/24"/>
      </openflow:packet>
    </openflow:match>

  </openflow:sliver>
</rspec>
}}}

Create a sliver at the GPO InstaGENI rack FOAM to allow the traffic exchange:
{{{
$ omni.py createsliver -a ig-of-gpo IG-MON-3-OF ./instageni-openflow-at-gpo.rspec     
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Substituting AM nickname ig-of-gpo with URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1, URN unspecified_AM_URN
WARNING:omni:You asked to use AM API 2, but the AM(s) you are contacting do not all speak that version.
WARNING:omni:At the URLs you are contacting, all your AMs speak AM API v1. 
WARNING:omni:Switching to AM API v1. Next time call Omni with '-V1'.
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF expires on 2013-03-07 16:26:04 UTC
INFO:omni:Substituting AM nickname ig-of-gpo with URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1, URN unspecified_AM_URN
INFO:omni:Substituting AM nickname ig-of-gpo with URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1, URN unspecified_AM_URN
INFO:omni:Creating sliver(s) from rspec file ./instageni-openflow-at-gpo.rspec for slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF
INFO:omni:Got return from CreateSliver for slice IG-MON-3-OF at https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1:
INFO:omni:<?xml version="1.0" encoding="UTF-8"?>
INFO:omni:  <!-- Reserved resources for:
	Slice: IG-MON-3-OF
	at AM:
	URN: unspecified_AM_URN
	URL: https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1
 -->
INFO:omni:  
<rspec xmlns="http://www.geni.net/resources/rspec/3"
       xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:openflow="http://www.geni.net/resources/rspec/ext/openflow/3"
       xs:schemaLocation="http://www.geni.net/resources/rspec/3
           http://www.geni.net/resources/rspec/3/manifest.xsd
           http://www.geni.net/resources/rspec/ext/openflow/3
           http://www.geni.net/resources/rspec/ext/openflow/3/of-resv.xsd"
       type="manifest">

  <openflow:sliver description=" InstaGENI OpenFlow" email="lnevers@bbn.com">
    <openflow:controller url="tcp:mallorea.gpolab.bbn.com:33018" type="primary" />

    <openflow:group name="bbn-instageni-1750">
      <openflow:datapath component_id="urn:publicid:IDN+openflow:foam:foam.instageni.gpolab.bbn.com+datapath+06:d6:84:34:97:c6:c9:00" 
	component_manager_id="urn:publicid:IDN+openflow:foam:foam.instageni.gpolab.bbn.com+authority+am"  />
    </openflow:group>

    <openflow:match>

      <openflow:use-group name="bbn-instageni-1750" />

      <openflow:packet>
        <openflow:dl_type value="0x800,0x806"/>
        <openflow:nw_dst value="10.42.18.0/24"/>
        <openflow:nw_src value="10.42.18.0/24"/>
      </openflow:packet>

    </openflow:match>

  </openflow:sliver>

</rspec>


INFO:omni: ------------------------------------------------------------
INFO:omni: Completed createsliver:

  Options as run:
		aggregate: ['ig-of-gpo']
		api_version: 1
		framework: pg

  Args: createsliver IG-MON-3-OF ./instageni-openflow-at-gpo.rspec

  Result Summary: Your AMs do not all speak requested API v2. At the URLs you are contacting, all your AMs speak AM API v1. Switching to AM API v1. Next time call Omni with '-V1'.
Got Reserved resources RSpec from foam-instageni-gpolab-bbn-com 
INFO:omni: ============================================================
}}}
This sliver is auto approved.  State can be confirmed with omni command:
{{{
$ omni.py sliverstatus -a ig-of-gpo IG-MON-3-OF -V1                               
INFO:omni:Loading config file /home/lnevers/.gcf/omni_config
INFO:omni:Using control framework pg
INFO:omni:Substituting AM nickname ig-of-gpo with URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1, URN unspecified_AM_URN
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF expires on 2013-03-07 16:26:04 UTC
INFO:omni:Substituting AM nickname ig-of-gpo with URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1, URN unspecified_AM_URN
INFO:omni:Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF:
INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF at AM https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1 has overall SliverStatus: ready
INFO:omni:Sliver status for Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF at AM URL https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1
INFO:omni:{
  "geni_status": "ready", 
  "geni_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF:8070b745-84c7-47eb-8e04-63a764997f3a", 
  "foam_pend_reason": null, 
  "foam_expires": "2013-03-07 16:26:04+00:00", 
  "geni_resources": [
    {
      "geni_urn": "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF:8070b745-84c7-47eb-8e04-63a764997f3a", 
      "geni_error": "", 
      "geni_status": "ready"
    }
  ], 
  "foam_status": "Approved"
}
INFO:omni: ------------------------------------------------------------
INFO:omni: Completed sliverstatus:
  Options as run:
		aggregate: ['ig-of-gpo']
		api_version: 1
		framework: pg
  Args: sliverstatus IG-MON-3-OF
  Result Summary: Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF expires on 2013-03-07 16:26:04 UTC
Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF at AM https://foam.instageni.gpolab.bbn.com:3626/foam/gapi/1 has overall SliverStatus: ready.
 Returned status of slivers on 1 of 1 possible aggregates. 
INFO:omni: ============================================================
}}}

Determine login for assigned nodes:
{{{
$ readyToLogin.py -a ig-gpo IG-MON-3-OF                
<...>
gpo-ig's geni_status is: ready (am_status:ready) 
User lnevers logins to gpo-ig using:
	xterm -e ssh -p 32058  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com &

gpo-ig2's geni_status is: ready (am_status:ready) 
User lnevers logins to gpo-ig2 using:
	xterm -e ssh -p 32059  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com &
}}}


Login to VM named 'gpo-ig' and send traffic to remote:
{{{
$ ssh -p 32058  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com 

[lnevers@ig-gpo ~]$ ping 10.42.18.42 -c 5
PING 10.42.18.42 (10.42.18.42) 56(84) bytes of data.
64 bytes from 10.42.18.42: icmp_req=1 ttl=64 time=6.73 ms
64 bytes from 10.42.18.42: icmp_req=2 ttl=64 time=0.077 ms
64 bytes from 10.42.18.42: icmp_req=3 ttl=64 time=0.075 ms
64 bytes from 10.42.18.42: icmp_req=4 ttl=64 time=0.075 ms
64 bytes from 10.42.18.42: icmp_req=5 ttl=64 time=0.075 ms

--- 10.42.18.42 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 0.075/1.407/6.735/2.664 ms
}}}

Login to VM named 'gpo-ig2' and send traffic to remote:
{{{
$ ssh -p 32059  -i /home/lnevers/.ssh/id_rsa lnevers@pc2.instageni.gpolab.bbn.com

[lnevers@ig-gpo2 ~]$ ping 10.42.18.43 -c 5
PING 10.42.18.43 (10.42.18.43) 56(84) bytes of data.
64 bytes from 10.42.18.43: icmp_req=1 ttl=64 time=6.33 ms
64 bytes from 10.42.18.43: icmp_req=2 ttl=64 time=0.075 ms
64 bytes from 10.42.18.43: icmp_req=3 ttl=64 time=0.074 ms
64 bytes from 10.42.18.43: icmp_req=4 ttl=64 time=0.076 ms
64 bytes from 10.42.18.43: icmp_req=5 ttl=64 time=0.074 ms

--- 10.42.18.43 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.074/1.327/6.337/2.505 ms
[lnevers@ig-gpo2 ~]$ 
}}}

Leave experiment running


== Step 4. Administrator review of running experiments ==
 
As administrator determine experiments status, resource assignment, and configuration for the 3 experiments that are running from previous steps:
  
 - EG-MON-3 - a 2 VM experiment 
 - IG-MON-3 - a 2 raw pc experiment
 - IG-MON-3-OF - a 2 VM !OpenFlow experiment
 
Determine list of active experiments, by selecting "Experimentation" pulldown in "red dot" mode, and choosing "Experiment list", which maps to the 
URL https://boss.instageni.gpolab.bbn.com/showexp_list.php3.  A capture of the active experiment page shows:

[[Image(IG-MON-3-experiments.jpg)]]
[[BR]]
Each of the 3 experiment created for this test case (EG-MON-3, IG-MON-3, and IG-MON-3-OF) are present in the list.

[[BR]]
In the Active Experiment table, select the experiment EG-MON-3 (2 VM experiment) and this shows show all know details for the experiment:

[[BR]]
[[Image(IG-MON-3-2vm.jpg)]]

[[BR]]
In the Active Experiment table, select the experiment IG-MON-3 (2 raw pc experiment) and this shows show all know details for the experiment:

[[BR]]
[[Image(IG-MON-3-2pc.jpg)]]

[[BR]]
In the Active Experiment table, select the experiment IG-MON-3-OF (2 VM !OpenFlow experiment)and this shows show all know details for the experiment:

[[BR]]
[[Image(IG-MON-3-2vm-of.jpg)]]

[[BR]]

For each of the above experiment clicking on the topology diagram on the left hand side panel will result in a details page that includes specific configuration and allocation for the experiment.

Clicking on the topology diagram for the experiment IG-MON-3 (2 raw pc experiment), this  shows node mapping, IP Port allocation, and other useful information:

[[Image(IG-MON-3-2pc-detail.jpg)]]

For the experiment IG-MON-3-OF (2 VM !OpenFlow experiment), click the topology diagram and the experiment topology along with the experiment details for Virtual Nodes, Physical Nodes Mapping, Port Allocation, Virtual Links, Physical Links, Links Mappings, and VLAN settings are shown:

[[Image(IG-MON-3-2vm-of-detail.jpg)]] 

[[BR]]

Compared the MAC addresses show on this page with the ones collected in earlier steps from logging into the experiment nodes and found them to be the same.


As administrator, log into the OpenVZ host pc2 and determine list of VMs running:

{{{
[lnevers@boss ~]$ ssh pc2
Last login: Wed Feb  6 15:10:36 2013 from boss.instageni.gpolab.bbn.com
[lnevers@vhost2 ~]$ 
[lnevers@vhost2 ~]$ sudo vzlist -a
      CTID      NPROC STATUS    IP_ADDR         HOSTNAME
         1         16 running   -               VM-1.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         2         16 running   -               VM-1.ln-pgeni-cred.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         3         37 running   -               gpo-ig.IG-CT-4.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         4         25 running   -               gpo-ig.SHORT-IG-CT-4.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         5         16 running   -               gpo-ig2.SHORT-IG-CT-4.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         6         22 running   -               VM-2.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         7         22 running   -               gpo-ig2.IG-CT-4.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
         8         23 running   -               server.portal01.panther.instageni.gpolab.bbn.com
         9         16 running   -               gpo-ig.IG-MON-3-OF.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
        10         22 running   -               gpo-ig2.IG-MON-3-OF.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com
[lnevers@vhost2 ~]$ 
}}}

While logged in to the OpenVZ host pc2, determine which container is running a given PID:
{{{
   UID          PID    PPID  C STIME TTY          TIME CMD
   20001      15017   13395  0 10:07 ?        00:00:07 /usr/bin/iperf -s

  [lnevers@vhost2 ~]$ sudo vzpid 15017
  Pid	CTID	Name
  15017	6	iperf
}}}

From the container list we can see that the CID 6 is hostname VM-2.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com.

Look up container experiment information, for example pc-vm2-6 (CID 6), see start-up attributes in ''/var/emulab/boot/tmcc.pcvm2-5''.
{{{
[lnevers@vhost2 ~]$ ls /var/emulab/boot/tmcc.pcvm2-5
accounts  creator    disks         ifconfig    linkdelay  mounts          nodeuuid    programs  startupcmd  tiptunnels  traceinfo  userenv
blobs     delay      eventkey      jailconfig  manifest   nodeattributes  nseconfigs  routing   syncserver  tpmblob     trafgens   watchdoginfo
bridges   dhcpdconf  firewallinfo  keyhash     motelog    nodeid          plabconfig  rpms      tarballs    tpmpubkey   tunnels
[lnevers@vhost2 ~]$ more /var/emulab/boot/tmcc.pcvm2-5/ifconfig 
INTERFACE IFACETYPE=vlan INET=10.42.13.42 MASK=255.255.255.0 ID=11 VMAC=02907d4e0e71 PMAC=10604B9B8216 RTABID=200 ENCAPSULATE=0 LAN=openflow-mesoscale-1 VT
AG=1750
}}}

Also review container for experiment logs:
{{{
[lnevers@vhost2 ~]$ ls /var/emulab/logs
bootsetup.debug          tbvnode-pcvm2-10.all  tbvnode-pcvm2-14.log  tbvnode-pcvm2-4.log  tbvnode-pcvm2-9.all     tmccproxy.pcvm2-2.log
dhclient-enter.log       tbvnode-pcvm2-10.log  tbvnode-pcvm2-15.log  tbvnode-pcvm2-5.all  tbvnode-pcvm2-9.log     tmccproxy.pcvm2-3.log
dhclient-exit.log        tbvnode-pcvm2-11.all  tbvnode-pcvm2-1.all   tbvnode-pcvm2-5.log  tmccproxy.pcvm2-10.log  tmccproxy.pcvm2-4.log
dhclient.log             tbvnode-pcvm2-11.log  tbvnode-pcvm2-1.log   tbvnode-pcvm2-6.all  tmccproxy.pcvm2-11.log  tmccproxy.pcvm2-5.log
emulab-systemd-swap.log  tbvnode-pcvm2-12.all  tbvnode-pcvm2-2.all   tbvnode-pcvm2-6.log  tmccproxy.pcvm2-12.log  tmccproxy.pcvm2-6.log
emulab-watchdog.log      tbvnode-pcvm2-12.log  tbvnode-pcvm2-2.log   tbvnode-pcvm2-7.all  tmccproxy.pcvm2-13.log  tmccproxy.pcvm2-7.log
progagent.debug          tbvnode-pcvm2-13.all  tbvnode-pcvm2-3.all   tbvnode-pcvm2-7.log  tmccproxy.pcvm2-14.log  tmccproxy.pcvm2-8.log
progwrap.debug           tbvnode-pcvm2-13.log  tbvnode-pcvm2-3.log   tbvnode-pcvm2-8.all  tmccproxy.pcvm2-15.log  tmccproxy.pcvm2-9.log
prunelvms                tbvnode-pcvm2-14.all  tbvnode-pcvm2-4.all   tbvnode-pcvm2-8.log  tmccproxy.pcvm2-1.log

[lnevers@vhost2 ~]$ ls -l /var/emulab/logs/tbvnode-pcvm2-6*
-rw-r--r-- 1 root root 433483 Mar  6 10:02 /var/emulab/logs/tbvnode-pcvm2-6.all
-rw-r--r-- 1 root root  13756 Mar  6 10:02 /var/emulab/logs/tbvnode-pcvm2-6.log
[lnevers@vhost2 ~]$ 

[lnevers@vhost2 ~]$ more /var/emulab/logs/tbvnode-pcvm2-6.log
TIMESTAMP: 10:02:23:478087 vnodesetup starting the real work
TIMESTAMP: 10:02:23:487690 vnodesetup calling into libsetup
Checking Testbed generic VM configuration ...
TIMESTAMP: 10:02:23:542273 vnodesetup back from libsetup
TIMESTAMP: 10:02:23:580189 vnodesetup (12570) waiting on vnode (12595)
TIMESTAMP: 10:02:23:580331 vnodesetup (12595) execing mkvnode.pl
libvnode: debug=1
TIMESTAMP: 03/06/2013 10:02:23:764243 starting openvz rootPreConfig()
TIMESTAMP: 03/06/2013 10:02:23:764494 finished openvz rootPreConfig()
GENVNODETYPE openvz
libops:
$VAR1 = 'openvz';
$VAR2 = {
          'vnodeUnmount' => sub { "DUMMY" },
          'vnodeDestroy' => sub { "DUMMY" },
          'init' => sub { "DUMMY" },
          'rootPostConfig' => sub { "DUMMY" },
          'vnodeCreate' => sub { "DUMMY" },
          'vnodeConfigResources' => sub { "DUMMY" },
          'setDebug' => sub { "DUMMY" },
          'vnodePreConfig' => sub { "DUMMY" },
          'vnodeConfigDevices' => sub { "DUMMY" },
          'vnodeReboot' => sub { "DUMMY" },
          'vnodeState' => sub { "DUMMY" },
          'vnodeExec' => sub { "DUMMY" },
          'rootPreConfig' => sub { "DUMMY" },
          'rootPreConfigNetwork' => sub { "DUMMY" },
          'vnodePreConfigExpNetwork' => sub { "DUMMY" },
           ....

}}}


== Step 5. Determine information for terminated experiments ==

As experimenter terminate the experiments EG-MON-3 (2 VMs experiment) and IG-MON-3 (2 raw pc experiment). 

Once the 2 experiments are terminated, as administrator use AM state, logs, or administrator interfaces to determine the following:
 - how many other experiments were run in the past day.
 - which GENI user created each of the terminated experiments.
 - mapping of experiments to OpenVZ or exclusive hosts for each of the terminated experiments.
 - control and dataplane MAC addresses assigned to each VM in each terminated experiment.
 - IP addresses assigned by InstaGENI to each VM in each terminated experiment. ???

Additionally, verify resource allocation, ownership information, MAC Address and IP Address information for terminated experiments whether using OpenVZ VMs or Dedicate PCs. 

Start by logging into https://boss.instageni.gpolab.bbn.com and switching to "red dot" mode. From the "Administration" pull-down choose  "ProtoGENI History", this brings a page of Aggregate History that contains the following type of information:

[[Image(IG-MON-3-AggHistory.jpg)]]

First clicked on "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3" to get history for the experiment EG-MON-3 (2 VMs experiment):

[[BR]]
[[Image(IG-MON-3-SliceHistory.jpg)]].  
[[BR]]

Also verified the "download all logs" link on the Log Files page and reviewed the content of files, which accurately capture the experiment EG-MON-3.

Back on the Aggregate History page, reviewed the manifest for the slice URN "urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3" , which showed the following manifest that accurately captures the resources in the sliver:

{{{
<rspec xmlns="http://www.geni.net/resources/rspec/3" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" 
xmlns:planetlab="http://www.planet-lab.org/resources/sfa/ext/planetlab/1" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="manifest" 
xsi:schemalocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/manifest.xsd"> 

<node client_id="VM-1" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false"   
   component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2036">
  <sliver_type name="emulab-openvz"> 
      <interface client_id="VM-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:lo0" 
        sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2040" mac_address="028fc5afa160"> 
        <ip address="192.168.1.1" netmask="255.255.255.0" type="ipv4"> 
        </ip>
     </interface> 
   <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm1-1">
   <host name="VM-1.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com">
     <services>
      <login authentication="ssh-keys" hostname="pc1.instageni.gpolab.bbn.com" port="31290" username="lnevers">
      </login>
     </services>
   </host>
   </rs:vnode>
  </sliver_type>
</node> 

<node client_id="VM-2" component_manager_id="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm" exclusive="false" 
   component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2037"> 
  <sliver_type name="emulab-openvz"> 
      <interface client_id="VM-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:lo0" 
        sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2041" mac_address="02fe79621711"> 
        <ip address="192.168.1.2" netmask="255.255.255.0" type="ipv4"> 
        </ip>
      </interface> 
   <rs:vnode xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1" name="pcvm1-5">
   <host name="VM-2.EG-MON-3.pgeni-gpolab-bbn-com.instageni.gpolab.bbn.com">
    <services>
     <login authentication="ssh-keys" hostname="pc1.instageni.gpolab.bbn.com" port="31291" username="lnevers">
     </login>
    </services>
   </host>
   </rs:vnode>
  </sliver_type>
</node> 

<link client_id="lan0" sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2039"> 
  <component_manager name="urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm"> 
    <interface_ref client_id="VM-1:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:lo0" 
      sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2040"> 
    <interface_ref client_id="VM-2:if0" component_id="urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:lo0" 
      sliver_id="urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+2041"> 
    <link_type name="lan">
   </link_type>
   </interface_ref>
   </interface_ref> 
  </component_manager>
</rspec> 
}}}

Using the above approach determined the following information for the three experiments in this test case:

|| '''Slice ID''' || '''Sliver URN''' || '''Physical nodes'''  || '''Component and hostname'''  || '''Dataplane IPs and MACs'''|| Create/Terminated||
||2067||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF|| pc2  ||pcvm2-9 (gpo-ig)  ||10.42.18.43(02e922fceb01) ||2013-03-06 10:37:53/not terminated||
||2067||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3-OF|| pc2  ||pcvm2-10 (gpo-ig2)||10.42.18.42(0240e5291a6f) ||2013-03-06 10:37:53/not terminated||
||2050||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3   || pc2  ||pcvm2-1 (VM-1)    ||192.168.1.1(02104639f347) ||2013-03-06 10:02:16/2013-03-06 13:45:27||
||2050||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+EG-MON-3   || pc2  ||pcvm2-6 (VM-2)    ||192.168.1.2(020063c74361) ||2013-03-06 10:02:16/2013-03-06 13:45:27||
||2044||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3   || pc5  ||pc5 (PC-1)        ||192.168.1.1(10604B9C476A) ||2013-03-06 09:50:29/2013-03-06 13:44:14||
||2044||urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+IG-MON-3   || pc4  ||pc4 (PC-2)        ||192.168.1.2(10604B9600D6) ||2013-03-06 09:50:29/2013-03-06 13:44:14||



== Step 6: get !OpenFlow state information ==

'''Using:'''
 * On the dataplane switch, get a list of controllers, and see if any additional controllers are serving experiments.
 * On the flowvisor VM, get a list of active FV slices from the !FlowVisor
 * On the FOAM VM, get a list of active slivers from FOAM
 * Use FV, FOAM, or the switch to list the flowspace of a running !OpenFlow experiment.

'''Verify:'''
 * A site administrator can get information about the !OpenFlow resources used by running experiments.
 * When an !OpenFlow experiment is started by InstaGENI, a new controller is added directly to the switch.
 * No new !FlowVisor slices are added for new !OpenFlow experiments started by InstaGENI.
 * No new FOAM slivers are added for new !OpenFlow experiments started by InstaGENI.

== Step 7: verify MAC addresses on the rack dataplane switch ==

'''Using:'''
 * Establish a privileged login to the dataplane switch
 * Obtain a list of the full MAC address table of the switch
 * On boss and the experimental hosts, use available data sources to determine which host or VM owns each MAC address.

'''Verify:'''
 * It is possible to identify and classify every MAC address visible on the switch 

== Step 8: verify active dataplane traffic ==

'''Using:'''
 * Establish a privileged login to the dataplane switch
 * Based on the information from Step 7, determine which interfaces are carrying traffic between the experimental VMs
 * Collect interface counters for those interfaces over a period of 10 minutes
 * Estimate the rate at which the experiment is sending traffic

'''Verify:'''
 * The switch reports interface counters, and an administrator can obtain plausible estimates of dataplane traffic quantities by looking at them.