Changes between Version 14 and Version 15 of OpenFlow/Slicer/TestPlan


Ignore:
Timestamp:
11/12/14 13:44:17 (5 years ago)
Author:
tupty@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OpenFlow/Slicer/TestPlan

    v14 v15  
    3636
    3737=== Step 2: Establish that basic connectivity works ===
    38  * Run a modified pox forwarding.l2_learning controller for 'samevlan' that installs a send-to-controller rule for unmatched traffic within the VLAN.
    39  * Start a packet capture on the slicer facing both the switch and the controller.
    40  * Using a host that is not in a slice, send ICMP ping traffic on the data plane and make sure it never reaches the other host. ''(002)'' ''(004-a)'' ''(004-c)''
    41  * Using 'samevlan', send ICMP ping data plane traffic between the two hosts.  Ensure that it gets through to the two hosts in 'samevlan', but not to the hosts in 'diffvlans'. ''(001)'' ''(003)''
    42  * Stop the packet capture, and check for the following:
    43   * Check that traffic from the hosts that don't belong to a slice never shows up at the slicer. ''(004-a)'' ''(004-c)''
    44   * Check that traffic from the slicer destined for a controller is only sent to a single controller ''(004-b)'' ''(004-d)''
    45   * Check that packet-ins from the switch have a VLAN tag, but packet-ins going to the controller do not.  Also ensure that nothing else is different. ''(014-b)'' ''(009)''
    46   * Check that packet-outs from the controller have no VLAN tag, but packet-outs going to the switch do.  Also ensure that nothing else is different. ''(014-a)'' ''(009)''
    47   * Traffic sent to the OFPP_ALL port from the controller is funnelled into the set of ports in the slice by the slicer. ''(003-c)''
     38 a. Run a modified pox forwarding.l2_learning controller for 'samevlan' that installs a send-to-controller rule for unmatched traffic within the VLAN.
     39 b. Start a packet capture on the slicer facing both the switch and the controller.
     40 c. Using a host that is not in a slice, send ICMP ping traffic on the data plane and make sure it never reaches the other host. ''(002)'' ''(004-a)'' ''(004-c)''
     41 d. Using 'samevlan', send ICMP ping data plane traffic between the two hosts.  Ensure that it gets through to the two hosts in 'samevlan', but not to the hosts in 'diffvlans'. ''(001)'' ''(003)''
     42 e. Stop the packet capture, and check for the following:
     43  i. Check that traffic from the hosts that don't belong to a slice never shows up at the slicer. ''(004-a)'' ''(004-c)''
     44  ii. Check that traffic from the slicer destined for a controller is only sent to a single controller ''(004-b)'' ''(004-d)''
     45  iii. Check that packet-ins from the switch have a VLAN tag, but packet-ins going to the controller do not.  Also ensure that nothing else is different. ''(014-b)'' ''(009)''
     46  iv. Check that packet-outs from the controller have no VLAN tag, but packet-outs going to the switch do.  Also ensure that nothing else is different. ''(014-a)'' ''(009)''
     47  v. Traffic sent to the OFPP_ALL port from the controller is funnelled into the set of ports in the slice by the slicer. ''(003-c)''
    4848
    4949=== Step 3: Test for VLAN enforcement and translation ===
    50  * Stop the pox forwarding.l2_learning controller for 'samevlan'.  Start a floodlight instance in its place with static flow pusher enabled.
    51  * Push a flow that matches one of the ports and VLANs, and tries to do a VLAN rewrite to a different VLAN ID.  Ensure this gets rejected by the slicer, and an error message is returned. ''(003-b)'' ''(004-e)'' ''(008-a)''
    52  * Push a flow that matches one of the ports and VLANs, and tries to do a strip-VLAN action.  Ensure this gets rejected by the slicer, and an error message is returned. ''(003-a)'' ''(004-e)'' ''(008-a)''
    53  * Run a modified pox forwarding.l2_learning controller for 'diffvlans' that installs a send-to-controller rule for unmatched traffic within the VLAN.
    54  * Send ICMP ping traffic between the 'diffvlans' hosts and make sure it gets exchanged properly. ''(014-d)''
     50 a. Stop the pox forwarding.l2_learning controller for 'samevlan'.  Start a floodlight instance in its place with static flow pusher enabled.
     51 b. Push a flow that matches one of the ports and VLANs, and tries to do a VLAN rewrite to a different VLAN ID.  Ensure this gets rejected by the slicer, and an error message is returned. ''(003-b)'' ''(004-e)'' ''(008-a)''
     52 c. Push a flow that matches one of the ports and VLANs, and tries to do a strip-VLAN action.  Ensure this gets rejected by the slicer, and an error message is returned. ''(003-a)'' ''(004-e)'' ''(008-a)''
     53 d. Run a modified pox forwarding.l2_learning controller for 'diffvlans' that installs a send-to-controller rule for unmatched traffic within the VLAN.
     54 e. Send ICMP ping traffic between the 'diffvlans' hosts and make sure it gets exchanged properly. ''(014-d)''
    5555
    5656=== Step 4: Test for OF functionality ===
    57  * Start a packet capture on the slicer facing both the switch and the controller.
    58  * Push a flow_stats request from the controller.
    59  * Stop the packet capture, and check for the following: the flow_stats reply only went to the controller that requested it. ''(004-g)''
    60  * Run some OFTest checks to make sure that ''(004-f)'' and ''(009)'' are met.
    61  * Test a few synchronous messages using OFTest to make sure ''(006)'' is met.
    62  * Test that ''(007-a)'', ''(007-b)'', and ''(007-c)'' all generate error messages.
     57 a. Start a packet capture on the slicer facing both the switch and the controller.
     58 b. Push a flow_stats request from the controller.
     59 c. Stop the packet capture, and check for the following: the flow_stats reply only went to the controller that requested it. ''(004-g)''
     60 d. Run some OFTest checks to make sure that ''(004-f)'' and ''(009)'' are met.
     61 e. Test a few synchronous messages using OFTest to make sure ''(006)'' is met.
     62 f. Test that ''(007-a)'', ''(007-b)'', and ''(007-c)'' all generate error messages.
    6363
    6464=== Step 5: Test for stacked slicer support ===
    65  * For 'diffvlans' stop the pox forwarding.l2_learning controller.  In its place, stand up another instance of the slicer.  In this higher-level slicer, create an identical slice for 'diffvlans' (can be done manually), and point it at a new controller URL.  At that new controller URL, run an instance of floodlight forwarding with static flow pusher.  Push down a flow that matches on the slice's VLAN for the switch and forwards traffic to the controller.
    66  * Send ICMP ping data plane traffic between the 'diffvlans' hosts. ''(010)''
     65 a. For 'diffvlans' stop the pox forwarding.l2_learning controller.  In its place, stand up another instance of the slicer.  In this higher-level slicer, create an identical slice for 'diffvlans' (can be done manually), and point it at a new controller URL.  At that new controller URL, run an instance of floodlight forwarding with static flow pusher.  Push down a flow that matches on the slice's VLAN for the switch and forwards traffic to the controller.
     66 b. Send ICMP ping data plane traffic between the 'diffvlans' hosts. ''(010)''
    6767
    6868
    6969=== Step 6: Test for any extra fucntionality ===
    70  * Test that any documented control plane performance isoaltion solution works as expected. ''(011)''
    71  * Test that any documented data plane performance isoaltion solution works as expected. ''(012)''
    72  * Test that any per-slice flow limits work as expected. ''(012)''
     70 a. Test that any documented control plane performance isoaltion solution works as expected. ''(011)''
     71 b. Test that any documented data plane performance isoaltion solution works as expected. ''(012)''
     72 c. Test that any per-slice flow limits work as expected. ''(012)''
    7373
    7474=== Step 7: Test API ===
    75  * Call dump_slices and make sure it works as expected.  Ensure it returns which slices are active and which are disabled along with however the AM identifies the slices. ''(005-d)'' ''(008-b)'' ''(008-c)''
    76  * Call change_controller and make sure it is reflected in the slicer state ''(005-c)''
    77  * Call update_slice if possible and report back what happens ''(005-e)''
    78  * Call delete_slice and clean up all slices ''(005-b)''
     75 a. Call dump_slices and make sure it works as expected.  Ensure it returns which slices are active and which are disabled along with however the AM identifies the slices. ''(005-d)'' ''(008-b)'' ''(008-c)''
     76 b. Call change_controller and make sure it is reflected in the slicer state ''(005-c)''
     77 c. Call update_slice if possible and report back what happens ''(005-e)''
     78 d. Call delete_slice and clean up all slices ''(005-b)''
    7979
    8080== OF-OPF-SLCR-OP Testing Scenario ==
    8181
    8282=== Step 1: Setup ===
    83  * Clear any existing log files that the slicer may have created.
    84  * Create 10 slices named slice{1-10}.  Each slice should have a single unique VLAN ID spanning across 10 switches between two hosts.  Generate ping traffic from a host at the end of each of the slices and make sure it is working.  Leave this traffic running for the duration of this scenario. ''(003)''
     83 a. Clear any existing log files that the slicer may have created.
     84 b. Create 10 slices named slice{1-10}.  Each slice should have a single unique VLAN ID spanning across 10 switches between two hosts.  Generate ping traffic from a host at the end of each of the slices and make sure it is working.  Leave this traffic running for the duration of this scenario. ''(003)''
    8585
    8686=== Step 2: Collect monitoring data ===
    87  * Make sure there is some way to collect monitoring data for all of the slices that have been set up. ''(002)'' ''(010)''
     87 a. Make sure there is some way to collect monitoring data for all of the slices that have been set up. ''(002)'' ''(010)''
    8888
    8989=== Step 3: Test slice operations ===
    90  * Create a new slice 'testslice', use the modified pox forwarding.l2_learning controller, and ensure that pings in the other slices continue to flow.  Ensure that this slice is added with no operator intervention. ''(004)'' ''(001-a)''
    91  * Change the controller URL for 'testslice'.  Ensure that the change takes place with no operator intervention. ''(001-c)''
    92  * If a slicer is capable of disabling a slice, trigger 'testslice' to get disabled.  Ensure that the slicer owner either gets notified, or that at the very least, 'dump_slices' shows that the slice is disabled.  Re-enable the slice if it was disabled. ''(001-d)''
    93  * Disconnect the controller.  Wait for about a minute, and try sending traffic.  Ensure that it doesn't get through. ''(008)''
    94  * Delete the slice and make sure that doing so does not require operator intervention.  Check right away and make sure all flows for that slice have been removed. ''(001-b)'' ''(009)''
     90 a. Create a new slice 'testslice', use the modified pox forwarding.l2_learning controller, and ensure that pings in the other slices continue to flow.  Ensure that this slice is added with no operator intervention. ''(004)'' ''(001-a)''
     91 b. Change the controller URL for 'testslice'.  Ensure that the change takes place with no operator intervention. ''(001-c)''
     92 c. If a slicer is capable of disabling a slice, trigger 'testslice' to get disabled.  Ensure that the slicer owner either gets notified, or that at the very least, 'dump_slices' shows that the slice is disabled.  Re-enable the slice if it was disabled. ''(001-d)''
     93 d. Disconnect the controller.  Wait for about a minute, and try sending traffic.  Ensure that it doesn't get through. ''(008)''
     94 e. Delete the slice and make sure that doing so does not require operator intervention.  Check right away and make sure all flows for that slice have been removed. ''(001-b)'' ''(009)''
    9595
    9696=== Step 4: Substrate-generated traffic ===
    97  * Connect a host to the substrate that is not part of a slice, and that does not share a data plane port with the other slices.  Send a high rate of traffic out of its data plane interface and ensure that the other slices remain stable. ''(007)''
    98  * Try sending traffic that the slicer cannot process (e.g. non-OpenFlow traffic or an OpenFlow version that the slicer doesn't currently understand) directly to the southbound interface of the slicer.  Make sure the slicer remains stable. ''(011)''
     97 a. Connect a host to the substrate that is not part of a slice, and that does not share a data plane port with the other slices.  Send a high rate of traffic out of its data plane interface and ensure that the other slices remain stable. ''(007)''
     98 b. Try sending traffic that the slicer cannot process (e.g. non-OpenFlow traffic or an OpenFlow version that the slicer doesn't currently understand) directly to the southbound interface of the slicer.  Make sure the slicer remains stable. ''(011)''
    9999
    100100=== Step 5: Test logging ===
    101  * Check that a log exists for the slicer. ''(005)''
    102  * Configure the logging to rotate on a daily basis.
    103  * Look at the entries for the log.  Any entries that exist in the default case should be reasonable.  At this point, the size of the log should also be reasonable. ''(006-a)''
    104  * Try changing the log level while the slicer is running to the most verbose level.  Verify that the logs are showing more data. ''(006-b)''
    105  * Set the logging back to a standard level.  Leave the setup running overnight.  In the morning, verify that the logs have been rotated, and that logging is still working. ''(006-c)''
     101 a. Check that a log exists for the slicer. ''(005)''
     102 b. Configure the logging to rotate on a daily basis.
     103 c. Look at the entries for the log.  Any entries that exist in the default case should be reasonable.  At this point, the size of the log should also be reasonable. ''(006-a)''
     104 d. Try changing the log level while the slicer is running to the most verbose level.  Verify that the logs are showing more data. ''(006-b)''
     105 e. Set the logging back to a standard level.  Leave the setup running overnight.  In the morning, verify that the logs have been rotated, and that logging is still working. ''(006-c)''
    106106
    107107== OF-OPR-SLCR-SW Testing ==
    108  * Ensure there is a support mailing list and some level of documentation for users and administrators. ''(001)'' ''(002)'' ''(009)''
    109  * Ensure with the development team that there is a plan to support OpenFlow 1.3 in the future. ''(003)''
    110  * Ensure that there is a repository which we can acccess. ''(004)''
    111  * Ensure that upgrades of software don't destroy previous configuration. ''(007)''
    112  * Work with the development team to make sure they can satisfy remaining software requirements. ''(005)'' ''(006)'' ''(008)''
     108 a. Ensure there is a support mailing list and some level of documentation for users and administrators. ''(001)'' ''(002)'' ''(009)''
     109 b. Ensure with the development team that there is a plan to support OpenFlow 1.3 in the future. ''(003)''
     110 c. Ensure that there is a repository which we can acccess. ''(004)''
     111 d. Ensure that upgrades of software don't destroy previous configuration. ''(007)''
     112 e. Work with the development team to make sure they can satisfy remaining software requirements. ''(005)'' ''(006)'' ''(008)''
    113113
    114114== Wishlist Testing ==