[[PageOutline]] = GENI OpenFlow Slicer Test Plan = Intro describing goals of testing (restate the goals from the requirements but as validation). Make sure to provide a link to the [wiki:OpenFlow/Slicer/Requirements GENI Slicer Requirements] that the plan is based on. If specific types of traffic are verified for each test case (ex. UDP, TCP, ICMP..) then make sure it is stated as a validation goal. Describe the expected outcome: - validation of all requirement to determine which are met (must, may, and wishlist). - validation of interactions with currently supported substrate (vendors, versions) - validation of interactions with currently available Service abstraction layer. - description of how status will be made available (periodic emails or wiki pages updates?) (where?) - final test report (wiki pages?) (where?) == Assumptions and Dependencies == Describe any assumptions, for example: - slicer software must interact with OF v1.0 - slicer software should interact with OF v1.3 - Slicer must interact with specific OF switches: x, y, z... - Slicer must interact with service abstraction layer: x, y, z.... - Documentation must is available describing configuration and usage. Describe dependencies, for example: - Tests will use the GENI AM API where possible. Most tests cannot be completed without the AM API = Slicer Test Cases = == OF-OPR-SLCR-FN Testing Scenario: == === Step 1: Setup === * Reserve 5 slices (using the add_slice function) [005-a]: * Slice 'samevlan' with two ports on a single switch, each on the same VLAN. Ensure the slicer state is accurate. [001-a] * Slice 'diffvlans' with two ports on the same switch, each associated with a different VLAN. Ensure the slicer state is accurate. [001-a] * Slice 'tenvlans' with 10 VLANs spanning across 10 switches. Ensure the slicer state is accurate. [001-a] * Slice 'untaggedports' with two untagged ports on a single switch (doesn't need to be possible to pass). Ensure the slicer state is accurate. [001-b] * Slice 'fullports' with two full ports on a single switch (doesn't need to be possible to pass). Ensure the slicer state is accurate. [001-c] * Add a hosts attached to each port with the appropriate data plane VLAN connections for 'samevlan', 'diffvlans', and hosts for at least two of the VLANs for 'tenvlans' * Add two hosts which are on the same VLAN but do not correspond to an existing slice. === Step 2: Establish that basic connectivity works === * Run a modified pox forwarding.l2_learning controller for 'samevlan' that installs a send-to-controller rule for unmatched traffic within the VLAN. * Start a packet capture on the slicer facing both the switch and the controller. * Using a host that is not in a slice, send traffic on the data plane and make sure it never reaches the other host. [002] [004-a] [004-c] * Using 'samevlan', send data plane traffic between the two hosts. Ensure that it gets through to the two hosts in 'samevlan', but not to the hosts in 'diffvlans'. [001] [003] * Stop the packet capture, and check for the following: * Check that traffic from the hosts that don't belong to a slice never shows up at the slicer. [004-a] [004-c] * Check that traffic from the slicer destined for a controller is only sent to a single controller [004-b] [004-d] * Check that packet-ins from the switch have a VLAN tag, but packet-ins going to the controller do not. Also ensure that nothing else is different. [014-b] [009] * Check that packet-outs from the controller have no VLAN tag, but packet-outs going to the switch do. Also ensure that nothing else is different. [014-a] [009] * Traffic sent to the OFPP_ALL port from the controller is funnelled into the set of ports in the slice by the slicer. [003-c] === Step 3: Test for VLAN enforcement and translation === * Stop the pox forwarding.l2_learning controller for 'samevlan'. Start a floodlight instance in its place with static flow pusher enabled. * Push a flow that matches one of the ports and VLANs, and tries to do a VLAN rewrite to a different VLAN ID. Ensure this gets rejected by the slicer, and an error message is returned. [003-b] [004-e] [008-a] * Push a flow that matches one of the ports and VLANs, and tries to do a strip-VLAN action. Ensure this gets rejected by the slicer, and an error message is returned. [003-a] [004-e] [008-a] * Run a modified pox forwarding.l2_learning controller for 'diffvlans' that installs a send-to-controller rule for unmatched traffic within the VLAN. * Send ping traffic between the 'diffvlans' hosts and make sure it gets exchanged properly. [014-d] === Step 4: Test for OF functionality === * Start a packet capture on the slicer facing both the switch and the controller. * Push a flow_stats request from the controller. * Stop the packet capture, and check for the following: the flow_stats reply only went to the controller that requested it. [004-g] * Run some OFTest checks to make sure that [004-f] and [009] are met. * Test a few synchronous messages using OFTest to make sure [006] is met. * Test that [007-a], [007-b], and [007-c] all generate error messages. === Step 5: Test for stacked slicer support === * For 'diffvlans' stop the pox forwarding.l2_learning controller. In its place, stand up another instance of the slicer. In this higher-level slicer, create an identical slice for 'diffvlans' (can be done manually), and point it at a new controller URL. At that new controller URL, run an instance of floodlight forwarding with static flow pusher. Push down a flow that matches on the slice's VLAN for the switch and forwards traffic to the controller. * Send data plane traffic between the 'diffvlans' hosts. === Step 6: Test for any extra fucntionality === * Test that any documented control plane performance isoaltion solution works as expected. [011] * Test that any documented data plane performance isoaltion solution works as expected. [012] * Test that any per-slice flow limits work as expected. [012] === Step 7: Test API === * Call dump_slices and make sure it works as expected. Ensure it returns which slices are active and which are disabled along with however the AM identifies the slices. [005-d] [008-b] [008-c] * Call change_controller and make sure it is reflected in the slicer state [005-c] * Call update_slice if possible and report back what happens [005-e] * Call delete_slice and clean up all slices [005-b] == OF-OPF-SLCR-OP Testing Scenario == === Step 1: Setup === * Clear any existing log files that the slicer may have created. * Create 10 slices named slice{1-10}. Each slice should have a single unique VLAN ID spanning across 10 switches between two hosts. Generate ping traffic from a host at the end of each of the slices and make sure it is working. Leave this traffic running for the duration of this scenario. [003] === Step 2: Collect monitoring data === * Make sure there is some way to collect monitoring data for all of the slices that have been set up. [002] === Step 3: Test slice operations === * Create a new slice 'testslice', use the modified pox forwarding.l2_learning controller, and ensure that pings in the other slices continue to flow. Ensure that this slice is added with no operator intervention. [004] [001-a] * Change the controller URL for 'testslice'. Ensure that the change takes place with no operator intervention. [001-c] * If a slicer is capable of disabling a slice, trigger 'testslice' to get disabled. Ensure that the slicer owner either gets notified, or that at the very least, 'dump_slices' shows that the slice is disabled. Re-enable the slice if it was disabled. [001-d] * Disconnect the controller. Wait for about a minute, and try sending traffic. Ensure that it doesn't get through. [008] * Delete the slice and make sure that doing so does not require operator intervention. Check right away and make sure all flows for that slice have been removed. [001-b] [009] === Step 4: Substrate-generated traffic === * Connect a host to the substrate that is not part of a slice, and that does not share a data plane port with the other slices. Send a high rate of traffic out of its data plane interface and ensure that the other slices remain stable. [007] === Step 5: Test logging === * Check that a log exists for the slicer. [005] * Configure the logging to rotate on a daily basis. * Look at the entries for the log. Any entries that exist in the default case should be reasonable. At this point, the size of the log should also be reasonable. [006-a] * Try changing the log level while the slicer is running to the most verbose level. Verify that the logs are showing more data. [006-b] * Set the logging back to a standard level. Leave the setup running overnight. In the morning, verify that the logs have been rotated, and that logging is still working. [006-c] Suggested 1 software development/support test case: - Make sure that there is a support mail-list, use list during testing, and make sure access to development/support is documented for users (OF-OPR-SLCR-SW-001, OF-OPR-SLCR-SW-002). - Verify existence of migration plan from OF 1.0 to OF 1.3. (OF-OPR-SLCR-SW-003) - Repository access validation (OF-OPR-SLCR-SW-004, OF-OPR-SLCR-SW-005, OF-OPR-SLCR-SW-006) - Software installation/installation and runtime (OF-OPR-SLCR-SW-007) '' Note 3: Note sure how to track OF-OPR-SLCR-SW-008? '' Define test cases that capture supplementary test cases for the [wiki:OpenFlow/Slicer/Requirements#Wishlist Wishlist Functionality] that may be delivered. = Test Methodology = Describe how testing is to be conducted, how issues are to be tracked, how status will be made available. Describe how test cases are named. = Glossary = (if needed)