Version 32 (modified by 9 years ago) (diff) | ,
---|
OpenFlow Slicer Test Plan Results
This page tracks the results of a piece of software run through the OpenFlow Slicer Test Plan.
Software | Version |
Flowspace Firewall | 1.0.4b |
Results Legend
Individual test case status may be one of the following:
- Color(green,Pass)? - Test case completed and met all criteria
- Color(#98FB98,Pass: most criteria)? - Test case completed and met most criteria with exceptions documented
- Color(red,Fail)? - Test case completed and failed.
- Color(orange,Blocked)? - Remaining testing blocked by ticketed issues.
- Color(#AAAAAA,N/A)? - Tests which are not applicable (only used sparingly).
OF-OPR-SLCR-FN Testing
Results
Step 1: Setup
Test Case | Result | Note |
1.a.i. | Color(green,Pass)? | |
1.a.ii. | Color(green,Pass)? | |
1.a.iii. | Color(green,Pass)? | |
1.a.iv. | Color(green,Pass)? | |
1.a.v. | Color(green,Pass)? | |
1.b. | Color(green,Pass)? | |
1.c. | Color(green,Pass)? |
Step 2: Establish that basic connectivity works
Test Case | Result | Note |
2.a. | Color(green,Pass)? | |
2.b. | Color(green,Pass)? | |
2.c. | Color(green,Pass)? | |
2.d. | Color(green,Pass)? | |
2.e. | Color(green,Pass)? | |
2.f.i. | Color(red,Fail)? | Slicer capture. This failure might be an issue with environment setup. Is slicer responsible for installing a "drop unmatched" rule? |
2.f.ii. | Color(green,Pass)? | Slicer capture |
2.f.iii. | Color(green,Pass)? | Slicer capture, Controller capture |
2.f.iv. | Color(green,Pass)? | Slicer capture, Controller capture |
2.f.v. | Color(green,Pass)? | Slicer capture, Controller capture |
2.g. | Color(green,Pass)? |
Step 3: Test for VLAN enforcement and translation
Test Case | Result | Note |
3.a. | Color(green,Pass)? | |
3.b. | Color(green,Pass)? | Controller capture |
3.c. | Color(green,Pass)? | Controller capture |
3.d. | Color(green,Pass)? | |
3.e. | Color(green,Pass)? |
Step 4: Check data plane throughput
Test Case | Result | Note |
4.a. | Color(green,Pass)? | |
4.b. | Color(green,Pass)? | |
4.c. | Color(green,Pass)? | |
4.d. | Color(green,Pass)? | |
4.e. | Color(green,Pass)? |
Step 5: Test for OF functionality
Test Case | Result | Note |
5.a. | Color(green,Pass)? | |
5.b. | Color(green,Pass)? | |
5.c. | Color(green,Pass)? | |
5.d.i. | Color(red,Fail)? | Known issue, will be fixed in next release. We should look into other similar instances where symmetric messages which are served by FSFW (and not the substrate) appropriately set the XID. |
5.d.ii. | Color(green,Pass)? | |
5.d.iii. | Color(green,Pass)? | |
5.e.i. | Color(green,Pass)? | Slicer capture, Slicer responds but never sends to substrate. |
5.e.ii. | Color(green,Pass)? | Slicer capture, Slicer doesn't respond, message never sent to substrate. |
Test 6: Test for stacked slicer support
Test Case | Result | Note |
6.a. | Color(green,Pass)? | |
6.b. | Color(green,Pass)? |
Test 7: Test for any extra functionality
Test Case | Result | Note |
7.a. | Color(#98FB98,Pass: most criteria)? | Controller capture, Limits seem to be enforced. Cases in which slice is disabled don't seem to be documented. Specific types of OF error messages sent in different cases don't seem to be documented. |
7.b. | Color(#AAAAAA,N/A)? | No data plane limiters. |
7.c. | Color(#98FB98,Pass: most criteria)? | Controller capture, Limits seem to be enforced. Cases in which slice is disabled don't seem to be documented. Specific types of OF error messages sent in different cases don't seem to be documented. |
Test 8: Test management API
Test Case | Result | Note |
8.a. | Color(orange,Blocked)? | Management API software under development. |
8.b. | Color(orange,Blocked)? | Management API software under development. |
8.c. | Color(orange,Blocked)? | Management API software under development. |
8.d. | Color(orange,Blocked)? | Management API software under development. |
Logs
This test was performed in GENI using vSwitches.
OF-OPR-SLCR-OP Testing (in GENI)
Results
Step 1: Setup
Test Case | Result | Note |
1.a. | Color(green,Pass)? | |
1.b. | Color(green,Pass)? |
Step 2: Collect monitoring data
Test Case | Result | Note |
2.a. | Color(green,Pass)? | Slice list, Slice1 status, Slice1 flows |
Step 3: Test slice operations
Test Case | Result | Note |
3.a. | Color(green,Pass)? | |
3.b. | Color(green,Pass)? | |
3.c. | Color(green,Pass)? | |
3.d. | Color(green,Pass)? | |
3.e. | Color(green,Pass)? | Substrate flows, no flows with testslice's VLAN ID of 200 |
Step 4: Substrate-generated traffic
Test Case | Result | Note |
4.a. | Color(green,Pass)? | |
4.b. | Color(green,Pass)? |
Step 5: Test logging
Test Case | Result | Note |
5.a. | Color(green,Pass)? | |
5.b. | Color(green,Pass)? | |
5.c. | Color(green,Pass)? | |
5.d. | Color(green,Pass)? | |
5.e. | Color(green,Pass)? |
Logs
This test was run using GENI and virtual switches. Parts of test case 3 were changed after this run.
OF-OPR-SLCR-OP Testing (in hardware testbed)
Results
Step 1: Setup
Test Case | Result | Note |
1.a. | Color(green,Pass)? | |
1.b. | Color(green,Pass)? |
Step 2: Collect monitoring data
Test Case | Result | Note |
2.a. | Color(green,Pass)? | Slice list, Slice1 status, Slice1 flows |
Step 3: Test slice operations
Test Case | Result | Note |
3.a. | Color(green,Pass)? | |
3.b. | Color(green,Pass)? | |
3.c. | Color(green,Pass)? | |
3.d. | Color(green,Pass)? | |
3.e. | Color(red,Fail)? | Substrate flows, flows with testslice's VLAN ID of 200 still exist. This should be somewhat fixed in the next release with a granularity of the FSFW flow stat polling rate. |
Step 4: Substrate-generated traffic
Test Case | Result | Note |
4.a. | Color(green,Pass)? | |
4.b. | Color(green,Pass)? |
Step 5: Test logging
Test Case | Result | Note |
5.a. | Color(green,Pass)? | |
5.b. | Color(green,Pass)? | |
5.c. | Color(green,Pass)? | |
5.d. | Color(green,Pass)? | |
5.e. |
Logs
This test was run with a hardware substrate, including an HP 3800 in OF 1.0 mode. The control plane elements were in the GPO InstaGENI rack.
OF-OPR-SLCR-SW Testing
Results
Step 1: Perform validation steps
Test Case | Result | Note |
1.a. | Color(green,Pass)? | See http://globalnoc.iu.edu/sdn/fsfw.html |
1.b. | Color(green,Pass)? | We've heard this verbally before. It will be a total rewrite. |
1.c. | Color(green,Pass)? | See https://github.com/GlobalNOC/FlowSpaceFirewall |
1.d. | ||
1.e. | Color(green,Pass)? | We're getting general agreement to fix reported issues. |
Logs
Fill these in as needed
Integration Testing
Results
Step 1: Setup
Test Case | Result | Note |
1.a. | Color(green,Pass)? | |
1.b. | Color(green,Pass)? | |
1.c. | Color(green,Pass)? | |
1.d. | Color(green,Pass)? |
Step 2: Collect monitoring data
Test Case | Result | Note |
2.a. | Color(green,Pass)? |
Wishlist Testing
Results
Step 1: Check documentation
Test Case | Result | Note |
1.a. | Color(#98FB98,Pass: most criteria)? | Software comes as a package, and there is installation documentation, but it is pretty loose. It should be sufficient for anyone with experience using RPM packages. |
1.b. | Color(green,Pass)? | |
1.c. | Color(#AAAAAA,N/A)? | No data plane limiters. |
Step 2: Set up the environment
Test Case | Result | Note |
2.a. | ||
2.b. | ||
2.c. |
Step 3: Test that priority setting works
Test Case | Result | Note |
3.a. | ||
3.b. | ||
3.c. | ||
3.d. |
Logs
Fill these in as needed
Attachments (19)
-
slicer.pcap (35.5 KB) - added by 9 years ago.
Step 2 slicer capture
-
dataplane.pcap (3.6 KB) - added by 9 years ago.
Step 2 data plane capture
-
controller.pcap (4.4 KB) - added by 9 years ago.
Step 2 controller capture
-
dataplane.2.pcap (3.6 KB) - added by 9 years ago.
Step 3 data plane capture
-
controller.2.pcap (16.0 KB) - added by 9 years ago.
Step 3 controller capture
-
controller.3.pcap (31.5 KB) - added by 9 years ago.
Step 7 controller capture
-
slicer.2.pcap (47.6 KB) - added by 9 years ago.
Step 5 slicer capture
-
functional_results.tar.gz (22.0 KB) - added by 9 years ago.
Results of functional test cases (doesn't include second half of step 5 test cases)
-
integration_results.tar.gz (284 bytes) - added by 9 years ago.
Integration testing results
-
slices.json (78 bytes) - added by 9 years ago.
Slice data from slicer
-
slice1-status.json (182 bytes) - added by 9 years ago.
Step 2 slice1 status from slicer
-
slice1-flows.json (3.5 KB) - added by 9 years ago.
Step 2 slice1 flows from slicer
-
flows.json (2.6 KB) - added by 9 years ago.
Step 3 substrate flows with no testslice flows
-
operations_results.tar.gz (1.3 KB) - added by 9 years ago.
Operations testing results
-
slice1-flows.2.json (2.8 KB) - added by 9 years ago.
HW testbed step 2 flows file
-
slice1-status.2.json (183 bytes) - added by 9 years ago.
HW testbed step 2 slice status file
-
slices.2.json (78 bytes) - added by 9 years ago.
HW testbed step 2 slices file
-
flows.2.json (5.8 KB) - added by 9 years ago.
HW testbed step 3 flows file
- operations_hw_results.tar.gz (1.3 KB) - added by 9 years ago.
Download all attachments as: .zip