wiki:OpenFlow/Slicer/TestPlan/Results/FlowspaceFirewall/1.0.4b

Version 32 (modified by tupty@bbn.com, 9 years ago) (diff)

--

OpenFlow Slicer Test Plan Results

This page tracks the results of a piece of software run through the OpenFlow Slicer Test Plan.

Software Version
Flowspace Firewall 1.0.4b

Results Legend

Individual test case status may be one of the following:

OF-OPR-SLCR-FN Testing

Results

Step 1: Setup

Test CaseResultNote
1.a.i. Color(green,Pass)?
1.a.ii. Color(green,Pass)?
1.a.iii. Color(green,Pass)?
1.a.iv. Color(green,Pass)?
1.a.v. Color(green,Pass)?
1.b. Color(green,Pass)?
1.c. Color(green,Pass)?

Step 2: Establish that basic connectivity works

Test CaseResultNote
2.a. Color(green,Pass)?
2.b. Color(green,Pass)?
2.c. Color(green,Pass)?
2.d. Color(green,Pass)?
2.e. Color(green,Pass)?
2.f.i. Color(red,Fail)? Slicer capture. This failure might be an issue with environment setup. Is slicer responsible for installing a "drop unmatched" rule?
2.f.ii. Color(green,Pass)?Slicer capture
2.f.iii. Color(green,Pass)?Slicer capture, Controller capture
2.f.iv. Color(green,Pass)?Slicer capture, Controller capture
2.f.v. Color(green,Pass)?Slicer capture, Controller capture
2.g. Color(green,Pass)?

Step 3: Test for VLAN enforcement and translation

Test CaseResultNote
3.a. Color(green,Pass)?
3.b. Color(green,Pass)?Controller capture
3.c. Color(green,Pass)?Controller capture
3.d. Color(green,Pass)?
3.e. Color(green,Pass)?

Step 4: Check data plane throughput

Test CaseResultNote
4.a. Color(green,Pass)?
4.b. Color(green,Pass)?
4.c. Color(green,Pass)?
4.d. Color(green,Pass)?
4.e. Color(green,Pass)?

Step 5: Test for OF functionality

Test CaseResultNote
5.a. Color(green,Pass)?
5.b. Color(green,Pass)?
5.c. Color(green,Pass)?
5.d.i. Color(red,Fail)?Known issue, will be fixed in next release. We should look into other similar instances where symmetric messages which are served by FSFW (and not the substrate) appropriately set the XID.
5.d.ii. Color(green,Pass)?
5.d.iii. Color(green,Pass)?
5.e.i. Color(green,Pass)?Slicer capture, Slicer responds but never sends to substrate.
5.e.ii. Color(green,Pass)?Slicer capture, Slicer doesn't respond, message never sent to substrate.

Test 6: Test for stacked slicer support

Test CaseResultNote
6.a. Color(green,Pass)?
6.b. Color(green,Pass)?

Test 7: Test for any extra functionality

Test CaseResultNote
7.a. Color(#98FB98,Pass: most criteria)?Controller capture, Limits seem to be enforced. Cases in which slice is disabled don't seem to be documented. Specific types of OF error messages sent in different cases don't seem to be documented.
7.b. Color(#AAAAAA,N/A)? No data plane limiters.
7.c. Color(#98FB98,Pass: most criteria)?Controller capture, Limits seem to be enforced. Cases in which slice is disabled don't seem to be documented. Specific types of OF error messages sent in different cases don't seem to be documented.

Test 8: Test management API

Test CaseResultNote
8.a. Color(orange,Blocked)? Management API software under development.
8.b. Color(orange,Blocked)? Management API software under development.
8.c. Color(orange,Blocked)? Management API software under development.
8.d. Color(orange,Blocked)? Management API software under development.

Logs

This test was performed in GENI using vSwitches.

OF-OPR-SLCR-OP Testing (in GENI)

Results

Step 1: Setup

Test CaseResultNote
1.a. Color(green,Pass)?
1.b. Color(green,Pass)?

Step 2: Collect monitoring data

Test CaseResultNote
2.a. Color(green,Pass)?Slice list, Slice1 status, Slice1 flows

Step 3: Test slice operations

Test CaseResultNote
3.a. Color(green,Pass)?
3.b. Color(green,Pass)?
3.c. Color(green,Pass)?
3.d. Color(green,Pass)?
3.e. Color(green,Pass)?Substrate flows, no flows with testslice's VLAN ID of 200

Step 4: Substrate-generated traffic

Test CaseResultNote
4.a. Color(green,Pass)?
4.b. Color(green,Pass)?

Step 5: Test logging

Test CaseResultNote
5.a. Color(green,Pass)?
5.b. Color(green,Pass)?
5.c. Color(green,Pass)?
5.d. Color(green,Pass)?
5.e. Color(green,Pass)?

Logs

This test was run using GENI and virtual switches. Parts of test case 3 were changed after this run.

OF-OPR-SLCR-OP Testing (in hardware testbed)

Results

Step 1: Setup

Test CaseResultNote
1.a. Color(green,Pass)?
1.b. Color(green,Pass)?

Step 2: Collect monitoring data

Test CaseResultNote
2.a. Color(green,Pass)?Slice list, Slice1 status, Slice1 flows

Step 3: Test slice operations

Test CaseResultNote
3.a. Color(green,Pass)?
3.b. Color(green,Pass)?
3.c. Color(green,Pass)?
3.d. Color(green,Pass)?
3.e. Color(red,Fail)?Substrate flows, flows with testslice's VLAN ID of 200 still exist. This should be somewhat fixed in the next release with a granularity of the FSFW flow stat polling rate.

Step 4: Substrate-generated traffic

Test CaseResultNote
4.a. Color(green,Pass)?
4.b. Color(green,Pass)?

Step 5: Test logging

Test CaseResultNote
5.a. Color(green,Pass)?
5.b. Color(green,Pass)?
5.c. Color(green,Pass)?
5.d. Color(green,Pass)?
5.e.

Logs

This test was run with a hardware substrate, including an HP 3800 in OF 1.0 mode. The control plane elements were in the GPO InstaGENI rack.

OF-OPR-SLCR-SW Testing

Results

Step 1: Perform validation steps

Test CaseResultNote
1.a. Color(green,Pass)? See http://globalnoc.iu.edu/sdn/fsfw.html
1.b. Color(green,Pass)? We've heard this verbally before. It will be a total rewrite.
1.c. Color(green,Pass)? See https://github.com/GlobalNOC/FlowSpaceFirewall
1.d.
1.e. Color(green,Pass)? We're getting general agreement to fix reported issues.

Logs

Fill these in as needed

Integration Testing

Results

Step 1: Setup

Test CaseResultNote
1.a. Color(green,Pass)?
1.b. Color(green,Pass)?
1.c. Color(green,Pass)?
1.d. Color(green,Pass)?

Step 2: Collect monitoring data

Test CaseResultNote
2.a. Color(green,Pass)?

Wishlist Testing

Results

Step 1: Check documentation

Test CaseResultNote
1.a. Color(#98FB98,Pass: most criteria)? Software comes as a package, and there is installation documentation, but it is pretty loose. It should be sufficient for anyone with experience using RPM packages.
1.b. Color(green,Pass)?
1.c. Color(#AAAAAA,N/A)? No data plane limiters.

Step 2: Set up the environment

Test CaseResultNote
2.a.
2.b.
2.c.

Step 3: Test that priority setting works

Test CaseResultNote
3.a.
3.b.
3.c.
3.d.

Logs

Fill these in as needed

Attachments (19)

Download all attachments as: .zip