Changes between Initial Version and Version 1 of HowTo/ConfigureOVSWithLayer3Routing

07/18/14 09:30:49 (8 years ago)



  • HowTo/ConfigureOVSWithLayer3Routing

    v1 v1  
     2= Configure OVS With Layer 3 Routing =
     4Open vSwitch (OVS) acts as a Layer 2 device when it's not connected to its controller (and its `fail-safe-mode` is set to standalone [default]). If we want to do Layer 3 control, we need to write the control logic in its controller. However, sometimes we want to Layer 3 control while there is other Layer 3 control. For example, we want to do firewall or NAT while there is IP routing managed by XORP or Kernel Static Routing. A straightforward solution is to write your own IP forwarding logic in your controller (Yeah, practice of writing your own XORP is fun!). This is really violating the reusability principle of software engineering. Another solution is to cross your finger and hope there is someone has written it for the controller framework you use.
     6In this page, we are going to show you how to configure OVS to work with Linux kernel static IP routing. The method is largely from the help of Ryan Izard,, from Clemson University.
     8The topology we want to have is shown in below figure.
     10[[Image(OVS Routing abstract.png, 30%, nolink)]]
     12The configuration we want to have is shown in below figure.
     14[[Image(OVS Routing detail.png, 30%, nolink)]]
     16In summary the idea is to create a single OVS bridge for each interface on your machine that you want to assign an IP; pass the packet between the interface and the network stack through the LOCAL port of OVS; and let Linux routing handle the rest of the part.
     17This wiki page provides step-by-step instructions.
     191. Create 2 OVS bridges.
     21ovs-vsctl add-br OVSbr1
     22ovs-vsctl add-br OVSbr2
     252. Zero out the IP of interfaces as you will assign it to the OVS bridges (your interface names may vary). On GENI, be careful not to bring down eth0, because it is your control interface, if you bring that interface down you won't be able to login to your host!
     27ifconfig eth1 0
     28ifconfig eth2 0
     313. Attach interfaces to according OVS bridges. Again, don't attach control plane interface.
     33ovs-vsctl add-port OVSbr1 eth1
     34ovs-vsctl add-port OVSbr2 eth2
     374.Verify the configurations by:
     39ovs-ofctl show OVSbr1
     40ovs-ofctl show OVSbr1
     41ovs-vsctl show
     445. Assign the IP addresses to the OVS bridges, and add routing entries (clean up ones if needed).
     46ifconfig OVSbr1 up
     47ifconfig OVSbr2 up
     49These will insert the corresponding routes automatically for you, and you can verify it via:
     51route -n
     53Alternatively, you could do:
     55ifconfig OVSbr1 up
     56ifconfig OVSbr1 up
     57route add -net netmask dev OVSbr1
     58route add -net netmask dev OVSbr1
     59route -n
     626. When an OVS bridge is installed in the Linux OS, it is wired such that any application packets or packets routed via Linux will be sent to the LOCAL port of the OVS bridge, assuming a route to that bridge exists. And, in the reverse direction, any packets sent out an OVS bridge's LOCAL port will be received by the local networking stack.
     64Let's do an example of how a packet would traverse our network from to
     6610.10.10.1 sends the packet with destination The packet arrives at eth1. There is an OpenFlow flow in place on OVSbr1 between eth1 and it's LOCAL port, so the packet will match this flow and be sent out the LOCAL port of OVSbr1 (i.e. port 65534). The packet is then received by the local machine's network stack. It has a destination IP of, so the routing table we have established will send the packet to the OVSbr2 network interface. OVSbr2 will receive this packet from the local network stack via it's LOCAL port (i.e. port 65534). There is an OpenFlow flow in place on OVS2 between OVS2's LOCAL port and eth2, so this packet will match that flow and be sent out eth2. From there, it will arrive at The same process will occur in reverse. This assumes though that you insert the flows between the physical interfaces (eth1 and eth2) and the OVS LOCAL ports. That's the key to handing packet to and receiving packets from the local OS.
     687. If you want to insert these flows with OVS itself, you can do something like the following:
     70ovs-ofctl add-flow OVSbr1 in_port=port_number_of_eth1,actions=output:LOCAL
     71ovs-ofctl add-flow OVSbr1 in_port=LOCAL,actions=output:port_number_of_eth1
     72ovs-ofctl add-flow OVSbr2 in_port=port_number_of_eth2,actions=output:LOCAL
     73ovs-ofctl add-flow OVSbr2 in_port=LOCAL,actions=output:port_number_of_eth2
     75In some cases, you must specifically use the keyword LOCAL or the port number 65534 (both mean the same thing, but OVS is particular about it). You can determine port_number_of_eth1 via:
     77ovs-ofctl show OVSbr1
     808. If you want to insert these flows via your controller, you will need to either specify port 65534 explicitly or use whatever convention your controller uses to specify the LOCAL port of a bridge.
     81Remember to set your controller to all OVS bridges you want to control:
     83ovs-vsctl set-controller OVSbr1 tcp: ptcp:6634:
     84ovs-vsctl set-controller OVSbr2 tcp: ptcp:6634:
     879. Apart from these, remember to verify IP forwarding is enabled:
     89cat /proc/sys/net/ipv4/ip_forward
     90echo 1 > /proc/sys/net/ipv4/ip_forward
     93For an example of using this configuration, you may want to try out the [ OpenFlow NAT Example].