Version 7 (modified by Nabeel Akhtar, 2 years ago) (diff)


Experiment 2: Load Balancing using Proportional Integral (PI) Control with Ryu Controller


In this experiment, we will use a control theoretic approach to do load balancing. An overview of the system is shown in below figure.

In this control theoretic approach, the load on VNF1 and VNF2 is monitored, and flow-forwarding decisions are made based on the load information retrieved from the hosting VMs. We will run a RINA distributed application to get the state (load) of the VNFs to the controller VM. Once the Ryu controller has the IDS load information, it will use the Proportional Integral (PI) load balancer to balance the load across the VNF instances based on the load information. This load balancing information is then provided to the Ryu controller, which updates the OpenFlow rules on the OVS switch to balance the load.

Note: another version of this experiment using POX controller can be found here?.

(1) RINA Distributed Application

First we will run a RINA distributed application to collect the VNF load information on the controller node.

We need Java installed on the nodes to run the RINA application. Install java on VNF1, VNF2 and controller nodes in new windows (Type Ctrl-C to exit netcat on the sources and destination). To install Java, execute:  sudo apt-get install openjdk-7-jdk
(If the install fails, you may first run: sudo apt-get update)
  1. In the controller window, download the RINA controller code:
  • cd ~
  • wget
  • tar -xvf Control.tar.gz
  1. Type ifconfig to get the IP address of the controller. Save this address as we will need this address to direct the RINA processes on the VNFs to the RINA process on the controller.
  1. In a new VNF1 window, download the RINA VNF1 code:
  • cd ~
  • wget
  • tar -xvf VNF1.tar.gz
  1. In a new VNF2 window, download the RINA VNF2 code.
  • cd ~
  • wget
  • tar -xvf VNF2.tar.gz
  1. Now we will change the IP address in the RINA configuration files for VNF1, VNF2 and controller, so these RINA processes can talk to each other. In the VNF1 window, execute:
  • cd ~/VNF1
  • nano

At the bottom of the file, change the and to the IP address of the controller. The following screenshot shows an example.

In the VNF2 window, execute:

  • cd ~/VNF2
  • nano

At the bottom of the file, again change the and to the IP address of the controller.

In the controller window, execute:

  • cd ~/Control/RINA
  • nano

At the bottom of the file, again change the '' and '' to the IP address of the controller.

  1. To run the RINA application, follow these steps (make sure you installed Java as noted above):

o In the controller window, execute the following commands:

  • cd ~/Control/RINA/
  • ./

o In the VNF1 window, execute the following commands:

  • cd ~/VNF1/
  • ./

o In the VNF2 window, execute the following commands:

  • cd ~/VNF2/
  • ./

You should see output on the controller window as shown below:

The RINA application on VNF1 and VNF2 should be run as soon as possible after the RINA application on the controller is started. If you wait for too long, you will get null values for CPU usage, as the controller`s RINA app is not able to subscribe to the CPU load of the VNFs. If this is the case, you should restart all RINA processes.

Note: the RINA distributed application also pass the intrusion detection results from VNFs to the Ryu controller, and this information is not used in this experiment, but will be used in Experiment 3 when handling intrusion traffic.

To stop all RINA processes running on a VM, run killall -v java

(2) PI Controller

The PI-controller gets the load information of VNF1 and VNF2 using RINA`s distributed application and makes the load balancing decision.

The figure below shows the block diagram of the Proportional Integral (PI) controlled NFV system.

Block diagram of the PI-controller NFV system. System load L and target load T(s)=T/s of VNF1 is used to compute X, i.e. ratio of traffic diverted to VNF2. K` = K/T.

The RINA-based distributed monitoring application provides the VNF1 state (average CPU load) information L(t) to the PI controller. The maximum capacity of a VNF instance is T. If the load on VNF1 exceeds T, new traffic flows are forwarded to a second VNF instance, VNF2. Assuming instantaneous feedback / measured load L(t), the PI control equation is given by:

The code for the PI controller is based on the following algorithm. Input IDSload.txt is the file generated by the RINA distributed application. This file has load information of the VNFs.

  1. To run the PI-controller, open a new controller window and execute:
  • cd ~/Control/PI_controller
  • python ~/Control/RINA/NFV1.txt

Note that here we are directing to the NFV1.txt file that is constantly updated by the RINA distributed application with the load information of VNFs.

  1. You should see the VNF state information printed on the screen. A sample output is shown below.

Here the target load on VNF1 is 30.0% of CPU usage, i.e. if the CPU load on VNF1 is more than 30.0%, traffic flows will be diverted to VNF2. The current CPU load shows the load on VNF1. The next line of the output shows the percentage of flows that will be directed to VNF2 and the last line shows the flows that were being directed to VNF2 before the current control update.

Do not close this window; leave the PI controller running.

(3) PI-based Ryu Controller

Now we will run the Ryu controller that will get the load balancing decision from the PI-controller and direct the flows accordingly.

  1. First we will update the port.config file to direct the controller to the NFV_ratio_PI.txt file generated by the PI-controller, which has the load balancing decision information. In a new controller window, execute:
  • nano /tmp/ryu/ryu/app/nfv.config

o Change the value of controller_type to PI
o Change the value of file_path_pi to the text file that has the PI controller`s output.

/users/<UserName>/Control/PI_controller /NFV_ratio_PI.txt
Change the <UserName> to your user name.

  1. Now we can run the Ryu controller. Execute
  • /tmp/ryu/bin/ryu-manager --verbose /tmp/ryu/ryu/app/

(4) Run Snort and Generate Traffic

  1. First we will run Snort IDS on VNF1 and VNF2. In separate windows for VNF1 and VNF2, execute the following command:
  • sudo /usr/local/bin/snort -A full -dev -i eth1
  1. We will use the iperf application to generate flows between a source and destination. If iperf is not installed on your nodes, execute:
  • sudo apt-get install iperf
  1. Run iperf server on the destination node:
  • iperf -u -s
  1. Now we will generate traffic from the sources (s1 and s2) to the destination node using iperf and see how it effects the CPU utilization at VNF1 and VNF2 running Snort IDS. Note that if we run multiple instances of iperf, we can generate significant load on the VNF instances. To run iperf client on a source, execute:
  • iperf -u -c destination -t 500 &
Note that you can run multiple instances of iperf by running iperf -c destination -t 500 & multiple times in s1 and s2 nodes. This flow lasts for 500 seconds. To kill all the flows generated at a node, run killall –v iperf
  1. Now if you look at the controller window, which is running the PI-controller, you can see the load on VNF1 has significantly increased. If the load is more than 30%, some percentage of the traffic flows will be diverted to VNF2.
OPTIONAL: Review the PI Ryu Controller Code

The Ryu controller gets the load balancing information from the output text file generated by the PI-controller. Based on the value of the control variable (variable X), it sends each new flow to either VNF1 or VNF2. The algorithm for the PI-based OVS controller is shown below.

(5) Real Time Graphs

You can draw real-time CPU usage graphs for both VNF1 and VNF2 nodes. A Python script is provided to produce these real-time graphs. The script periodically retrieves the CPU usage file from the Controller node and plots the graph for it.

  1. Download the python script to your laptop using the following link.
  1. Run the script and direct it to the CPU usage files (NFV1.txt and NFV2.txt) present at the controller. To run the Python script, type the following in the folder where you saved the file:
         python -n <username>@<controller IP address>

Change <username> to your user name and <controller IP address> to the IP address of the controller.

  1. You should see real-time CPU graphs as shown below:
To run the Python script to plot graphs, you need the python plotting library matplotlib. If you do not have this library on your laptop, you can use the following link to download it to your computer:

Next Experiment

Experiment 3: Handing Intrusion with Ryu Controller