wiki:GENIEducation/HyperNets

Version 22 (modified by shuang@bbn.com, 11 years ago) (diff)

--

All About HyperNet Architecture

What I have achieved so far

  • 1. set up Network Hypervisor server on andor.gpolab.bbn.com
  • 2. run HyperNets and Hypervisor server on difference end system and let them communicate via XML-RPC
  • 3. add support for protoGENI, instaGENI and exoGENI in the HyperNet config file
    • 3.1 user can specify either the AM nickname as defined in omni_config (e.g., eg-gpo, ig-utah, pg-ky), or directly using protoGENI/instaGENI/exoGENI in the "am" field of the config file
    • 3.2 user can further issue whether he/she wants vm or pc from GENI by specifying "vm"/"pc" in the "nodeType" field of the config file
  • 4. users' credential will be copied to the hypervisor on registerHyperNet() API call (including ssl private key, will have to change this when "speaks-for" is available *)
  • 5. build a web page that shows Network Hypervisor server logs: http://andor.gpolab.bbn.com/log/log.html
  • 6. concurrent tests: Sarah and I tried to run the same HyperNet package at the same time, reserving resources from both exoGENI and instaGENI. Network Hypervisor is able to handle the XML-RPC calls simultaneously (one does not have to wait until the other finishes)
  • 7. An example HyperNet tutorial: http://groups.geni.net/geni/wiki/GENIEducation/SampleAssignments/HyperNetRingDemo

To use a HyperNet App

  • 1. What the user needs
    • 1.1 credentials: more precisely, the following files: omni_config, private/public key pairs for GENI (can get from GENI Portal, e.g., geni_key_portal and geni_key_portal.pub in your ~/.ssh/ directory), the ssl certificate for GENI (can get from GENI Portal, i.e., geni_cert_portal.pem in your ~/.ssl/ directory)
    • 1.2 JRE (Java Runtime Environment) - do not know the version requirement yet, assume the default version on your Windows/Ubuntu/Mac would work as long as your PC/Laptop is not 20 years old :-)
  • 2. How to get the files needed above?
    • 2.1 Option A. Get a GENI Portal account, log on your portal account, click "profile", then download public key and private key to ~/.ssh/ directory; SSL certificate to ~/.ssl/ directory, then download and customize a template omni config file and customize it.
    • 2.2 Option B. Get a GENI Portal Account, download a "omni-bundle" script from somewhere, and run it using the following command:
        omni-configure.py
      
  • 3. An example?

To Build a Network Hypervisor Server

For Ubuntu:

  • 0. Since the Hypervisor server uses port 8001 to do XML-RPC, it also builds a website to show the server logs. Open Port 80 and 8001 on the server first.
  • 1. install apache:
     sudo apt-get install apache2
    
  • 2. add a new user:
     sudo adduser hypernet (password is hypernet)
    
  • 3. install JDK:
     sudo apt-get install openjdk-7-jdk
    
  • 4. copy hypervisor source code as user "hypernet" in "hypernet's" home directory:
     su hypernet
     wget http://www.gpolab.bbn.com/experiment-support/HyperNet/Hypervisor.tar.gz
     tar xzvf Hypervisor.tar.gz
    
  • 5. set environment path: add the following line to ~/.bashrc (as user "hypernet")
     export CLASSPATH=.:~/PVNLibrary/*:$CLASSPATH
    
    then log out and log in again as user "hypernet" (you can do "echo $CLASSPATH" to confirm that the classpath is set properly)
  • 6. install omni following the tutorial here: http://trac.gpolab.bbn.com/gcf/wiki (remember to set the $PATH variable so that the system can call omni.py directory instead of having to use the full path)
  • 7. change corresponding fields in the hypervisor source code to point to the correct directory:
    • 7.1 in "PVNLib.java" change the following so that it points to the default GENI user (unlikely to be called, but just in case)
              //default user credentials
      	private final String defaultUserName = "shuang";                                                  //default hypervisor's username, needs to match with the default omni_config in your ~/.gcf/ directory
                         //on andor the above is shufeng
      	private final String defaultOmniConfigFile = "/Users/shuang/.gcf/omni_config";  //default location for hypervisor's omni_config file
                         //on andor, the above is /home/hypernet/.gcf/omni_config
      	private final String hypervisorRoot = "/Users/shuang/Sites/";                               //default web site location on hypervisor
                         //on andor, the above is /var/www/
      	private final String webRoot = "http://128.89.81.82/~shuang/";                          //default web url for hypervisor service
                         //on andor, the above is http://128.89.91.33/
      
    • 7.2 in "pvnlib/RspecBuilder.java", change the following to point to the correct resource folder:
      public final static String resourceFolder = "/Users/shuang/Sites/resources/";            //default folder to hold physical topology xml files
                       //on andor, the above is /var/www/resources/
      
    • 7.3 in "pvnlib/TopologyMgr/Graph.java", change the following to point to the right omni.py:
      private static String omni = "/Users/shuang/omni/gcf/src/omni.py";                         //path for omni.py
                       //on andor, the above is changed to /home/hypernet/omni/gcf/src/omni.py
      
    • 7.4 recompile the source code, in folder "RPCServer/", do "make clean", then do "make". you might need to install make first.
  • 8. copy "log.html" in RPCServer/ folder to your web directory. In our case, since the web directory is at "/var/www/", we first change the permission on that folder to allow user "hypernet" to write, then put log.html in "/var/www/log/" folder:
    sudo chmod -R 777 /var/www/
    mkdir /var/www/log/
    cp log.html /var/www/log/
    
  • 9. in folder RPCServer/, change runServer.sh accordingly. since the log file is in /var/www/log/ folder, you do not need to change runServer.sh
  • 10. You are all set. In folder RPCServer/, run the hypervisor service by running:
    ./runServer.sh
    
    And you will see the logs from the hypervisor Log page: http://128.89.91.33/log/log.html

HyperNet Config file

  • 1. default config file is myConfig.txt, in the same directory of your HyperNet appliance
  • 2. by default, it uses "~/.gcf/omni_config" as the omni configuration file (mainly to search for credential files. i.e., ssh keys, ssl certificates and keys)
  • 3. by default, it uses pg-utah as the aggregate manager, if users specify to use instaGENI, it uses ig-utah; if users specify to use exoGENI, it uses eg-gpo
  • 4. by default, it uses vm as the nodeType (i.e., emulab-openvz for protoGENI and instaGENI AMs, ox-small for exoGENI AMs); user can specify "pc" for protoGENI AMs; or "Xen" for instaGENI AMs; or "xo-medium"/"xo-large" for exoGENI AMs
  • 5. "HyperNetName" is the name of the HyperNet, we actually pre-pend the username before the HyperNetName to form the name for the slice in GENI. e.g., username is "shufeng", HyperNetName is "Ring", then slice name is "shufeng-Ring"
  • 6. server is the IP address of the Network Hypervisor service hosing node.
  • 7. an example HyperNet config file can be found here: http://www.gpolab.bbn.com/experiment-support/HyperNet/Examples/myConfig.txt

Network Hypervisor API calls

  • Config getConfig(String configFile): get configuration from configFile. e.g.,
     #import pvnlib.*
     ...
     Config myConfig = PVNLib.getConfig("myConfig.txt");
    
  • APIMessage registerPVN(Config myConfig): registers a HyperNet, with configuration specified in myConfig. e.g.,
     #import pvnlib.*
     ...
     private PVNLib myPVN = new PVNLib("andor.gpolab.bbn.com");
     private static boolean regPVN(Config myConfig) {
    	APIMessage retValue = myPVN.registerPVN(myConfig);
    	System.out.println(retValue.message + "\n");
    	return retValue.result;
     }
    
  • APIMessage buildTopo(String pvnName): build the topology on GENI with the given HyperNet name. e.g.,
     #import pvnlib.*
     ...
     private PVNLib myPVN = new PVNLib("andor.gpolab.bbn.com");
     private static void build() {
    	APIMessage ret = myPVN.buildTopo(myConfig.PVNName);
    	System.out.println("\n"+ret.message + "\n");
    	myPVN.parseLogin(ret);
     }
    
  • APIMessage tearDown(String pvnName): tears down the topology on GENI with given HyperNet name. e.g.,
     #import pvnlib.*
     ...
     private PVNLib myPVN = new PVNLib("andor.gpolab.bbn.com");
     private static void tearDown() {
    	APIMessage ret = myPVN.tearDown(myConfig);
    	textArea.append("\n"+ret.message + "\n");
    	myPVN.parseLogin(ret);
     }
    
  • Topology buildRandomRing(String pvnName, int nodeNum): builds a random ring with nodeNum of nodes on the ring, returns the topology of the ring.
  • Topology buildRing(String pvnName, ArrayList<Node> nodeList): builds a ring with connecting the given nodeList, returns the topology of the ring.
  • Topology buildRandomStar(String pvnName, int nodeNum): builds a random star with nodeNum of nodes in the star, excluding the central node (so there are nodeNum+1 total number of nodes), returns the topology of the star.
  • Topology buildStar(String pvnName, Node center, ArrayList<Node> nodeList): builds a star with center the center of the star, and nodeList are the nodes connecting the center node. Returns the topology of the star.
  • Topology buildRandomMesh(String pvnName, int nodeNum, int degree): builds a random connected mesh network, with nodeNum of nodes in the network. Degree defines the least number of links each node in the topology has. i.e., if degree = nodeNum-1, then it is a full mesh.
  • Topology buildMesh(String pvnName, ArrayList<Node> nodeList, int degree): builds a mesh network connecting the given list of nodes
  • Topology buildRandomFullMesh(String pvnName, int nodeNum): builds a full mesh network connecting nodeNum of random nodes.
  • Topology buildFullMesh(String pvnName, ArrayList<Node> nodeList): builds a full mesh with the given list of nodes.
     #import pvnlib.*
     ...
     Topology myTopo;
     //myTopo = myPVN.buildRandomRing(myConfig.PVNName, myConfig.nodeNum);
     //myTopo = myPVN.buildRandomStar(myConfig.PVNName, myConfig.nodeNum);
     //myTopo = myPVN.buildRandomMesh(myConfig.PVNName, myConfig.nodeNum, 2);
     myPVN.buildRandomFullMesh(myConfig.PVNName, myConfig.nodeNum);
    
  • Node addPR(String pvnName, Node pr, String type): adds a programmable router to your topology. pvnName is the name of your HyperNet. You can specify Node pr's virtualization type, which specific physical node you want (bounded node), and from which aggregate manager. Alternatively, you can specify the virtualization type of the pr in String type.
     #import pvnlib.*
     ...
     Node myNode = new Node();
     myNode.virtuali_id = "fancy-node-name";
     myNode.node_type = "xo.small";
     myNode.image = "http://emmy9.casa.umass.edu/Disk_Images/ExoGENI/exogeni-umass-ovs-1.0.4.xml";
     myNode.version = "375d5861080b85c9b17e8d4c6c431e955e1d72fd";
     myNode.tarfiles = "http://www.gpolab.bbn.com/experiment-support/OpenFlowExampleExperiment/software/install-script-exo.tar.gz";
     myNode.install_path = "/tmp/";
     myNode.startup_command = "/tmp/postboot_script_exo.sh $sliceName $self.Name()";
     //you can even specify which physical node it is by: myNode.address = "pc321.emulab.net";
     myPVN.addPR(myNode);
    
  • ArrayList<Node> addTunnel(String pvnName, Node pr1, Node pr2): adds a link between two Nodes pr1 and pr2. This link can be a VLAN link or a GRE tunnel, depending on the two nodes it is connecting. (e.g., if the two nodes are from different aggregates, then a GRE tunnel is created; otherwise a normal VLAN link is created)
     Node node1 = new Node(); //node1 is in KY
     node1.node_type = "pc";
     node1.node_cm = "urn:publicid:IDN+uky.emulab.net+authority+cm";
     Node node2 = new Node(); //node2 is in Utah
     node2.node_type = "pc";
     node2.node_cm = "urn:publicid:IDN+emulab.net+authority+cm";
    
     node1 = myPVN.addPR(myConfig.PVNName, node1, "pc");
     node2 = myPVN.addPR(myConfig.PVNName, node2, "pc");
     myPVN.addTunnel(myConfig.PVNName, node1, node2); //this will create a GRE tunnel between node1 and node2 since they are from different aggregates. 
    

Compose your own HyperNet to deploy a topology on GENI

  • 1. Pre-requisites:
    • 1.0. Make sure you have access to the Hypervisor server. Right now it is andor.gpolab.bbn.com (try ping it to make sure that there is no firewall issue).
    • 1.1. Of course when you built your own HyperNet, you want to try it out to debug it. For this purpose, you do need to have a GENI account. Follow the section about "To use a HyperNet app" to get account and set up credentials.
    • 1.2. It will be best if you know some Java, but it is totally fine if you don't know. The examples are quite easy to follow.
    • 1.3. Download and install JDK 1.6 or higher.
    • 1.4. It is recommended that you use some sort of IDE for Java, such as eclipse, instead of writing the code in vim and writing your own Makefile.
    • 1.5. The most important: download HyperNet library from http://www.gpolab.bbn.com/experiment-support/HyperNet/PVNLibrary.tar.gz
      • 1.5.1. extract it to any directory (e.g., your home directory), and add the library folder to your "CLASSPATH" varialbe by adding the following line to your ~/.bashrc file: (in eclipse, you need to include the .jar files in that folder as external JARs in the Build Path of your Java project)
        export CLASSPATH=.:~/PVNLibrary/*:$CLASSPATH
        
  • 2. Coding:
    • 2.1. Your HyperNet code should follow these steps:
      • 2.1.1. Read Config file (by default, it is "myConfig.txt" in the same directory as your HyperNet app) using getConfig()
      • 2.1.2. Register HyperNet network using registerPVN()
      • 2.1.3. Compose your network using the APIs described earlier
      • 2.1.4. Build your network using buildTopo()
  • 3. A simple example to follow:
    • The best practice is to start with an example: http://www.gpolab.bbn.com/experiment-support/HyperNet/Examples/sourceCode/ExampleRingGui.tar.gz
    • Run the example and make sure it works (remember to deletesliver after you built the topology, via flack) -- if it does not work, there must be something wrong with your setup (not the code), e.g., connection to Hypervisor server, myConfig.txt, your credential files (omni_config), etc.
    • Try change the code in the example so that it creates a random Star topology (the code is already there, you just need to comment and uncomment)
  • 4. A more complicated example (that only works on exoGENI since only exoGENI recognize the image name -- you know what that means when you see the code :-)):
  • 5. Since GENI resources are scarce, it is recommended that you build the RSpec, check the RSpec first, then deploy the topology on GENI (i.e., reserve GENI resources)
    • 5.1. What should I do if I only want to build the RSpec instead of deploying it on GENI? -- Check the next section

What if I only want to create the RSpec file instead of actually building/deploying it on GENI?

  • 1. Simple: use showRspec() instead of buildTopo()
  • 2. If you only want to see RSpec file, you do not even need your credential (i.e., you do not need to have "userCredential" in your config file). As a result, a non-GENI user can use our Hypervisor server to create Rspec files that can be understood by GENI AMs.
  • 3. An example: http://www.gpolab.bbn.com/experiment-support/HyperNet/Examples/sourceCode/ExampleRingRspec.tar.gz
    • 3.1. This example does not create a GUI window so the code is much simpler than the ExampleRingGui example.
    • 3.2. Try change the code as well as the config file so that it creates the RSpec for a full mesh network with 6 nodes.