Changes between Initial Version and Version 1 of OMF5.3Installation


Ignore:
Timestamp:
09/07/11 17:24:34 (10 years ago)
Author:
agosain@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OMF5.3Installation

    v1 v1  
     1[[PageOutline]]
     2
     3== 6.  Building and installing software on the WiMAX mobile node  ==
     4
     5
     6This page describes the installation of a stand-alone OMF (cOntrol and Management Framework) setup on a single node, and the setup required to perform basic WiMAX experiments on a mobile node. Experiments can also be performed using a custom built image which includes the entire software suite and is available by contacting the GPO at gpo-infra@geni.net.
     7
     8We refer to corresponding pages on http://mytestbed.net/ occasionally in this document, so that the interested reader could compare/follow the related procedures. However, there are important differences between what we have here and the pages on http://mytestbed.net/ . We believe that having all the steps we took to create WiMAX experiments in one place reduces possible confusions for a new experimenter who wants to replicate our setup.
     9
     10
     11=== Table of code, versions;  issues ===
     12
     13The following table shows the hardware/software versions used in the image available at "FILLME".
     14
     15||'''Hardware'''||'''WiMAX Card'''||'''OMF Version'''||'''OML Version'''||'''OS'''||
     16||Dell 1012 Netbook||  Internal Intel 6250  || 5.3 || 2.5.0 || Ubuntu 11.04 ||
     17
     18
     19=== Software components ===
     20
     21The following shows the interaction amongst the different components of the OMF/OML system for the "roaming" MS image.
     22
     23[[Image(OMFCase3Diagram.png, 50%)]]
     24
     25To get a detailed look at OMF please refer to http://mytestbed.net/projects/omf .
     26
     27
     28
     29=== Include OMF/OML services ===
     30
     31
     32
     33In this section we describe how to install all of the OMF components on a single machine, as opposed to the traditional use of OMF managing a testbed of many different nodes. This is required because there is no reliable control channel available for the mobile node that only has a wimax interface. The "roaming" node wired connection is unplugged after the image load process to enable mobile experiments.
     34
     35This section is '''OPTIONAL''' as a fully customized image is available with all components installed. The image can be found at "FILLME"
     36
     37
     38==== Setting up the network connections ====
     39
     40- Ensure you have a working Internet connection. (ifconfig ethX)
     41
     42- Remove the network-manager:
     43{{{
     44apt-get remove network-manager
     45}}}
     46 
     47(do not run this via SSH since you might lose connectivity). Make sure you are able to assign static IP addresses to the interfaces manually, and that the changes remain in place after reboot.
     48
     49- Become superuser by entering
     50{{{
     51sudo -s
     52}}}
     53
     54
     55- Open the file /etc/network/interfaces and configure the loopback interface as follows:
     56
     57
     58{{{
     59        auto lo
     60        iface lo inet loopback
     61
     62}}}
     63
     64- In the same file, configure your wimax interface(e.g. wmx0). Here's an example:
     65
     66{{{
     67        auto wmx0
     68        iface wmx0 inet static
     69        address <ip_addr>
     70        netmask <netmask>
     71        gateway <gateway>
     72
     73
     74}}}
     75
     76- Save your changes, bring down lo and wmx0 interfaces, bring them back on and check to see if your changes have taken effect (ifconfig wmx0, ifconfig lo).
     77
     78- You do not need to configure "dnsmasq" for stand-alone OMF installation. If you're also following the installation guide on OMF website, skip the dnsmasq part.
     79
     80- You do not need to setup "PXE booting" for stand-alone OMF installation. If you're also following the installation guide on OMF website, skip the "Setup PXE booting" part.
     81       
     82
     83==== Configuring XMPP ====
     84
     85The communication of the "node" and the "Experiment Controller" (EC) happens through via XMPP. Consequently, you'll need to have an XMPP server running on the machine as well. For OMF-5.3, Openfire 3.7 XMPP server is recommended.
     86
     87Installing Openfire
     88
     89- Uninstall any other XMPP servers first
     90
     91- Make sure ports TCP 5222, 5269 and 9090 are open in your firewall
     92
     93- Install Java packages
     94
     95{{{
     96apt-get install sun-java6-jre
     97}}}
     98
     99If this package cannot be found, make sure you have enabled the "partner" repository in /etc/apt/sources.list. Then run
     100
     101{{{
     102apt-get update
     103}}}
     104 and retry.
     105
     106Install XMPP openfire software
     107{{{
     108wget http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_3.7.0_all.deb
     109dpkg -i downloadServlet?filename=openfire%2Fopenfire_3.7.0_all.deb
     110}}}
     111
     112
     113- Check if openfire is running:
     114{{{
     115 ps aux | grep openfire
     116}}}
     117
     118Startup can take a while, please be patient.
     119
     120- Direct your web browser to http://localhost:9090 and begin the setup wizard
     121
     122- Choose your language and click continue
     123
     124- Enter the AM's hostname in the Domain field and click continue
     125
     126- Choose the embedded database. You can also use mysql, but you will have to create a mysql user and a database manually first.
     127
     128- Choose the default profile and click continue
     129
     130- Enter an admin password and click continue, then wait until the installation is finished
     131
     132- Log on to the web GUI at http://localhost:9090 with the user/password you set in the wizard
     133
     134- For server2server connectivity, you need to set up a DNS name "pubsub.yourhost.name" which can be resolved from the public internet. It can be an A record pointing to the same IP address as your host, or a CNAME pointing to the hostname. There is no need for a DNS SRV record.
     135
     136
     137====    Configuring the Aggregate Manager (AM)  ====
     138
     139- add this line to your /etc/apt/sources.list:
     140
     141{{{
     142        deb http://pkg.mytestbed.net/ubuntu lucid/
     143}}}
     144 
     145
     146- NOTE: If you are using a different version of Ubuntu you might be tempted to replace "lucid" with your version here(e.g. maverick). That repository however may contain a different version of OMF. This document only applies if you use the packages from the "lucid" source. We recommend that you use these packages in combination with Ubuntu lucid only.
     147
     148 
     149{{{
     150apt-get update
     151apt-get install omf-aggmgr-5.3 oml2-server
     152}}}
     153
     154
     155NOTE: (the OML server is installed to collect measurement data for the result service. You don't need to install it if you are not planning to use OML and the result service.)
     156
     157- The omf-aggmgr service will fail to start. This is normal as we have to configure it first.
     158
     159- Copy the example configuration file /usr/share/doc/omf-aggmgr-5.3/examples/omf-aggmgr.yaml to /etc/omf-aggmgr-5.3/ and edit it to enter the correct XMPP server name. The XMPP user account (aggmgr) is created automatically. An example of for the ':server' field, under ':xmpp' would be "wimax-cl-1", that is our hostname.
     160
     161Be careful to use only 'spaces' to indent. 'tab' indents are not supported by the ruby yaml parser used to read the file.
     162
     163- If you are running multiple AMs, please configure different XMPP user names for them in omf-aggmgr.yaml.
     164
     165- In the directory /etc/omf-aggmgr-5.3/available, copy each file that is named file.yaml.nicta to file.yaml, unless you are at Winlab, where you copy each file.yaml.winlab to file.yaml
     166
     167- Enable the following services by creating the following symlinks:
     168       
     169{{{
     170        cd /etc/omf-aggmgr-5.3/enabled
     171        ln -s ../available/cmcStub.yaml
     172        ln -s ../available/inventory.yaml
     173        ln -s ../available/result.yaml
     174
     175}}}
     176
     177- Modify the configuration file called /etc/omf-aggmgr-5.3/available/inventory.yaml to reflect your mySQL database configuration
     178
     179
     180====  Inventory Installation  ====
     181
     182-
     183{{{
     184apt-get install mysql-server libdb4.6 phpmyadmin
     185
     186}}}
     187
     188- When prompted, enter a mysql root password of your choice or just press enter for no password
     189 
     190{{{
     191mysql -u root -p
     192}}}
     193
     194
     195- Paste the following command sequence:
     196       
     197{{{
     198use mysql;
     199create user 'omf'@'localhost';
     200grant all on *.* to 'omf'@'localhost';
     201set password for 'omf'@'localhost'=password('omf');
     202quit;
     203
     204}}}
     205
     206- You may obviously use a different user and password, but don't forget to add these to the aggregate manager configuration files later
     207
     208- Start mysql server
     209{{{
     210mysql -u omf -pomf
     211}}}
     212 
     213-Create inventory database
     214       
     215{{{
     216create database inventory;
     217quit;
     218}}}
     219
     220
     221- To initially populate the inventory database, you may import NICTA's sample inventory SQL file. It contains all the necessary tables and is filled with sample entries from one of our sandboxes. You may edit it to your needs before or after importing it.
     222
     223- To import the NICTA sample inventory
     224
     225{{{
     226zcat /usr/share/doc/omf-aggmgr-5.3/examples/inventory.sql.gz | mysql -u omf -pomf inventory
     227
     228}}}
     229
     230- You should now be able to open your browser at http://localhost/phpmyadmin/ and log on as omf/omf. Have a look at the inventory database tables to get a feel for what goes in there. Check "Installation Inventory_" for a detailed description of each table. Modify the inventory contents to match your setup accordingly.
     231
     232- Your inventory database must hold information about the testbed name (e.g. the 'testbeds' table must have an entry corresponding to this name)
     233
     234- For every node in your testbed, you need to add a database record in each of the tables ''locations, motherboards and nodes''
     235
     236- The inventory requires some manual work, and describing each step would take this quick installation guide too far. If you are reasonably convinced that you've filled the tables correctly, please proceed with the guide. You can optimize the inventory at a later stage.
     237
     238(This needs to be modified with stand-alone inventory sample)
     239
     240
     241====  Creating one pubsub node  ====
     242
     243- OMF entities use certain Pubsub nodes on the XMPP server to communicate. These nodes must be created before experiments can be run. In our case, which is an stand-alone installation, we only need to create one such node.
     244
     245- To create the system side of the Pubsub node tree
     246{{{
     247omf_create_psnode-5.3 <AM_name> mksys
     248}}}
     249
     250Replace the first argument with your XMPP server domain, if different from the one used here.
     251
     252- In non-stand-alone setups of OMF, in order to image the nodes, a slice named "pxe_slice" must be created and all nodes must be added to it. We are not interested in that when considering a stand-alone installation so we skip that at this stage. If you want to have this option available for any reason, consult OMF-5.3 installation guide at "Creating the pubsub nodes" section.
     253
     254- Regular experiments are executed in a slice as well. To run an experiment on a set of nodes, they must be added to that slice first. Use the following command for this purpose:
     255{{{
     256omf_create_psnode-5.3 <AM_name> mkslice default_slice <wimax_client_name>
     257}}}
     258 to create a "default_slice" containing your single node. Replace the first argument with your XMPP domain.
     259
     260- If you want to remove the node from your stand-alone single-node testbed or you've made a mistake creating the system node, don't worry about deleting the system node from the XMPP server. It does no harm to keep them there.
     261
     262- Restart OMF aggregate manager
     263{{{
     264/etc/init.d/omf-aggmgr-5.3 restart
     265}}}
     266
     267Restart every time you edited a .yaml file to apply the changes
     268
     269
     270====  Installing the Experiment Controller (EC)  ====
     271
     272- Before installing the EC on the local machine, make sure that the AM services (TCP port 5053) and XMPP (TCP port 5222) are accessible to it. Follow the following steps for installation of ECs:
     273
     274- Ensure the machine you are installing the EC on has internet access
     275
     276- Add this line to your /etc/apt/sources.list:
     277       
     278{{{
     279deb http://pkg.mytestbed.net/ubuntu lucid/
     280}}}
     281 
     282
     283- NOTE: If you are using a different version of Ubuntu you might be tempted to replace "lucid" with your version here. That repository however may contain a different version of OMF. This document only applies if you use the packages from the "lucid" source. We recommend that you use these packages in combination with Ubuntu lucid only.
     284
     285
     286{{{
     287apt-get update
     288apt-get install omf-expctl-5.3
     289}}}
     290
     291
     292
     293- Copy the default configuration file for the EC
     294{{{
     295zcat /usr/share/doc/omf-expctl-5.3/examples/omf-expctl.yaml.gz > /etc/omf-expctl-5.3/omf-expctl.yaml
     296}}}
     297
     298
     299- The file /etc/omf-expctl-5.3/omf-expctl.yaml has multiple configuration sections for different testbeds. In this guide we will only look at section :default:
     300
     301- Add your testbed domain and experiment slice name after ':domain:' and ':slice:', e.g. "wimaxlocal" and "default_slice"
     302
     303- Add your XMPP domain in ':communicator:xmpp:pubsub_gateway:', e.g. wimax-cl-1
     304
     305- Add the IP address of a local interface in ':web:host:' to bind the EC's webserver. Use 127.0.0.1 in stand-alone installation.
     306
     307- Add the contact details of your OML server (if you followed this guide it is running on the AM machine) in ':omluri:' in format 'protocol:hostname:port', e.g. tcp:localhost:3003
     308
     309- If you have legacy HTTP AM services running on this machine or other machines, add their URLs to 'inventory:url:', ':cmc:url' and so on
     310
     311- If you are unsure about the configuration parameters, please contact your testbed administrator. If you are the administrator and you are still unsure, please do not hesitate to contact us.
     312
     313
     314====   Installing the Resource Controller (RC)  ====
     315
     316- Ensure the machine you are installing the RC on has network access to your XMPP server
     317
     318- Add this line to your /etc/apt/sources.list if it is not there already:
     319
     320{{{
     321        deb http://pkg.mytestbed.net/ubuntu lucid/
     322}}}
     323 
     324
     325- NOTE: If you are using a different version of Ubuntu you might be tempted to replace "lucid" with your version here. That repository however may contain a different version of OMF. This document only applies if you use the packages from the "lucid" source. We recommend that you use these packages in combination with Ubuntu lucid only.
     326
     327
     328{{{
     329 apt-get update
     330 apt-get install omf-resctl-5.3
     331}}}
     332
     333
     334- The RC will fail to start. This is normal as we have to configure it first.
     335
     336- Copy the example configuration file /usr/share/doc/omf-resctl-5.3/examples/omf-resctl.nicta.yaml to /etc/omf-resctl-5.3/omf-resctl.yaml
     337
     338- Edit /etc/omf-resctl-5.3/omf-resctl.yaml
     339
     340- Enter the name of your control network interface after ':control_if:' (in our case, the loopback interface: lo)
     341
     342- Enter the FQDN or IP address of your XMPP server after ':pubsub_gateway:',
     343
     344- Enter the node's HRN after ':name:', (e.g. 'omf.bbn.node1')
     345
     346- Enter the name of the experiment slice this node is participating in after ':slice:', (e.g. default_slice)
     347
     348- Run /etc/init.d/omf-resctl-5.3 start to start the RC
     349
     350- Check the log file /var/log/omf-resctl-5.3 and see if it successfully registers with the XMPP server
     351
     352
     353
     354
     355
     356=== Include apps for throughput experiment  === 
     357                 
     358The OML developers have modified the source code of the following applications to include Measurement Points (MP) that can be stored in OML databases. Our experiments use the following applications that are available in the image "FILLME":
     359         
     360- Gpslogger_oml2: This is used to capture gps coordinates while conducting outdoor experiments. The application is available at git://git.mytestbed.net/oml-apps.git  [[BR]]
     361 
     362- oml2_iperf: This is an instrumented version of iperf that sends traffic over the wimax interface to measure throughput. Currently there are two versions of oml2_iperf.
     363         
     364  - An OML version that does not support the dual mode of operation of iperf.
     365  - An OML version that supports the dual mode operation of iperf.
     366The two versions are not compatible. We have made available separate images with the two different iperf versions at : "FILLME"
     367         
     368- wimaxcu: This is an instrumented version of the wimax utility available with the wimax driver that reports the radio parameters on the client side.
     369         
     370In order to be able to properly use wimaxcu link status reports in the experiment, we will have to write an OMF wrapper around it as 'wimaxcu' does not come with OML support. To create the wrapper:
     371         
     372  - Create a folder named wiwrapper
     373  - Put the following code segment in a separate file named wiwrapper.rb, and place it in the wiwrapper folder
     374  - Get the file 'oml4r.rb' from http://mytestbed.net/repositories/changes/oml/lib/client/ruby/oml4r.rb
     375  - Put the file 'oml4r.rb' in wiwrapper folder
     376  - Make a tar ball of wiwrapper folder(wiwrapper.tar).
     377  - Put the tar ball, wiwrapper.rb, and oml4r.rb in the same directory as that of your experiment (we assume this is the directory you run your experiments from).
     378         
     379This is the code for wiwrapper.rb:
     380                 
     381{{{
     382
     383#----------------------------------------------------------------------
     384# Copyright (c) 2011 Raytheon BBN Technologies
     385#
     386# Permission is hereby granted, free of charge, to any person obtaining
     387# a copy of this software and/or hardware specification (the "Work") to
     388# deal in the Work without restriction, including without limitation the
     389# rights to use, copy, modify, merge, publish, distribute, sublicense,
     390# and/or sell copies of the Work, and to permit persons to whom the Work
     391# is furnished to do so, subject to the following conditions:
     392#
     393# The above copyright notice and this permission notice shall be
     394# included in all copies or substantial portions of the Work.
     395#
     396# THE WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
     397# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
     398# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
     399# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
     400# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
     401# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
     402# OUT OF OR IN CONNECTION WITH THE WORK OR THE USE OR OTHER DEALINGS
     403# IN THE WORK.
     404#----------------------------------------------------------------------
     405
     406#!/usr/bin/env ruby
     407$LOAD_PATH<<'/usr/bin'
     408require 'oml4r'
     409                 
     410                 
     411APPNAME = "wimaxcu"
     412APPPATH = "/usr/bin/wimaxcu"
     413APPVERSION = "1.0"
     414                 
     415                 
     416                 
     417class MPStat < OML4R::MPBase
     418    name :wimaxstat
     419    param :CenterFrequency, :type => :long
     420    param :RSSI, :type => :long
     421    param :CINR, :type => :long
     422    param :TXPWR,:type => :long
     423                 
     424    # wimaxcu returns other metrics which we ignore here
     425end
     426                 
     427class Wrapper
     428         
     429  def process_output(output)
     430    # wimaxcu returns a sequence of lines
     431    # The 1st line is a list of labels for the fields of the remaining lines
     432    # Each remaining line is for a detected stations, and follows the format:
     433    # ADDR AID CHAN RATE RSSI DBM IDLE TXSEQ RXSEQ CAPS ACAPS ERP STATE MODE
     434    lines = output.split("\n")
     435    lines.delete_at(0)
     436    lines.delete_at(1)
     437    lines.delete_at(4)
     438    column = Array.new
     439    lines.each { |row|
     440      nums = row.scan(/\d+|-\d+/)
     441      column << nums
     442    }
     443         
     444    # Inject the measurements into OML 
     445    MPStat.inject(column[0], column[1], column[2], column[3])
     446  end
     447                 
     448         
     449  def initialize(args)
     450             
     451    # Initialise some variable specific to this wrapper
     452    @interval = 5
     453         
     454    # Now call the Init of OML4R with the command line arguments (args)
     455    # and a block defining the arguments specific to this wrapper
     456    OML4R::init(args, :appID => APPNAME) { |argParser|
     457      argParser.banner = "\nExecute a wrapper around #{APPNAME}\n" +
     458                         "Use -h or --help for a list of options\n\n" 
     459      argParser.on("-s", "--sampling DURATION", 
     460                   "Interval in second between collected samples") { |time| 
     461                      @interval = time 
     462                   }
     463      argParser.on_tail("-v", "--version", 
     464                        "Show the version\n") { |v| 
     465                           puts "Version: #{APPVERSION}"; exit 
     466                        }
     467    }
     468                 
     469    # Finally do some checking specific to this wrapper
     470    # e.g. here we do not proceed if the user did not give us a 
     471    # valid interface to monitor
     472    #unless @interface != nil
     473    #  raise "You did not specify an interface to monitor! (-i option)"
     474    #end
     475  end
     476         
     477  def start()
     478    while true
     479    # Run the wlanconfig command with the following syntax
     480    #   "wlanconfig <interface> list"
     481    cmd = "#{APPPATH} status link"
     482    output = `#{cmd}`
     483    # Process its output
     484    process_output(output)
     485    # Wait for a given duration and loop again
     486    sleep @interval.to_i
     487   end
     488  end
     489end
     490         
     491begin
     492  app = Wrapper.new(ARGV)
     493  app.start()
     494rescue Exception => ex
     495  puts "Received an Exception when executing the wrapper!"
     496  puts "The Exception is: #{ex}\n"
     497end
     498}}}
     499         
     500        For more information on writing wrappers in OMF/OML see: http://mytestbed.net/wiki/omf/OMLWrapperHowTo
     501         
     502         
     503                 
     504                 
     505                 
     506        === Include one or more experiment scripts and "prototypes" === 
     507         
     508        Please Refer to the "Writing experiment script" Section: 
     509        http://groups.geni.net/syseng/wiki/OMF5.3Experiment
     510         
     511        ===  Complete build process ===
     512                 
     513                 
     514        === Test ===
     515         
     516         
     517        === OMF Save for image, to archive ===
     518                 
     519         
     520        The experiment described above is a sample. The motivation is for experimenters to develop new experiments using the framework described above. Once the experimenters have modified the scripts and created new experiments on the Wimax MS, they will need to save their work for repeating experiments.
     521         
     522        - On the OMF aggregate manager machine
     523                 
     524        {{{
     525        omf save -n <node_name>
     526        }}}
     527                 
     528        - Restart the Wimax MS and the MS will boot from PXE
     529         
     530        - The image will be saved on the OMF aggregate manager machine under /var/lib/omf-images-5.3
     531                 
     532        - Rename the image and re-use if needed.
     533 
     534
     535                 
     536
     537===  Known Issues  ===
     538
     539 - There is an incompatibility with OMF and OML at the moment. Newer version of OML uses more precise data types, but OMF is not yet aware of them. NICTA's aware of the problem and is working on a unification solution. In the mean-time, a short fix is to use ":long" instead of "uint32" and ":float" instead of ":double", similar to what we did with the new iperf OMF application. OML still supports these old types, but this is for transition only, and will at some point be deprecated.
     540
     541 - If you receive the following error:  'undefined method `stdin' for nil:NilClass' after using the new iperf in your experiment, you might need to make a small change in your OMF Code. Check http://mytestbed.net/issues/526 and http://mytestbed.net/projects/omf/repository/revisions/331c8c6ecedd02d07b7f60c72f6bffa0c0febbdf
     542 
     543 - We have not yet investigated the use of OMF Disconnected Mode to perform mobile experiments. However, there seems to be issues when one wants to direct the results of an OMF experiment to multiple OML servers/proxies. This feature has to be operational if we want to use the visualization service while performing an experiment in disconnected mode.
     544
     545 - The recent version of OML is 2.6, but as you've probably noticed, we have used OML 2.5.0 in our setup. The reason is that we were facing problems compiling the new, comprehensive iperf code when OML 2.6. was installed/used. The main problem was that OML library files did not seem to be correctly installed.
     546
     547 - Our set up does not yet work with external wimax cards. This is because OMF internally uses lsusb to detect whether a wimax device is available and sometimes external devices (such as an external wimax card) are not listed by that command. Attempts are being made to resolve the issue.
     548
     549
     550
     551
     552
     553
     554
     555
     556