wiki:GIR2.1_MillionNodeGENI

Version 1 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

Million Node GENI Evaluation

The ProtoGENI XMLRPC server integration delivers an approach to integrate the SeattleGENI (Million-node GENI) project with the ProtoGENI project. This package uses Emulab user SSL certificate credentials to access the XMLRPC server to acquire and release SeattleGENI resources.

The software for Million Node GENI Integration is available at XML-RPC Server for Protogeni Integration web site.

Other software available but not evaluated:

Time frame: This evaluation took place July 15-20, 2010.

Million Node GENI Findings

Initial version of instructions were not complete, after feedback instructions were overhauled and are now complete.

Million Node GENI How-to

Download XMLRPC server integration package and untar the tgz file protogeni_integration.tgz. The package includes the following README:

Protogeni XMLRPC Server Integration:

The Protogeni XMLRPC server integration is a way to integrate the 
SeattleGENI (Million-node GENI) project with the protogeni project.
It allows emulab users to to acquire and release SeattleGENI
resources. An emulab user can use their credential to access the 
XMLRPC server to acquire resources. The user can only use their 
emulab ssl certificate in order to be granted access.


Files that are required to run XMLRPC server:

  1. seattlegeni_xmlrpc_server.pl (included)
  2. generate_pubkey.py (included)
  3. protogeni reference component files.
  4. seattlegeni_xmlrpc.logfile
  5. protogeni_user_file.txt
  6. protogeni_vessel_handle.txt
  7. genica.bundle
  8. genicrl.bundle
  9. __lockfile__
  10. seattlegeni_apache.conf
  11. Seattle library files.

Setting up XMLRPC server:

  1. Create a folder where you want to have the XMLRPC server running
     from.

  2. Make sure that you have either all the seattle
     library files in the folder or have the seattle libraries in your
     PYTHONPATH. If you do not have the Seattle library files you can get it
     from our svn at https://seattle.cs.washington.edu/svn/seattle/trunk/
     Once you have checked out our repository, go into trunk and run the
     command:
    
     $> python preparetest.py /folder/to/copy/seattle_libraries_to

     (Note that running this command on a folder will delete any existing
     file already in that folder. It is best to do this on an empty folder,
     like in the one created in step 1.)   

  3. Download the Reference Component files from the Protogeni site and make
     sure that all the required libraries are downloaded and installed. 
     Instructions for all of this can be found at:
     https://www.protogeni.net/trac/protogeni/wiki/ReferenceCM

  4. Run the script prepare_seattlegeni_server.py with the command below
     where the target folder should be the folder created in step 1. This
     will copy/create/download the files that are required to run the xmlrpc
     server.
  
     $> python prepare_seattlegeni_server.py target_folder  

  5. Set up the .conf file for apache. An example of what the 
     .conf file should look like is provided (seattlegeni_apache.conf).
     A more detailed description on how to edit this file is located at the
     bottom of this file.

  6. Edit the file protogeni_user_file.txt and add all the seattlegeni
     users that have been specifically allocated for the protogeni integration.
     You can acquire seattlegeni usernames by registering an username through 
     the SeattleGENI website at: https://seattlegeni.cs.washington.edu
     The format of each line in protogeni_user_file.txt is:

     seattlegeni_username:password_for_user:1:0:0

     The last 3 values are set by default to 1:0:0 but this will get changed
     to some other value after the XMLRPC server has run.

  7. Edit the seattlegeni_xmlrpc_server.pl file to make sure some of the global
     variables are set correctly if not done so already. The variables that 
     need to be changed if necessary are:
      * $generate_pubkey_path
      * $server_url
      * $protogeni_user_filename
      * $protogeni_vessel_handle_filename
      * $lockfile_path

   8. Edit the generate_pubkey.py file to make sure that the correct path has
      been set for the Seattle library files. The directory path for the Seattle
 
      library files is the same path as the directory where Seattle files 
      were extracted to in Step 2.

Api calls provided in the XMLRPC server:
  1. CreateSliver()
  2. DeleteSlice()

You can visit our wiki page to learn more about the two calls and our XMLRPC 
server. The wiki page is located at: https://seattle.cs.washington.edu/wiki/ProtogeniIntegration

Configuring the seattlegeni_apache.conf file:

This is a just a very simple description of how to configure the .conf file. More complicated
configuration could be done if necessary. In order to configure the .conf file properly, the
following variables must be modified:

  * The path of the log files must be set correctly. So any errors and access logs are 
    recorded properly.
  * You must have a valid SSL Certificate and set the path to the Certificate file (Line 42)
  * The path for SSLCACertificateFile and SSLCARevocationFile must be set in line 45-46. 
    When the prepare_seattlegeni_server.py file was ran in Step 4, the two files genica.bundle
    and genicrl.bundle should have been downloaded to the target_folder. Use the file path of 
    these two files as the path for SSLCACertificateFile and SSLCARevocationFile. If the two
    files were not downloaded, then they could be downloaded here:
  
      https://www.emulab.net/genica.bundle
      https://www.emulab.net/genicrl.bundle
   
    These files are used to grant access to emulab users.
  * The path to the seattlegeni_xmlrpc_server.pl should be set for the variable ScriptAlias
    in line 49.
  * The directory where the  XMLRPC server resides should be set at line 51 as the directory path.
  * The location of the Seattle library files that were extracted in Step 2, should be added to 
    the pythonpath in line 61 of the .conf file.

More tutorials on how to configure apache files can be found at the apache website at
http://httpd.apache.org/ under the documentation section.

Testing the XMLRPC server:

We have provided the a test file that tests the two api calls. In order to run
it successfully, you will need a valid emulab certificate (emulab.pem) and must
have it in the same folder as the test file. The test file must also be modified
slightly to include the right certificate and the right filepath for certificate.


If you have any questions please contact Monzur Muhammad at monzum@cs.washington.edu 
or Justin Cappos at justinc@cs.washington.edu

Reference Component Manager Installation

The ProtoGENI XMLRPC Server requires the Reference Component Manager package from the ProtoGENI Reference Component Manager page. Installation instructions are Fedora specific, and so are the installation scripts. The installation tested was Ubuntu 9.10 Karmic. The Reference Component Manager requires the following pre-requisites:

  • gcc
  • Apache HTTP server, and mod_ssl
  • curl
  • libvirt (installed libvirt-bin)
  • mysql-server
  • Perl (version 5 or newer), the suidperl wrapper (perl-suid):
    • Crypt::SSLeay
    • Frontier::RPC
    • RPC::XML
    • Time:: and Date::
    • XML::LibXML and XML::Simple
  • qemu
  • xmlsec1
  • xmlsec1-openssl

Once installation is completed for the above, the following is required:

 $ sudo ln -s libxmlsec1-openssl.so.1 /usr/lib/libxmlsec1-openssl.so
 $ sudo /etc/rc.d/init.d/mysqld start
 $ sudo /etc/init.d/libvirt-bin start

A local configuration file must be created by copying a file named defs-example to a file named defs-local. The defs-local must be modified to contain local settings, evaluation used the following settings:

SSLCERT_COUNTRY="US"
SSLCERT_STATE="MA"
SSLCERT_LOCALITY="Cambridge"
SSLCERT_ORGNAME="BBN"

PROTOGENI_DOMAIN="gpolab.bbn.com"
PROTOGENI_AUTHORITY="bbn.com"

ADMIN_EMAIL="lnevers@bbn.com"

NUM_VIRTUAL_NODES=4
IMAGE_DIR="/var/vm"
BASE_VIRTUAL_IMAGE="/var/vm/base.img"

To build the Reference Component Manager:

  $ cd reference-cm-2.0.2/
  $ ./configure --with-DEFS=defs-local
  $ make

Note: The make install did not work due to various errors, but based on input from SeattleGENI folks, there is no need to install the Reference Component Manager, the SeattleGENI software has variables to set the location of the Reference Component Manager.

Installing the Protogeni XMLRPC Server

Once Reference Component Manages is built, the system is ready to install the ProtoGENI XMLRPC Server software. Executed instructions in README.txt:

$ tar xvzf  protogeni_integration.tgz
$ svn co  https://seattle.cs.washington.edu/svn/seattle/trunk
$ mkdir /home/lnevers/millionnode/seattleGENI
$ cd trunk 
$ python preparetest.py /home/lnevers/millionnode/seattleGENI
$  cd ../protogeni_integration
$ python prepare_seattlegeni_server.py /home/lnevers/millionnode/seattleGENI/

Preparing to copy necessary files to /home/lnevers/millionnode/seattleGENI
Copying directory.....xml_manifest_schema
Copying file.....seattlegeni_xmlrpc_server.pl
Copying file.....README.txt
Copying directory.....test
Copying file.....seattlegeni_apache.conf
Copying file.....generate_pubkey.py
Copying file.....prepare_seattlegeni_server.py
Copying directory.....seattle_libraries
Copying file.....LICENSE.txt

Preparing to create files needed by the XMLRPC server to /home/lnevers/millionnode/seattleGENI
Creating file.....seattlegeni_xmlrpc.logfile
Creating file.....protogeni_user_file.txt
Creating file.....protogeni_vessel_handle.txt
Creating file.....__lockfile__

Downloading required files to /home/lnevers/millionnode/seattleGENI
Downloading file.....genica.bundle
Downloading file.....genicrl.bundle
Changing mode of files in directory: /home/lnevers/millionnode/seattleGENI
##################################################################################
#  All the necessary files for the XMLRPC server to run has been copied          
#  over and downloaded to the folder: /home/lnevers/millionnode/seattleGENI                      
#                                                                                
#  In order to finish the setup please edit the files listed below and            
#  set the appropriate variable values as listed below.            
#
#  Before modifying the files, the protogeni Reference Component Manager files 
#  and Seattle library files should already be downloaded to some folder. This             
#  should have been done in Step 2 and 3 in the README.txt             
#                                                                 
#  seattlegeni_xmlrpc_server.pl:                                                  
#    * use lib '/path/to/protogeni/reference-cm-2.0/xmlrpc'; (Line 12)
#    * use lib '/path/to/protogeni/reference-cm-2.0/lib'; (Line 13)   
#    * $directory_prefix = '/home/lnevers/millionnode/seattleGENI'; (Line 36)                    
#    * $server_url = 'https://seattlegeni.cs.washington.edu/xmlrpc/'; (Line 40)   
#                                                                                
#  generate_pubkey.py:                                                           
#    * sys.path.append("where/the/seattle/library/files/are/located) (Line 2)    
#
#  protogeni_user_file.txt:
#    * Add all the seatlegeni users that have been allocated for this server.
#      Each of the line in this file should follow the format:
#  
#        seattlegeni_username:password_for_user:1:0:0
#     
#      The last 3 values are set by default to 1:0:0 but this will get changed
#      to some other value after the XMLRPC server has run.
#
# Please contact Monzur Muhammad (monzum@cs.washington.edu) with any questions! 
##################################################################################

Executed instructions from prepare_seattlegeni_server.py install output:

  1. Modified the seattlegeni_xmlrpc_server.pl script:
    • set line 12 and 13to :
      use lib '/home/lnevers/reference-cm-2.0.2/xmlrpc';
      use lib '/home/lnevers/reference-cm-2.0.2/lib';
      
    • set line 36 to the target directory:
      $directory_prefix = '/home/lnevers/millionnode/seattleGENI'; 
      
    • Set line 40 to my server's URL:
      my $server_url = 'https://sendaria.gpolab.bbn.com/xmlrpc/';
      
  1. Modified the generate_pubkey.py script:
  • set line 5 to define the target directory
    sys.path.append("/home/lnevers/millionnode/seattleGENI")
    
  1. Create a file protogeni_user_file.txt containing the following entry:
    lnevers:password:1:0:0
    

Returned to instruction in README.txt to modify the file seattlegeni_apache.conf file.

Made the following modification in the <VirtualHost *:80> section modified the redirect to point to my server:

    RedirectMatch ^/xmlrpc/?$ https://sendaria.gpolab.bbn.com/xmlrpc/

Made the following modifications in the <VirtualHost *:443> section provided:

    ServerName sendaria.gpolab.bbn.com
    SSLCertificateFile /etc/apache2/cert.pem
    SSLCertificateKeyFile /etc/apache2/cert.key
    SSLCACertificateFile /home/lnevers/millionnode/seattleGENI/genica.bundle
    SSLCARevocationFile /home/lnevers/millionnode/seattleGENI/genicrl.bundle
    ScriptAlias /xmlrpc/ /home/lnevers/millionnode/seattleGENI/seattlegeni_xmlrpc_server.pl
    PythonPath "['/home/lnevers/millionnode/seattleGENI'] + sys.path"

Once modifications were made, inserted content of seattlegeni_apache.conf at the end of the existing /etc/apache2/apache2.conf file and restarted the apache server:

$ sudo /etc/init.d/apache2 restart
 * Restarting web server apache2
 [Fri Jul 16 11:07:45 2010] [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [Fri Jul 16 11:07:46 2010] [warn] NameVirtualHost *:80 has no VirtualHosts

Using SeattleGENI to get resources

A script test_seattlegeni_xmlrpc.py is provided in the test directory that can connect to the Seattle XMLRPC Server? that requires modifications to add local values:

   14  certificate_file = "/home/lnevers/.ssl/cert.pem"
   15  certificate_key = "/home/lnevers/.ssl/cert.pem"
   68       <owner_urn>urn:publicid:IDN+emulab.net+user+lnevers</owner_urn>
    79      <signatures>-----BEGIN CERTIFICATE-----
    80  MIIDUTCCAroCAmqTMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJVUzENMAsG
    81  A1UECBMEVXRhaDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHTAbBgNVBAoTFFV0
    82  YWggTmV0d29yayBUZXN0YmVkMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBBdXRob3Jp
    83  dHkxGDAWBgNVBAMTD2Jvc3MuZW11bGFiLm5ldDEoMCYGCSqGSIb3DQEJARYZdGVz
    84  dGJlZC1vcHNAZmx1eC51dGFoLmVkdTAeFw0xMDA3MTYxOTEwMDVaFw0xMTA3MTYx
    85  OTEwMDVaMIGqMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEdMBsGA1UEChMU
    86  VXRhaCBOZXR3b3JrIFRlc3RiZWQxGzAZBgNVBAsTEnV0YWhlbXVsYWIubG5ldmVy
    87  czEtMCsGA1UEAxMkZWUzYjNiMWEtMWNkNC0xMWRmLTg2MzMtMDAxMTQzZTQ1M2Zl
    88  MSEwHwYJKoZIhvcNAQkBFhJsbmV2ZXJzQGVtdWxhYi5uZXQwgZ8wDQYJKoZIhvcN
    89  AQEBBQADgY0AMIGJAoGBALK88CkpB82ElRkqApjci+HfYFeLtfuJ8dIU0DnhdbOd
    90  g1SB5K8RsGw3DGMmpWYDv5AUMbGZnFsw9ob2u47Y2polg8+7sG8NG//ZIx47m2Dj
    91  BCDvk95t2zb4K4M6Dn2eGXljv7LEg4Kx5AXEQLQdLHO/VuS10OqnLgKF8a68wGkv
    92  AgMBAAGjezB5MA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFBk2a7X6jRTjlVn3
    93  Pf5O6P2YPg+AMEcGA1UdEQRAMD6GKHVybjpwdWJsaWNpZDpJRE4rZW11bGFiLm5l
    94  dCt1c2VyK2xuZXZlcnOBEmxuZXZlcnNAZW11bGFiLm5ldDANBgkqhkiG9w0BAQQF
    95  AAOBgQA76kObk77BaI5ZpHf62bSek/hCCTjv520p3Cp29s9N2EPnIwSs82GoPFXH
    96  7pIaU6evs7ZD9b0ZnDegAHDv8Zeqz0Fn+2cwEbhWYzxWgav/hebQYL5wUd5ezLlb
    97  yxce15rpSlCQv7oaC8DBqJctxt66XrKqmPExXS2nxrnZV4QwEw==
    98  -----END CERTIFICATE-----</signatures>

To run the test script:

 $ cd test
 $ python test_seattlegeni_xmlrpc.py 
Enter passphrase:
urn:publicid:IDN+SeattleGENI+user+protogeni1
<?xml version="1.0" encoding="UTF-8"?>
<rspec generated="today" generated_by="SeattleGENI" type="response" xmlns="https://blackbox.cs.washington.edu/xmlrpc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://blackbox.cs.washington.edu/~geni/seattle_manifest.xsd">
  <node api_key="MNWBL2SYFDZCJH8RKPA9I0XEQ7GV3165" publickey_string="65537 120512685441740842577373708819853931919825986406212955251278187150733421961521346496763914232583237622839226083360962446467364575063949367258436537472495303123088081828744228500546956746065704870103042710822372960689981873465298497023441025867905151253217158391319083999233266757582399827051457444655446660647
">
    <vesselhandle_list>
      <vesselhandle>65537 135486028496902009512665627603761673058893411503400933667144174323377387824725672176557951389838224463301751548053770501211092945741696900250872017990448234196912042892224707867237605444969401667381528409670296897206150316535751501256273682695198645865943714625042708603924924372971196635859701046616587919739:v8</vesselhandle>
    </vesselhandle_list>
  </node>
</rspec>

urn:publicid:IDN+SeattleGENI+user+protogeni2
<?xml version="1.0" encoding="UTF-8"?>
<rspec generated="today" generated_by="SeattleGENI" type="response" xmlns="https://blackbox.cs.washington.edu/xmlrpc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://blackbox.cs.washington.edu/~geni/seattle_manifest.xsd">
  <node api_key="3FJ1L42UZB6CEG0WSXKIDTNYRAHV857Q" publickey_string="65537 99137223245520444922747703237270954294414249004376965224379445123161250311865998850953070377780649840154443924528730300268496550282290679936805925390531494429626384108485699399044957286921103880646612751794829788055457659515012067777365590984166289334136254695024516376725106328234936490259448720070322813127
">
    <vesselhandle_list>
      <vesselhandle>65537 109703901645392842475261885067191637931597554235162826134115110078745096229852809462726461439171148503919390693276391823816120735831387189021620686461578725093140964433578875735519779895910204956609725146841517639798091531536350008111732174175278425175920055620762913246177804641266921000408015879257586199017:v8</vesselhandle>
      <vesselhandle>65537 118876430724935094861347028121907352394591897835719150725044631542141161230341160500078765878664347130963266276779794892905188782977933756290093612572777315305199863840307517780462673519335307459392968414438548472639508124059941157199585131184579519315308964055730784861577688204234542534434013944747163006451:v10</vesselhandle>
    </vesselhandle_list>
  </node>
</rspec>

urn:publicid:IDN+SeattleGENI+user+protogeni3
<?xml version="1.0" encoding="UTF-8"?>
<rspec generated="today" generated_by="SeattleGENI" type="response" xmlns="https://blackbox.cs.washington.edu/xmlrpc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://blackbox.cs.washington.edu/~geni/seattle_manifest.xsd">
  <node api_key="Q9507RNPB14MIJ2DLUFY68SHTVGAXZWK" publickey_string="65537 112029918886290129218790289019320255873188979273207259403054602598966103342936766017663567519702754843054423339917072931557372062715143025514538057390537951314538394766941533782231863750081920741115452498577986870914283783676356318451665601655312120647237563019182070314099910259714194019116264964124226166057
">
    <vesselhandle_list>
      <vesselhandle>65537 94699844046035047708449864504292714466944613878938290306448572818929724230196983934650297672246876930569769740095893099321685410762456567226774356736962062273385737755236596156922390743692189931029148246106432507253281350760415347268769723462446607932704140086892670989264760930086180628622097221633067587913:v8</vesselhandle>
      <vesselhandle>65537 145749200566611248551053550298841530701845278117896789236165118489405005669440664679671633922652054178898157375760064324034403660809793813251254621087799819243994017413124521695839916031412774334860456409926229981851487986697706084295095621388739432279991381166510607525838232620532890555908141989087672027137:v4</vesselhandle>
      <vesselhandle>65537 115799825556065081469969196532374085924278990947875934693283909109419871665310551693354500523982001928801880754775426023747963095942825040603391606821938997804096435215984693170706922119196987460547224047274188560957641142910008833255931777432864229660851704751124299092619560208267382191288828214755689339471:v4</vesselhandle>
    </vesselhandle_list>
  </node>
</rspec>

{'output': 'Resource already allocated for requested for slice_urn: urn:publicid:IDN+SeattleGENI+slice+mytestslice0', 'code': 7, 'value': 0}
{'output': 'Resource already allocated for requested for slice_urn: urn:publicid:IDN+SeattleGENI+slice+mytestslice1', 'code': 7, 'value': 0}
{'output': 'Resource already allocated for requested for slice_urn: urn:publicid:IDN+SeattleGENI+slice+mytestslice2', 'code': 7, 'value': 0}
{'output': '', 'code': 0, 'value': 0}
{'output': '', 'code': 0, 'value': 0}
{'output': '', 'code': 0, 'value': 0}
{'output': 'The slice urn:publicid:IDN+SeattleGENI+slice+mytestslice0 does not exist!', 'code': 12, 'value': 0}
{'output': 'The slice urn:publicid:IDN+SeattleGENI+slice+mytestslice1 does not exist!', 'code': 12, 'value': 0}
{'output': 'The slice urn:publicid:IDN+SeattleGENI+slice+mytestslice2 does not exist!', 'code': 12, 'value': 0}