Changes between Initial Version and Version 1 of GENIExperimenter/ExperimentExample


Ignore:
Timestamp:
03/08/11 10:11:35 (13 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • GENIExperimenter/ExperimentExample

    v1 v1  
     1[[PageOutline]]
     2
     3= GENI Experiment Example 1 =
     4
     5This page illustrates an operational example for the [wiki:GENIExperimenter GENI Experimenter How-to] page capturing the actions required to run an experiment using multiple resource aggregates and with user credentials from the GPO ProtoGENI Clearinghouse pgeni.gpolab.bbn.com. Details on how credentials are set up can be found in the [wiki:GENIExperimenter#a1.GetGENICredentials GENI Credentials] section of the main GENI Experimenter page. In this experiment, a total of 5 nodes are requested, with 2 nodes from the Emulab(PG) aggregate and 3 nodes from the !PlanetLab (PL) aggregate.   
     6
     7This page does not cover the installation of the OMNI client, as it is well documented, please see the following pages to get the [http://trac.gpolab.bbn.com/gcf/wiki/GettingGcf GCF package] and to [http://trac.gpolab.bbn.com/gcf/wiki/QuickStart install] it. 
     8
     9== Omni Configuration ==
     10
     11For GENI Experiment Example defined above, the following omni_config settings were used:
     12
     13{{{
     14[omni]
     15default_cf = pgeni
     16users = lnevers
     17
     18# ---------- Users ----------
     19[lnevers]
     20urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+lnevers
     21keys = ~/.ssh/id_rsa.pub
     22
     23# ---------- Frameworks ----------
     24[pgeni]
     25type = pg
     26ch = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/ch
     27sa = https://www.pgeni.gpolab.bbn.com:443/protogeni/xmlrpc/sa
     28cert = ~/.ssl/encrypted-cleartext.pem
     29key = ~/.ssl/encrypted-cleartext.pem
     30verbose=false
     31}}}
     32
     33The above configuration is specific to the experiment captured in this page, please see the [http://trac.gpolab.bbn.com/gcf/wiki/Omni Omni] page for more details. The omni_configuration parameters shown define the following:  [[BR]]
     34|| '''Parameter'''   || ''' Use'''  ||
     35|| default_cf||  Default aggregate used if not specified on the command line.  ||
     36|| users||  users setting used when reserving slices. ||
     37|| urn||  The username URN, see how to determing [wiki:GENIExperimenter/ExperimentCredentials#DetermineyourURN your URN] ||
     38|| keys|| A comma separated list of public ssh key files to be uploaded to the Clearinghouse ||
     39|| type||  Resource aggregate "pg" indicates ProtoGENI, sse the [http://trac.gpolab.bbn.com/gcf/wiki/Omni Omni] page. ||
     40|| ch||  ProtoGENI Clearinghouse URL. ||
     41|| sa|| Slice Authority URL. ||
     42|| cert||  Location of the SSL encrypted certificate (contains both a certificate and a key). ||
     43|| key||  Location of your SSL encrypted certificate (contains both a certificate and a key). ||
     44
     45
     46== List available resources ==
     47
     48With the GPO Protogeni setting defined as shown in the omni_config above, you can get a list of available resources from each of the other 2 aggregates.
     49
     50Getting a list of resources from Emulab Protogeni aggregate in native format:
     51{{{
     52$  src/omni.py -n -a  https://www.emulab.net/protogeni/xmlrpc/am listresources >resources-PG-emulab.xml
     53INFO:omni:Loading config file omni_config
     54INFO:omni:Using control framework pgeni
     55INFO:omni:Resources at https://www.emulab.net/protogeni/xmlrpc/am:
     56}}}
     57Getting a list of resources from !PlanetLab aggregate:
     58{{{
     59$  src/omni.py listresources -n -a http://www.planet-lab.org:12346 >resources-PL.xml
     60INFO:omni:Loading config file omni_config
     61INFO:omni:Using control framework pgeni
     62INFO:omni:Resources at http://www.planet-lab.org:12346:
     63}}}
     64
     65You may use the output of two ''listresources'' commands to generate rspecs to request resources. The above ''listresources'' commands use the  ''-n'' flag to request output in native rspec format rather than the default omnispec format. Note, the ''-n'' flag requires the ''-a'' flag also be used to specify an aggregate manager.   
     66
     67If you are planning to use MyPLC resources, you may also query the MyPLC installation to get a list of available resources as follows:
     68{{{
     69$ ./src/omni.py -n -a http://myplc.gpolab.bbn.com:12346 listresources >resources-MyPLC.xml
     70INFO:omni:Loading config file omni_config
     71INFO:omni:Using control framework pgeni
     72INFO:omni:Resources at http://myplc.gpolab.bbn.com:12346:
     73
     74}}}
     75
     76== Create a slice ==
     77
     78Since the [wiki:GENIExperimenter/ExperimentExample#OmniConfiguration omni_config] defines the credentials and the Clearinghouse for the GPO Protogeni, your slice will be created in pgeni.gpolab.bbn.com aggregate.  For this example a slice named ''lnevers_slice'' is created, you should choose a slice name that is meaningful to you:
     79
     80{{{
     81$ ./src/omni.py createslice lnevers_slice
     82INFO:omni:Loading config file omni_config
     83INFO:omni:Using control framework pgeni
     84Created slice with Name lnevers_slice, URN urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice
     85}}}
     86
     87== Add Emulab PG resources to GPO PG slice ==
     88
     89To request two Emulab ProtoGENI nodes, the following rspec named emulab.rspec was used for this example, your rspec will be specific to your experiment requirements:
     90{{{
     91<?xml version="1.0" encoding="UTF-8"?>
     92<rspec xmlns="http://protogeni.net/resources/rspec/0.2">
     93  <node virtual_id="geni1"
     94        virtualization_type="emulab-vnode">
     95  </node>
     96  <node virtual_id="geni2"
     97        virtualization_type="emulab-vnode">
     98  </node>
     99</rspec>
     100}}}
     101
     102To request two Protogeni nodes, a sliver was created within the Emulab Protogeni with the rpec above using the following command:
     103{{{
     104$ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am -n createsliver lnevers_slice emulab.rspec
     105INFO:omni:Loading config file omni_config
     106INFO:omni:Using control framework pgeni
     107Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result:
     108<?xml version="1.0" ?>
     109<rspec xmlns="http://protogeni.net/resources/rspec/0.2"> 
     110    <node component_manager_urn="urn:publicid:IDN+emulab.net+authority+cm" component_manager_uuid="28a10955-aa00-11dd-ad1f-001143e453fe" component_urn="urn:publicid:IDN+emulab.net+node+pc303" component_uuid="de9e6e61-773e-102b-8eb4-001143e453fe" exclusive="1" hostname="pc303.emulab.net" sliver_urn="urn:publicid:IDN+emulab.net+sliver+36084" sliver_uuid="a8e9898e-48df-11e0-b35d-001143e453fe" sshdport="22" virtual_id="geni1" virtualization_type="raw">   
     111      <services>      <login authentication="ssh-keys" hostname="pc303.emulab.net" port="22"/>    </services>  </node> 
     112    <node component_manager_urn="urn:publicid:IDN+emulab.net+authority+cm" component_manager_uuid="28a10955-aa00-11dd-ad1f-001143e453fe" component_urn="urn:publicid:IDN+emulab.net+node+pc203" component_uuid="de9e66fd-773e-102b-8eb4-001143e453fe" exclusive="1" hostname="pc203.emulab.net" sliver_urn="urn:publicid:IDN+emulab.net+sliver+36085" sliver_uuid="a95e6eef-48df-11e0-b35d-001143e453fe" sshdport="22" virtual_id="geni2" virtualization_type="raw">   
     113      <services>      <login authentication="ssh-keys" hostname="pc203.emulab.net" port="22"/>    </services>  </node> 
     114</rspec>
     115
     116}}}
     117
     118== Add PlanetLab resources to GPO PG slice ==
     119
     120To request three Planetlab nodes, the following rspec named planet.rspec was used for this example, your rspec will be specific to your experiment requirements:
     121
     122{{{
     123<?xml version="1.0"?>
     124<RSpec type="SFA">
     125  <network name="plc">
     126    <site id="rit">
     127      <name>RIT</name>
     128      <node id="n11503">
     129        <hostname>"planet2.cs.rit.edu</hostname>
     130        <sliver />
     131      </node>
     132    </site>
     133    <site id="williams">
     134      <name>Williams College</name>
     135      <node id="n10860">
     136        <hostname>planetlab3.williams.edu</hostname>
     137        <sliver />
     138      </node>
     139    </site>
     140    <site id="bbn">
     141      <name>BBN Technologies</name>
     142      <node id="n13627">
     143        <hostname>plnode-03.gpolab.bbn.com</hostname>
     144        <sliver />
     145      </node>
     146    </site>
     147  </network>
     148</RSpec>
     149}}}
     150
     151To request the three Planetlab nodes, a sliver was created within Planetlab with the rpec above using the following command:
     152
     153{{{
     154$ ./src/omni.py -a http://www.planet-lab.org:12346 -n createsliver lnevers_slice planet.rspec
     155INFO:omni:Loading config file omni_config
     156INFO:omni:Using control framework pgeni
     157Asked http://www.planet-lab.org:12346 to reserve resources. Result: <?xml version="1.0"?>
     158<RSpec type="SFA">
     159  <network name="plc">
     160    <site id="rit">
     161      <name>RIT</name>
     162      <node id="n11503">
     163        <hostname>"planet2.cs.rit.edu</hostname>
     164        <sliver />
     165      </node>
     166    </site>
     167    <site id="williams">
     168      <name>Williams College</name>
     169      <node id="n10860">
     170        <hostname>planetlab3.williams.edu</hostname>
     171        <sliver />
     172      </node>
     173    </site>
     174    <site id="bbn">
     175      <name>BBN Technologies</name>
     176      <node id="n13627">
     177        <hostname>plnode-03.gpolab.bbn.com</hostname>
     178        <sliver />
     179      </node>
     180    </site>
     181  </network>
     182</RSpec>
     183
     184
     185INFO:omni:Please run the omni sliverstatus call on your slice to determine your login name to PL resources
     186}}}
     187
     188As part of the ''createsliver'' operation for PL nodes, Omni tools upload the user SSH keys based on the [users] section of the omni_config. If you have problems in setting up your slice, contact us at [mailto:help@geni.net].
     189
     190== Checking sliver resource status ==
     191
     192Once you have requested the resources you may check the sliver status for PG and PL nodes. As you review the results, note the expiration time to make sure that it meets your experiment running time requirement. To get ''sliverstatus'' for the example experiment on this page the following command was issued for the PG resources:
     193
     194{{{
     195$ ./src/omni.py sliverstatus -a https://www.emulab.net/protogeni/xmlrpc/am lnevers_slice
     196INFO:omni:Loading config file omni_config
     197INFO:omni:Using control framework pgeni
     198Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice:
     199Sliver at https://www.emulab.net/protogeni/xmlrpc/am:
     200{'geni_resources': [{'geni_error': '',
     201                     'geni_status': 'ready',
     202                     'geni_urn': 'urn:publicid:IDN+emulab.net+sliver+36085',
     203                     'pg_manifest': {'attributes': {'component_manager_urn': 'urn:publicid:IDN+emulab.net+authority+cm',
     204                                                    'component_manager_uuid': '28a10955-aa00-11dd-ad1f-001143e453fe',
     205                                                    'component_urn': 'urn:publicid:IDN+emulab.net+node+pc203',
     206                                                    'component_uuid': 'de9e66fd-773e-102b-8eb4-001143e453fe',
     207                                                    'exclusive': 1,
     208                                                    'hostname': 'pc203.emulab.net',
     209                                                    'sliver_urn': 'urn:publicid:IDN+emulab.net+sliver+36085',
     210                                                    'sliver_uuid': 'a95e6eef-48df-11e0-b35d-001143e453fe',
     211                                                    'sshdport': 22,
     212                                                    'virtual_id': 'geni2',
     213                                                    'virtualization_type': 'raw',
     214                                                    'xmlns': 'http://protogeni.net/resources/rspec/0.2'},
     215                                     'children': [{'attributes': {},
     216                                                   'children': [{'attributes': {'authentication': 'ssh-keys',
     217                                                                                'hostname': 'pc203.emulab.net',
     218                                                                                'port': 22},
     219                                                                 'children': [],
     220                                                                 'name': 'login'}],
     221                                                   'name': 'services'}],
     222                                     'name': 'node'},
     223                     'pg_status': 'ready'},
     224                    {'geni_error': '',
     225                     'geni_status': 'ready',
     226                     'geni_urn': 'urn:publicid:IDN+emulab.net+sliver+36084',
     227                     'pg_manifest': {'attributes': {'component_manager_urn': 'urn:publicid:IDN+emulab.net+authority+cm',
     228                                                    'component_manager_uuid': '28a10955-aa00-11dd-ad1f-001143e453fe',
     229                                                    'component_urn': 'urn:publicid:IDN+emulab.net+node+pc303',
     230                                                    'component_uuid': 'de9e6e61-773e-102b-8eb4-001143e453fe',
     231                                                    'exclusive': 1,
     232                                                    'hostname': 'pc303.emulab.net',
     233                                                    'sliver_urn': 'urn:publicid:IDN+emulab.net+sliver+36084',
     234                                                    'sliver_uuid': 'a8e9898e-48df-11e0-b35d-001143e453fe',
     235                                                    'sshdport': 22,
     236                                                    'virtual_id': 'geni1',
     237                                                    'virtualization_type': 'raw',
     238                                                    'xmlns': 'http://protogeni.net/resources/rspec/0.2'},
     239                                     'children': [{'attributes': {},
     240                                                   'children': [{'attributes': {'authentication': 'ssh-keys',
     241                                                                                'hostname': 'pc303.emulab.net',
     242                                                                                'port': 22},
     243                                                                 'children': [],
     244                                                                 'name': 'login'}],
     245                                                   'name': 'services'}],
     246                                     'name': 'node'},
     247                     'pg_status': 'ready'}],
     248 'geni_status': 'ready',
     249 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice',
     250 'pg_expires': '2011-03-07 16:23:05',
     251 'pg_status': 'ready'}
     252}}}
     253
     254To Check the ''sliverstatus'' for PL resources, the following command was issued:
     255
     256{{{
     257$ ./src/omni.py sliverstatus -a http://www.planet-lab.org:12346 lnevers_slice
     258INFO:omni:Loading config file omni_config
     259INFO:omni:Using control framework pgeni
     260Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice:
     261Sliver at http://www.planet-lab.org:12346:
     262{'geni_resources': [{'geni_error': '',
     263                     'geni_status': 'ready',
     264                     'geni_urn': 'urn:publicid:IDN+plc:bbn+node+plnode-03.gpolab.bbn.com',
     265                     'pl_boot_state': 'boot',
     266                     'pl_hostname': 'plnode-03.gpolab.bbn.com',
     267                     'pl_last_contact': 'Mon Mar  7 12:17:51 2011'},
     268                    {'geni_error': '',
     269                     'geni_status': 'ready',
     270                     'geni_urn': 'urn:publicid:IDN+plc:williams+node+planetlab3.williams.edu',
     271                     'pl_boot_state': 'boot',
     272                     'pl_hostname': 'planetlab3.williams.edu',
     273                     'pl_last_contact': 'Mon Mar  7 12:23:21 2011'},
     274                    {'geni_error': '',
     275                     'geni_status': 'ready',
     276                     'geni_urn': 'urn:publicid:IDN+plc:rit+node+planet2.cs.rit.edu',
     277                     'pl_boot_state': 'boot',
     278                     'pl_hostname': 'planet2.cs.rit.edu',
     279                     'pl_last_contact': 'Tue Mar  1 05:49:20 2011'}],
     280 'geni_status': 'ready',
     281 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice',
     282 'pl_expires': 'Mon Mar 21 13:24:29 2011',
     283 'pl_login': 'pgenigpolabbbncom_lneversslice'}
     284
     285}}}
     286
     287A lot of very useful information is reported for the ''sliverstatus'' output including which hosts are assigned to you and for how long. For PlanetLab, notice the value in the 'pl_login' field, which is the login to be used to access the nodes. For a slice across multiple aggregates, you should pay close attention to the expiration time for each aggregate sliver (pg_expires & pl_expires).  For this example the pg_expires is ''2011-03-07 16:23:05'' while the pl_expires is ''Mon Mar 21 13:24:29 2011'', make sure that each expiration time meets your requirements for the experiment duration. If you need to extend the expiration, you can use the Omni ''renewsliver'' to expand the time for each sliver and for the slice. The following command is issued to modify the expiration time for the ProtoGENI resources:
     288
     289{{{
     290$ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am renewslice lnevers_slice "03/21/11 13:00:00"
     291INFO:omni:Loading config file omni_config
     292INFO:omni:Using control framework pgeni
     293INFO:omni.protogeni:requesting new expiration '2011-03-21T13:00:00'
     294Slice lnevers_slice now expires at 2011-03-21 13:00:00
     295
     296}}}
     297
     298Check status again and you will notice that the ''pg_expires'' has been modified and is more in line with the ''pg_expires'':
     299
     300{{{
     301$ ./src/omni.py sliverstatus -a https://www.emulab.net/protogeni/xmlrpc/am lnevers_slice
     302INFO:omni:Loading config file omni_config
     303INFO:omni:Using control framework pgeni
     304Status of Slice urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice:
     305Sliver at https://www.emulab.net/protogeni/xmlrpc/am:
     306{'geni_resources': [{'geni_error': '',
     307                     'geni_status': 'ready',
     308                     'geni_urn': 'urn:publicid:IDN+emulab.net+sliver+36085',
     309                     'pg_manifest': {'attributes': {'component_manager_urn': 'urn:publicid:IDN+emulab.net+authority+cm',
     310                                                    'component_manager_uuid': '28a10955-aa00-11dd-ad1f-001143e453fe',
     311                                                    'component_urn': 'urn:publicid:IDN+emulab.net+node+pc203',
     312                                                    'component_uuid': 'de9e66fd-773e-102b-8eb4-001143e453fe',
     313                                                    'exclusive': 1,
     314                                                    'hostname': 'pc203.emulab.net',
     315                                                    'sliver_urn': 'urn:publicid:IDN+emulab.net+sliver+36085',
     316                                                    'sliver_uuid': 'a95e6eef-48df-11e0-b35d-001143e453fe',
     317                                                    'sshdport': 22,
     318                                                    'virtual_id': 'geni2',
     319                                                    'virtualization_type': 'raw',
     320                                                    'xmlns': 'http://protogeni.net/resources/rspec/0.2'},
     321                                     'children': [{'attributes': {},
     322                                                   'children': [{'attributes': {'authentication': 'ssh-keys',
     323                                                                                'hostname': 'pc203.emulab.net',
     324                                                                                'port': 22},
     325                                                                 'children': [],
     326                                                                 'name': 'login'}],
     327                                                   'name': 'services'}],
     328                                     'name': 'node'},
     329                     'pg_status': 'ready'},
     330                    {'geni_error': '',
     331                     'geni_status': 'ready',
     332                     'geni_urn': 'urn:publicid:IDN+emulab.net+sliver+36084',
     333                     'pg_manifest': {'attributes': {'component_manager_urn': 'urn:publicid:IDN+emulab.net+authority+cm',
     334                                                    'component_manager_uuid': '28a10955-aa00-11dd-ad1f-001143e453fe',
     335                                                    'component_urn': 'urn:publicid:IDN+emulab.net+node+pc303',
     336                                                    'component_uuid': 'de9e6e61-773e-102b-8eb4-001143e453fe',
     337                                                    'exclusive': 1,
     338                                                    'hostname': 'pc303.emulab.net',
     339                                                    'sliver_urn': 'urn:publicid:IDN+emulab.net+sliver+36084',
     340                                                    'sliver_uuid': 'a8e9898e-48df-11e0-b35d-001143e453fe',
     341                                                    'sshdport': 22,
     342                                                    'virtual_id': 'geni1',
     343                                                    'virtualization_type': 'raw',
     344                                                    'xmlns': 'http://protogeni.net/resources/rspec/0.2'},
     345                                     'children': [{'attributes': {},
     346                                                   'children': [{'attributes': {'authentication': 'ssh-keys',
     347                                                                                'hostname': 'pc303.emulab.net',
     348                                                                                'port': 22},
     349                                                                 'children': [],
     350                                                                 'name': 'login'}],
     351                                                   'name': 'services'}],
     352                                     'name': 'node'},
     353                     'pg_status': 'ready'}],
     354 'geni_status': 'ready',
     355 'geni_urn': 'urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lnevers_slice',
     356 'pg_expires': '2011-03-21 07:00:00',
     357 'pg_status': 'ready'}
     358}}}
     359
     360Both sliver now have similar expiration time and resources are available for the same timeframe. 
     361
     362== Verify Resources and run your Experiment ==
     363
     364Once you have the ''sliverstatus'', you have enough information to access the hosts allocated for you experiment.  Simply SSH into one of the hosts and ping any of the other hosts in your experiment.  Notice that the ''pl_login'' is being used to get access.  Now verify that there is connectivity between to allocated resource. In this example experiment, one of the !PlanetLab resources allocated is used to ping a !ProtoGENI resource also part of the experiment:
     365
     366{{{
     367$ ssh pgenigpolabbbncom_lneversslice@plnode-03.gpolab.bbn.com
     368Last login: Mon Mar  7 18:20:48 2011 from sendaria.gpolab.bbn.com
     369[pgenigpolabbbncom_lneversslice@plnode-03 ~]$ ping pc203.emulab.net -c 3
     370PING pc203.emulab.net (155.98.39.3) 56(84) bytes of data.
     37164 bytes from pc203.emulab.net (155.98.39.3): icmp_seq=1 ttl=44 time=116 ms
     37264 bytes from pc203.emulab.net (155.98.39.3): icmp_seq=2 ttl=44 time=116 ms
     37364 bytes from pc203.emulab.net (155.98.39.3): icmp_seq=3 ttl=44 time=116 ms
     374
     375--- pc203.emulab.net ping statistics ---
     3763 packets transmitted, 3 received, 0% packet loss, time 2001ms
     377rtt min/avg/max/mdev = 116.385/116.595/116.997/0.398 ms
     378[pgenigpolabbbncom_lneversslice@plnode-03 ~]$
     379}}}
     380
     381Once you have verified connectivity between these hosts that have been allocated, you may start your experiment.
     382
     383== Delete sliver resources ==
     384
     385When the experiment is completed you should release the resources by deleting the resource sliver for each the Resource Aggregates used in the Experiment. In the example experiment used in this page, the ProtoGENI sliver was deleted as follows:
     386{{{
     387$ ./src/omni.py -a https://www.emulab.net/protogeni/xmlrpc/am  deletesliver lnevers_slice
     388INFO:omni:Loading config file omni_config
     389INFO:omni:Using control framework pgeni
     390Deleted sliver urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lneversslice on unspecified_AM_URN at https://www.emulab.net/protogeni/xmlrpc/am
     391}}}
     392
     393Also the Planetlab sliver was deleted as follows:
     394{{{
     395$ ./src/omni.py -a http://www.planet-lab.org:12346  deletesliver lnevers_slice
     396INFO:omni:Loading config file omni_config
     397INFO:omni:Using control framework pgeni
     398Deleted sliver urn:publicid:IDN+pgeni.gpolab.bbn.com+slice+lneversslice on unspecified_AM_URN at http://www.planet-lab.org:12346
     399
     400}}}
     401
     402
     403----
     404{{{
     405#!html
     406<a href="mailto:lnevers@bbn.com">Email us</a> with any questions and feedback on this page!
     407}}}