wiki:sol4/IDMS/ExpTutorial

Version 30 (modified by ezkissel@indiana.edu, 10 years ago) (diff)

--

IDMS Experiment in GENI

The persistent IDMS experiment allows other users to attach at shared VLAN sites, granting access to the available storage resources. Users may then upload and download their experiment data using a set of client tools. IDMS provides:

  • A custom RSpec that configures a node to access the IDMS experiment
  • Nodes use DHCP to lease an address within the IDMS data plane subnet
  • Automatically installed client tools to upload/download data to available storage nodes

Base Experiment Topology

Shared VLANs at UCLA, Missouri, Nysernet, GPO and large storage at RENCI, UCD, and TAMU

  • Stitching between the shared VLAN sites to create a cross-country data plane
  • Phoebus Gateways on ProtoGENI backbone nodes for external access to the experiment and to improve WAN transfer performance
  • DHCP server at nysernet-ig

IDMS ip subnet

IDMS experiment is implemented on 10.10.0.0/16 ip subnet. IP range is fixed for each part of experiment.

Base experiment nodes 10.10.100.{0-255}
IBP resource allocated dynamically 10.10.{50-99}.0
External user host 10.10.{150-250}.{2-0}
DHCP server 10.10.150.1

IDMS topology

Prerequisite

In order to attach external user to base experiment. You need to reserve resources. You can use either Flack or Omni to do that. If you haven't used any of these tools before, please go through the introductory tutorials first.

Step 1: Create Slice

Create the slice in the project. Name the slice something like xxxxIdmsUser.

     omni createslice <slice_name> -r <project_name>

Step 2: Reserve Resources

  1. Download the rspec . For more information on rspec refer Inside rspec

  1. Base experiment can only be joined at shared VLAN nodes. Currently shared VLAN nodes are at
    • ucla-ig
    • missouri-ig
    • nysernet-ig
    • gpo-ig
  1. Select aggregate manager from list above.
       omni createsliver -a <aggregate_manager> <slice_name> <path_to_rspec_file> -r <project_name>
    
  1. Poll the resource status to make sure overall 'geni_status' is 'ready'
       omni sliverstatus -a <aggregate_manager> <slice_name> -r <project_name>
    
  1. If 'geni_status' is not 'ready' try using another aggregate manager

Step 3: Login to Reserved Host

  1. Login to reserved host. If you are unaware of this, please follow instruction here
  1. Test configuration : DHCP server allocates ip to each host connected to base experiment. To ensure your host is connected to IDMS base experiment data plane, ping the DHCP server.
        ping 10.10.150.0
    
    lbone server advertises list of ip addresses of free IBP server which in turns Lors tool use to stash data. In case of IDMS experiment lbone server will avertise list of ip addresses of IBP server in IDMS subnet. To test connectivity with lbone server, telnet to lbone server on 6767 port. If communication is successful, you will get telnet prompt. To escape the prompt just hit Enter
       telnet monitor.incntre.iu.edu 6767
    
    You can also ensure connectivity to each IBP server in subnet by pinging the ip address
       ping <IBP_server_IP_address>
    
  1. To download / upload data Lors tool is already installed. Run the lors_setup.sh script to setup the ~/.xndrc config file for the LoRS

Command Line Tools.

    lors_setup.sh

Step 4: Upload Data

  1. To upload data use lors_upload command. By default depot store file for 1 day. If you want to change the allocation time use '-d' option
       Command :
           lors_upload -H monitor.incntre.iu.edu -f <file_to_be_uploaded> <options>
    
       Sample Execution:
           [adorwat@user-2 ~]$ lors_upload -V 1 -H monitor.incntre.iu.edu -f sample -n
           TOOL Upload
           SIZE 47
           TITLE Filename sample
           EXNODEFILE sample.xnd
           No	ID	Depot	Port	Score	Bandwidth	Prox	NFailure	NThread	NThreadDone
           ================================================================================
           0	1	10.10.100.16	6714	0.000000	0.000000	0.000000	0	0	0
           1	2	10.10.100.3	6714	0.000000	0.000000	0.000000	0	0	0
           2	3	10.10.150.2	6714	0.000000	0.000000	0.000000	0	0	0
           3	4	10.10.150.21	6714	0.000000	0.000000	0.000000	0	0	0
           4	5	10.10.150.25	6714	0.000000	0.000000	0.000000	0	0	0
           5	6	10.10.150.4	6714	0.000000	0.000000	0.000000	0	0	0
           6	0	10.10.100.15	6714	0.000000	0.000000	0.000000	0	0	0
           entering while: 0
           Capability               10.10.100.3:6714        0       47
           Store                    10.10.100.3:6714        0       47
           min_unique_depots: 12
           ALLOCATION: 0
           ALLOCATION: 0
           Capability              10.10.150.2:6714        0       47
           Capability             10.10.150.21:6714        0       47
           Copy              10.10.100.3:6714 to              10.10.150.2:6714 0 47
           Copy              10.10.100.3:6714 to             10.10.150.21:6714 0 47
           End Success
    
    
    .xnd file will be generated when you upload the file using lors_upload. This exnode file contains metadata about how your file has been distributed across available depots. If you wish to share your data with other hosts connected on IDMS data plane. Just share the .xnd file. More advanced options to explore.
   lors_upload --help
   Usage: lors_upload <filename>
     -f, --samename                 Specify the exNode filename by appending .xnd to the original filename.
     -i, --stdin                    Read from standard input rather than a specified filename.
     -o, --outputfile=STRING        Specify a specific name for the output exNode file.
     -v, --version                  Display Version information about this package.
     -V, --verbose=0                Modify the level or mode of Verbosity in output messages.
     -H, --lbone-host=STRING        Specify an L-Bone Server for resource discover and proxmity resolution.
     -P, --lbone-port=6767          Specify the port number when using an L-Bone Server on a non standard port.
     -l, --location="state= TN"     Specify a location hint to pass the L-Bone Query.
     -d, --duration=STRING          Specify the amount of time to allocate storage. The available modifiers are 'm' 'h' 'd' for 'minute' 'hour' and 'days' respectively. Default is seconds. (e.g. -d 1d or -d
                                 1h)
     -s, --soft                     Specify SOFT storage for storage allocations.
     -h, --hard                     Specify HARD storage for storage allocations.
     -m, --maxdepots=6              Specify the maximum number of depots returned by the 'location' hint.
     -b, -bs='5M'                   Specify the logical data blocksize of input file.
     -c, --copies=1                 Specify the number of copies.
     -F, --fragments=4              Rather than specifying the logical data blocksize, specify the number of blocks into which the file should be divided.
    --depot-list                   Only use depots specified in .xndrc file
     -t, --threads=INT              Specify the maximum number of threads to use to perform Upload.
     -T, --timeout='3m'             Specify the maximum time to allow for all operations.
     -M, --buffersize=STRING        Specify the maximum internal buffer to use on operations.
     -O, --offset=STRING            Specify an offset relative to the exNode from which download will begin
     -L, --length=STRING            Specify a length other than the logical extent of the exNode.
     -n, --none                     Turn off all e2e services.
     -e, --des                      Turn on des encryption.
     -a, --aes                      Turn on aes encryption.
     -z, --compress                 Turn on zlib compression.
     -x, --xor                      Turn on xor encryption.
     -k, --checksum                 Turn on checksum.
     -E, --e2e-blocksize=STRING     When specifying e2e conditioning, select an e2e bocksize which will evenly fit into your chosen Mapping Blocksize.
     -X, --xndrc=<filename>         You may specify an alternate source for the upload DepotList.
     -D, --demo                     Add the messages understood by the LoRS View visualization.
     --lodn-allocate[=STRING]       Turn on LoDN proxy allocation. The STRING value contains authentication credentials separated by an &.
     -n, --normal-allocate          Override LoDN proxy allocation if specified in the xndrc file.

   Help options:
     -?, --help                     Show this help message
     --usage                        Display brief usage message

Step 5: Download Data

To download data stashed on storage node use .xnd file generated during the upload.

   command :
        lors_download -o <file_name> <xnd_file_location>

   Sample Execution:
        [adorwat@user-2 ~]$ lors_download -o sample sample.xnd 
        TOOL Download
        EXNODEFILE sample
        TITLE Filename sample
        MAXLENGTH 47
        Load             10.10.150.21:6714        0       47
        Load              10.10.150.2:6714        0       47
        End Success

More advanced options to explore

   lors_download --help
   Usage: lors_download <filename>
     -f, --samename              Specify the exNode filename by appending .xnd to the original filename.
     -o, --outputfile=STRING     Specify a specific name for the output exNode file.
     -v, --version               Display Version information about this package.
     -V, --verbose=0             Modify the level or mode of Verbosity in output messages.
     -a, --thread-depot=-1       Specify the maxmium connection to any single IBP depot.
     -r, --redundancy=1          Specify the maxmium number of threads working on a single download job.
     -R, --resume                Download will resume working on a partially downloaded file if one is detected.
     -p, --progress=0            The progress number specifies how many completed blocks are allowed before a download of a pending block is duplicated.
     -b, -bs='5M'                Specify the logical blocksize of transfers.
     -t, --threads=INT           Specify the maximum number of threads to use to perform Download.
     -T, --timeout='3m'          Specify the maximum time to allow for all operations.
     -q, --prebuffer=INT         Specify the number of blocks to prebuffer before outputting to file.
     -C, --cache=INT             Specify the number of block-sized memory buffers to use during download.
     -M, --buffersize=STRING     Specify the maximum internal buffer size to use on operations.
     -O, --offset=LONG           Specify an offset relative to the exNode from which download will begin
     -L, --length=LONG           Specify a length other than the logical extent of the exNode.
     -D, --demo                  Add the messages understood by the LoRS View visualization.
   Help options:
     -?, --help                  Show this help message
     --usage                     Display brief usage message

Inside rspec

Customized rspec is provided to reserve resource on the Instageni aggregate manager.

  • Customized rspec will find the node with sliver type emulab-openvz which comes by default with Fedora 15 disk image.
  • Upon successful booting of image, Data plane interface with default 10.10.1.1 ip address will be connected to shared VLAN. It will also install wget to download configuration files and dhclient to dynamically configure network interfaces.
  • ip_config script will find the interface name for default ip address and configure that interface for dhcp. When interface is restarted the data plane will be connected to IDMS subnet. Now it can talk to other nodes in the subnet.
  • At last Lors rpm will be downloaded and installed.

This rspec can easily be customized for any other experiment to connect with IDMS experiment. Important step would be connecting data plane interface on shared vlan and then renewing ip through DHCP.

Contact

In case of any issue or bug. Please contact

Attachments (3)

Download all attachments as: .zip