| Version 26 (modified by , 10 years ago) (diff) |
|---|
IDMS Experiment in GENI
The persistent IDMS experiment allows other users to attach at shared VLAN sites, granting access to the available storage resources. Users may then upload and download their experiment data using a set of client tools. IDMS provides:
- A custom RSpec that configures a node to access the IDMS experiment
- Nodes use DHCP to lease an address within the IDMS data plane subnet
- Automatically installed client tools to upload/download data to available storage nodes
Base Experiment Topology
Shared VLANs at UCLA, Missouri, Nysernet, GPO and large storage at RENCI, UCD, and TAMU
- Stitching between the shared VLAN sites to create a cross-country data plane
- Phoebus Gateways on ProtoGENI backbone nodes for external access to the experiment and to improve WAN transfer performance
- DHCP server at nysernet-ig
IDMS ip subnet
IDMS experiment is implemented on 10.10.0.0/16 ip subnet. IP range is fixed for each part of experiment.
| Base experiment nodes | 10.10.100.{0-255} |
| IBP resource allocated dynamically | 10.10.{50-99}.0 |
| External user host | 10.10.{150-250}.{2-0} |
| DHCP server | 10.10.150.1 |
Prerequisite
In order to attach external user to base experiment. You need to reserve resources. You can use either Flack or Omni to do that. If you haven't used any of these tools before, please go through the introductory tutorials first.
- Flack tutorial. Flack is graphical web tool.
- Omni tutorial. Omni is a command line tool.
Step 1: Create Slice
Create the slice in the project. Name the slice something like xxxxIdmsUser.
omni createslice <slice_name> -r <project_name>
Step 2: Reserve Resources
- Download the rspec . For more information on rspec refer Inside rspec
- Base experiment can only be joined at shared VLAN nodes. Currently shared VLAN nodes are at
- ucla-ig
- missouri-ig
- nysernet-ig
- gpo-ig
- Select aggregate manager from list above.
omni createsliver -a <aggregate_manager> <slice_name> <path_to_rspec_file> -r <project_name>
- Poll the resource status to make sure overall 'geni_status' is 'ready'
omni sliverstatus -a <aggregate_manager> <slice_name> -r <project_name>
- If 'geni_status' is not 'ready' try using another aggregate manager
Step 3: Login to Reserved Host
- Login to reserved host. If you are unaware of this, please follow instruction here
- Test configuration : DHCP server allocates ip to each host connected to base experiment. To ensure your host is connected to IDMS base experiment data plane, ping the DHCP server.
ping 10.10.150.0
lbone server advertises list of ip addresses of free IBP server which in turns Lors tool use to stash data. In case of IDMS experiment lbone server will avertise list of ip addresses of IBP server in IDMS subnet. To test connectivity with lbone server, telnet to lbone server on 6767 port. If communication is successful, you will get telnet prompt. To escape the prompt just hit Entertelnet monitor.incntre.iu.edu 6767
You can also ensure connectivity to each IBP server in subnet by pinging the ip addressping <IBP_server_IP_address>
- To download / upload data Lors tool is already installed. Run the lors_setup.sh script to setup the ~/.xndrc config file for the LoRS
Command Line Tools.
lors_setup.sh
Step 4: Upload Data
- To upload data use lors_upload command. By default depot store file for 1 day. If you want to change the allocation time use '-d' option
Command : lors_upload –H monitor.incntre.iu.edu -f <file_to_be_uploaded> <options> Sample Execution: [adorwat@user-2 ~]$ lors_upload -V 1 -H monitor.incntre.iu.edu -f sample TOOL Upload SIZE 47 TITLE Filename sample EXNODEFILE sample.xnd No ID Depot Port Score Bandwidth Prox NFailure NThread NThreadDone ================================================================================ 0 1 10.10.100.16 6714 0.000000 0.000000 0.000000 0 0 0 1 2 10.10.100.3 6714 0.000000 0.000000 0.000000 0 0 0 2 3 10.10.150.2 6714 0.000000 0.000000 0.000000 0 0 0 3 4 10.10.150.21 6714 0.000000 0.000000 0.000000 0 0 0 4 5 10.10.150.25 6714 0.000000 0.000000 0.000000 0 0 0 5 6 10.10.150.4 6714 0.000000 0.000000 0.000000 0 0 0 6 0 10.10.100.15 6714 0.000000 0.000000 0.000000 0 0 0 entering while: 0 Capability 10.10.100.3:6714 0 47 Store 10.10.100.3:6714 0 47 min_unique_depots: 12 ALLOCATION: 0 ALLOCATION: 0 Capability 10.10.150.2:6714 0 47 Capability 10.10.150.21:6714 0 47 Copy 10.10.100.3:6714 to 10.10.150.2:6714 0 47 Copy 10.10.100.3:6714 to 10.10.150.21:6714 0 47 End Success.xnd file will be generated when you upload the file using lors_upload. This exnode file contains metadata about how your file has been distributed across available depots. If you wish to share your data with other hosts connected on IDMS data plane. Just share the .xnd file. More advanced options to explore.
lors_upload --help
Usage: lors_upload <filename>
-f, --samename Specify the exNode filename by appending .xnd to the original filename.
-i, --stdin Read from standard input rather than a specified filename.
-o, --outputfile=STRING Specify a specific name for the output exNode file.
-v, --version Display Version information about this package.
-V, --verbose=0 Modify the level or mode of Verbosity in output messages.
-H, --lbone-host=STRING Specify an L-Bone Server for resource discover and proxmity resolution.
-P, --lbone-port=6767 Specify the port number when using an L-Bone Server on a non standard port.
-l, --location="state= TN" Specify a location hint to pass the L-Bone Query.
-d, --duration=STRING Specify the amount of time to allocate storage. The available modifiers are 'm' 'h' 'd' for 'minute' 'hour' and 'days' respectively. Default is seconds. (e.g. -d 1d or -d
1h)
-s, --soft Specify SOFT storage for storage allocations.
-h, --hard Specify HARD storage for storage allocations.
-m, --maxdepots=6 Specify the maximum number of depots returned by the 'location' hint.
-b, -bs='5M' Specify the logical data blocksize of input file.
-c, --copies=1 Specify the number of copies.
-F, --fragments=4 Rather than specifying the logical data blocksize, specify the number of blocks into which the file should be divided.
--depot-list Only use depots specified in .xndrc file
-t, --threads=INT Specify the maximum number of threads to use to perform Upload.
-T, --timeout='3m' Specify the maximum time to allow for all operations.
-M, --buffersize=STRING Specify the maximum internal buffer to use on operations.
-O, --offset=STRING Specify an offset relative to the exNode from which download will begin
-L, --length=STRING Specify a length other than the logical extent of the exNode.
-n, --none Turn off all e2e services.
-e, --des Turn on des encryption.
-a, --aes Turn on aes encryption.
-z, --compress Turn on zlib compression.
-x, --xor Turn on xor encryption.
-k, --checksum Turn on checksum.
-E, --e2e-blocksize=STRING When specifying e2e conditioning, select an e2e bocksize which will evenly fit into your chosen Mapping Blocksize.
-X, --xndrc=<filename> You may specify an alternate source for the upload DepotList.
-D, --demo Add the messages understood by the LoRS View visualization.
--lodn-allocate[=STRING] Turn on LoDN proxy allocation. The STRING value contains authentication credentials separated by an &.
-n, --normal-allocate Override LoDN proxy allocation if specified in the xndrc file.
Help options:
-?, --help Show this help message
--usage Display brief usage message
Step 5: Download Data
To download data stashed on storage node use .xnd file generated during the upload.
command :
lors_download -f <xnd_file_location> -o <file_name>
Sample Execution:
[adorwat@user-2 ~]$ lors_download -f sample.xnd -o sample
TOOL Download
EXNODEFILE sample
TITLE Filename sample
MAXLENGTH 47
Load 10.10.150.21:6714 0 47
Load 10.10.150.2:6714 0 47
End Success
More advanced options to explore
lors_download --help
Usage: lors_download <filename>
-f, --samename Specify the exNode filename by appending .xnd to the original filename.
-o, --outputfile=STRING Specify a specific name for the output exNode file.
-v, --version Display Version information about this package.
-V, --verbose=0 Modify the level or mode of Verbosity in output messages.
-a, --thread-depot=-1 Specify the maxmium connection to any single IBP depot.
-r, --redundancy=1 Specify the maxmium number of threads working on a single download job.
-R, --resume Download will resume working on a partially downloaded file if one is detected.
-p, --progress=0 The progress number specifies how many completed blocks are allowed before a download of a pending block is duplicated.
-b, -bs='5M' Specify the logical blocksize of transfers.
-t, --threads=INT Specify the maximum number of threads to use to perform Download.
-T, --timeout='3m' Specify the maximum time to allow for all operations.
-q, --prebuffer=INT Specify the number of blocks to prebuffer before outputting to file.
-C, --cache=INT Specify the number of block-sized memory buffers to use during download.
-M, --buffersize=STRING Specify the maximum internal buffer size to use on operations.
-O, --offset=LONG Specify an offset relative to the exNode from which download will begin
-L, --length=LONG Specify a length other than the logical extent of the exNode.
-D, --demo Add the messages understood by the LoRS View visualization.
Help options:
-?, --help Show this help message
--usage Display brief usage message
Inside rspec
Customized rspec is provided to reserve resource on the Instageni aggregate manager.
- Customized rspec will find the node with sliver type emulab-openvz which comes by default with Fedora 15 disk image.
- Upon successful booting of image, Data plane interface with default 10.10.1.1 ip address will be connected to shared VLAN. It will also install wget to download configuration files and dhclient to dynamically configure network interfaces.
- ip_config script will find the interface name for default ip address and configure that interface for dhcp. When interface is restarted the data plane will be connected to IDMS subnet. Now it can talk to other nodes in the subnet.
- At last Lors rpm will be downloaded and installed.
This rspec can easily be customized for any other experiment to connect with IDMS experiment. Important step would be connecting data plane interface on shared vlan and then renewing ip through DHCP.
Contact
In case of any issue or bug. Please contact
- Akshay Dorwat <adorwat@indiana.edu>
- Ezra Kissel <ezkissel@indiana.edu>
- Martin Swany <swany@iu.edu>
Attachments (3)
- idms_gec201.jpg (40.5 KB) - added by 10 years ago.
- ig_idms_user.xml (1.8 KB) - added by 10 years ago.
- idms_gec22.png (213.6 KB) - added by 10 years ago.
Download all attachments as: .zip
