wiki:sol4/IDMS

Version 4 (modified by ezkissel@indiana.edu, 6 years ago) (diff)

--

Project Number

1956

Project Title

GENI Shakedown Expimeriments -- Intelligent Data Movement Service (IDMS)

Technical Contacts

PI Ezra Kissel, Indiana University (mailto:ezkissel@indiana.edu)

Participating Organizations

Indiana University

Related Projects

GEMINI I&M

GPO Engineers

Project Manager: Niky Riga

Scope

This project will develop a dynamic storage and networking using data logistics that will be deployed and evaluated as experiments on the GENI infrastructure. The functionality of the system will resemble that of existing cloud-based storage and content distribution networks; however, the dynamic and unique nature of GENI resources will allow for the development and evaluation of new approaches to adaptive data dissemination. The work to be carried out has the following major objectives:

  • Develop and deploy an experimental data movement service that will make use of scalable and configurable GENI resources across control frameworks.
  • Leverage existing and proven technologies to form a packaged system that may be run as repeatable and dynamic GENI experiments with variable resource constraints.
  • Take advantage of GENI layer-2 networking infrastructure, including SDN capabilities that allow for optimizing data movement performance.
  • Develop network “appliances” that can automatically configure themselves and scale across available resources according to demand.
  • Instantiate and evaluate a GENI experiment that runs as a long-lived service with the potential for opt-in users.

Taken together, this project aims to investigate an intelligent data movement and storage (IDMS) service within GENI. At a fundamental level, an IDMS service provides storage to users via networked computer systems, similar to many distributed storage solutions such as Dropbox and Box. Beyond basic storage capacity, our IDMS system will include mechanisms for optimizing data distribution throughout the network while also taking into account the desired locality of user data. A key goal is to create a set of repeatable and customizable experiments that will allow us to evaluate the extent of scalability, optimization, and usability achievable with IDMS. The IDMS system will make use of a number of existing components including the Internet Backplane Protocol (IBP) depot software, eXtensible Session Protocol (XSP), Phoebus, and the Unified Network Information Service (UNIS).

Strategy and Goals

This project will use existing software, including the IBP Depot implementation, XSP, and Phoebus. None of this software is currently under any license, and only the IBP Depot is from a non-IU institution. In particular, the IBP Depot release author commented that their software would fall under the GPL or BSD license for any future releases. We (Indiana) also have plans to have XSP and Phoebus releases fall under the GPL or Apache license, alleviating any concerns for wider distribution and use. The source, and any binary packages, tarballs, etc., will be made publicly available on this project page.

As noted in the milestones below, we expect to create “network appliances” that run one or more of our software packages as small-footprint disk images that can be loaded onto GENI node, or “PC”, resources. At a minimum, we expect these to run on GENI Racks, both in ExoGENI and InstaGENI aggregates. The list of targeted aggregates may be expanded given available resources and node types.

The availability of these appliance images will be made public on the project wiki page. The expected release date for the initial set of images usable by others in GENI is expected by GEC20 (July 2014), including supporting documentation. As project gets underway, specific release dates will be posted here.

Operational Capabilities

Milestones

GEC18 Milestones
GEC19 Milestones
GEC20 Milestones

Quarterly Status Reports

Resources

Attachments (3)

Download all attachments as: .zip