wiki:GIMIv3/Documentation

Version 10 (modified by shuang@renci.org, 5 years ago) (diff)

--

GIMI Documentation

Services

For each service we need documentation for installation and administration.

Labwiki

[Jack]

Job Service

[Thierry]

OML server

[Olivier]

AMQP

[Thierry]

GIMI Experiment Service

[Jack or Thierry?]

iRODS

iRODS, which stands for integrated Rule-Oriented Digital System, is an open-source data management software in use at research organizations and government agencies worldwide. iRODS acts as middleware sitting on top of the distributed storage systems and underneath the user applications that require archiving, processing and disseminating data in a transparent and automated fashion. For more detailed and comprehensive descriptions, please refer to http://irods.org/.

In GIMI, we use iRODS as the back-end system to provide archive and automated processing functions to experimenters. We configured two iRODS instances, one as the production data system, the other as the environment for development purposes. The production system has the iCAT running on geni-gimi.renci.org, and two data servers running on geni-gimi.renci.org and emmy9.casa.umass.edu. geni-gimi.renci.org is currently allocated with ~2TB disk space, which can be further increased depending on its usage. To interact with iRODS, various user interfaces are provided. For example, iCommand (https://wiki.irods.org/index.php/icommands) provides a UNIX-like command line interface to users who are already familiar with UNIX commands (such as ls, cp, rm, etc). iRODS also provides a rich set of APIs. For example, Jargon, a JAVA client API for iRODS, provides the REST functions so that GENI experimenters can easily create their iRODS accounts via the GENI portal. iRODS is equipped with a powerful rule language and rule engine. We developed iRODS policies and rules that recognize metadata files created based on specific schemas and automate data ingestion to help experimenters more efficiently organize and share their data.

Currently our iRODS instances are running release 3.3.

Images

[Divya]

What is installed in a GIMI image?

Postboot scripts

[Divya]

What is done in the postboot script? What may need to change over time?

Tool Usage

[Divya/Jeanne]

  • Links to tutorials
  • Links to instructions on how to write OEDL scripts
  • Links to instructions on how to add OML capability to applications