New requirements for iRODS and git repositories

[johren@bbn.com]

Currently, the Labwiki scripts are being stored and accessed in git repositories on emmy9. This was done for at least two reasons:

1. The ODBC issues in iRODS (now resolved)
2. Performance of listing scripts stored in iRODS

One of the problems with using the git repositories on emmy9 is that users do not have access to their scripts other than through Labwiki. If they want to save them off, they have to cut and paste them from Labwiki.

One of the advantages of using git is that the scripts can be versioned. However, this is not a real benefit current because the users do not have direct access to the git repository.

A few options were discussed at the 4/15 meeting. This ticket will be used to track the discussions of these requirements to solve these problems.

Option 1: possibly rsync git repository on emmy9 to iRODS

• this will not really give versioning benefit

Option 2: dump script and config parameters to iRODS at the same time as dumping the data

• this keeps the scripts archived with the data
• still does not provide versioning

Jack agreed to take a look at effort required for Option #2.

[johren@bbn.com]

Did some reconfiguration of Labwiki that may now allow specifying repository configuration per user. This requires per-user state to be maintained.

Config parameters are saved in the database that is already archived. Does not include the script itself. Issue with recursive scripts.

[johren@bbn.com]

Max suggests having the EC do the push of script to OML so it will be pushed into iRODS with the dump (hook script). We would have to extend the hook script to pull it out of the DB and push separately into iRODS.

Jack will work on this.

[jack.hong@nicta.com.au]

EC as in version 6.1.2.pre.4 now archive OEDL content to OML directly.

loadOEDL DSL method will archive relative scripts to OML as well.

if loadOEDL DSL is used to load a file from OmfEC gem, it won't archive such file, because 1: it is part of the library; 2: library version is archived in OML db.

[johren@bbn.com]

scripts will now be in the database encoded BASE64. gimi4 needs to be updated to EC version 6.1.2.pre.4 gem file needs to be changed to point to new version. Jack will send instructions to Divya. Test this week.

[johren@bbn.com]

Divya will sync up with Jack and update the ticket on 6/10.

[dbhat@bbn.com]

OEDL scripts stored in experiment database in a table called "omf_ec_meta_data"

OEDL script is compressed using zlib and encoded using base64 before being stored in table.