Version 2 (modified by 15 years ago) (diff) | ,
---|
Project Number
1709
Project Title
Data-Intensive Cloud Control for GENI
a.k.a. DICLOUD
Technical Contacts
Michael Zink, Principal Investigator
University of Massachusetts, Amherst
zink@cs.umass.edu
http://www-net.cs.umass.edu/~zink/umasshome/pmwiki.php
Prashant Shenoy, Co-Principal Investigator
University of Massachusetts, Amherst
shenoy@cs.umass.edu
http://www.cs.umass.edu/~shenoy/
Jim Kurose, Co-Principal Investigator
University of Massachusetts, Amherst
kurose@cs.umass.edu
http://www-net.cs.umass.edu/personnel/kurose.html
David Irwin, Post-doctoral Research Associate
University of Massachusetts, Amherst
David Irwin
http://www.cs.umass.edu/~irwin/
Emmanuel Cecchet, Senior Research Fellow
University of Massachusetts, Amherst
Amherst, MA 01003-9264
http://www.cs.umass.edu/~cecchet/
Participating Organizations
GPO Liason System Engineer
Harry Mussman hmussman@geni.net
Scope
This project will develop a complete environment for researchers to conduct data-intensive experiments in GENI from start (the data collection point) to finish (processing and archiving).
To do so, this project will extend the GENI/ViSE sensor network (sensornet) testbed at UMass-Amherst and augment GENI Cluster D’s Orca control framework with capabilities for researchers to
(i) obtain data-centric slices that span core sensornet nodes, data center nodes, and, importantly, storage volumes “in the cloud,” (ii) deploy popular cloud computing programming paradigms to enable simple, but powerful, distributed data processing, and (iii) execute experiment workflows to explicitly control experiment data flow and resource allocation across a network of components/aggregates.
The project will build on existing software artifacts in the GENI “ecosystem” and tailor them to the distinct requirements of data-intensive experiments. While the enhanced software artifacts will generalize to any high-bandwidth data-intensive experiments, the GENI/ViSE sensornet testbed, which collects high-bandwidth data from multiple high-power (virtualized) sensor/actuators, will be the initial data source.
Our goal by year one is to incorporate commercial cloud computing services, including storage services, as GENI substrates available for researchers.
Our goal by year two is to enhance GENI’s usefulness by testing and hardening the capability for researchers to request (or load) distributed software platforms on commercial clouds. We will demonstrate the capability using both an MPI stack and Apache’s Hadoop framework, an open-source version of MapReduce and Google File System (GFS).
Our goal by year three is to complete the integration of Gush to discover resources and deploy experiment workflows across data-centric slices in the Orca CF.
Current Capabilities
Milestones
MilestoneDate(ORCAAUG: S2.a Rel 2.1_2 ORCA code to projects)?
MilestoneDate(ORCAAUG: S2.b Rel 2.2_2 ORCA code to projects)?
MilestoneDate(ORCAAUG: S2.c Rel 2.3_2 ORCA code to projects)?
MilestoneDate(ORCAAUG: S2.d Rel 2.x_1 ORCA code, docs to GPO)?
MilestoneDate(ORCAAUG: S2.e Single registry of brokers)?
MilestoneDate(ORCAAUG: S2.f Query broker for available resources)?
MilestoneDate(ORCAAUG: S2.g Broker authorization based on attributes from Shibboleth)?
MilestoneDate(ORCAAUG: S2.h Eucalyptus VM provisioning)?
MilestoneDate(ORCAAUG: S2.i First ontology workshop)?
MilestoneDate(ORCAAUG: S2.j First draft of the ontology schema)?
MilestoneDate(ORCAAUG: S2.k Second ontology workshop)?
Project Technical Documents
Quarterly Status Reports
Spiral 2 Connectivity
Related Projects
Attachments (25)
- 2009-12-31 GENI DiCloud q1 report final.pdf (36.6 KB) - added by 15 years ago.
-
orca_amazon_handlers.tgz (37.7 KB) - added by 15 years ago.
Orca Amazon Cloud Handlers
-
2009-01-29 S2c Orca-Amazon Cloud handlers.pdf (64.1 KB) - added by 15 years ago.
Orca Amazon Handler Guide
-
2009-02-10 S2d Broker policy.pdf (72.9 KB) - added by 15 years ago.
Broker Policy for Monitoring Cloud Usage
-
GEC7 demo.pdf (143.6 KB) - added by 14 years ago.
GEC7 demo description (missing the EC2 resource bridged through ViSE)
- 2010-03-31 GENI DiCloud q2 report.pdf (41.4 KB) - added by 14 years ago.
-
2009-05-15 S2.f CloudWatch monitoring.pdf (864.9 KB) - added by 14 years ago.
Deliverable S2.f
- 2010-06-31 GENI DiCloud q3 report.pdf (32.1 KB) - added by 14 years ago.
-
2010-07-20 GEC8 Cluster D Session - DiCloud.pdf (280.8 KB) - added by 14 years ago.
Slides presented at the Cluster D session during GEC8
-
2010-07-20 GEC8 DiCloud demo.pdf (647.4 KB) - added by 14 years ago.
Slides of the DiCloud demo presented at GEC8
-
2010-08-06 DiCloud - Spiral 2 Project Review v2.pptx (146.8 KB) - added by 14 years ago.
DiCloud - Spiral 2 review slides
-
dicloud-src-dist.tar.gz (27.2 KB) - added by 14 years ago.
Updated handlers for Orca bella and aggregate manager demo-ed at GEC8 (source code)
-
dicloud-dist.tar.gz (3.5 MB) - added by 14 years ago.
Updated handlers for Orca bella and aggregate manager demo-ed at GEC8 (binary distribution)
-
gec8_demo_scripts.tgz (5.1 KB) - added by 14 years ago.
Scripts used for the GEC8 demo
-
2010-11-01 dicloud-web-portal.tar.gz (4.0 MB) - added by 14 years ago.
DiCloud Web portal source code
-
2010-11-01 dicloud-src-dist.tar.gz (45.6 KB) - added by 14 years ago.
DiCloud software source code (Orca handlers, console and server)
-
2010-11-01 dicloud-dist.tar.gz (3.7 MB) - added by 14 years ago.
DiCloud software binary distribution (Orca handlers, console and server)
-
DiCloudUserguide.pdf (757.9 KB) - added by 14 years ago.
DiCloud User Guide (Oct 26, 2010 version)
-
2010-10 DiCloud Software architecure.pdf (508.9 KB) - added by 14 years ago.
DiCloud software architecture and administration document
-
2010-11-05 GENI DiCloud GEC9 report.pdf (37.9 KB) - added by 14 years ago.
DiCloud progress report (GEC 9)
-
2011-03-29 GENI DiCloud GEC10 report.pdf (16.0 KB) - added by 13 years ago.
Quarterly progress report
- nowcasting gec9 demos.pptx (2.0 MB) - added by 13 years ago.
-
2011-07-29 GENI DiCloud GEC11 report.pdf (25.7 KB) - added by 13 years ago.
DiCloud report post GEC 11
-
2011-11-01 GENI DiCloud GEC12 report.doc (49.0 KB) - added by 13 years ago.
Post-GEC12 Report
-
GEC13-report.docx (130.7 KB) - added by 12 years ago.
Post-GEC13 DiCloud Report