| | 1 | |
| | 2 | [[PageOutline]] |
| | 3 | |
| | 4 | = GENICloud Project Status Report = |
| | 5 | |
| | 6 | Period: November 18, 2010 - March 18, 2011 |
| | 7 | |
| | 8 | == I. Major accomplishments == |
| | 9 | Built (with the collaboration of the iGENI project (Joe Mambretti and Tom !DeFanti) and the !GLab project (Paul Mueller) the !TransCloud, a transcontinental cloud operating at four sites over two continents, connected by 10 Gb/s connections over the CaveWAVE, National Lambda Rail, and the Global Lambda Interchange Facility (GLIF). Network connectivity is due to the facilities at !StarLight, !NetherLight, and DFM. The !TransCloud became operational February 15, 2011, and is open for use by GENI, FIRE, and !GLab researchers. |
| | 10 | |
| | 11 | Demonstrated !TransCloud at GEC-10, including a transcoding service and a prototype query system across distributed clusters. |
| | 12 | |
| | 13 | === A. Milestones achieved === |
| | 14 | |
| | 15 | Deployed and operated an SFA-compliant multi-site cloud infrastructure |
| | 16 | |
| | 17 | === B. Deliverables made === |
| | 18 | |
| | 19 | Developed SFA-comnpliant Eucalyptus-based cloud manager, and added support in Eucaltyptus for SFA 2.0 |
| | 20 | |
| | 21 | == II. Description of work performed during last quarter == |
| | 22 | |
| | 23 | === A. Activities and findings === |
| | 24 | |
| | 25 | 1. Designed and implemented virtual Hadoop cluster over the wide-area; specifically, each cluster component ran on a VM, with VMs residing in physical hosts from HP Palo Alto (OpenCirrus), Northwestern University and Kaiserslautern (University collaborators). |
| | 26 | 2. Built a proof of concept distributed query extension to the Pig declarative query language. Pig was developed for generating mapreduce jobs on Hadoop for the querying of data, and this extension allows for the declaration of multiple sites for the processing of data, along with a site to collate those results. |
| | 27 | 3. Ran brief performance test using different VM images and found that pure CPU performance (not just Hadoop specific) for each VM is highly dependent on the particular image, even if they fit the typical partitioning of medium, large and extra-large defined by Eucalyptus. |
| | 28 | 4. Developed a sub-virtual machine isolation layer for Cloud Programming, based on Google Native Client (!NaCl) and the Seattle project's Restricted Python (!RePy) |
| | 29 | 5. Brought up and installed a cluster monitoring infrastructure on each !TransCloud cluster, based on the Ganglia cluster monitoring system |
| | 30 | 6. Designed developed, and demonstrated a monitoring and visualization system for the progress of Hadoop jobs |
| | 31 | 7. Secured the domain name trans-cloud.net so that !TransCloud slivers will be contained in this name domain. |
| | 32 | |
| | 33 | === B. Project participants === |
| | 34 | |
| | 35 | ==== HP ==== |
| | 36 | * Rick !McGeer[[BR]]Principal Investigator[[BR]][mailto:rick.mcgeer@hp.com] |
| | 37 | * Jessica Blaine [[BR]]Project Manager[[BR]][mailto:jessica-ann.blaine@hp.com] |
| | 38 | * Alvin !AuYoung[[BR]]Post-doctoral Fellow[[BR]][mailto:alvin.auyoung@hp.com] |
| | 39 | |
| | 40 | ==== UCSD ==== |
| | 41 | * Alex Snoeren[[BR]]Principal Investigator[BR][[mailto:snoeren@cs.ucsd.edu] |
| | 42 | |
| | 43 | ==== Princeton ==== |
| | 44 | * Andy Bavier[[BR]]Principal Investigator[[BR]][mailto:acb@cs.princeton.edu] |
| | 45 | * Mrco Yuen[[BR]][mailto:marcoy@gmail.com] |
| | 46 | |
| | 47 | ==== University of Victoria ==== |
| | 48 | * Yvonne Coady[[BR]]Professor[[BR]][mailto:ycoady@cs.uvic.ca] |
| | 49 | * Chris Matthews[[BR]][Graduate Student[[BR]][mailto:cmatthew@cs.uvic.ca] |
| | 50 | * Chris Pearson[[BR]][Graduate Student[[BR]][mailto:pearson@cs.uvic.ca |
| | 51 | |
| | 52 | === C. Publications (individual and organizational) === |
| | 53 | 1. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick McGeer, "NanoXen : Better Systems Through Rigorous Containment and Active Modeling", OSDI 2010 (Poster). |
| | 54 | 2. Rick McGeer, Alvin AuYoung, Andy Bavier, Jessica Blaine, Yvonne Coady, Joe Mambretti, Chris Matthews, Chris Pearson, Alex Snoeren, Marco Yuen, "TRANSCLOUD:: Design Considerations for a High-Performance Cloud Architecture Across Multiple Administrative Domains", Proceedings CLOSER, 2011 |
| | 55 | 3. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick McGeer, "NanoXen : Better Systems Through Rigorous Containment and Active Modeling", Proceedings SAVCBS, 2010. |
| | 56 | |
| | 57 | === D. Outreach activities === |
| | 58 | |
| | 59 | === E. Collaborations === |
| | 60 | 1. iGeni, Prof. Joe Mambretti, Prof. Tom !DeFanti |
| | 61 | 2. Prof. Michael Zink, !UMass (provided data repository for Hadoop application) |
| | 62 | 3. Seattle project, Dr. Justin Cappos (collaborates on Cloud programming environments) |
| | 63 | 4. !PlanetLab, Prof. Larry Peterson |
| | 64 | |
| | 65 | ==== Other (non-GENI) collaborators ==== |
| | 66 | 1. Mathematics and Information Technology Applications in Complex Systems (MITACS), Government of Canada (co-sponsor). Duncan Phillips, collaborator contact |
| | 67 | 2. G-Lab, Prof. Paul Mueller |
| | 68 | 3. University of Amsterdam, Prof. Cees de Laat (provides |
| | 69 | connectivity and will join !TransCloud as a site) |
| | 70 | 4. VICCI, Prof. Larry Peterson |
| | 71 | |
| | 72 | === F. Other Contributions === |
