Changes between Version 1 and Version 2 of GENICloud-Post-GEC10-status


Ignore:
Timestamp:
04/05/11 15:49:42 (14 years ago)
Author:
Vic Thomas
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GENICloud-Post-GEC10-status

    v1 v2  
    77
    88== I. Major accomplishments ==
    9 Built (with the collaboration of the iGENI project (Joe Mambretti and Tom !DeFanti) and the !GLab project (Paul Mueller) the !TransCloud, a transcontinental cloud operating at four sites over two continents, connected by 10  Gb/s connections over the    CaveWAVE, National Lambda Rail, and the Global Lambda Interchange    Facility (GLIF).  Network connectivity is due to the facilities at    !StarLight, !NetherLight, and DFM.  The !TransCloud became   operational February 15, 2011, and is open for use by GENI, FIRE,     and !GLab researchers.
     9Built (with the collaboration of the iGENI project (Joe Mambretti and Tom !DeFanti) and the GLab project (Paul Mueller) the !TransCloud, a transcontinental cloud operating at four sites over two continents, connected by 10  Gb/s connections over the    CaveWAVE, National Lambda Rail, and the Global Lambda Interchange    Facility (GLIF).  Network connectivity is due to the facilities at    !StarLight, !NetherLight, and DFM.  The !TransCloud became   operational February 15, 2011, and is open for use by GENI, FIRE,     and GLab researchers.
    1010
    1111Demonstrated !TransCloud at GEC-10, including a transcoding   service and a prototype query system across distributed clusters.
     
    2323=== A. Activities and findings ===
    2424
    25 1. Designed and implemented virtual Hadoop cluster over the   wide-area; specifically, each cluster component ran on a VM, with   VMs residing in physical hosts from HP Palo Alto (OpenCirrus),   Northwestern University and Kaiserslautern (University   collaborators).
    26 2. Built a proof of concept distributed query extension to the Pig declarative query language. Pig was developed for generating mapreduce jobs on Hadoop for the querying of data, and this extension allows for the declaration of multiple sites for the processing of data, along with a site to collate those results.
    27 3. Ran brief performance test using different VM images and found that pure CPU performance (not just Hadoop specific) for each VM is highly dependent on the particular image, even if they fit the typical partitioning of medium, large and extra-large defined by Eucalyptus.
    28 4. Developed  a sub-virtual machine isolation layer for Cloud Programming, based on Google Native Client (!NaCl) and the Seattle project's Restricted Python (!RePy)
    29 5. Brought up and installed a cluster monitoring infrastructure on each !TransCloud cluster, based on the Ganglia cluster monitoring system
    30 6. Designed developed, and demonstrated a monitoring and visualization system for the progress of Hadoop jobs
    31 7. Secured the domain name trans-cloud.net so that !TransCloud slivers will be contained in this name domain.
     25 1. Designed and implemented virtual Hadoop cluster over the   wide-area; specifically, each cluster component ran on a VM, with   VMs residing in physical hosts from HP Palo Alto (!OpenCirrus),   Northwestern University and Kaiserslautern (University   collaborators).
     26 2. Built a proof of concept distributed query extension to the Pig declarative query language. Pig was developed for generating mapreduce jobs on Hadoop for the querying of data, and this extension allows for the declaration of multiple sites for the processing of data, along with a site to collate those results.
     27 3. Ran brief performance test using different VM images and found that pure CPU performance (not just Hadoop specific) for each VM is highly dependent on the particular image, even if they fit the typical partitioning of medium, large and extra-large defined by Eucalyptus.
     28 4. Developed  a sub-virtual machine isolation layer for Cloud Programming, based on Google Native Client (!NaCl) and the Seattle project's Restricted Python (!RePy)
     29 5. Brought up and installed a cluster monitoring infrastructure on each !TransCloud cluster, based on the Ganglia cluster monitoring system
     30 6. Designed developed, and demonstrated a monitoring and visualization system for the progress of Hadoop jobs
     31 7. Secured the domain name trans-cloud.net so that !TransCloud slivers will be contained in this name domain.
    3232
    3333=== B. Project participants ===
    3434
    3535==== HP ====
    36 * Rick !McGeer[[BR]]Principal Investigator[[BR]][mailto:rick.mcgeer@hp.com]
    37 * Jessica Blaine [[BR]]Project Manager[[BR]][mailto:jessica-ann.blaine@hp.com]
    38 * Alvin !AuYoung[[BR]]Post-doctoral Fellow[[BR]][mailto:alvin.auyoung@hp.com]
     36 * Rick !McGeer[[BR]]Principal Investigator[[BR]][mailto:rick.mcgeer@hp.com]
     37 * Jessica Blaine [[BR]]Project Manager[[BR]][mailto:jessica-ann.blaine@hp.com]
     38 * Alvin !AuYoung[[BR]]Post-doctoral Fellow[[BR]][mailto:alvin.auyoung@hp.com]
    3939
    4040==== UCSD ====
    41 * Alex Snoeren[[BR]]Principal Investigator[BR][[mailto:snoeren@cs.ucsd.edu]
     41 * Alex Snoeren[[BR]]Principal Investigator[[BR]][mailto:snoeren@cs.ucsd.edu]
    4242
    4343==== Princeton ====
    44 * Andy Bavier[[BR]]Principal Investigator[[BR]][mailto:acb@cs.princeton.edu]
    45 * Mrco Yuen[[BR]][mailto:marcoy@gmail.com]
     44 * Andy Bavier[[BR]]Principal Investigator[[BR]][mailto:acb@cs.princeton.edu]
     45 * Mrco Yuen[[BR]][mailto:marcoy@gmail.com]
    4646
    4747==== University of Victoria ====
    48 * Yvonne Coady[[BR]]Professor[[BR]][mailto:ycoady@cs.uvic.ca]
    49 * Chris Matthews[[BR]][Graduate Student[[BR]][mailto:cmatthew@cs.uvic.ca]
    50 * Chris Pearson[[BR]][Graduate Student[[BR]][mailto:pearson@cs.uvic.ca
     48 * Yvonne Coady[[BR]]Professor[[BR]][mailto:ycoady@cs.uvic.ca]
     49 * Chris Matthews[[BR]]]Graduate Student[[BR]][mailto:cmatthew@cs.uvic.ca]
     50 * Chris Pearson[[BR]]Graduate Student[[BR]][mailto:pearson@cs.uvic.ca]
    5151
    5252=== C. Publications (individual and organizational) ===
    53 1. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick McGeer, "NanoXen : Better Systems Through Rigorous Containment and Active Modeling", OSDI 2010 (Poster).
    54 2. Rick McGeer,  Alvin AuYoung, Andy Bavier, Jessica Blaine,  Yvonne Coady,  Joe Mambretti,  Chris Matthews, Chris Pearson, Alex Snoeren, Marco Yuen, "TRANSCLOUD:: Design Considerations for a High-Performance Cloud Architecture Across Multiple Administrative Domains",  Proceedings CLOSER, 2011
    55 3. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick McGeer, "NanoXen : Better Systems Through Rigorous Containment and Active Modeling", Proceedings SAVCBS, 2010.
     53 1. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick !McGeer, "!NanoXen : Better Systems Through Rigorous Containment and Active Modeling", OSDI 2010 (Poster).
     54 2. Rick !McGeer,  Alvin !AuYoung, Andy Bavier, Jessica Blaine,  Yvonne Coady,  Joe Mambretti,  Chris Matthews, Chris Pearson, Alex Snoeren, Marco Yuen, "TRANSCLOUD:: Design Considerations for a High-Performance Cloud Architecture Across Multiple Administrative Domains",  Proceedings CLOSER, 2011
     55 3. Chris Matthews, Justin Cappos, Yvonne Coady, John Hartman, Jonathan Jacky and Rick !McGeer, "!NanoXen : Better Systems Through Rigorous Containment and Active Modeling", Proceedings SAVCBS, 2010.
    5656
    5757=== D. Outreach activities ===
    5858
    5959=== E. Collaborations ===
    60 1. iGeni, Prof. Joe Mambretti, Prof. Tom !DeFanti
    61 2. Prof. Michael Zink, !UMass (provided data repository for Hadoop application)
    62 3. Seattle project, Dr. Justin Cappos (collaborates on Cloud programming environments)
    63 4. !PlanetLab, Prof. Larry Peterson
     60 1. iGeni, Prof. Joe Mambretti, Prof. Tom !DeFanti
     61 2. Prof. Michael Zink, !UMass (provided data repository for Hadoop application)
     62 3. Seattle project, Dr. Justin Cappos (collaborates on Cloud programming environments)
     63 4. !PlanetLab, Prof. Larry Peterson
    6464
    6565==== Other (non-GENI) collaborators ====
    66 1. Mathematics and Information Technology Applications in Complex Systems (MITACS), Government of Canada (co-sponsor).  Duncan Phillips, collaborator contact
    67 2. G-Lab, Prof. Paul Mueller
    68 3. University of Amsterdam, Prof. Cees de Laat (provides
    69 connectivity and will join !TransCloud as a site)
    70 4. VICCI, Prof. Larry Peterson
     66 1. Mathematics and Information Technology Applications in Complex Systems (MITACS), Government of Canada (co-sponsor).  Duncan Phillips, collaborator contact
     67 2. G-Lab, Prof. Paul Mueller
     68 3. University of Amsterdam, Prof. Cees de Laat (provides connectivity and will join !TransCloud as a site)
     69 4. VICCI, Prof. Larry Peterson
    7170
    7271=== F. Other Contributions ===