Changes between Version 1 and Version 2 of GIMI

03/01/12 12:20:57 (7 years ago)



  • GIMI

    v1 v2  
    33== Project Number ==
    5 1663
    77== Project Title ==
    9 GENI Registry Services[[BR]]
    10 a.k.a Digital Object Registry, DIGOBREG
     9GIMI: Large-scale GENI Instrumentation and Measurement Infrastructure [[BR]]
     10a.k.a GIMI
    1212=== Technical Contacts ===
    14 '''PI''' Larry Lannom ( / 703 620 8990) [[BR]]
    15 Giridhar Manepalli ( / 703 620 8990) [[BR]]
    16 Jim French ( / 703 620 8990) [[BR]]
    17 Christophe Blanchi ( / 703 620 8990)
     14'''PI''' Michael Zink,  University of Massachusetts at Amherst  (  [[BR]]
     15'''Co-PI'''  Max Ott, NICTA  ( [[BR]]
     16'''Key personnel''' Ilia Baldine, RENCI  ( [[BR]]
    1920=== Participating Organizations ===
    21 Corporation for National Research Initiatives (CNRI)[[BR]]
    22 1895 Preston White Drive, Suite 100[[BR]]
    23 Reston, VA 20191 - 5434
     22University of Massachusetts at Amherst [[BR]]
     24NICTA [[BR]]
     26RENCI [[BR]]
    2530=== Related Projects ===
     32ExoGENI [[BR]] 
     34WiMAX at UMass  [[BR]]
     36DICLOUD  [[BR]]
    2739=== GPO Liaison System Engineer ===
    3345== Scope ==
    35 The scope of work on this project is to adapt the Handle System and/or
    36 a Digital Object Registry to realize a clearinghouse registry for
    37 principals, slices, and/or components in a selected GENI Spiral 1 control
    38 framework  (possibly using XML RPC), and capable of
    39 supporting limited operations in Year 1.  Also, to analyze how the
    40 Handle System and/or a Digital Object Registry could be used to
    41 identify and register GENI software, including experimenter’s tools,
    42 test images and configurations, and test results.  And finally, to
    43 define the operational, scaling, security, and management
    44 requirements, plus recommended design approaches, for implementing
    45 GENI clearinghouse and software registry services.
     47This project will develop and deploy the GIMI instrumentation and measurement framework, capable of supporting the needs of both GENI experimenters and GENI infrastructure operators.  It uses the ORBIT Measurement Library (OML) and integrated Rule Oriented Data System (iRODS) as its basis.  It will provide libraries to instrument resources, to filter and process measurement flows, and to consume measurement flows.  It will use the iRODS data grid for archiving and further processing.  It will include access control based on accepted GENI policy and authorization mechanisms. [[BR]]
     49== Strategy and Goals ==
     51Its first goal is to provide easy-to-use I&M services for experimenters, who are deploying slices in GENI on selected types of servers, VMs and racks, interconnected by various types of network paths.  [[BR]]
     53Its second goal is to provide comprehensive infrastructure measurement services for infrastructure operators, who are deploying measurement slices on selected types of servers, VMs and racks, interconnected by a various types of network paths.  In addition, slices established by other infrastructure operators, or by experimenters, can be authorized to gather data from the measurement slices. [[BR]]
     55This project will build and operate two persistent services:   the GENI Measurement Data Archive service (i.e.,  iRODs)and the GENI Experimenter Portal Service.  These can be utilized by all types of GENI I&M services, including those developed in the GEMINI project.  The GENI Experimenter Portal Service will be developed in cooperation with the GEMINI project, to support both GIMI and GEMINI tools.
     56This project will use the GENI Measurement Data Object Descriptor (MDOD) schema, defined for use by all GENI I&M services. [[BR]]
     58It may utilize two persistent services provided by the GEMINI project:  the GENI Global I&M Registry (i.e., UNIS), and the GENI Event Messaging Service.   [[BR]]
     60The GIMI project will use the configuration shown in the diagram below. [[BR]]
     62GIMI I&M Tools Configuration (p6) [[BR]]
    4766== Operational Capabilities ==
    50 GENI Federated Clearinghouse Prototype[[BR]]
    52 [  Link to GENI Federated Clearinghouse]  [[BR]]
    53 [attachment:"GENI Federated Clearing House Performance report.pdf" Evaluation of the GENI Federated Clearinghouse] [[BR]]
    5770=== Milestones ===
     73a)      Define MeasurementDataObjectDescriptor (MDOD) schema, plus naming of objects and object registry, in collaboration with I&M teams and the GPO.   (March 2012, GEC13) [[BR]]
     76b)      Define v0.5 I&M use cases for experiments and infrastructure monitoring, in collaboration with I&M teams, the GPO, experimenters and operators/GMOC.  (February 2012) [[BR]]
     79c)      Develop reference v1.0 experiment control tools based on OMF and Gush (including OMNI) and document the interfaces, protocols, and APIs of these tools, in collaboration with I&M teams and the GPO.  (March 2012, GEC13) [[BR]]
     82d)      Description of ontology extensions to cover measurement point resources (software and hardware). This will be integrated into ORCA per the ORCA-AUG SOW.   (February 2012) [[BR]]
     85e)      Identify use cases and concept of operations for GENI messaging services, in cooperation with the IMF and GEMINI projects.  (March 2012, GEC13) [[BR]]
     88f)      Deliver  v0.5 definition of GIMI architecture and software modules for year 1, documented on GIMI web page.  (February 2012) [[BR]]
     91g)      OML Release 2.7.0:  (February 2012) [[BR]]
     931.      Fixing of current bugs affecting client apps or user's interpretation of data [[BR]]
     952.      Fixing of current deployment and user development issues  [[BR]]
     973.      New features to improve measurement reporting and future analysis  [[BR]]
     994.      New features to ease current usage and enable future features [[BR]]
     1015.      Improved documentation [[BR]]
     104h)      Agree on v1.0 I&M use cases for experiments and infrastructure monitoring, in collaboration with I&M teams and the GPO, and document, including:  (March 2012, GEC13) [[BR]]
     1061.      WiMax [[BR]]
     1082.      LEARN/BEN layer 0,1 or 2 real-time measurements [[BR]]
     1103.      Sensor network (radars, power monitors) [[BR]]
     1124.      Measure DiCloud workflow [[BR]]
     115i)      Compare available technology and prototypes for GENI messaging services, in cooperation with the IMF and GEMINI projects.  (March 2012, GEC13) [[BR]]
     1171.      Decide whether GENI Event Messaging Service being developed for GEMINI project could be used by GIMI tools. [[BR]]
     120j)      Agree on v1.0 definition of GIMI architecture and software modules for year 1 with the GPO, and document on GIMI web page.   (March 2012, GEC13) [[BR]]
     124k)      Identify necessary mappings between iRods metadata catalog information and MDOD.  (March 2012, GEC13) [[BR]]
     1261.      MDOD will be created and edited as part of measurement orchestration, to be completed in Year 2 (Spiral 5). [[BR]]
     130l)      Agree on v1.0 target GENI environment for GIMI tools with I&M teams and the GPO  (March 2012, GEC13) [[BR]]
     1321.      Nominal v1.0 target aggregate is ORCA and ExoGENI racks;  both servers and VMs;  when VMs have private IP addresses, make use of RENCI’s proxy mechanism, which basically works like an automated NAT. [[BR]]
     1342.      GIMI tools will come with an image;  create template images with a basic configuration of tools which can be used and modified by other experimenters. [[BR]]
     1363.      GIMI tools will be configured via OMF. [[BR]]
     139m)      Deliver  Beta 1.0 of GIMI software  (June 2012) [[BR]]
     1411.      Utilize XML messaging service based on XMPP [[BR]]
     1432.      Utilize OML servers that are part of the software release, and are run as part of the slice. [[BR]]
     1453.      Visualization is provided by a standalone tool, that is run as part of the slice.  [[BR]]
     1474.      Demonstrate GIMI working on target aggregates, including  ORCA and ExoGENI racks (if available), both servers and VMs.
     1485.      Demonstrate some WiMAX measurement capabilities.
     1496.      Demonstrate Layer 0,1 or 2 measurement capabilities  (potentially a combined LEARN/CASA measurement depending on radar deployment in DFW)
     1507.      If installed, demonstrate initial capabilities to monitor racks.
     152n)      Develop use cases and best practices for Experimenter Portal Service, and document (based on LabWiki, Max’s portal) (March 2012, GEC13) 
     1531.      In cooperation with the GEMINI project. 
     1542.      Goal is to have one portal service support both GIMI and GEMINI tools.
     157o)      Stand up a v1.0 prototype iRods datagrid.  (March 2012, GEC13)
     1581.      Ideally across at least two GIMI institutions
     1592.      Identify/document the critical iRods microservices needed to support GENI.
     1603.      Identify/document mechanisms to map GENI authorization onto iRods datagrid.
     1614.      Share with GEMINI project
     163p)      Deliver Release 1.0 of GIMI software after functional and performance testing.  (July 2012)
     1641.      Document on GIMI web page
     1652.      Provide support to early users
     167q)      OML Release 2.8.0:  (June 2012)
     1681.      Fixing of new bugs from previous release 2.7.0
     1692.      Fixing of user development issues
     1703.      New features to improve data analysis
     1714.      New features to improve filtering capabilities and re-design of filter handling system
     1725.      New binding for additional languages (Python)
     1736.      New OML application to collect measurements from SNMP agent (NICTA);  should provide measurements equivalent to INSTOOLS     
     1747.      Improved documentation
     176r)      Provide v1.0 persistent Experimenter Portal Service (based on NICTA LabWiki) that can be used by GENI experimenters. (June 2012)
     1771.      Allows users to keep a persistent log of their experiments
     1782.      Includes OML server
     1793.      May push measurement data to the iRODS archive
     1804.      May retrieve measurement data stored in iRODs archive, and visualize it.  (requires sql query support in iRODS, where an sql database can be “actively” hosted in iRODS, and someone can run queries against it.
     1815.      Provision of a set of visualization tools to graphically present measurement data
     1826.      Share with GEMINI project
     184s)      Demonstrate Release 1.0 GIMI tools for experimenter use cases  (July 2012, GEC14)
     1851.      Can push measurement data from OML server run as part of the slice, to the iRODS archive.
     1862.      Feedback on use of v1.0 software by GPO and early users
     1873.      Tutorial on v1.0 software for all users
     1884.      Agree on definition of V1.1 release
     190t)      Demonstrate tool-based setup of basic set of measurements using Release 1.0 of GIMI tools  (July 2012, GEC14)
     1911.      WiFi
     1922.      LEARN/BEN
     1933.      Sensors
     195u)      Best-effort operation of persistent OML server in Experimenter Portal Service, for prototyping and testing.  (July 2012, GEC14)
     1961.      Can be accessed through Experimenter Portal Service
     198v)      Prototype orchestration capability, using OMF EC and RC  (July 2012, GEC14)
     200w)      Deliver Release 1.1 of GIMI software, with fixes and small features.  (October 2012)
     202x)      OML Release 2.9.0:  (October 2012)
     2031.      Fixing of new bugs from previous release 2.8.0, stress test disconnection mode
     2042.      New features to improve data analysis
     2053.      New features to support IPv6 and IPv4/v6 address in databases
     2064.      New features to improve filtering capabilities: filter composition/stacking
     2075.      New binding for additional languages (Java/JNI)
     2086.      New platform support: Android (experimental)
     2097.      Improved documentation
    61213=== Status Reports and Demonstrations ===
    69 === Software Releases ===
    74 === Connectivity ===
    76 Expect to use IP addresses for registry.  Part of ProtoGENI cluster.
    78 No special GENI layer 2 connectivity needs for virtual Ethernet connections.
     221=== Software Releases and Acceptance Tests ===
     225=== Tutorials and Experimenter Support ===