Changes between Version 1 and Version 2 of GIMI


Ignore:
Timestamp:
03/01/12 12:20:57 (8 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GIMI

    v1 v2  
    33== Project Number ==
    44
    5 1663
     51856
    66
    77== Project Title ==
    88
    9 GENI Registry Services[[BR]]
    10 a.k.a Digital Object Registry, DIGOBREG
     9GIMI: Large-scale GENI Instrumentation and Measurement Infrastructure [[BR]]
     10a.k.a GIMI
    1111
    1212=== Technical Contacts ===
    1313
    14 '''PI''' Larry Lannom (mailto:llannom@cnri.reston.va.us / 703 620 8990) [[BR]]
    15 Giridhar Manepalli (mailto:gmanepalli@cnri.reston.va.us / 703 620 8990) [[BR]]
    16 Jim French (mailto:jfrench@cnri.reston.va.us / 703 620 8990) [[BR]]
    17 Christophe Blanchi (mailto:cblanchi@cnri.reston.va.us / 703 620 8990)
     14'''PI''' Michael Zink,  University of Massachusetts at Amherst  (mailto:zink@ecs.umass.edu)  [[BR]]
     15'''Co-PI'''  Max Ott, NICTA  (mailto:max.ott@nicta.com.au) [[BR]]
     16'''Key personnel''' Ilia Baldine, RENCI  (mailto:ibaldin@renci.org) [[BR]]
     17
     18
    1819
    1920=== Participating Organizations ===
    2021
    21 Corporation for National Research Initiatives (CNRI)[[BR]]
    22 1895 Preston White Drive, Suite 100[[BR]]
    23 Reston, VA 20191 - 5434
     22University of Massachusetts at Amherst [[BR]]
     23
     24NICTA [[BR]]
     25
     26RENCI [[BR]]
     27
     28
    2429
    2530=== Related Projects ===
    2631
     32ExoGENI [[BR]] 
     33
     34WiMAX at UMass  [[BR]]
     35
     36DICLOUD  [[BR]]
     37
     38
    2739=== GPO Liaison System Engineer ===
    2840
     
    3345== Scope ==
    3446
    35 The scope of work on this project is to adapt the Handle System and/or
    36 a Digital Object Registry to realize a clearinghouse registry for
    37 principals, slices, and/or components in a selected GENI Spiral 1 control
    38 framework  (possibly using XML RPC), and capable of
    39 supporting limited operations in Year 1.  Also, to analyze how the
    40 Handle System and/or a Digital Object Registry could be used to
    41 identify and register GENI software, including experimenter’s tools,
    42 test images and configurations, and test results.  And finally, to
    43 define the operational, scaling, security, and management
    44 requirements, plus recommended design approaches, for implementing
    45 GENI clearinghouse and software registry services.
     47This project will develop and deploy the GIMI instrumentation and measurement framework, capable of supporting the needs of both GENI experimenters and GENI infrastructure operators.  It uses the ORBIT Measurement Library (OML) and integrated Rule Oriented Data System (iRODS) as its basis.  It will provide libraries to instrument resources, to filter and process measurement flows, and to consume measurement flows.  It will use the iRODS data grid for archiving and further processing.  It will include access control based on accepted GENI policy and authorization mechanisms. [[BR]]
     48
     49== Strategy and Goals ==
     50
     51Its first goal is to provide easy-to-use I&M services for experimenters, who are deploying slices in GENI on selected types of servers, VMs and racks, interconnected by various types of network paths.  [[BR]]
     52   
     53Its second goal is to provide comprehensive infrastructure measurement services for infrastructure operators, who are deploying measurement slices on selected types of servers, VMs and racks, interconnected by a various types of network paths.  In addition, slices established by other infrastructure operators, or by experimenters, can be authorized to gather data from the measurement slices. [[BR]]
     54
     55This project will build and operate two persistent services:   the GENI Measurement Data Archive service (i.e.,  iRODs)and the GENI Experimenter Portal Service.  These can be utilized by all types of GENI I&M services, including those developed in the GEMINI project.  The GENI Experimenter Portal Service will be developed in cooperation with the GEMINI project, to support both GIMI and GEMINI tools.
     56This project will use the GENI Measurement Data Object Descriptor (MDOD) schema, defined for use by all GENI I&M services. [[BR]]
     57
     58It may utilize two persistent services provided by the GEMINI project:  the GENI Global I&M Registry (i.e., UNIS), and the GENI Event Messaging Service.   [[BR]]
     59
     60The GIMI project will use the configuration shown in the diagram below. [[BR]]
     61
     62GIMI I&M Tools Configuration (p6) [[BR]]
     63
     64
    4665
    4766== Operational Capabilities ==
    4867
    4968
    50 GENI Federated Clearinghouse Prototype[[BR]]
    51 
    52 [http://geni.doregistry.org/GFC/  Link to GENI Federated Clearinghouse]  [[BR]]
    53 [attachment:"GENI Federated Clearing House Performance report.pdf" Evaluation of the GENI Federated Clearinghouse] [[BR]]
    54 
    55 
    5669
    5770=== Milestones ===
    5871
    5972
     73a)      Define MeasurementDataObjectDescriptor (MDOD) schema, plus naming of objects and object registry, in collaboration with I&M teams and the GPO.   (March 2012, GEC13) [[BR]]
     74
     75
     76b)      Define v0.5 I&M use cases for experiments and infrastructure monitoring, in collaboration with I&M teams, the GPO, experimenters and operators/GMOC.  (February 2012) [[BR]]
     77
     78
     79c)      Develop reference v1.0 experiment control tools based on OMF and Gush (including OMNI) and document the interfaces, protocols, and APIs of these tools, in collaboration with I&M teams and the GPO.  (March 2012, GEC13) [[BR]]
     80
     81
     82d)      Description of ontology extensions to cover measurement point resources (software and hardware). This will be integrated into ORCA per the ORCA-AUG SOW.   (February 2012) [[BR]]
     83
     84
     85e)      Identify use cases and concept of operations for GENI messaging services, in cooperation with the IMF and GEMINI projects.  (March 2012, GEC13) [[BR]]
     86
     87
     88f)      Deliver  v0.5 definition of GIMI architecture and software modules for year 1, documented on GIMI web page.  (February 2012) [[BR]]
     89
     90
     91g)      OML Release 2.7.0:  (February 2012) [[BR]]
     92
     931.      Fixing of current bugs affecting client apps or user's interpretation of data [[BR]]
     94
     952.      Fixing of current deployment and user development issues  [[BR]]
     96
     973.      New features to improve measurement reporting and future analysis  [[BR]]
     98
     994.      New features to ease current usage and enable future features [[BR]]
     100
     1015.      Improved documentation [[BR]]
     102
     103
     104h)      Agree on v1.0 I&M use cases for experiments and infrastructure monitoring, in collaboration with I&M teams and the GPO, and document, including:  (March 2012, GEC13) [[BR]]
     105
     1061.      WiMax [[BR]]
     107
     1082.      LEARN/BEN layer 0,1 or 2 real-time measurements [[BR]]
     109
     1103.      Sensor network (radars, power monitors) [[BR]]
     111
     1124.      Measure DiCloud workflow [[BR]]
     113
     114
     115i)      Compare available technology and prototypes for GENI messaging services, in cooperation with the IMF and GEMINI projects.  (March 2012, GEC13) [[BR]]
     116
     1171.      Decide whether GENI Event Messaging Service being developed for GEMINI project could be used by GIMI tools. [[BR]]
     118
     119
     120j)      Agree on v1.0 definition of GIMI architecture and software modules for year 1 with the GPO, and document on GIMI web page.   (March 2012, GEC13) [[BR]]
     121
     122
     123
     124k)      Identify necessary mappings between iRods metadata catalog information and MDOD.  (March 2012, GEC13) [[BR]]
     125 
     1261.      MDOD will be created and edited as part of measurement orchestration, to be completed in Year 2 (Spiral 5). [[BR]]
     127
     128
     129
     130l)      Agree on v1.0 target GENI environment for GIMI tools with I&M teams and the GPO  (March 2012, GEC13) [[BR]]
     131
     1321.      Nominal v1.0 target aggregate is ORCA and ExoGENI racks;  both servers and VMs;  when VMs have private IP addresses, make use of RENCI’s proxy mechanism, which basically works like an automated NAT. [[BR]]
     133
     1342.      GIMI tools will come with an image;  create template images with a basic configuration of tools which can be used and modified by other experimenters. [[BR]]
     135
     1363.      GIMI tools will be configured via OMF. [[BR]]
     137
     138
     139m)      Deliver  Beta 1.0 of GIMI software  (June 2012) [[BR]]
     140
     1411.      Utilize XML messaging service based on XMPP [[BR]]
     142
     1432.      Utilize OML servers that are part of the software release, and are run as part of the slice. [[BR]]
     144 
     1453.      Visualization is provided by a standalone tool, that is run as part of the slice.  [[BR]]
     146 
     1474.      Demonstrate GIMI working on target aggregates, including  ORCA and ExoGENI racks (if available), both servers and VMs.
     1485.      Demonstrate some WiMAX measurement capabilities.
     1496.      Demonstrate Layer 0,1 or 2 measurement capabilities  (potentially a combined LEARN/CASA measurement depending on radar deployment in DFW)
     1507.      If installed, demonstrate initial capabilities to monitor racks.
     151
     152n)      Develop use cases and best practices for Experimenter Portal Service, and document (based on LabWiki, Max’s portal) (March 2012, GEC13) 
     1531.      In cooperation with the GEMINI project. 
     1542.      Goal is to have one portal service support both GIMI and GEMINI tools.
     155
     156
     157o)      Stand up a v1.0 prototype iRods datagrid.  (March 2012, GEC13)
     1581.      Ideally across at least two GIMI institutions
     1592.      Identify/document the critical iRods microservices needed to support GENI.
     1603.      Identify/document mechanisms to map GENI authorization onto iRods datagrid.
     1614.      Share with GEMINI project
     162
     163p)      Deliver Release 1.0 of GIMI software after functional and performance testing.  (July 2012)
     1641.      Document on GIMI web page
     1652.      Provide support to early users
     166
     167q)      OML Release 2.8.0:  (June 2012)
     1681.      Fixing of new bugs from previous release 2.7.0
     1692.      Fixing of user development issues
     1703.      New features to improve data analysis
     1714.      New features to improve filtering capabilities and re-design of filter handling system
     1725.      New binding for additional languages (Python)
     1736.      New OML application to collect measurements from SNMP agent (NICTA);  should provide measurements equivalent to INSTOOLS     
     1747.      Improved documentation
     175
     176r)      Provide v1.0 persistent Experimenter Portal Service (based on NICTA LabWiki) that can be used by GENI experimenters. (June 2012)
     1771.      Allows users to keep a persistent log of their experiments
     1782.      Includes OML server
     1793.      May push measurement data to the iRODS archive
     1804.      May retrieve measurement data stored in iRODs archive, and visualize it.  (requires sql query support in iRODS, where an sql database can be “actively” hosted in iRODS, and someone can run queries against it.
     1815.      Provision of a set of visualization tools to graphically present measurement data
     1826.      Share with GEMINI project
     183
     184s)      Demonstrate Release 1.0 GIMI tools for experimenter use cases  (July 2012, GEC14)
     1851.      Can push measurement data from OML server run as part of the slice, to the iRODS archive.
     1862.      Feedback on use of v1.0 software by GPO and early users
     1873.      Tutorial on v1.0 software for all users
     1884.      Agree on definition of V1.1 release
     189
     190t)      Demonstrate tool-based setup of basic set of measurements using Release 1.0 of GIMI tools  (July 2012, GEC14)
     1911.      WiFi
     1922.      LEARN/BEN
     1933.      Sensors
     194
     195u)      Best-effort operation of persistent OML server in Experimenter Portal Service, for prototyping and testing.  (July 2012, GEC14)
     1961.      Can be accessed through Experimenter Portal Service
     197
     198v)      Prototype orchestration capability, using OMF EC and RC  (July 2012, GEC14)
     199
     200w)      Deliver Release 1.1 of GIMI software, with fixes and small features.  (October 2012)
     201
     202x)      OML Release 2.9.0:  (October 2012)
     2031.      Fixing of new bugs from previous release 2.8.0, stress test disconnection mode
     2042.      New features to improve data analysis
     2053.      New features to support IPv6 and IPv4/v6 address in databases
     2064.      New features to improve filtering capabilities: filter composition/stacking
     2075.      New binding for additional languages (Java/JNI)
     2086.      New platform support: Android (experimental)
     2097.      Improved documentation
     210
     211
    60212
    61213=== Status Reports and Demonstrations ===
     
    67219
    68220
    69 === Software Releases ===
    70 
    71 
    72 
    73 
    74 === Connectivity ===
    75 
    76 Expect to use IP addresses for registry.  Part of ProtoGENI cluster.
    77 
    78 No special GENI layer 2 connectivity needs for virtual Ethernet connections.
     221=== Software Releases and Acceptance Tests ===
     222
     223
     224
     225=== Tutorials and Experimenter Support ===