Changes between Version 16 and Version 17 of GEC13Agenda/InstrumentationAndMeasurement


Ignore:
Timestamp:
03/02/12 10:53:51 (7 years ago)
Author:
hmussman@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEC13Agenda/InstrumentationAndMeasurement

    v16 v17  
    3636
    3737
    38 
     38== T2)  I&M Use Cases for Experiments, and Support for Experimenters  ==
     3910:05am  [[BR]]
     40
     41Team [[BR]]
     42 CO-LEAD  Jim Griffioen (U Kentucky)  [[BR]]
     43 Hussamuddin Nasir  (U Kentucky)  [[BR]]
     44 CO-LEAD  Mike Zink (UMass Amherst)  [[BR]]
     45 David Irwin (UMass Amherst)  [[BR]]
     46 Max Ott (NICTA)  [[BR]]
     47 Prasad Calyam (Ohio Super Center)   [[BR]]
     48 Harry Mussman  (GPO) [[BR]]
     49
     50Goals  [[BR]]
     51 Define use cases for a wide range of experiments:  complex (by skilled experimenters) to simple (by novice experimenter, i.e., a student)  [[BR]]
     52
     53Tasks  [[BR]]
     54 Provide a concise but complete definition of I&M Use Cases for Experiments [[BR]]
     55 Identify the support that should be available to experimenters, i.e., on-line instructions, examples and help desks.  [[BR]]
     56 Update the [http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture  GENI I&M Architecture document]:  [[BR]]
     57  Sec. 3.1.  I&M Use cases for Experimenters  [[BR]]
     58  Sec. 4.2.1  Typical Arrangements of I&M Services:  For Experimenter Gathering MD from their Slice  [[BR]]
     59  Sec. 4.3.1  Type 1 I&M Service:  Service Contained within a Slice  [[BR]]
     60 Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects  [[BR]]
     61
     62[wiki:InstMeasTopic_4.2UseCasesExperimenters  Work in Progress]  [[BR]]
     63
     64Summary of current status
     65
     66
     67
     68== T3)  I&M Use Cases for Infrastructure Measurement, and Support for Operators ==
     69I&M Use Cases for Infrastructure Measurement  [[BR]]
     70Support for Operators   [[BR]]
     71
     72[wiki:InstMeasTopic_4.3UseCasesInfrastructure  Work in Progress]  [[BR]]
     73
     741)  Goals  [[BR]]
     75
     76 From Sec. 2 of  the GENI I&M Architecture document:  [[BR]]
     77
     78  In addition, the GENI operations staff require extensive and reliable instrumentation and measurement capabilities to monitor and troubleshoot the GENI suite and its constituent entities.  Some of this data will be made available to experimenters, to help them conduct useful and repeatable experiments.   [[BR]]
     79
     80  The GMOC, providing GENI-wide operator services, needs to monitor essentially all GENI infrastructure on a 24x7 basis.  In this case, the GMOC Operator will gather, analyze and present MD that monitors hundreds of infrastructure elements.  [[BR]]
     81
     82
     832)  Tasks  [[BR]]
     84
     85 Provide a concise but complete definition of I&M Use Cases for Infrastructure Measurement [[BR]]
     86
     87 Identify the support that should be available to operators  [[BR]]
     88
     89 Update the [http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture  GENI I&M Architecture document]:  [[BR]]
     90   Sec. 3.3.  I&M Use cases for Central Operators (i.e., GMOC)  [[BR]]
     91   Sec. 3.4.  I&M Use cases for Aggregate Providers and Operators  [[BR]]
     92   Sec. 4.2.2  Typical Arrangements of I&M Services:  For Operator Gathering MD from GENI Infrastructure   [[BR]]
     93   Sec. 4.2.3  Typical Arrangements of I&M Services:  For Experimenters Gathering MD from their Slice and from GENI Infrastructure [[BR]]
     94   Sec. 4.3.3  Type 3 I&M Service:  Common Service with MD for Multiple Slices  [[BR]]
     95
     96 Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects  [[BR]]
     97
     98
     99
     1003)  Team  [[BR]]
     101 
     102 LEAD  Martin Swany (Indiana U) [[BR]]
     103 Guilherme Fernandes  (?)  [[BR]]
     104 Eric Boyd (Internet2) [[BR]]
     105 Jason Zurawski (Internet2) [[BR]]
     106 Prasad Calyam  (Ohio Super Center) [[BR]]
     107 Chris Small, for NetKarma (Indiana U) [[BR]]
     108 Ilia Baldine, for ExoGENI racks (RENCI) [[BR]]
     109 Jonathan Mills (RENCI)  [[BR]]
     110 ?, for InstaGENI racks (HP) [[BR]]
     111 ?, for GMOC [[BR]]
     112 Sarah Edwards (GPO) [[BR]]
     113 Chaos Golubitski (GPO) [[BR]]
     114 Harry Mussman  (GPO) [[BR]]
     115
     116== T4)  GENI Environment  ==
     117Platform Targets  [[BR]]
     118Experiment and Measurement Orchestration Tools  [[BR]]
     119Interfaces/Protocols Between Tools and Services  [[BR]]
     120
     121[wiki:InstMeasTopic_4.4GENIEnvironment  Work in Progress]  [[BR]]
     122
     1231)  Definition of topic:  [[BR]]
     124
     125a)  Platform targets: [[BR]]
     126  1)  protoGENI servers/VMs [[BR]]
     127  2)  ORCA servers/VMs [[BR]]
     128  3)  WiMAX sites [[BR]]
     129  4)  ExoGENI (RENCI) racks;  like ORCA [[BR]]
     130  5)  TransCloud (HP) racks;  like protoGENI? [[BR]]
     131  6)  PlanetLab
     132
     133b) Experiment and Measurement Orchestration tools: [[BR]]
     134  1)  FLACK [[BR]]
     135  2)  GUSH (with Omni) [[BR]]
     136  3)  SFACE + RAVEN [[BR]]
     137
     138c) Interfaces/protocols between tools and services: [[BR]]
     139  1)  protoGENI API, with protoGENI RSpec v2, for resource assignment [[BR]]
     140  2)  GENI AM API, with protoGENI RSpec v2, for resource assignment [[BR]]
     141  3)  ns for topology [[BR]]
     142  4)  ssh for image loading [[BR]]
     143  5)  OMF messages for orchestration [[BR]]
     144
     1452)  Who: Martin Swany (IU);  Jim Griffioen (U Kentucky);  Ilia Baldine, for ExoGENI racks (RENCI);  ?, for InstaGENI racks (?);  David Irwin (UMass Amherst);  Mike Zink (UMass Amherst);  Christoph Dwertmann (NICTA);  LEAD  Harry Mussman  (GPO);  Tom Mitchell (GPO);  Aaron Helsinger (GPO);  Marshall Brinn (GPO);  Vic Thomas (GPO)  [[BR]]
     146
     147
     148
     149== T5)  Descriptors, Objects and Registries  ==
     150Descriptor Schema and Registry Service  [[BR]]
     151Object Names and Registry Service  [[BR]]
     152
     153[wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService  Work in Progress]  [[BR]]
     154
     155
     1561) Goals  [[BR]]
     157
     158
     159
     1602) Tasks  [[BR]]
     161
     162a)  Need to finalize MDOD schema, for XML file.  [http://groups.geni.net/geni/wiki/GEC11InstMeasWorkingSession#a2MeasurementDataObjectDescriptorMDOD  References]  [[BR]]
     163b)  Want to extend MDOD to cover all types of objects, i.e., software images.  (NetKarma)[[BR]]
     164c)  Want to use MDOD schema to define Event Record schema.  (NetKarma)[[BR]]
     165d)  Do we need MDOD registry?  Use UNIS lookup service?  Use DOR registry? Include in iRODS? Consider IF-MAP server? [[BR]]
     166e)  Need MDOD creation and editing service. (CNRI) [[BR]]
     167f)  Need Measurement Data Object identifiers (names);  sometimes need a persistent, public reference;  consider DataCite approach, which uses handle  [[BR]]
     168
     169
     170
     1713) Team  [[BR]]
     172
     173Jason Zurawski (Internet2) [[BR]]
     174Ezra Kissel (U Delaware) [[BR]]
     175Eric Boyd (Internet2) [[BR]]
     176Beth Plale (IU) [[BR]]
     177Chris Small (GEMINI, IU) [[BR]]
     178Scott Jensen (Indiana U) [[BR]]
     179Larry Lannom (CNRI) [[BR]]
     180LEAD Giridhar Manepalli (CNRI) [[BR]]
     181Deniz Gurken (GIMI, UH) [[BR]]
     182Harry Mussman (GPO)  [[BR]] [[BR]]
     183
     184
     185
     186== T6)  XML Messaging Service ==
     187XML Messaging Service  [[BR]]
     188
     189[wiki:InstMeasTopic_4.6MessagingService  Work in Progress]  [[BR]]
     190
     191
     192
     1931)  Goals   [[BR]]
     194
     195 In I&M, a Measurement Orchestration (MO) service needs to interact with the I&M services;  one way to do so is to use the OMF interface, currently used (for example) by the OMF Experiment Controller to command an OMF Resource Controller (RC) that is installed next to an application.  [[BR]]
     196
     197 The OMF messages are carried between an EC and RC by an XML pub/sub messaging service, based on an XMPP server.  [[BR]]
     198
     199 For GENI, an XML Messaging service will be provided to carry these OMF messages, and thus enable orchestration.  [[BR]]
     200
     201 However, such a service may have other useful functions, such as transporting Event Records (ERs).  [[BR]]
     202
     203 To be generally useful, authentication and authorization functions must be provided.   [[BR]]
     204 
     205 For a detailed list of all goals defined for the XML Messaging service, see the Spiral 4 SOW of the IMF project.  [[BR]]
     206
     207 
     2082)  Tasks   [[BR]]
     209
     210 Define, prototype, demonstrate and operate a GENI Messaging service, starting at GEC13. [[BR]]
     211
     212 The XML Mesaaging service will be built by the IMF project.  For a detailed task list, see the Spiral 4 SOW of the IMF project. [[BR]]
     213
     214 Support use in GENI by many tools, including  GEMINI and GIMI I&M tools  [[BR]]
     215
     216 Define operations plan for XML Messaging service.  [[BR]]
     217
     218
     2193)  Team   [[BR]]
     220 
     221 Rudra Dutta (NCSU) [[BR]]
     222 LEAD Anirban Mandal (RENCI) [[BR]]
     223 Christoph Dwertmann (NICTA) [[BR]]
     224 Ahmed El-Hassany (IU) [[BR]]
     225 Harry Mussman (GPO) [[BR]]
     226
     227
     228== T7)  Measurement Information (MI) Service ==
     229Lookup Service  [[BR]]
     230
     231[wiki:InstMeasTopic_4.7LookupService  Work in Progress]  [[BR]]
     232
     2331)  Goals  [[BR]]
     234
     235 A Lookup Service is a type of registry defined in the perfSONAR toolset;  the availability of MD at perfSONAR interfaces MC services is registered there, using metadata describing the data and its location.
     236
     237 In addition, a similar Topology Service is used to store the topology of the network being examined. 
     238
     239 Together, these services are combined in the Unified Network Information Service (UNIS). 
     240
     241 The GEMINI project includes perfSONAR tools, and requires UNIS functions.  Some will be provided locally (within the slice), but a global service is required if available MD is to be registered, so that users can find it and get it.  [[BR]]
     242
     243 For GENI, the Measurement Information service (MI) will be provided to serve the need for a global UNIS service.
     244
     245 Can the MI service be used for other registry functions?
     246
     247 Can the MI service be used for finding web interfaces, and other types of portals?
     248
     249 Can the MI service be used to provide topology information to assist resource assignment and stitching in GENI? 
     250
     2512)  Tasks   [[BR]]
     252
     253 The MI service will be built by the GEMINI project using UNIS technology;  see the Spiral 4 SOW of the GEMINI project for the detailed steps, which include:  [[BR]]
     254
     255 1) Refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances: [[BR]]
     256 2) Update topology model to Rspec version 3 and AM API (February 2012) [[BR]]
     257 3) Modifications to allow local UNIS to register with global UNIS (March 2012) [[BR]]
     258 4) Initial deployment of persistent GENI Global I&M Registry (GGR) service, based on UNIS. (May 2012) [[BR]]
     259 5) Complete noSQL (MongoDB) backend (September 2012)  [[BR]]
     260 6) Unify Perl and Python implementations (September 2012) [[BR]]
     261
     262 Define, prototype, demonstrate and operate a MI service, starting in May, 2012. [[BR]]
     263
     264 Support use in GENI by many tools, including  GEMINI and GIMI I&M tools  [[BR]]
     265
     266 Define operations plan for MI service.  [[BR]]
     267
     268
     2693)  Team  [[BR]]
     270 
     271 ? (GEMINI)  [[BR]]
     272 Giridhar Manepalli (CNRI)  [[BR]]
     273 Harry Mussman (GPO)  [[BR]]
     274
     275
     276
     277
     278== T8) GENI User Services ==
     279Portal Service  [[BR]]
     280
     281[wiki:InstMeasTopic_4.8PortalService  Work in Progress]  [[BR]]
     282
     2831)  Goals [[BR]]
     284
     285 Work towards Max Ott's vision for experiment support  [[BR]]
     286
     287 Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user.  Those user services with a GUI (web) interface are often called "portal services".[[BR]]
     288
     289 Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data.   [[BR]]
     290
     291 These services should work together via APIs, etc., to streamline the experiment process. [[BR]]
     292
     2932)  Tasks [[BR]]
     294
     295 Based upon the configuration defined below, the implementation is split into: [[BR]]
     296
     297 1)  A GENI User Workspace Service, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools [[BR]]
     298
     299 2)  Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user. [[BR]]
     300
     301 Define, prototype, deploy and operate a GENI User Workspace Service. [[BR]]
     302
     303 Gather the various "user tools" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure. [[BR]]
     304
     305 Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).  [[BR]]
     306
     3073)  Team [[BR]]
     308 
     309  Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky)  [[BR]]
     310  Max Ott and/or Christoph Dwertmann (NICTA)  [[BR]]
     311  Chris Small  (NetKarma,  IU)  [[BR]]
     312  Ahmed El-Hassany (IU) [[BR]]
     313  Jeanne Ohren (GPO)[[BR]]
     314  Harry Mussman  (GPO) [[BR]]
     315
     316
     317
     318
     319
     320== T9)  Digital Object Archive (DOA) Service ==
     321User Workspace and Archive Services  [[BR]]
     322
     323[wiki:InstMeasTopic_4.9ArchiveService  Work in Progress]  [[BR]]
     324
     3251)  Definition of topic:  [[BR]]
     326
     327 a)  Consider CNRI prototype user workspace and archive service [[BR]]
     328 b)  Consider U Kentucky archive service [[BR]]
     329 c)  Use iRODS service? [[BR]]
     330 d)  How to move data to service:  move file;  move SQL DB;  or? [[BR]]
     331 e)  How are partitions for different users managed? [[BR]]
     332 f)  How is data in MDOD used to set archive sharing policy?  what is required in MDOD? [[BR]]
     333 g)  Operations plan for Archive Service. [[BR]]
     334
     3352)  Who: Giridhar Manepalli (CNRI);  LEAD  Shu Huang (RENCI) ;  Larry Lannom (CNRI);  Wesley Wu (U Kentucky);  Hussam Nasir (U Kentucky);  Chris Small  (NetKarma, IU) [[BR]]
     336
     337
     338== T10) List of Operational Services ==
     339Persistent Operational Services  [[BR]]
     340Services Shared by GIMI and GEMINI  [[BR]]
     341
     342[wiki:InstMeasTopic_4.10OperationalServices  Work in Progress]  [[BR]]
     343
     3441)  What persistent operational services must be supported: [[BR]]
     345
     346  a)  Descriptor registry   required?  use UNIS?  DOR?  iRODS? IF-MAP? [[BR]]
     347  b)  Object registry   required?  use DOR? [[BR]]
     348  c)  XML Messaging service [[BR]]
     349  d)  Measurement Information (MI)service  [[BR]]
     350  e)  GENI User services [[BR]]
     351  f) Digital Object Archive (DOA) service [[BR]]
     352
     3532)  Can all services can be shared by GIMI and GEMINI tools? [[BR]]
     354
     3553)  How do these relate to other GENI services/functions, including clearinghouse?
     356
     3574)  Who: Mike Zink (UMass Amherst);  Martin Swany (Inidana U);  Jim Griffioen (U Kentucky);  Giridhar Manepalli (CNRI);  Aaron Helsinger (GPO);  Jeanne Ohren (GPO);  Harry Mussman (GPO)   [[BR]]
    39358
    40359