Changes between Version 16 and Version 17 of GEC13Agenda/InstrumentationAndMeasurement

03/02/12 10:53:51 (9 years ago)



  • GEC13Agenda/InstrumentationAndMeasurement

    v16 v17  
     38== T2)  I&M Use Cases for Experiments, and Support for Experimenters  ==
     3910:05am  [[BR]]
     41Team [[BR]]
     42 CO-LEAD  Jim Griffioen (U Kentucky)  [[BR]]
     43 Hussamuddin Nasir  (U Kentucky)  [[BR]]
     44 CO-LEAD  Mike Zink (UMass Amherst)  [[BR]]
     45 David Irwin (UMass Amherst)  [[BR]]
     46 Max Ott (NICTA)  [[BR]]
     47 Prasad Calyam (Ohio Super Center)   [[BR]]
     48 Harry Mussman  (GPO) [[BR]]
     50Goals  [[BR]]
     51 Define use cases for a wide range of experiments:  complex (by skilled experimenters) to simple (by novice experimenter, i.e., a student)  [[BR]]
     53Tasks  [[BR]]
     54 Provide a concise but complete definition of I&M Use Cases for Experiments [[BR]]
     55 Identify the support that should be available to experimenters, i.e., on-line instructions, examples and help desks.  [[BR]]
     56 Update the [  GENI I&M Architecture document]:  [[BR]]
     57  Sec. 3.1.  I&M Use cases for Experimenters  [[BR]]
     58  Sec. 4.2.1  Typical Arrangements of I&M Services:  For Experimenter Gathering MD from their Slice  [[BR]]
     59  Sec. 4.3.1  Type 1 I&M Service:  Service Contained within a Slice  [[BR]]
     60 Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects  [[BR]]
     62[wiki:InstMeasTopic_4.2UseCasesExperimenters  Work in Progress]  [[BR]]
     64Summary of current status
     68== T3)  I&M Use Cases for Infrastructure Measurement, and Support for Operators ==
     69I&M Use Cases for Infrastructure Measurement  [[BR]]
     70Support for Operators   [[BR]]
     72[wiki:InstMeasTopic_4.3UseCasesInfrastructure  Work in Progress]  [[BR]]
     741)  Goals  [[BR]]
     76 From Sec. 2 of  the GENI I&M Architecture document:  [[BR]]
     78  In addition, the GENI operations staff require extensive and reliable instrumentation and measurement capabilities to monitor and troubleshoot the GENI suite and its constituent entities.  Some of this data will be made available to experimenters, to help them conduct useful and repeatable experiments.   [[BR]]
     80  The GMOC, providing GENI-wide operator services, needs to monitor essentially all GENI infrastructure on a 24x7 basis.  In this case, the GMOC Operator will gather, analyze and present MD that monitors hundreds of infrastructure elements.  [[BR]]
     832)  Tasks  [[BR]]
     85 Provide a concise but complete definition of I&M Use Cases for Infrastructure Measurement [[BR]]
     87 Identify the support that should be available to operators  [[BR]]
     89 Update the [  GENI I&M Architecture document]:  [[BR]]
     90   Sec. 3.3.  I&M Use cases for Central Operators (i.e., GMOC)  [[BR]]
     91   Sec. 3.4.  I&M Use cases for Aggregate Providers and Operators  [[BR]]
     92   Sec. 4.2.2  Typical Arrangements of I&M Services:  For Operator Gathering MD from GENI Infrastructure   [[BR]]
     93   Sec. 4.2.3  Typical Arrangements of I&M Services:  For Experimenters Gathering MD from their Slice and from GENI Infrastructure [[BR]]
     94   Sec. 4.3.3  Type 3 I&M Service:  Common Service with MD for Multiple Slices  [[BR]]
     96 Use as guidance in the design of GENI I&M tools, particularly for the GEMINI and GIMI projects  [[BR]]
     1003)  Team  [[BR]]
     102 LEAD  Martin Swany (Indiana U) [[BR]]
     103 Guilherme Fernandes  (?)  [[BR]]
     104 Eric Boyd (Internet2) [[BR]]
     105 Jason Zurawski (Internet2) [[BR]]
     106 Prasad Calyam  (Ohio Super Center) [[BR]]
     107 Chris Small, for NetKarma (Indiana U) [[BR]]
     108 Ilia Baldine, for ExoGENI racks (RENCI) [[BR]]
     109 Jonathan Mills (RENCI)  [[BR]]
     110 ?, for InstaGENI racks (HP) [[BR]]
     111 ?, for GMOC [[BR]]
     112 Sarah Edwards (GPO) [[BR]]
     113 Chaos Golubitski (GPO) [[BR]]
     114 Harry Mussman  (GPO) [[BR]]
     116== T4)  GENI Environment  ==
     117Platform Targets  [[BR]]
     118Experiment and Measurement Orchestration Tools  [[BR]]
     119Interfaces/Protocols Between Tools and Services  [[BR]]
     121[wiki:InstMeasTopic_4.4GENIEnvironment  Work in Progress]  [[BR]]
     1231)  Definition of topic:  [[BR]]
     125a)  Platform targets: [[BR]]
     126  1)  protoGENI servers/VMs [[BR]]
     127  2)  ORCA servers/VMs [[BR]]
     128  3)  WiMAX sites [[BR]]
     129  4)  ExoGENI (RENCI) racks;  like ORCA [[BR]]
     130  5)  TransCloud (HP) racks;  like protoGENI? [[BR]]
     131  6)  PlanetLab
     133b) Experiment and Measurement Orchestration tools: [[BR]]
     134  1)  FLACK [[BR]]
     135  2)  GUSH (with Omni) [[BR]]
     136  3)  SFACE + RAVEN [[BR]]
     138c) Interfaces/protocols between tools and services: [[BR]]
     139  1)  protoGENI API, with protoGENI RSpec v2, for resource assignment [[BR]]
     140  2)  GENI AM API, with protoGENI RSpec v2, for resource assignment [[BR]]
     141  3)  ns for topology [[BR]]
     142  4)  ssh for image loading [[BR]]
     143  5)  OMF messages for orchestration [[BR]]
     1452)  Who: Martin Swany (IU);  Jim Griffioen (U Kentucky);  Ilia Baldine, for ExoGENI racks (RENCI);  ?, for InstaGENI racks (?);  David Irwin (UMass Amherst);  Mike Zink (UMass Amherst);  Christoph Dwertmann (NICTA);  LEAD  Harry Mussman  (GPO);  Tom Mitchell (GPO);  Aaron Helsinger (GPO);  Marshall Brinn (GPO);  Vic Thomas (GPO)  [[BR]]
     149== T5)  Descriptors, Objects and Registries  ==
     150Descriptor Schema and Registry Service  [[BR]]
     151Object Names and Registry Service  [[BR]]
     153[wiki:InstMeasTopic_4.5DescriptorsObjectsRegistriesLookupService  Work in Progress]  [[BR]]
     1561) Goals  [[BR]]
     1602) Tasks  [[BR]]
     162a)  Need to finalize MDOD schema, for XML file.  [  References]  [[BR]]
     163b)  Want to extend MDOD to cover all types of objects, i.e., software images.  (NetKarma)[[BR]]
     164c)  Want to use MDOD schema to define Event Record schema.  (NetKarma)[[BR]]
     165d)  Do we need MDOD registry?  Use UNIS lookup service?  Use DOR registry? Include in iRODS? Consider IF-MAP server? [[BR]]
     166e)  Need MDOD creation and editing service. (CNRI) [[BR]]
     167f)  Need Measurement Data Object identifiers (names);  sometimes need a persistent, public reference;  consider DataCite approach, which uses handle  [[BR]]
     1713) Team  [[BR]]
     173Jason Zurawski (Internet2) [[BR]]
     174Ezra Kissel (U Delaware) [[BR]]
     175Eric Boyd (Internet2) [[BR]]
     176Beth Plale (IU) [[BR]]
     177Chris Small (GEMINI, IU) [[BR]]
     178Scott Jensen (Indiana U) [[BR]]
     179Larry Lannom (CNRI) [[BR]]
     180LEAD Giridhar Manepalli (CNRI) [[BR]]
     181Deniz Gurken (GIMI, UH) [[BR]]
     182Harry Mussman (GPO)  [[BR]] [[BR]]
     186== T6)  XML Messaging Service ==
     187XML Messaging Service  [[BR]]
     189[wiki:InstMeasTopic_4.6MessagingService  Work in Progress]  [[BR]]
     1931)  Goals   [[BR]]
     195 In I&M, a Measurement Orchestration (MO) service needs to interact with the I&M services;  one way to do so is to use the OMF interface, currently used (for example) by the OMF Experiment Controller to command an OMF Resource Controller (RC) that is installed next to an application.  [[BR]]
     197 The OMF messages are carried between an EC and RC by an XML pub/sub messaging service, based on an XMPP server.  [[BR]]
     199 For GENI, an XML Messaging service will be provided to carry these OMF messages, and thus enable orchestration.  [[BR]]
     201 However, such a service may have other useful functions, such as transporting Event Records (ERs).  [[BR]]
     203 To be generally useful, authentication and authorization functions must be provided.   [[BR]]
     205 For a detailed list of all goals defined for the XML Messaging service, see the Spiral 4 SOW of the IMF project.  [[BR]]
     2082)  Tasks   [[BR]]
     210 Define, prototype, demonstrate and operate a GENI Messaging service, starting at GEC13. [[BR]]
     212 The XML Mesaaging service will be built by the IMF project.  For a detailed task list, see the Spiral 4 SOW of the IMF project. [[BR]]
     214 Support use in GENI by many tools, including  GEMINI and GIMI I&M tools  [[BR]]
     216 Define operations plan for XML Messaging service.  [[BR]]
     2193)  Team   [[BR]]
     221 Rudra Dutta (NCSU) [[BR]]
     222 LEAD Anirban Mandal (RENCI) [[BR]]
     223 Christoph Dwertmann (NICTA) [[BR]]
     224 Ahmed El-Hassany (IU) [[BR]]
     225 Harry Mussman (GPO) [[BR]]
     228== T7)  Measurement Information (MI) Service ==
     229Lookup Service  [[BR]]
     231[wiki:InstMeasTopic_4.7LookupService  Work in Progress]  [[BR]]
     2331)  Goals  [[BR]]
     235 A Lookup Service is a type of registry defined in the perfSONAR toolset;  the availability of MD at perfSONAR interfaces MC services is registered there, using metadata describing the data and its location.
     237 In addition, a similar Topology Service is used to store the topology of the network being examined. 
     239 Together, these services are combined in the Unified Network Information Service (UNIS). 
     241 The GEMINI project includes perfSONAR tools, and requires UNIS functions.  Some will be provided locally (within the slice), but a global service is required if available MD is to be registered, so that users can find it and get it.  [[BR]]
     243 For GENI, the Measurement Information service (MI) will be provided to serve the need for a global UNIS service.
     245 Can the MI service be used for other registry functions?
     247 Can the MI service be used for finding web interfaces, and other types of portals?
     249 Can the MI service be used to provide topology information to assist resource assignment and stitching in GENI? 
     2512)  Tasks   [[BR]]
     253 The MI service will be built by the GEMINI project using UNIS technology;  see the Spiral 4 SOW of the GEMINI project for the detailed steps, which include:  [[BR]]
     255 1) Refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances: [[BR]]
     256 2) Update topology model to Rspec version 3 and AM API (February 2012) [[BR]]
     257 3) Modifications to allow local UNIS to register with global UNIS (March 2012) [[BR]]
     258 4) Initial deployment of persistent GENI Global I&M Registry (GGR) service, based on UNIS. (May 2012) [[BR]]
     259 5) Complete noSQL (MongoDB) backend (September 2012)  [[BR]]
     260 6) Unify Perl and Python implementations (September 2012) [[BR]]
     262 Define, prototype, demonstrate and operate a MI service, starting in May, 2012. [[BR]]
     264 Support use in GENI by many tools, including  GEMINI and GIMI I&M tools  [[BR]]
     266 Define operations plan for MI service.  [[BR]]
     2693)  Team  [[BR]]
     271 ? (GEMINI)  [[BR]]
     272 Giridhar Manepalli (CNRI)  [[BR]]
     273 Harry Mussman (GPO)  [[BR]]
     278== T8) GENI User Services ==
     279Portal Service  [[BR]]
     281[wiki:InstMeasTopic_4.8PortalService  Work in Progress]  [[BR]]
     2831)  Goals [[BR]]
     285 Work towards Max Ott's vision for experiment support  [[BR]]
     287 Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user.  Those user services with a GUI (web) interface are often called "portal services".[[BR]]
     289 Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data.   [[BR]]
     291 These services should work together via APIs, etc., to streamline the experiment process. [[BR]]
     2932)  Tasks [[BR]]
     295 Based upon the configuration defined below, the implementation is split into: [[BR]]
     297 1)  A GENI User Workspace Service, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools [[BR]]
     299 2)  Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user. [[BR]]
     301 Define, prototype, deploy and operate a GENI User Workspace Service. [[BR]]
     303 Gather the various "user tools" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure. [[BR]]
     305 Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).  [[BR]]
     3073)  Team [[BR]]
     309  Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky)  [[BR]]
     310  Max Ott and/or Christoph Dwertmann (NICTA)  [[BR]]
     311  Chris Small  (NetKarma,  IU)  [[BR]]
     312  Ahmed El-Hassany (IU) [[BR]]
     313  Jeanne Ohren (GPO)[[BR]]
     314  Harry Mussman  (GPO) [[BR]]
     320== T9)  Digital Object Archive (DOA) Service ==
     321User Workspace and Archive Services  [[BR]]
     323[wiki:InstMeasTopic_4.9ArchiveService  Work in Progress]  [[BR]]
     3251)  Definition of topic:  [[BR]]
     327 a)  Consider CNRI prototype user workspace and archive service [[BR]]
     328 b)  Consider U Kentucky archive service [[BR]]
     329 c)  Use iRODS service? [[BR]]
     330 d)  How to move data to service:  move file;  move SQL DB;  or? [[BR]]
     331 e)  How are partitions for different users managed? [[BR]]
     332 f)  How is data in MDOD used to set archive sharing policy?  what is required in MDOD? [[BR]]
     333 g)  Operations plan for Archive Service. [[BR]]
     3352)  Who: Giridhar Manepalli (CNRI);  LEAD  Shu Huang (RENCI) ;  Larry Lannom (CNRI);  Wesley Wu (U Kentucky);  Hussam Nasir (U Kentucky);  Chris Small  (NetKarma, IU) [[BR]]
     338== T10) List of Operational Services ==
     339Persistent Operational Services  [[BR]]
     340Services Shared by GIMI and GEMINI  [[BR]]
     342[wiki:InstMeasTopic_4.10OperationalServices  Work in Progress]  [[BR]]
     3441)  What persistent operational services must be supported: [[BR]]
     346  a)  Descriptor registry   required?  use UNIS?  DOR?  iRODS? IF-MAP? [[BR]]
     347  b)  Object registry   required?  use DOR? [[BR]]
     348  c)  XML Messaging service [[BR]]
     349  d)  Measurement Information (MI)service  [[BR]]
     350  e)  GENI User services [[BR]]
     351  f) Digital Object Archive (DOA) service [[BR]]
     3532)  Can all services can be shared by GIMI and GEMINI tools? [[BR]]
     3553)  How do these relate to other GENI services/functions, including clearinghouse?
     3574)  Who: Mike Zink (UMass Amherst);  Martin Swany (Inidana U);  Jim Griffioen (U Kentucky);  Giridhar Manepalli (CNRI);  Aaron Helsinger (GPO);  Jeanne Ohren (GPO);  Harry Mussman (GPO)   [[BR]]