[[PageOutline]] = Service Developers Roundtable = == Schedule == Tuesday, 8:30am - 10:00 am == Session Leaders == Harry Mussman. GENI Project Officee == Agenda / Details == == Pre-Requisites == None [[PageOutline]] = Instrumentation and Measurement (I&M): GIMI and GEMINI Design Topics at GEC15 = Tuesday, October 23, 8:30am - 10am [[BR]] Room: [[BR]] Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies) [[BR]] == Description == This I&M session will focus on the key issues of interest to both the GIMI and GEMINI projects, as they continue to work towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement. In particular, we will consider the services and functions that can be used by both the GIMI and GEMINI projects.[[BR]] We will form teams that can resolve these issues. [[BR]] Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement. [[BR]] == Agenda == This is a tentative agenda, which may change. [[BR]] == 1) Introductions == 8:30am [[BR]] == 2) Topics for this meeting == === T9) GENI Storage and Archive Service (based on iRODS) === 8:35am [[BR]] Shu Huang (RENCI) [[BR]] Tasks * Design and deploy GENI Storage and Archive Service for use by all GENI services. * Provide functions similar to the Measurement Data Archive (MDA) service prototype, designed by CNRI. * Use to store all types of experiment artifacts (not just measurement data), and identify objects to be moved to a long-term archive. * Implement using the iRODS data grid service, by defining rules (interfaces) to provide the necessary GENI functions. [[BR]] Design goals and approach * [wiki:InstMeasTopic_4.9ArchiveService status at GEC13] [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T9b%29%20%20gec13_irods_im_anirban.pptx slides at GEC13] [[BR]] * iRODS service has been established at RENCI [[BR]] * iRODS service has been established at Univ of Kentucky [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/irods-gec14-1.pptx iRODS configuration at GEC14] [[BR]] * Proposed configuration? [[BR]] * Operations plan? [[BR]] Current implementation and next steps * [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/iRODS%20GENI%20plan.docx iRODS GENI Plan] [[BR]] Open issues * Who is lead? [[BR]] * Will all iRODS services be federated, for redundancy? * How to move data to service: move file; move SQL DB; or? [[BR]] * How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.[[BR]] * How are partitions for different users managed? [[BR]] * How is authorization for a user done? [[BR]] * How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] * How will the GEMINI project use the iRODS service? [[BR]] * Will other parts of GENI want to use the iRODS service? [[BR]] * Do we need a breakout session for further discussion? when? [[BR]] Team * Shu Huang (RENCI), GIMI project, Lead * ? , GEMINI project * Giridhar Manepalli (CNRI) * Harry Mussman and Jeanne Ohren (GPO) a) Introductions 5min b) Meeting Introduction Harry Mussman 10min Goals and Approach Work towards a "unified experiment environment". Help groups prepare responses to Sol 4 Current overall experiment configuration Summary of current experiment steps, and possible next steps towards a “unified experiment environment”. Today’s discussion topics: Common graphical resource assignment tool Rationalization of all graphical portals, and all graphical tools Introduction of OMF to config/orchestrate I&M and ExpApp services c) Common graphical resource assignment tool 25min * Overview Harry Mussman S3: Obtain slice, install software Now: GIMI using ExoGENI, Flukes; loads images/pkgs Now: GEMINI using InstaGENI, FLACK; loads images/pkgs Next: move towards common graphical resource assignment tool for both ExoGENI and InstaGENI racks Suggest: discuss common graphical resource assignment tool, and relationship to portals, at Roundtable session * Current Flukes tool (Ilia Baldine) 5min 3 slides: current config; current capabilities and issues; future possibilities * Current FLACK tool (Rob Ricci) 5min 3 slides: current config; current capabilities and issues; future possibilities * Roundtable discussion 15min (Ilia Baldine); (Rob Ricci); (Tom Mitchell or Aaron Helsinger); Niky Riga [Any topics, suggestions and issues suggested AHEAD OF TIME by members of roundtable] [At the end: any conclusions, list of issues, team to continue discussion] d) Rationalization of all graphical portals, and all graphical tools * Overview Harry Mussman S2: Establish exper mgmt environment Next: GENI CH Portal, to establish credentials, and enter GENI S3: Obtain slice, install software Now: GIMI using ExoGENI, Flukes; loads images/pkgs Now: GEMINI using InstaGENI, FLACK; loads images/pkgs Next: move towards common graphical resource assignment tool for both ExoGENI and InstaGENI racks S5: Run/orchestrate I&M and exp app Now: GEMINI uses portal to GUI, to config/orchestrate Now: GEMINI uses portal to GUI, to configure and present graphs Now: GIMI uses portal incl GUI, to configure and present graphs S6: Analyze Now: Both use portal on GENI Storage Service (iRODS) to curate files (objects) Next: Need to rationalize all graphical portals, and all graphical tools, to better unify experiment environment Suggest: Consider single sign-on at GENI CH portal, then link to GEMINI Portal (GENI Desktop) for entry into I&M tools, then link to I&M GUIs (including the GIMI portal/GUI and the iRODS portal/GUI); discuss at Rountable * Current GENI CH Portal (Tom Mitchell or Aaron Helsinger) 5min 3 slides: current config; current capabilities and issues; future possibilities * Current GEMINI Portal (GENI Desktop) (Charles Carpenter or Jim Griffioen) 5min 3 slides: current config; current capabilities and issues; future possibilities * Current GIMI Portal/GUI (Jeanne Ohren or Divya) 5min 3 slides: current config; current capabilities and issues; future possibilities * Roundtable discussion 15min (Tom Mitchell or Aaron Helsinger); (Charles Carpenter or Jim Griffioen); (Jeanne Ohren or Divya); Niky Riga [Any topics, suggestions and issues suggested AHEAD OF TIME by members of roundtable] [At the end: any conclusions, list of issues, team to continue discussion] e) Introduction of OMF to config/orchestrate I&M and ExpApp services * Overview Harry Mussman S5: Run/orchestrate I&M and exp app Now: GEMINI uses portal to GUI, to config/orchestrate Now: GIMI uses OMF EC with Ruby script to config/orchestrate Gap: no common way to config/orchestrate exp apps Next: Need to rationalize orchestration of all services, including GIMI, GEMINI and Exp Apps, to better unify experiment environment Suggest: Introduce OMF for config/orchestration of all I&M tools and Exp Apps; later, add optional graphical interface; discuss at Roundtable * Summarize OMF capabilities Max Ott 5min 3 slides: current config; current capabilities and issues; future possibilities * Roundtable discussion 10min Max Ott; Marshall Brinn; Martin Swany; Fraida Fund [Any topics, suggestions and issues suggested AHEAD OF TIME by members of roundtable] [At the end: any conclusions, list of issues, team to continue discussion] === T5) Digital Objects, Identifiers and Descriptors === 9:05am [[BR]] Giridhar Manepalli (CNRI) [[BR]] Tasks * Define a comprehensive system for identifying and describing a wide range of digital objects, including all types of experiment artifacts (e.g., rspecs, scripts, measurement data). * Show how to group objects into one combined object (e.g., all artifacts associated with one experiment), and then identify, describe adn archive this combined object Design goals Approach and next steps * [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/MD_Workspace_And_Archive_GEC15.pptx Archive, DOI (handle), Descriptors] [[BR]] Open issues Team * Giridhar Manepalli and Larry Lannom (CNRI) * Harry Mussman and Jeanne Ohren (GPO) * ?, GIMI project * ?, GEMINI project === T10) I&M Service-to-Service Authentication and Authorization === 9:25am [[BR]] Ezra Kissel (IU) [[BR]] Tasks * Establish a approach for providing authentication and authorization between GENI I&M service when they are required. * Note: One approach included in LAMP tools from IU * Note: Most authentication and authorization work in GENI has been focused on resource assignment, between an experimenter's tolls and an aggregate manager Design goals and approach Current implementation and next steps [http://groups.geni.net/geni/attachment/wiki/GEMINI/AA-workflow.png GEMINI AA Workflow Configuration] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEMINI/101112%20%20gemini_aa.txt GEMINI AA Workflow Process] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEMINI/092812b_gemini_aa_continued.txt Status] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEMINI/Visio-083112_AuthenticationAuthorization_Figures.pdf AA Reference Configurations] [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/IM_AA.pptx I&M A&A Considerations (slides)] [[BR]] Open issues Team * Ezra Kissel (IU), GEMINI project, Lead * ?, GIMI project * Harry Mussman and Jeanne Ohren (GPO) == 3) Goals for the next 4 months == 9:45am [[BR]] Harry Mussman (GPO) [[BR]] Goals * Both GIMI and GEMINI tools should be available to Users during this period [[BR]] * Acceptance testing continued by the GPO (Jeanne Ohren) [[BR]] * Bug tracking system, plus periodic bug fixes, continued (who are leads?) [[BR]] * How can we facilitate feedback from Users, after EVERY use? [[BR]] * Both GIMI and GEMINI must provide a high level of support to Users [[BR]] * Tutorials must be kept up-to-date and operational, so that Users can teach themselves. (who are leads?) [[BR]] * Instructions must be continuously refined, to make the tools easier to use. (who are leads?) [[BR]] * Requested tool extensions must be gathered from users, based upon their experience. (who are leads?) [[BR]] * Users with advanced requirements must be found, better identify gaps in the tools. How? (who are leads?) [[BR]] * Tool extensions must be proposed by the teams, and reviewed with the users. How? (who are leads?) [[BR]] * A list of tool extensions needs to be developed for inclusion into v1.1, due at GEC15. (who are leads?) [[BR]] * Persistent multi-user services must be reliably available to all Users [[BR]] * User Worksapce (Jeanne Ohren) [[BR]] * XML messaging service (who is lead?) [[BR]] * iRODS archive service (who is lead?) [[BR]] * IREEL analysis and presentation service (who is lead?) [[BR]] * INSTOOLS portal-to-GUIs service (who is lead?) [[BR]] * Unified Network Information Service (UNIS) (who is lead?) [[BR]] Discussion * Do we have leads identified for all items? [[BR]] == 4) Adjourn == 10:00am [[BR]] = I&M References = == GENI I&M Architecture and Tools: Goals, Overview and Status == * GIMI I&M tool set design [[BR]] * Based on OML, provides basic host and active network measurements [[BR]] * Ready for use on ExoGENI racks; successful tutorial [[BR]] * Introduces XML messaging service, iRODS archive service, IREEL(lab wiki) portal service; use by others? iRODS by GEMINI [[BR]] * Spiral 5 goals: also work with InstaGENI racks, WiMAX sites [[BR]] * GEMINI I&M tool set design [[BR]] * Based on INSTOOLS and perfSONAR/LAMP, provides basic host and active network measurements [[BR]] * Ready for use on InstaGENI/protoGENI racks ; successful tutorial [[BR]] * Introduces portal to GUIs service, UNIS unified ntwk info service; use by others? [[BR]] * Spiral 5 goals: also work with ExoGENI racks [[BR]] * Need simplified MDOD measurement data object descriptor schema [[BR]] * Also service for creation and editing of MDOD [[BR]] * Possible extensions to other GENI objects, events [[BR]] [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/071112%20I%26M%20GEC14%20Wrap-Up%20Agenda.ppt slides] [[BR]] === GIMI I&M Tools: Goals, Overview and Status === '''PI''' Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu) [[BR]] '''Co-PI''' Max Ott, NICTA (mailto:max.ott@nicta.com.au) [[BR]] '''Key personnel''' Ilia Baldine, RENCI (mailto:ibaldin@renci.org) [[BR]] [wiki:GIMI GIMI wiki] [[BR]] [http://gimi.ecs.umass.edu/ GIMI website at UMass Amherst] [[BR]] Goals [[BR]] * Starts with OML, and provides easy-to-use tools for GENI environment [[BR]] * First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf [[BR]] * Second focus: I&M use cases for infrastructure measurement [[BR]] * Later focus: add sensor measurement data, carried in data network [[BR]] * Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] * Later, support: WiMAX sites [[BR]] * Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] * Introduces XML messaging service, with pub/sub, in public IP space [[BR]] * Introduces iRODS service, as a measurement data archive. [[BR]] * Introduces IREEL portal service, for measurement data analysys and presentation. [[BR]] Summary: [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GIMI%20GEC%2014%20I%26M%20Session.pptx GIMI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5] [[BR]] * For more information, attend or audit [http://groups.geni.net/geni/wiki/GIMIv1.0Tutorial GIMI v1.0 tutorial] [[BR]] === GEMINI I&M Tools: Goals, Overview and Status === '''PI''' Martin Swany, Indiana University (mailto:swany@iu.edu) [[BR]] '''Co-PI''' Chris Small, Indiana University (mailto:chsmall@indiana.edu) [[BR]] '''Co-PI''' James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu) [[BR]] '''Co-PI''' Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu) [[BR]] [wiki:GEMINI GEMINI wiki] [[BR]] Goals [[BR]] * Includes LAMP code and features in GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements [[BR]] * Includes INSTOOLS code and features in GEMINI tools, to gather basic host measurements [[BR]] * Later, integrates LAMP and INSTOOLS code and features into new code base for GEMINI tools. [[BR]] * First focus: I&M use cases for infrastructure measurement [[BR]] * Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential [[BR]] * Show how to monitor clusters/racks [[BR]] * Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project) [[BR]] * Second focus: I&M use cases for experiments [[BR]] * Gather basic host measurements (as was done by INSTOOLS using SNMP) [[BR]] * Show how to gather customized data from applications (as provided by OML Client) [[BR]] * Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] * Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] * Continues INSTOOLS portal service, to find GUIs [[BR]] * Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology [[BR]] Summary [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GEMINI%20Overview%20and%20Direction-GEC14.pptx GEMINI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5] [[BR]] * For more information, attend or audit [http://groups.geni.net/geni/wiki/GEMINITutorial GEMINI v1.0 tutorial] [[BR]] == Persistent Multi-user I&M Services == Goals [[BR]] * Use by both GIMI and GEMINI tools sets [[BR]] * Use by other parts of GENI [[BR]] * Identify need for further reviews and discussions [[BR]] === S1) GENI (XML) Messaging Service === === S2) GENI Storage and Archive Service (based on iRODS) === 9:15am [[BR]] Shu Huang (RENCI) [[BR]] Tasks * Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions. [[BR]] Summary * [wiki:InstMeasTopic_4.9ArchiveService status at GEC13] [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T9b%29%20%20gec13_irods_im_anirban.pptx slides at GEC13] [[BR]] * iRODS service has been established at RENCI [[BR]] * iRODS service has been established at Univ of Kentucky [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/irods-gec14-1.pptx iRODS configuration at GEC14] [[BR]] * Proposed configuration? [[BR]] * Operations plan? [[BR]] Discussion * Who is lead? [[BR]] * Will all iRODS services be federated, for redundancy? * How to move data to service: move file; move SQL DB; or? [[BR]] * How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.[[BR]] * How are partitions for different users managed? [[BR]] * How is authorization for a user done? [[BR]] * How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] * How will the GEMINI project use the iRODS service? [[BR]] * Will other parts of GENI want to use the iRODS service? [[BR]] * Do we need a breakout session for further discussion? when? [[BR]] === S3) GIMI Portal Service === 9:30am [[BR]] Max Ott and Christoph Dwertmann (NICTA); Mike Zink and Cong Wang (UMass Amherst) [[BR]] Tasks * Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project [[BR]] * Extend/change to become Lab wiki service [[BR]] Summary * IREEL service has been established at NICTA [[BR]] * IREEL service has been established at UMassAmherst [[BR]] * Current configuration [ slides] [[BR]] * Proposed configuration [ slides] [[BR]] * Operations plan [ slides] [[BR]] Discussion * Who is lead? [[BR]] * How are partitions for different users managed? [[BR]] * How is authorization for a user done? [[BR]] * Can IREEL be used to provide other functions, e.g., experiment meangement? if so, how? [[BR]] * Will the GEMINI project want to use the IREEL service? [[BR]] * Will other parts of GENI want to use the IREEL service? [[BR]] * Do we need a breakout session for further discussion? when? [[BR]] === S4) GEMINI Portal Service === 9:40am [[BR]] Charles Carpenter (U Kentucky) [[BR]] Tasks: * Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools [[BR]] Summary * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GEMINI%20Portal.ppt current GEMINI portal service configuration] [[BR]] Discussion * Who is lead? [[BR]] * How are partitions for different users managed? [[BR]] * How is authorization for a user done? [[BR]] * What additional features are under consideration? [[BR]] * Will the GIMI project want to use the INSTOOLS portal-to-GUIs service? [[BR]] * Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service? [[BR]] * Do we need a breakout session for further discussion? when? [[BR]] === S5) Unified Network Information Service (UNIS) === 9:45am [[BR]] Ahmed El-Hassany (IU) [[BR]] Tasks [[BR]] * Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set. Summary * [wiki:InstMeasTopic_4.7LookupService status at GEC13] [[BR]] * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/UNIS_GEC14.pdf UNIS configuration at GEC14; proposed configuration] [[BR]] Discussion * Who is lead? [[BR]] * Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances? [[BR]] * What additional features are under consideration? [[BR]] * What new interfaces are under consideration? [[BR]] * Will the GIMI project want to use the UNIS service? [[BR]] * Will other parts of GENI want to use the UNIS service, e.g., stitching? [[BR]] * Do we need a breakout session for further discussion? when? [[BR]]