[[PageOutline]] = 4.8) GENI User Portal Services = == 1) Goals == Work towards Max Ott's vision for experiment support [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/Paper_-_Tridentcom.pdf TridentCom Portal Paper][[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/Will%20RSpec%20help%20us%20to%20do%20better%20science%20-%20GEC11%20-%20July%202011.pdf Vision Slides from GEC11] [[BR]] [http://omf.mytestbed.net/projects/omf/wiki/PortalDesignSystemRequirements#Overview Testbed Portal System Requirements] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/Requirements-2.1.pdf Redmine Portal Service ] [[BR]] [[Image(NICTALabNotebookServiceModel.jpg, 70%)]] [[BR]] [[Image(SystemCLI.jpg, 50%)]] [[BR]] [[Image(NICATRedminePortalTool.jpg, 40%)]] [[BR]] NICTA Lab Notebook Service capabilities [[BR]] – Wiki: Keep notes with references[[BR]] – Schedule & monitor runs[[BR]] – Analysis with integrated R server[[BR]] – Repository for all artifacts[[BR]] Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Portal Services", where each portal service provides an interface (e.g., API or GUI) to the user. [[BR]] Together, the "GENI User Portal Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data. [[BR]] These services should work together via APIs, etc., to streamline the experiment process. [[BR]] == 2) Tasks == Based upon the configuration defined below, the implementation is split into: [[BR]] 1) GENI User Workspace Service, which is a persistent Linux OS environment, that serves as a container for multiple portal tools [[BR]] 2) Multiple GENI User Portal Tools, where each provides an interface or "portal" to the user. [[BR]] Define, prototype, deploy and operate a GENI User Workspace Service. [[BR]] Gather the various "portal services" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure. [[BR]] Optimize "portal services" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators). [[BR]] == 3) Team == Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky) [[BR]] Max Ott and/or Christoph Dwertmann (NICTA) [[BR]] Chris Small (NetKarma, IU) [[BR]] Ahmed El-Hassany (IU) [[BR]] Jeanne Ohren (GPO)[[BR]] Harry Mussman (GPO) [[BR]] == 4) Meetings == == 5) Issues == == 6) GENI User Portal Services Uses Cases == [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/GENIExperimenterPortalServiceUseCases.pdf Portal services use cases] (Jeannie Ohren, GPO) [[BR]] Use Cases: – User’s window into GENI [[BR]] – Conveniently and easily manage GENI resources, I&M data, and experiment results [[BR]] – A place to control the experiment [[BR]] – Works for both short/small (weeks-months?) and long/large (months-years?) experiments [[BR]] – A place to get to necessary GENI experimenter tools such as Flack, GUSH, OMNI [[BR]] – A place to store and retrieve experiment results and analysis (Lab notebook?) [[BR]] – MAP (Measurement, Analysis, Presentation) [[BR]] – Manages moving I&M data to archive (iRODS?) with metadata (MDOD?) [[BR]] – Allows the public to access published experiment results (or is this a separate interface?) [[BR]] == 7) GENI User Workspace Service Configuration == [[Image(Visio-112811_UseCases_Projects_Figures_Page_05.jpg, 90%)]] [[BR]] Linux OS environment: [[BR]] - On server: [[BR]] - Dedicated to user, i.e., a desktop server [[BR]] - In an organization (e.g., BBN), shared by multiple users in the organization [[BR]] - In GENI infrastructure, shared by multiple users [[BR]] - How are partitions for different users assigned? using GENI resource assignment process? [[BR]] – With login for the user [[BR]] - Includes file system [[BR]] – Includes certificate and credential stores [[BR]] – Includes various services/tools dedicated to this user, with programmatic and web interfaces [[BR]] - With interfaces/APIs between services tools [[BR]] - See Testbed Portal System Requirements above for an example [[BR]] - Prefer programmatic interface/API to allow coordination between services, and scripts [[BR]] - Prefer web interfaces to allow users to view topology, measurement data, events and trends [[BR]] Included services/tools/capabilities: [[BR]] - User can load desired services/tools, and bind them together [[BR]] - Experiment Control services/tools (e.g., GUSH, OMNI, OMF Experiment Controller) used to: [[BR]] - Assign resources for application and I&M services [[BR]] - Load topology and load images into slice, including those required for I&M [[BR]] - Configure and orchestrate/manage experiment applications [[BR]] - Configure and orchestrate/manage I&M services [[BR]] - Measurement Data Object Descriptor (MDOD) creation and editing, including annotation [[BR]] - User Workspace (UW) service [[BR]] - Measurement Collection (MC) service [[BR]] - Measurement Analysis and Presentation (MAP) services [[BR]] - Topology Presentation service [[BR]] - What specific services/tools will be available? [[BR]] NOT included services/tools: [[BR]] - Shared archive service, i.e., iRODS service [[BR]] == 8) Current and planned GENI User Portal Services == === a) INSTOOLS portal and web services (U Kentucky) === [http://groups.geni.net/geni/wiki/InstrumentationTools INSTOOLS project wiki] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/griffioen%20%20instools_imwg_gec7.pdf INSTOOLS GEC7 slides] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/ref%20%20InstTools-1%20%20instools-design-doc.pdf INSTOOLS design document] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/INSTOOLS_IM_tutorial_gec9.pdf INSTOOLS tutorial] [[BR]] [http://groups.geni.net/geni/wiki/GIR3.2_INSTOOLS INSTOOLS evaluation] [[BR]] [[Image(instools_imwg_gec79.jpg, 50%)]] [[BR]] INSTOOLS Portal service capabilities: [[BR]] - Resident in browser, with FLACK experiment control tool [[BR]] - Provides a way for experimenter to find all web servers in experimenter's slice [[BR]] - Allows experimenter to move data to archive service (local (U Kentucky) or remote (CNRI) [[BR]] INSTOOLS Web service capabilities: [[BR]] - One is resident in each Measurement Controller (MC) [[BR]] - Allows experimenter to configure measurements [[BR]] - Allows experimenter to view topology and view measured data [[BR]] - Allows experimenter to open a command line into any node, using vnc protocol [[BR]] === b) LAMP Periscope services (U Delaware+) === [http://groups.geni.net/geni/wiki/LAMP LAMP project wiki] [[BR]] [http://groups.geni.net/geni/wiki/LAMP/Tutorial LAMP tutorial] [[BR]] [http://groups.geni.net/geni/wiki/GIR3.2_LAMP LAMP evaluation] [[BR]] [[Image(Lamp-portal-1.jpg, 50%)]] [[BR]] Periscope services capabilities: [[BR]] - Slice Overview [[BR]] - Configuration Status [[BR]] - Registered Services [[BR]] - Measurement Tools (Daemons) [[BR]] - perfSONAR Services [[BR]] - Configuration [[BR]] - Enabled Services [[BR]] - Clock Synchronization [[BR]] - Scheduled Tests [[BR]] - Visualization [[BR]] - Throughput [[BR]] - One-way Latency [[BR]] - Ping Latency [[BR]] - Host Monitoring [[BR]] === c) Measurement Data Archive services (CNRI) === Measurement Data Archive (MDA) service capabilities: [[BR]] - The MDA srvc allows users to store, retrieve, browse, search, share and archive measurement data files, including their associated metadata. [[BR]] - The MDA srvc is not limited to measurement data, and can be used for other types of files needed by a researcher. [[BR]] - The MDA srvc utilizes the mechanisms provided by the CF to authenticate and authorize users. [[BR]] - The MDA srvc is implemented by a User Workspace (UW) Srvc, which in turn utilizes an Digital Object Archive (DOA) Srvc. [[BR]] [[Image(Visio-112811_UseCases_Projects_Figures_Page_12.jpg, 90%)]] [[BR]] [[Image(Visio-112811_UseCases_Projects_Figures_Page_13.jpg, 90%)]] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/Measurement%20Data%20Archive%20Prototype.pdf Design of prototype MDA service] [[BR]] [http://mda.doregistry.org/ Prototype MDA web interface] [[BR]] Programmatic Linux OS interface to User Workspace service: [[BR]] - Users have accounts [[BR]] - Users can mount file system using SMB (Samba) protocol [[BR]] - Users may securely transfer files using SFTP [[BR]] - Users can access a secure shell (SSH) to modify permissions and group settings. [[BR]] Web interface to User Workspace service: [[BR]] - Data objects can be searched, browsed and retrieved using web front end [[BR]] - Data objects can be found using keywords or timestamps [[BR]] - Users may examine metadata for a data object [[BR]] - Users may download data or metadata [[BR]] - How is metadata entered? [[BR]] - Users may request that an object be sent to the persistent archive service; a persistent identifier is returned [[BR]] Web interface to Object Archive service: [[BR]] - Data objects can be searched, browsed and retrieved using web front end [[BR]] Programmatic HTTP interface to User Workspace service: [[BR]] - Uses may GetID [[BR]] - Users may Archive an object [[BR]] - Users may retrieve an object or metadata [[BR]] Programmatic HTTP interface to Object Archive service: [[BR]] - Uses may GetID [[BR]] - Users may retrieve an object or metadata [[BR]] === d) NICTA Lab Notebook services === [http://omf.mytestbed.net/projects/omf/wiki/PortalDesignSystemRequirements#Overview Testbed Portal System Requirements] [[BR]] [http://groups.geni.net/geni/attachment/wiki/InstMeasTopic_4.8PortalService/Requirements-2.1.pdf Redmine Portal Service ] [[BR]] [[Image(NICTALabNotebookServiceModel.jpg, 70%)]] [[BR]] [[Image(SystemCLI.jpg, 50%)]] [[BR]] [[Image(NICATRedminePortalTool.jpg, 40%)]] [[BR]] NICTA Lab Notebook Service capabilities [[BR]] – Wiki: Keep notes with references[[BR]] – Schedule & monitor runs[[BR]] – Analysis with integrated R server[[BR]] – Repository for all artifacts[[BR]] === e) GIMI Portal services === [ GIMI project wiki] [[BR]] [[Image(Visio-112811_UseCases_Projects_Figures_Page_06.jpg, 90%)]] [[BR]] Experiment Control service capabilities: [[BR]] – Assign resources to slice, including those required for I&M [[BR]] – Load topology and load images into slice, including those required for I&M [[BR]] - Orchestrate (manage) experiment applications (processes) [[BR]] Measurement Orchestration (MO) service capabilities: [[BR]] – Orchestrate (manage) I&M services [[BR]] – Create and edit Measurement Data Object Descriptor (MDOD), including annotation [[BR]] OML Server and Results service capabilities: [[BR]] – Collect data (MC function) [[BR]] – Analyze and Present data (MAP functions)[[BR]] - Web interface [[BR]] - pdf file output [[BR]] – Archive or retrieve SQL data object to iRODS archive service[[BR]] === f) GEMINI Portal services === Expected to include some or all of: [[BR]] - LAMP Periscope services [[BR]] - INSTOOLS Portal service [[BR]] - INSTOOLS web service [[BR]] - more? [[BR]]