Version 45 (modified by 13 years ago) (diff) | ,
---|
4.8) GENI User Services
1) Goals
Work towards Max Ott's vision for experiment support
NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts
Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".
Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data.
These services should work together via APIs, etc., to streamline the experiment process.
2) Tasks
Based upon the configuration defined below, the implementation is split into:
1) A GENI User Workspace Service, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools
2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.
Define, prototype, deploy and operate a GENI User Workspace Service.
Gather the various "user tools" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.
Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).
3) Team
Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky)
Max Ott and/or Christoph Dwertmann (NICTA)
Chris Small (NetKarma, IU)
Ahmed El-Hassany (IU)
Jeanne Ohren (GPO)
Harry Mussman (GPO)
4) Meetings
5) Issues
6) Uses Cases
User services use cases (Jeannie Ohren, GPO)
– User’s window into GENI
– Conveniently and easily manage GENI resources, I&M data, and experiment results
– A place to control the experiment
– Works for both short/small (weeks-months?) and long/large (months-years?) experiments
– A place to get to necessary GENI experimenter tools such as Flack, GUSH, OMNI
– A place to store and retrieve experiment results and analysis (Lab notebook?)
– MAP (Measurement, Analysis, Presentation)
– Manages moving I&M data to archive (iRODS?) with metadata (MDOD?)
– Allows the public to access published experiment results (or is this a separate interface?)
7) User Workspace Service Configuration
Persistent Linux OS environment dedicated to a user:
- On a server:
- Dedicated to user, i.e., a desktop server
- In an organization (e.g., BBN), shared by multiple users in the organization
- In GENI infrastructure, shared by multiple users
- How are partitions for different users assigned? using GENI resource assignment process?
- How are partitions for different users assigned? using GENI resource assignment process?
- Dedicated to user, i.e., a desktop server
– With a login for the user
- Includes a file system for the user
– Includes certificate and credential stores for the user
– Includes various services/tools dedicated to this user, with programmatic and web interfaces
- With interfaces/APIs between services tools
- See Testbed Portal System Requirements above for an example
- Prefer programmatic interface/API to allow coordination between services, and scripts
- Prefer web interfaces to allow users to view topology, measurement data, events and trends
- With interfaces/APIs between services tools
- Persistent data available to user includes:
- Certificates and credentials
- Experiment scripts
- Experiment topology descriptions (RSpecs)
- Measurement data and MDODs (metadata)
Included services/tools/capabilities:
- User can load desired services/tools, and bind them together
- Experiment Control services/tools (e.g., GUSH, OMNI, OMF Experiment Controller) used to:
- Assign resources for application and I&M services
- Load topology and load images into slice, including those required for I&M
- Configure and orchestrate/manage experiment applications
- Configure and orchestrate/manage I&M services
- Assign resources for application and I&M services
- Measurement Point (MP) services
- Measurement Collection (MC) service
- Measurement Analysis and Presentation (MAP) services
- Measurement Data Object Descriptor (MDOD) creation and editing, including annotation
- Directory Archive (DA) service
- Topology Presentation service
- What specific services/tools will be available?
NOT included services/tools:
- Shared Digital Object Archive (DOA) service, i.e., iRODS service
8) Current and planned GENI User Tools
a) INSTOOLS portal and web services (U Kentucky)
INSTOOLS Portal service capabilities:
- Resident in browser, with FLACK experiment control tool
- Provides a way for experimenter to find all web servers in experimenter's slice
- Allows experimenter to move data to archive service (local (U Kentucky) or remote (CNRI)
INSTOOLS Web service capabilities:
- One is resident in each Measurement Controller (MC)
- Allows experimenter to configure measurements
- Allows experimenter to view topology and view measured data
- Allows experimenter to open a command line into any node, using vnc protocol
b) LAMP Periscope services (U Delaware+)
Periscope services capabilities:
- Slice Overview
- Configuration Status
- Registered Services
- Measurement Tools (Daemons)
- perfSONAR Services
- Measurement Tools (Daemons)
- Configuration Status
- Configuration
- Enabled Services
- Clock Synchronization
- Scheduled Tests
- Enabled Services
- Visualization
- Throughput
- One-way Latency
- Ping Latency
- Host Monitoring
- Throughput
c) Prototype Measurement Data Archive (MDA) services (CNRI)
Measurement Data Archive (MDA) service capabilities:
- The MDA srvc allows users to store, retrieve, browse, search, share and archive measurement data files, including their associated metadata.
- The MDA srvc is not limited to measurement data, and can be used for other types of files needed by a researcher.
- The MDA srvc utilizes the mechanisms provided by the CF to authenticate and authorize users.
The prototype MDA services have been implemented by CNRI, using these components:
- A hosted Linux OS server that includes multiple services
- One User Workspace (UW) Srvc for each user, a Linux OS environment with a file system, etc.
- A common Directory Archive (DA) service, which runs in the Linux server has has special privileges so that it can read and write the directories of each user.
- One User Workspace (UW) Srvc for each user, a Linux OS environment with a file system, etc.
- A separate and common Digital Object Archive (DOA) Srvc.
More details are in design of prototype services
The (approximate) configuration of the prototype services is shown here:
Interfaces to these services:
Programmatic Linux OS interface to User Workspace service:
- Users have accounts
- Users can mount file system using SMB (Samba) protocol
- Users may securely transfer files using SFTP
- Users can access a secure shell (SSH) to modify permissions and group settings.
- Users can load code and run processes.
The Directory Archive (DA) service runs on the same server, and has permission to read and write the directories of all users.
Web interface to Directory Archive (DA) service:
- One or more directories can be designated to become a data object
- A Measurement Data Object Descriptor (MDOD) (metadata) can be created as a text file, and then stored with the data object
- Data objects can be searched, browsed and retrieved using web front end
- Data objects can be found using keywords or timestamps
- Users may examine metadata for a data object
- Users may download data or metadata
- Users may request that an object be sent to the persistent Digital Object Archive (DOA) service; a persistent identifier is returned
Programmatic HTTP interface to Directory Archive (DA) service:
- Uses may GetID
- Users may Archive an object
- Users may retrieve an object or metadata
Web interface to Digital Object Archive (DOA) service:
- Data objects can be searched, browsed and retrieved using web front end
Programmatic HTTP interface to Digital Object Archive (DOA) service:
- Uses may GetID
- Users may retrieve an object or metadata
Web interfaces to prototype DA and DOA services
d) NICTA Lab Notebook services
NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts
e) GIMI Portal services
[ GIMI project wiki]
Experiment Control service capabilities:
– Assign resources to slice, including those required for I&M
– Load topology and load images into slice, including those required for I&M
- Orchestrate (manage) experiment applications (processes)
Measurement Orchestration (MO) service capabilities:
– Orchestrate (manage) I&M services
– Create and edit Measurement Data Object Descriptor (MDOD), including annotation
OML Server and Results service capabilities:
– Collect data (MC function)
– Analyze and Present data (MAP functions)
- Web interface
- pdf file output
– Archive or retrieve SQL data object to iRODS archive service
f) GEMINI Portal services
Expected to include some or all of:
- LAMP Periscope services
- INSTOOLS Portal service
- INSTOOLS web service
- more?
Attachments (19)
- Paper_-_Tridentcom.pdf (833.8 KB) - added by 13 years ago.
- Will RSpec help us to do better science - GEC11 - July 2011.pdf (10.0 MB) - added by 13 years ago.
- Requirements-2.1.pdf (392.2 KB) - added by 13 years ago.
- GENIExperimenterPortalServiceUseCases.pdf (58.3 KB) - added by 13 years ago.
- griffioen instools_imwg_gec7.pdf (213.8 KB) - added by 13 years ago.
- ref InstTools-1 instools-design-doc.pdf (759.4 KB) - added by 13 years ago.
- NICTALabNotebookServiceModel.jpg (1.4 MB) - added by 13 years ago.
- NICATRedminePortalTool.jpg (65.0 KB) - added by 13 years ago.
- SystemCLI.jpg (34.0 KB) - added by 13 years ago.
- INSTOOLS_IM_tutorial_gec9.pdf (1.2 MB) - added by 13 years ago.
- Lamp-portal-1.jpg (138.4 KB) - added by 13 years ago.
- instools_imwg_gec79.jpg (115.0 KB) - added by 13 years ago.
- Visio-112811_UseCases_Projects_Figures_Page_12.jpg (496.2 KB) - added by 13 years ago.
- Visio-112811_UseCases_Projects_Figures_Page_13.jpg (556.8 KB) - added by 13 years ago.
- Measurement Data Archive Prototype.pdf (220.3 KB) - added by 13 years ago.
- Visio-112811_UseCases_Projects_Figures_Page_06.jpg (614.7 KB) - added by 13 years ago.
- Visio-112811_UseCases_Projects_Figures_Page_05.jpg (425.5 KB) - added by 13 years ago.
- Visio-030212_UseCases_Projects_Figures_Page_21.jpg (557.6 KB) - added by 13 years ago.
- Visio-030212_UseCases_Projects_Figures_Page_05.jpg (538.5 KB) - added by 13 years ago.