wiki:InstMeasTopic_4.8PortalService

Version 41 (modified by hmussman@bbn.com, 7 years ago) (diff)

--

4.8) GENI User Portal Services

1) Goals

Work towards Max Ott's vision for experiment support

TridentCom Portal Paper

Vision Slides from GEC11

Testbed Portal System Requirements

Redmine Portal Service




NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts

Provide a way for a GENI user (e.g., experimenter) to access a wide variety of "GENI User Portal Services", where each portal service provides an interface (e.g., API or GUI) to the user.

Together, the "GENI User Portal Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data.

These services should work together via APIs, etc., to streamline the experiment process.

2) Tasks

Based upon the configuration defined below, the implementation is split into:

1) GENI User Workspace Service, which is a persistent Linux OS environment, that serves as a container for multiple portal tools

2) Multiple GENI User Portal Tools, where each provides an interface or "portal" to the user.

Define, prototype, deploy and operate a GENI User Workspace Service.

Gather the various "portal services" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.

Optimize "portal services" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).

3) Team

Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky)
Max Ott and/or Christoph Dwertmann (NICTA)
Chris Small (NetKarma, IU)
Ahmed El-Hassany (IU)
Jeanne Ohren (GPO)
Harry Mussman (GPO)

4) Meetings

5) Issues

6) GENI User Workspace Service: Configuration


Linux OS environment:

  • On server:
    • Dedicated to user, i.e., a desktop server
    • In an organization (e.g., BBN), shared by multiple users in the organization
    • In GENI infrastructure, shared by multiple users
      • How are partitions for different users assigned? using GENI resource assignment process?

– With login for the user

  • Includes file system
    – Includes certificate and credential stores
    – Includes various services/tools dedicated to this user, with programmatic and web interfaces
    • With interfaces/APIs between services tools
    • See Testbed Portal System Requirements above for an example
    • Prefer programmatic interface/API to allow coordination between services, and scripts
    • Prefer web interfaces to allow users to view topology, measurement data, events and trends

Included services/tools/capabilities:

  • User can load desired services/tools, and bind them together
  • Experiment Control services/tools (e.g., GUSH, OMNI, OMF Experiment Controller) used to:
    • Assign resources for application and I&M services
    • Load topology and load images into slice, including those required for I&M
    • Configure and orchestrate/manage experiment applications
    • Configure and orchestrate/manage I&M services
  • Measurement Data Object Descriptor (MDOD) creation and editing, including annotation
  • User Workspace (UW) service
  • Measurement Collection (MC) service
  • Measurement Analysis and Presentation (MAP) services
  • Topology Presentation service
  • What specific services/tools will be available?

NOT included services/tools:

  • Shared archive service, i.e., iRODS service

4.8.4) Current and Planned Portal Services

a) INSTOOLS portal and web services (U Kentucky)

INSTOOLS project wiki

INSTOOLS GEC7 slides

INSTOOLS design document

INSTOOLS tutorial

INSTOOLS evaluation


INSTOOLS Portal service capabilities:

  • Resident in browser, with FLACK experiment control tool
  • Provides a way for experimenter to find all web servers in experimenter's slice
  • Allows experimenter to move data to archive service (local (U Kentucky) or remote (CNRI)

INSTOOLS Web service capabilities:

  • One is resident in each Measurement Controller (MC)
  • Allows experimenter to configure measurements
  • Allows experimenter to view topology and view measured data
  • Allows experimenter to open a command line into any node, using vnc protocol

b) LAMP Periscope services (U Delaware+)

LAMP project wiki

LAMP tutorial

LAMP evaluation


Periscope services capabilities:

  • Slice Overview
    • Configuration Status
    • Registered Services
      • Measurement Tools (Daemons)
      • perfSONAR Services
  • Configuration
    • Enabled Services
    • Clock Synchronization
    • Scheduled Tests
  • Visualization
    • Throughput
    • One-way Latency
    • Ping Latency
    • Host Monitoring

c) Measurement Data Archive services (CNRI)

Measurement Data Archive (MDA) service capabilities:

  • The MDA srvc allows users to store, retrieve, browse, search, share and archive measurement data files, including their associated metadata.
  • The MDA srvc is not limited to measurement data, and can be used for other types of files needed by a researcher.
  • The MDA srvc utilizes the mechanisms provided by the CF to authenticate and authorize users.
  • The MDA srvc is implemented by a User Workspace (UW) Srvc, which in turn utilizes an Digital Object Archive (DOA) Srvc.



Design of prototype MDA service

Prototype MDA web interface

Programmatic Linux OS interface to User Workspace service:

  • Users have accounts
  • Users can mount file system using SMB (Samba) protocol
  • Users may securely transfer files using SFTP
  • Users can access a secure shell (SSH) to modify permissions and group settings.

Web interface to User Workspace service:

  • Data objects can be searched, browsed and retrieved using web front end
  • Data objects can be found using keywords or timestamps
  • Users may examine metadata for a data object
  • Users may download data or metadata
  • How is metadata entered?
  • Users may request that an object be sent to the persistent archive service; a persistent identifier is returned

Web interface to Object Archive service:

  • Data objects can be searched, browsed and retrieved using web front end

Programmatic HTTP interface to User Workspace service:

  • Uses may GetID
  • Users may Archive an object
  • Users may retrieve an object or metadata

Programmatic HTTP interface to Object Archive service:

  • Uses may GetID
  • Users may retrieve an object or metadata

d) NICTA Lab Notebook services

Testbed Portal System Requirements

Redmine Portal Service




NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts

e) GIMI Portal services

[ GIMI project wiki]


Experiment Control service capabilities:
– Assign resources to slice, including those required for I&M
– Load topology and load images into slice, including those required for I&M

  • Orchestrate (manage) experiment applications (processes)

Measurement Orchestration (MO) service capabilities:
– Orchestrate (manage) I&M services
– Create and edit Measurement Data Object Descriptor (MDOD), including annotation

OML Server and Results service capabilities:
– Collect data (MC function)
– Analyze and Present data (MAP functions)

  • Web interface
  • pdf file output

– Archive or retrieve SQL data object to iRODS archive service

f) GEMINI Portal services

Expected to include some or all of:

  • LAMP Periscope services
  • INSTOOLS Portal service
  • INSTOOLS web service
  • more?

4.8.5) GENI User Portal: Uses Cases

Portal services use cases (Jeannie Ohren, GPO)

Use Cases: – User’s window into GENI
– Conveniently and easily manage GENI resources, I&M data, and experiment results
– A place to control the experiment
– Works for both short/small (weeks-months?) and long/large (months-years?) experiments
– A place to get to necessary GENI experimenter tools such as Flack, GUSH, OMNI
– A place to store and retrieve experiment results and analysis (Lab notebook?)
– MAP (Measurement, Analysis, Presentation)
– Manages moving I&M data to archive (iRODS?) with metadata (MDOD?)
– Allows the public to access published experiment results (or is this a separate interface?)

4.8.6) GENI User Portal: Configuration


Linux OS environment:

  • On server:
    • Dedicated to user, i.e., a desktop server
    • In an organization (e.g., BBN), shared by multiple users in the organization
    • In GENI infrastructure, shared by multiple users
      • How are partitions for different users assigned? using GENI resource assignment process?

– With login for the user

  • Includes file system
    – Includes certificate and credential stores
    – Includes various services/tools dedicated to this user, with programmatic and web interfaces
    • With interfaces/APIs between services tools
    • See Testbed Portal System Requirements above for an example
    • Prefer programmatic interface/API to allow coordination between services, and scripts
    • Prefer web interfaces to allow users to view topology, measurement data, events and trends

Included services/tools/capabilities:

  • User can load desired services/tools, and bind them together
  • Experiment Control services/tools (e.g., GUSH, OMNI, OMF Experiment Controller) used to:
    • Assign resources for application and I&M services
    • Load topology and load images into slice, including those required for I&M
    • Configure and orchestrate/manage experiment applications
    • Configure and orchestrate/manage I&M services
  • Measurement Data Object Descriptor (MDOD) creation and editing, including annotation
  • User Workspace (UW) service
  • Measurement Collection (MC) service
  • Measurement Analysis and Presentation (MAP) services
  • Topology Presentation service
  • What specific services/tools will be available?

NOT included services/tools:

  • Shared archive service, i.e., iRODS service

Attachments (19)