Version 44 (modified by, 10 years ago) (diff)


4.8) GENI User Services

1) Goals

Work towards Max Ott's vision for experiment support

TridentCom Portal Paper

Vision Slides from GEC11

Testbed Portal System Requirements

Redmine Portal Service

NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts

Provide a way for a GENI user (e.g., experimenter or operator) to access a wide variety of "GENI User Services", where each user service provides an interface (e.g., API or GUI) to the user. Those user services with a GUI (web) interface are often called "portal services".

Together, the "GENI User Services" should provide all of the functions the user needs to setup and run their experiment, then gather, analyze and present the measurement data.

These services should work together via APIs, etc., to streamline the experiment process.

2) Tasks

Based upon the configuration defined below, the implementation is split into:

1) A GENI User Workspace Service, which is a persistent Linux OS environment dedicated to the user, that serves as a container for multiple user tools

2) Multiple GENI User Tools, where each provides a service with an interface or a "portal" to the user.

Define, prototype, deploy and operate a GENI User Workspace Service.

Gather the various "user tools" that have been implemented to date, and fit into GENI User Workspace Service so that GENI I&M users can begin to conveniently conduct experiments or instrument infrastructure.

Optimize "user tools" and their interfaces to better meet the needs of GENI users (e.g., experimenters and operators).

3) Team

Jim Griffioen and/or Charles Carpenter (INSTOOLS and GEMINI, U Kentucky)
Max Ott and/or Christoph Dwertmann (NICTA)
Chris Small (NetKarma, IU)
Ahmed El-Hassany (IU)
Jeanne Ohren (GPO)
Harry Mussman (GPO)

4) Meetings

5) Issues

6) Uses Cases

User services use cases (Jeannie Ohren, GPO)

– User’s window into GENI
– Conveniently and easily manage GENI resources, I&M data, and experiment results
– A place to control the experiment
– Works for both short/small (weeks-months?) and long/large (months-years?) experiments
– A place to get to necessary GENI experimenter tools such as Flack, GUSH, OMNI
– A place to store and retrieve experiment results and analysis (Lab notebook?)
– MAP (Measurement, Analysis, Presentation)
– Manages moving I&M data to archive (iRODS?) with metadata (MDOD?)
– Allows the public to access published experiment results (or is this a separate interface?)

7) User Workspace Service Configuration

Persistent Linux OS environment dedicated to a user:

  • On a server:
    • Dedicated to user, i.e., a desktop server
    • In an organization (e.g., BBN), shared by multiple users in the organization
    • In GENI infrastructure, shared by multiple users
      • How are partitions for different users assigned? using GENI resource assignment process?

– With a login for the user

  • Includes a file system for the user
    – Includes certificate and credential stores for the user
    – Includes various services/tools dedicated to this user, with programmatic and web interfaces
    • With interfaces/APIs between services tools
    • See Testbed Portal System Requirements above for an example
    • Prefer programmatic interface/API to allow coordination between services, and scripts
    • Prefer web interfaces to allow users to view topology, measurement data, events and trends
  • Persistent data available to user includes:
    • Certificates and credentials
    • Experiment scripts
    • Experiment topology descriptions (RSpecs)
    • Measurement data and MDODs (metadata)

Included services/tools/capabilities:

  • User can load desired services/tools, and bind them together
  • Experiment Control services/tools (e.g., GUSH, OMNI, OMF Experiment Controller) used to:
    • Assign resources for application and I&M services
    • Load topology and load images into slice, including those required for I&M
    • Configure and orchestrate/manage experiment applications
    • Configure and orchestrate/manage I&M services
  • Measurement Data Object Descriptor (MDOD) creation and editing, including annotation
  • Measurement Collection (MC) service
  • Measurement Analysis and Presentation (MAP) services
  • Directory Archive (DA) service
  • Topology Presentation service
  • What specific services/tools will be available?

NOT included services/tools:

  • Shared Digital Object Archive (DOA) service, i.e., iRODS service

8) Current and planned GENI User Portal Services

a) INSTOOLS portal and web services (U Kentucky)

INSTOOLS project wiki


INSTOOLS design document

INSTOOLS tutorial

INSTOOLS evaluation

INSTOOLS Portal service capabilities:

  • Resident in browser, with FLACK experiment control tool
  • Provides a way for experimenter to find all web servers in experimenter's slice
  • Allows experimenter to move data to archive service (local (U Kentucky) or remote (CNRI)

INSTOOLS Web service capabilities:

  • One is resident in each Measurement Controller (MC)
  • Allows experimenter to configure measurements
  • Allows experimenter to view topology and view measured data
  • Allows experimenter to open a command line into any node, using vnc protocol

b) LAMP Periscope services (U Delaware+)

LAMP project wiki

LAMP tutorial

LAMP evaluation

Periscope services capabilities:

  • Slice Overview
    • Configuration Status
    • Registered Services
      • Measurement Tools (Daemons)
      • perfSONAR Services
  • Configuration
    • Enabled Services
    • Clock Synchronization
    • Scheduled Tests
  • Visualization
    • Throughput
    • One-way Latency
    • Ping Latency
    • Host Monitoring

c) Measurement Data Archive services (CNRI)

Measurement Data Archive (MDA) service capabilities:

  • The MDA srvc allows users to store, retrieve, browse, search, share and archive measurement data files, including their associated metadata.
  • The MDA srvc is not limited to measurement data, and can be used for other types of files needed by a researcher.
  • The MDA srvc utilizes the mechanisms provided by the CF to authenticate and authorize users.
  • The MDA srvc is implemented by a User Workspace (UW) Srvc, which in turn utilizes an Digital Object Archive (DOA) Srvc.

Design of prototype MDA service

Prototype MDA web interface

Programmatic Linux OS interface to User Workspace service:

  • Users have accounts
  • Users can mount file system using SMB (Samba) protocol
  • Users may securely transfer files using SFTP
  • Users can access a secure shell (SSH) to modify permissions and group settings.

Web interface to User Workspace service:

  • Data objects can be searched, browsed and retrieved using web front end
  • Data objects can be found using keywords or timestamps
  • Users may examine metadata for a data object
  • Users may download data or metadata
  • How is metadata entered?
  • Users may request that an object be sent to the persistent archive service; a persistent identifier is returned

Web interface to Object Archive service:

  • Data objects can be searched, browsed and retrieved using web front end

Programmatic HTTP interface to User Workspace service:

  • Uses may GetID
  • Users may Archive an object
  • Users may retrieve an object or metadata

Programmatic HTTP interface to Object Archive service:

  • Uses may GetID
  • Users may retrieve an object or metadata

d) NICTA Lab Notebook services

Testbed Portal System Requirements

Redmine Portal Service

NICTA Lab Notebook Service capabilities
– Wiki: Keep notes with references
– Schedule & monitor runs
– Analysis with integrated R server
– Repository for all artifacts

e) GIMI Portal services

[ GIMI project wiki]

Experiment Control service capabilities:
– Assign resources to slice, including those required for I&M
– Load topology and load images into slice, including those required for I&M

  • Orchestrate (manage) experiment applications (processes)

Measurement Orchestration (MO) service capabilities:
– Orchestrate (manage) I&M services
– Create and edit Measurement Data Object Descriptor (MDOD), including annotation

OML Server and Results service capabilities:
– Collect data (MC function)
– Analyze and Present data (MAP functions)

  • Web interface
  • pdf file output

– Archive or retrieve SQL data object to iRODS archive service

f) GEMINI Portal services

Expected to include some or all of:

  • LAMP Periscope services
  • INSTOOLS Portal service
  • INSTOOLS web service
  • more?

Attachments (19)