Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services
Team members:
Vic Thomas - BBN/GPO (yes)
Ivan Seskar – Rutgers WINLAB (yes)
Max Ott – NICTA (yes, by phone)
Sonia Fahmy* – Purdue (yes)
Giridhar Manepalli - CNRI (yes)
*agreed to organize first discussion and writing
Most instrumentation services follow the following steps: (1) request,
(2) measure, (3) process, (4) transport, and (5) retrieve. A user of
the instrumentation service will request certain measurements, which
results in the provisioning and configuration of measurement
probes. The probes generate a set or a stream of measurement data
which may be further processed and potentially transported to a
different location. Finally, the results of the requested measurements
are stored in one or more repositories from which they can be
retrieved and/or queried by authorized users, tools and services,
either immediately (real-time) or at a later time (history).
Depending on usage scenarios and objectives, several different
architectures for instrumentation and measurement can emerge. Current
GENI instrumentation projects leverage a number of popular tools to
construct their service. For example, the Scalable Sensing Service
project (S3) implements "sensor pods" (measurement points) as
cgi scripts accessible through any web-server that supports cgi. Boa,
a light-weight open source web-server, is currently used. Therefore,
starting/stopping the sensor pods is simply accomplished by starting
and stopping the server. This framework enables convenient third
party measurements; that is, measurements between two nodes can be
initiated by a third node. Periodic measurements are configured as
"cron" jobs on the sensor pods. A sensing information manager ensures
the liveness of the service using vxargs. The sensing information
manager also collects the measurements from the sensor pods using
rsync, and stores them in a data store (S3).
In contrast, OML (OMLTridentCom,OMLwiki) takes a distributed
stream-based approach. Probes are assumed to produce a stream of
measurement tuples which are streamed through a configurable set of
filters and caches to a repository. The repository, like S3, provides
a web service interface allowing users and other services to retrieve
the results (pull mode). However, users/services can also directly
subscribe to a stream for operation in push mode. OML is integrated
into the OMF control framework to simplify configuration of the entire
distributed setup, as well as provide to support for "steerable"
experiments, where the orchestration of an experiment can be
influenced by the simultaneously collected measurements.
Another instrumentation service currently in GENI is perfSONAR, which
sits somewhere in the middle. Various distributed monitoring services
are "wrapped" into individual web services called "Measurement Points"
(MPs) which return their results using the Open Grid Forum's Network
Measurement Working Group (NMWG) XML schema. A set of other services,
such as the Measurement Archive, Lookup, Authentication, Topology,
Transformation, and Resource Protector provide a comprehensive suite
of services. perfSONAR is migrating to Representational State
Transfer (REST), viewed as a lighter-weight service-oriented
architecture over existing web service technologies, e.g.,
SOAP. XML-RPC and SNMP are also leveraged in several current
measurement projects.
APIs
There are essentially two types of measurement APIs: one concerning
itself with the configuration of the various elements and services
participating in the measurement and monitoring activities, and the
other facilitating the transfer of the measurement data itself.
The provisioning and configuration of instrumentation functionality in
a large federated environment serving many experimenters and spanning
many countries with their respective legal frameworks is clearly a
complex undertaking. However, at a fundamental level, instrumentation
requires discovering, provisioning, and configuring resources, within
the constraints of policies and user privileges. This is essentially
functionality that the GENI control framework (CF) is (or will be)
providing. What is not covered by the CF is the protocol for encoding
and transporting measurement data between entities. Here, we must
differentiate between two types of operations: the push or streaming,
and the pull or requesting mode of operation. Both are not unique to
this domain and there are several standards and widely used solutions.
OML will soon be using the Internet Protocol Flow Information Export
(IPFIX) protocol standardized by the IETF for streaming measurements
from an exporter to a collector. perfSONAR, as discussed above, is
using SOAP and the Open Grid Forum's Network Measurement Working Group
(NMWG) XML schema for returning measurements back to a requester. In
OML, once an application has been modified to integrate OML
measurements, two possible configurations are possible: passing
general arguments on the command line of the enhanced application, or
using an XML configuration file. The XML configuration file allows
the user to multiplex the measurement stream and apply different
filters inside the application and between Measurement Points
(OMLTridentCom,OMLwiki).
References:
(S3) P. Yalagandula, P. Sharma, S. Banerjee, S. Basu, and S.-J Lee.
S3: A scalable sensing service for monitoring large networked
systems. In Proceedings of the ACM SIGCOMM Workshop on Internet
Network Management (INM), Sept. 2006.
(OMLTridentCom) J. White, G. Jourjon, T. Rakatoarivelo, M. Ott.
Measurement Architectures for Network Experiments with Disconnected
Mobile Nodes. In Proceedings of TridentCom, 2009.
