Version 1 (modified by, 14 years ago) (diff)


Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services

Team members:

Vic Thomas - BBN/GPO (yes)
Ivan Seskar – Rutgers WINLAB (yes)
Max Ott – NICTA (yes, by phone)
Sonia Fahmy* – Purdue (yes)
Giridhar Manepalli - CNRI (yes)

*agreed to organize first discussion and writing

Most instrumentation services follow the following steps: (1) request, (2) measure, (3) process, (4) transport, and (5) retrieve. A user of the instrumentation service will request certain measurements, which results in the provisioning and configuration of measurement probes. The probes generate a set or a stream of measurement data which may be further processed and potentially transported to a different location. Finally, the results of the requested measurements are stored in one or more repositories from which they can be retrieved and/or queried by authorized users, tools and services, either immediately (real-time) or at a later time (history).

Depending on usage scenarios and objectives, several different architectures for instrumentation and measurement can emerge. Current GENI instrumentation projects leverage a number of popular tools to construct their service. For example, the Scalable Sensing Service project (S3) implements "sensor pods" (measurement points) as cgi scripts accessible through any web-server that supports cgi. Boa, a light-weight open source web-server, is currently used. Therefore, starting/stopping the sensor pods is simply accomplished by starting and stopping the server. This framework enables convenient third party measurements; that is, measurements between two nodes can be initiated by a third node. Periodic measurements are configured as "cron" jobs on the sensor pods. A sensing information manager ensures the liveness of the service using vxargs. The sensing information manager also collects the measurements from the sensor pods using rsync, and stores them in a data store (S3).

In contrast, OML (OMLTridentCom,OMLwiki) takes a distributed stream-based approach. Probes are assumed to produce a stream of measurement tuples which are streamed through a configurable set of filters and caches to a repository. The repository, like S3, provides a web service interface allowing users and other services to retrieve the results (pull mode). However, users/services can also directly subscribe to a stream for operation in push mode. OML is integrated into the OMF control framework to simplify configuration of the entire distributed setup, as well as provide to support for "steerable" experiments, where the orchestration of an experiment can be influenced by the simultaneously collected measurements.

Another instrumentation service currently in GENI is perfSONAR, which sits somewhere in the middle. Various distributed monitoring services are "wrapped" into individual web services called "Measurement Points" (MPs) which return their results using the Open Grid Forum's Network Measurement Working Group (NMWG) XML schema. A set of other services, such as the Measurement Archive, Lookup, Authentication, Topology, Transformation, and Resource Protector provide a comprehensive suite of services. perfSONAR is migrating to Representational State Transfer (REST), viewed as a lighter-weight service-oriented architecture over existing web service technologies, e.g., SOAP. XML-RPC and SNMP are also leveraged in several current measurement projects.


There are essentially two types of measurement APIs: one concerning itself with the configuration of the various elements and services participating in the measurement and monitoring activities, and the other facilitating the transfer of the measurement data itself.

The provisioning and configuration of instrumentation functionality in a large federated environment serving many experimenters and spanning many countries with their respective legal frameworks is clearly a complex undertaking. However, at a fundamental level, instrumentation requires discovering, provisioning, and configuring resources, within the constraints of policies and user privileges. This is essentially functionality that the GENI control framework (CF) is (or will be) providing. What is not covered by the CF is the protocol for encoding and transporting measurement data between entities. Here, we must differentiate between two types of operations: the push or streaming, and the pull or requesting mode of operation. Both are not unique to this domain and there are several standards and widely used solutions.

OML will soon be using the Internet Protocol Flow Information Export (IPFIX) protocol standardized by the IETF for streaming measurements from an exporter to a collector. perfSONAR, as discussed above, is using SOAP and the Open Grid Forum's Network Measurement Working Group (NMWG) XML schema for returning measurements back to a requester. In OML, once an application has been modified to integrate OML measurements, two possible configurations are possible: passing general arguments on the command line of the enhanced application, or using an XML configuration file. The XML configuration file allows the user to multiplex the measurement stream and apply different filters inside the application and between Measurement Points (OMLTridentCom,OMLwiki).


(S3) P. Yalagandula, P. Sharma, S. Banerjee, S. Basu, and S.-J Lee. S3: A scalable sensing service for monitoring large networked systems. In Proceedings of the ACM SIGCOMM Workshop on Internet Network Management (INM), Sept. 2006.

(OMLTridentCom) J. White, G. Jourjon, T. Rakatoarivelo, M. Ott. Measurement Architectures for Network Experiments with Disconnected Mobile Nodes. In Proceedings of TridentCom, 2009.