| 1 | === Topic 5 GENI I&M Interfaces and Protocols (APIs): Manage Services === |
| 2 | |
| 3 | Team members: |
| 4 | Vic Thomas - BBN/GPO (yes)[[BR]] |
| 5 | Ivan Seskar – Rutgers WINLAB (yes)[[BR]] |
| 6 | Max Ott – NICTA (yes, by phone)[[BR]] |
| 7 | Sonia Fahmy* – Purdue (yes)[[BR]] |
| 8 | Giridhar Manepalli - CNRI (yes)[[BR]] |
| 9 | |
| 10 | *agreed to organize first discussion and writing |
| 11 | |
| 12 | Most instrumentation services follow the following steps: (1) request, |
| 13 | (2) measure, (3) process, (4) transport, and (5) retrieve. A user of |
| 14 | the instrumentation service will request certain measurements, which |
| 15 | results in the provisioning and configuration of measurement |
| 16 | probes. The probes generate a set or a stream of measurement data |
| 17 | which may be further processed and potentially transported to a |
| 18 | different location. Finally, the results of the requested measurements |
| 19 | are stored in one or more repositories from which they can be |
| 20 | retrieved and/or queried by authorized users, tools and services, |
| 21 | either immediately (real-time) or at a later time (history).[[BR]] |
| 22 | |
| 23 | Depending on usage scenarios and objectives, several different |
| 24 | architectures for instrumentation and measurement can emerge. Current |
| 25 | GENI instrumentation projects leverage a number of popular tools to |
| 26 | construct their service. For example, the Scalable Sensing Service |
| 27 | project (S3) implements "sensor pods" (measurement points) as |
| 28 | cgi scripts accessible through any web-server that supports cgi. Boa, |
| 29 | a light-weight open source web-server, is currently used. Therefore, |
| 30 | starting/stopping the sensor pods is simply accomplished by starting |
| 31 | and stopping the server. This framework enables convenient third |
| 32 | party measurements; that is, measurements between two nodes can be |
| 33 | initiated by a third node. Periodic measurements are configured as |
| 34 | "cron" jobs on the sensor pods. A sensing information manager ensures |
| 35 | the liveness of the service using vxargs. The sensing information |
| 36 | manager also collects the measurements from the sensor pods using |
| 37 | rsync, and stores them in a data store (S3).[[BR]] |
| 38 | |
| 39 | In contrast, OML (OMLTridentCom,OMLwiki) takes a distributed |
| 40 | stream-based approach. Probes are assumed to produce a stream of |
| 41 | measurement tuples which are streamed through a configurable set of |
| 42 | filters and caches to a repository. The repository, like S3, provides |
| 43 | a web service interface allowing users and other services to retrieve |
| 44 | the results (pull mode). However, users/services can also directly |
| 45 | subscribe to a stream for operation in push mode. OML is integrated |
| 46 | into the OMF control framework to simplify configuration of the entire |
| 47 | distributed setup, as well as provide to support for "steerable" |
| 48 | experiments, where the orchestration of an experiment can be |
| 49 | influenced by the simultaneously collected measurements.[[BR]] |
| 50 | |
| 51 | Another instrumentation service currently in GENI is perfSONAR, which |
| 52 | sits somewhere in the middle. Various distributed monitoring services |
| 53 | are "wrapped" into individual web services called "Measurement Points" |
| 54 | (MPs) which return their results using the Open Grid Forum's Network |
| 55 | Measurement Working Group (NMWG) XML schema. A set of other services, |
| 56 | such as the Measurement Archive, Lookup, Authentication, Topology, |
| 57 | Transformation, and Resource Protector provide a comprehensive suite |
| 58 | of services. perfSONAR is migrating to Representational State |
| 59 | Transfer (REST), viewed as a lighter-weight service-oriented |
| 60 | architecture over existing web service technologies, e.g., |
| 61 | SOAP. XML-RPC and SNMP are also leveraged in several current |
| 62 | measurement projects.[[BR]] |
| 63 | |
| 64 | APIs[[BR]] |
| 65 | |
| 66 | There are essentially two types of measurement APIs: one concerning |
| 67 | itself with the configuration of the various elements and services |
| 68 | participating in the measurement and monitoring activities, and the |
| 69 | other facilitating the transfer of the measurement data itself.[[BR]] |
| 70 | |
| 71 | The provisioning and configuration of instrumentation functionality in |
| 72 | a large federated environment serving many experimenters and spanning |
| 73 | many countries with their respective legal frameworks is clearly a |
| 74 | complex undertaking. However, at a fundamental level, instrumentation |
| 75 | requires discovering, provisioning, and configuring resources, within |
| 76 | the constraints of policies and user privileges. This is essentially |
| 77 | functionality that the GENI control framework (CF) is (or will be) |
| 78 | providing. What is not covered by the CF is the protocol for encoding |
| 79 | and transporting measurement data between entities. Here, we must |
| 80 | differentiate between two types of operations: the push or streaming, |
| 81 | and the pull or requesting mode of operation. Both are not unique to |
| 82 | this domain and there are several standards and widely used solutions.[[BR]] |
| 83 | |
| 84 | OML will soon be using the Internet Protocol Flow Information Export |
| 85 | (IPFIX) protocol standardized by the IETF for streaming measurements |
| 86 | from an exporter to a collector. perfSONAR, as discussed above, is |
| 87 | using SOAP and the Open Grid Forum's Network Measurement Working Group |
| 88 | (NMWG) XML schema for returning measurements back to a requester. In |
| 89 | OML, once an application has been modified to integrate OML |
| 90 | measurements, two possible configurations are possible: passing |
| 91 | general arguments on the command line of the enhanced application, or |
| 92 | using an XML configuration file. The XML configuration file allows |
| 93 | the user to multiplex the measurement stream and apply different |
| 94 | filters inside the application and between Measurement Points |
| 95 | (OMLTridentCom,OMLwiki).[[BR]] |
| 96 | |
| 97 | |
| 98 | References:[[BR]] |
| 99 | |
| 100 | (S3) P. Yalagandula, P. Sharma, S. Banerjee, S. Basu, and S.-J Lee. |
| 101 | S3: A scalable sensing service for monitoring large networked |
| 102 | systems. In Proceedings of the ACM SIGCOMM Workshop on Internet |
| 103 | Network Management (INM), Sept. 2006.[[BR]] |
| 104 | |
| 105 | (OMLTridentCom) J. White, G. Jourjon, T. Rakatoarivelo, M. Ott. |
| 106 | Measurement Architectures for Network Experiments with Disconnected |
| 107 | Mobile Nodes. In Proceedings of TridentCom, 2009.[[BR]] |
| 108 | |
| 109 | (OMLwiki) http://omf.mytestbed.net/wiki/oml/Documentation |
| 110 | |