GENI / FIRE Collaboration Workshop, September 2015, Session 6: Monitoring

Session chairs

  • Scott Kirkpatrick
  • Jim Griffioen

Session Summary

The session was separated into three parts. We had thought of them as first, monitoring the testbed networks we had heard about in the rest of the program, studying clouds, clusters and slices connecting them, then second monitoring issues in wireless, but new technology testbeds and high bandwidth smartphones serving to observe networks, and third, monitoring the internet from end to end. To accommodate some speakers’ travel schedules, we held the session in the order, topics 2, 3, then 1.

The discussion of wireless covered experimental design and protocols as seen in the iMinds testbeds, where the Wishfull project will make possible much more flexible experimental control, but the classic issue of the need for very controlled experiments to reproducibly characterize a technology in its very early stages remains. 5G, although an important goal, remains imperfectly specified and lacking in hardware to test. Interference effects that will become important as spectrum is increasingly shared among different technologies are observed and will be of interest. The second talk in the session, about the FCC’s comittment to transparency and efforts to put as many as 100,000 mobile applications performing standard edge internet performance efforts in the field to attain transparency, was disappointing. The FCC is still not completely through their process of determining robustness of their results and a privacy exposure-free means of disseminating them. They still hope to release a report of results by city and raw data sets for public inspection “soon,” but this would appear to not be likely until next year.

In the end-to end session, AMIS, a hardware and software platform for measurement of longer flows (>5 min) with modern privacy primitives ranging from full encryption to differential privacy (controlled noise). was described. Capturing and sharing information from sensor networks was covered as a component that will be architected and implemented as part of a standard substrate built with a city’s cooperation in a new project, EMBERS, and made available through open source mechanisms for more general use. Congestion in the internet has been claimed to be a non-problem, but studies done at M-Lab which compare RTTs and download performance from content sources in multiple eyeball ISP over extended periods of time show that in fact there are frequent and extended anomalies. Their methods are simple, and can be easily extended to include GENI clusters as participants in the measurement constellation. Finally, data captured for a month at a time in several cities from a widely distributed smartphone app was shown to provide the most extensive coverage of the internet’s edge to date. For example, 50,000 unique (anonymized) Android phones contributed measurements in the LA basin, and these could be interpreted to show difference in loading or contention at different times of day between applications deriving their data over distinct paths in the Internet. The discussion diverged briefly into a very animated discussion of the ethics of using so many people’s personal activities as part of an aggregated measurement tool.

The first session (which occurred last) began by looking at the problem of monitoring power usage information for testbeds networks and clouds. The session began with a description of the Cloudlab power monitoring system which uses the Intelligent Platform Management Interface (IPMI) to collect power measurements over time and make them available to experimenters via a web page. A related talk described the power monitoring service supported by the Kwapi system which was designed to offer high frequency data collection, allowing users to correlate and associate power measurements with other network behaviors (e.g., bandwidth usage). The federation of testbeds presents additional monitoring challenges ranging from the complexities of dealing with heterogeneous infrastructure, software systems, and data formats, to scalable data collection and aggregation, to trust/reliability issues. Example systems such as MODACLOUDS, SPECS, and DICE were described that use monitoring architectures consisting of many distinct components, each with a specific purpose, to address the complexity of these systems. The FELIX monitoring service handles the scalability challenge by arranging measurement services hierarchically at the highest level, but then using a distributed approach within testbed islands. The Federated Trust and User Experience framework addresses the problem of assessing a testbed's reliability and trustworthiness by combining conventional monitoring information (objective information) with reputation-based trust management systems (subjective information) to define a reputation value that can be used to compare testbeds. While much work has focused on monitoring testbed infrastructure, relatively few tools are available to users to measure their specific experiment or slice. Looking at experimenter mailing lists and discussion groups shows that the majority of problems that experimenters face would not be detected by monitoring the infrastructure, but rather require slice/experiment-specific measurement and validation tools designed to test and evaluate the performance of a user's slice or experiment.


Session 2: Monitoring Wireless Infrastructure

Discussion leader: Ivan Seskar

Author(s) Title Presentation
Ingrid Moerman Emphasizing 5G Systems Measurement slides
James Miller Measuring Broadband to the Sidewalk slides

Session 3: Monitoring the Internet as a Whole

Discussion leader: Scott Kirkpatrick

Author(s) Title Presentation
Yan Luo Privacy Preserving Network Measurement slides
Timur Friedman Smart Cities slides
Collin Anderson Internet Misbehavior as Seen from M-Lab slides
Scott Kirkpatrick Crowd-sourced Data Demo slides

Session 1: Monitoring Testbed Networks, Clouds, Clusters, and Slices

Discussion leader: Jim Griffioen

Author(s) Title Presentation
Mike Zink Power and Temperature Measurement Infrastructure for CloudLab slides
Yahya Al-Hazmi Monitoring Ontologies -- Demo slides
Brecht Vermeulen & Chrysa Papagianni Facility Monitoring for Federations and Measuring Reputation in Testbeds slides
Dana Petcu Monitoring in Multi-clouds slides
Bartek Belter Monitoring of Multi-domain Slices in FELIX slides
Lucas Nussbaum Kwapi Monitor for GRID 5000 slides
Jim Griffioen Monitoring Experiments vs Testbeds slides
Last modified 6 years ago Last modified on 10/06/15 23:53:21

Attachments (13)