Changes between Initial Version and Version 1 of Embedded Real-Time Measurements-2Q09-status

Timestamp:

05/28/10 09:30:37 (14 years ago)

Author:

jtaylor@bbn.com

Comment:

--

Embedded Real-Time Measurements-2Q09-status

@@ +1 @@
+[[PageOutline]]
+
+= ERM Project Status Report = 
+
+Period: 2Q09
+== I. Major accomplishments ==
+During this past quarter, our main accomplishments involved the completion of milestone 4 and
+the submission of the corresponding technical note. Building on the work previously completed
+under milestones 1, 2, and 3, we have also made progress on all other project milestones. Also of
+significant note: our ongoing interactions with the GENI teams involved in Cluster D (ORCA),
+primarily Renaissance Computing Institute (RENCI) and Duke University, have resulted in our
+project ERM officially joining Cluster D. This will provide us with a tangible way for our project
+endeavors to have a substantial impact on the (future) GENI network infrastructure, specifically
+going forward with implementation and prototyping efforts in the second year of Spiral 1. The
+optical networking equipment in the Breakable Experimental Network (BEN) (in Cluster D,
+ORCA) supports the acquisition of physical layer measurements that will provide us with a
+physical layer substrate that can be enabled with embedded real-time measurements.
+
+As our previous work and the following document show, we propose the design of a unified
+measurement framework (UMF) to ultimately limit the hardware and software overhead and
+complexity associated with accessing measurement data. The UMF will serve as a platform to
+gather physical layer measurements and to convey them to both the control framework and the
+GENI users/researchers who may then choose to conduct experiments in a cross-layer fashion.
+We are continuing to focus on the incorporation of a diverse set of real-time measurements in
+networking protocols, as can be seen specifically in the Milestone 4 technical note. This will
+facilitate an effective implementation of a real-time measurement based cross-layer
+communications systems and will ensure that the GENI infrastructure includes the technology to
+support efficient cross-layer communications and physical layer measurements.
+
+=== A. Milestones achieved ===
+The following section outlines the progress made on the pre-defined milestones as given in
+[erm09_1].[[BR]]
+
+'''''Milestone 1: GENI requirements for real-time measurements (completed 03/01/2009)'''''[[BR]]
+The capabilities of GENI’s infrastructure with respect to embedded real-time measurements were
+evaluated and we assessed the GENI requirements for real-time user access to data
+measurements across a diverse set of heterogeneous technologies. Additional information, as
+well as a technical note, on milestone 1 is available online [erm09_1], [fidler09_1].
+
+'''''Milestone 2: Develop specifications and networking protocols (completed 03/01/2009)'''''[[BR]]
+A set of specifications for supporting real-time measurements within the network substrate, as
+well as a set of specifications for networking protocols based on the GENI requirements for realtime
+user-accessed cross-layer measurements, were developed. We identified a set of technical
+specifications for the implementation of a unified, integrated measurement framework, with the
+ultimate goal of limiting the hardware and software overhead and complexity associated with accessing the measurement data. Additional information, as well as a technical note, on milestone 2 is available online [erm09_1], [lai09_1].
+
+'''''Milestone 3: Perform discrete-event network simulations (completed 03/01/2009)'''''[[BR]]
+In order to enable and perform discrete-event network simulations and quantitatively evaluate the
+performance impact of several scenarios of cross-layer information exchange based on real-time
+measurements, we have developed new simulation modules for the ns-2 open source network
+modeling environment. We have made these modules available to the GENI and networking
+community. The newly developed ns-2 ERM software modules, as well as the technical note, on
+milestone 3 are available online [erm09_1], [fidler09_2]. (The software modules were developed
+in cooperation with Alcatel-Lucent, Bell Labs.)
+
+'''''Milestone 4: Develop a software architecture (completed 06/01/2009)'''''[[BR]]
+Within the scope of this milestone, we have further developed our vision of the unified
+measurement framework (UMF) based on GENI real-time measurement requirements and other
+recent activities, developments, hardware/software resources, and products within other GENI
+prototyping efforts. We have interacted with other GENI prototyping groups, including existing
+software measurement architectures (e.g. SILO [silo09_1], perfSONAR [perfsonar09_1], OMF
+[rutgers09_1], etc.) and determined the potential of implementing a real-time measurement
+infrastructure within each of these architectures. In conjunction to our findings, we have
+analyzed the appropriate data exchange formats (e.g. XML, etc.) and several network
+management protocol languages (e.g. SNMP, TL1, etc.) within their applicability within our
+envisioned unified measurement framework [lai09_1]. Section 5 of this document outlines a
+summary of this analysis and our findings. Additional information, as well as a technical note, on
+milestone 4 is available online [erm09_1], [fidler09_3].
+
+'''''Milestone 5: Support the GPO in developing an experimental use-case (due 09/01/2009)'''''[[BR]]
+We plan to support the GPO in developing experimental designs for use-cases based on our work
+on a measurement-driven cross-layer communication system. Following the successful
+simulation work based on the modules developed in [fidler09_2], we will aim to work closely
+with the GPO to further validate the cross-layer communication schemes in developing the
+experimental use-case.
+
+'''''Milestone 6: Identify a candidate control framework (due 09/01/2009)'''''[[BR]]
+The goal of this milestone is to interact with the GPO to identify a candidate control framework
+and cluster for the integration of our real-time measurement system. We have made significant
+strides in this direction by recently joining the ORCA/BEN Cluster D [orcaben08_1]. This move
+was based on our continuing discussions with other GENI teams, particularly RENCI, Duke
+University, and North Carolina State University. We also have ongoing interactions with other
+GENI working groups and teams, such as the University of Houston [gurkan08_1] and the
+University of Wisconsin [barford06_1].
+
+=== B. Deliverables made ===
+  * Contributions to the GENI Wikipedia page on “Embedding real-time substrate measurements for cross-layer communications” (April 2009) 
+  * Oral presentation: F. Fidler, “Unified Measurements” at 4th GENI Engineering Conference, Miami, FL (April 2009)
+  * Technical Note Milestone 4 (May 2009)
+  * GENI quarterly report (July 2009)
+  * Oral presentation: F. Fidler, “Unified Measurement Framework” GENI Measurement Workshop, Madison, WI (June 2009)
+  * Conference submission: C. P. Lai et al., “Experimental Demonstration of QoS-Aware Cross-Layer Packet Protection Switching”, accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+  * Conference submission: F. Fidler et al., “Cross-Layer Simulations of Fast Packet Protection Mechanisms”, accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+  * Conference submission: F. Fidler et al., “Impairment-Aware Traffic Engineering Using Cross-Layer Protocols”, accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+
+== II. Description of work performed during last quarter ==
+  * Organizational work[[BR]]
+Working with the GPO (specifically John Jacob), details regarding our ongoing work and our
+contributions to GENI Spiral 1 were released on the GENI ERM Wikipage; this includes
+quarterly reports and the Milestone 4 technical note. An oral presentation, “Measurements as a
+GENI resource,” emphasizing the importance of real-time measurements and summarizing the
+results of our first milestones was presented at the 4th GENI Engineering Conference in April in
+the substrate working group session. An invitation to review our Milestone 4 document was sent
+out to the GENI substrate group mailing list in order to solicit other groups’ views and comments
+on our work on enabling real-time measurements. An oral presentation was also given at a
+Measurement Workshop at the University of Wisconsin, outlining our vision of a unified,
+integrated measurement framework that will be essential to realizing real-time measurements in
+GENI.
+
+  * Collaboration efforts[[BR]]
+We are currently engaging in collaboration efforts with several other GENI teams and groups,
+specifically RENCI, Duke University, and the University of Houston, with the overall endeavor
+of realizing real-time measurements in the future GENI network infrastructure. Our ongoing
+communications and discussions have led us to recently (May 2009) join Cluster D
+(ORCA/BEN). We have participated in multiple ORCA call meetings, including ORCA-fest, a
+day-long discussion among all Cluster D participants regarding the cluster’s Spiral 1 work and
+future ORCA roadmap. The Cluster D control framework recognizes the importance of enabling
+real-time measurements within BEN and test-beds, and will facilitate the integration of our work
+within their architecture and infrastructure.
+
+Furthermore, in order to find synergies with other measurement projects within GENI, we have
+also engaged in active discussions during multiple phone conferences (e.g. the review of
+[barford09_1] in June 2009) and at the GENI Measurement Workshop in Madison, Wisconsin (e.g. discussion with Matt Zekauskas from Internet2 on possible cooperation endeavors between
+our project and perfSONAR, June 2009).
+
+  * Real-time measurements software architecture[[BR]]
+Within milestone 4, we address the software architecture development of a measurement
+framework based on GENI real-time measurement requirements and other developments and
+resources available within the GENI prototyping activities. We have interacted with other
+prototype and measurement groups and consequently identified and leveraged ongoing relevant
+activities, software architectures, and protocols. We focused on a number of software
+architectures dedicated to network measurements which could serve as an interface between our
+proposed UMF, the control framework, and the GENI experimenter. By assessing several
+network management protocols and data exchange formats with respect to their capacity to
+exchange measurement and control information between the substrate’s performance monitors
+and the UMF, we propose a feasible way to implement the UMF within the GENI network
+infrastructure.
+
+=== A. Activities and findings ===
+''5.1 Unified measurement framework''[[BR]]
+Within the scope of our project, we have showed the advantages of a UMF and outlined how it
+would enable cross-layer communications based research on real-time measurements extracted
+from performance monitors within the substrate. As per our previous work and documents, the
+UMF is required if one wants to:
+  * control and acquire measurement data in a unified way,
+  * abstract measurement capabilities and equipment,
+  * provide a single point of access for the GENI researcher and the control framework,
+  * enable basic processing of the measurement data,
+  * provide some storage capacity,
+  * allow for easy reconfiguration, and
+  * reduce software, hardware, and design overhead when extending the measurement capabilities of the GENI network.
+
+''5.2 Real-time measurement software architecture''[[BR]]
+We have researched and analyzed existing GENI related software measurement prototyping
+efforts, such as GIMS, ORBIT, SILO, and perfSONAR, and evaluated the feasibility of realizing
+real-time physical layer measurement capabilities within these systems. We found that the
+architectures that are adequately flexible and might lend themselves particularly well to
+supporting real-time measurements and potentially also cross-layer communications are SILO
+and perfSONAR. SILO may be implemented using API calls, and the measurement data could
+function as an additional implemented service. In contrast, the perfSONAR measurement archive
+uses SNMP, which may be leveraged for measurement acquisition and network topology
+sharing.
+
+We have also analyzed several network management protocol languages (e.g. SNMP, TL1, etc.)
+and appropriate data exchange formats (e.g. XML, etc.) with respect to their applicability within
+the proposed unified measurement framework. We conclude that standard network management
+protocols such as SNMP or TL1 should be used for accessing individual performance monitoring devices to transmit physical layer measurement information to a hardware implementation of the
+unified measurement framework.
+
+One implementation example that we envision is using NetFPGA cards hosted in a server as
+recommended in [lai09_1]. The NetFPGA can pre-process the measurement information in order
+for the corresponding measurement data to be stored in databases (e.g. SQL), then accessed and
+exported to other services/software frameworks (e.g. SILO, perfSONAR, etc.) via XML based
+protocols. Furthermore, from the UMF, information about the resources, measurement
+capabilities, and network topology can be sent to the GENI control frameworks by means of
+XML encoded resource description languages such as NDL or RSpecs.
+
+''5.3 University of Wisconsin Measurement Workshop''[[BR]]
+In addition to participating in a phone call review of the Measurement System project
+specification document (June 16, 2009) [barford09_1], we also recently attended and presented
+at a GENI Measurement Workshop, hosted by the Measurement System project at the University
+of Wisconsin. Besides discussing the motivation for supporting physical layer real-time
+measurements, i.e. create increasingly flexible next-generation optical transport networks, we
+reported on the various hardware and software aspects of the proposed unified measurement
+framework and its interfaces.
+=== B. Project participants ===
+P.I. Keren Bergman, Columbia University New York, ''bergman@ee.columbia.edu''[[BR]]
+Caroline P. Lai, Columbia University New York, ''caroline@ee.columbia.edu''[[BR]]
+Franz Fidler, Columbia University New York, ''ffidler@ee.columbia.edu''[[BR]]
+Michael Wang, Columbia University New York, ''msw2138@columbia.edu''[[BR]]
+Peter Winzer, Alcatel-Lucent Bell Labs, ''winzer@alcatel-lucent.com''[[BR]]
+Marina K. Thottan, Alcatel-Lucent Bell Labs, ''marinat@alcatel-lucent.com''[[BR]]
+
+=== C. Publications (individual and organizational) ===
+  * C. P. Lai, F. Fidler, K. Bergman, “Experimental Demonstration of QoS-Aware Cross-Layer Packet Protection Switching” accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+  * F. Fidler, P. Winzer, C. P. Lai, M. K. Thottan, K. Bergman, “Cross-Layer Simulations of Fast Packet Protection Mechanisms” accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+  * F. Fidler, P. Winzer, M. K. Thottan, K. Bergman, “Impairment-Aware Traffic Engineering Using Cross-Layer Protocols” accepted at European Conference on Optical Communications (ECOC), Vienna, Austria (to be presented September 2009)
+  * F. Fidler, C.P. Lai, K. Bergman, “Milestone 4: GENI Real-Time Measurements Software Architecture,” Technical Note, May 2009
+  * F. Fidler, “Unified Measurements: GENI Requirements and Specifications for Embedded Real-Time Measurements,” Presentation at 4th GENI Engineering Conference, Miami, FL, 31 March – 2 April 2009.
+  * F. Fidler, “Unified Measurement Framework,” Presentation at GENI Measurement Workshop, Madison, WI, 26 June 2009.
+=== D. Outreach activities ===
+None. 
+
+=== E. Collaborations ===
+  * '''Cluster D:''' Renaissance Computing Institute (RENCI) and Duke University, Ilia Baldin [[Br]] We have been in frequent contact with Cluster D (primarily through RENCI) to discuss the possibilities of implementing real-time measurement capabilities in BEN. They fully support our realization of a unified measurement framework [lai09_1] in their network infrastructure within the scope of our ERM project.
+  * '''GENI Substrate Working Group:''' University of Houston, Deniz Gurkan[[BR]] We are also in ongoing active communications with Prof. Gurkan and her group at the University of Houston regarding their current GENI project of evaluating existing measurement capabilities in current GENI prototypes. They have recently assessed the physical layer test equipment available within other GENI groups [gurkan09_1].
+  * '''GPO:''' John Jacob [[BR]] We have engaged in e-mail discussions with John Jacob of the GPO regarding our latest milestone technical note releases and our recent joining of Cluster D.
+  * '''perfSONAR:''' Internet2, Matt Zekauskas[[BR]] We have been in contact with Matt Zekauskas, starting discussions on the possibility of exporting physical layer measurement data to the perfSONAR measurement framework.
+  * '''Vendor:''' Polatis, Jim Dertzbaugh[[BR]] We have been in contact with Polatis through Jim Dertzbaugh, obtaining detailed information about the power monitoring capabilities of the Polatis fiber switches. In the previous quarter, we participated in an on-site demonstration of these switches at the Optical Fiber Communications (OFC) conference. In this quarter, Jim Dertzbaugh also visited our lab at Columbia, and the outcome of our in-person discussions will be the acquisition of a 20-port fiber switch that we will obtain in the mid-July 2009 timeframe. The fiber switch is capable of outputting its power monitoring measurements via the network protocol language SNMP. Since the 24-port version of the switch is currently deployed in BEN, the Polatis switch will comprise a valuable hardware resource in our lab and will help us test and experiment potential future hardware implementations of the UMF.
+  * '''Vendor:''' Infinera, John Walker [[BR]] We have been in contact with Infinera regarding the BER monitoring capabilities of their DTN nodes. The process of establishing an NDA was previously initiated and we are following up on this matter.
+=== F. Other Contributions ===
+None.
+
+'''Bibliography'''
+  1. [erm09_1] Global Environment for Network Innovations – Wikipedia (2008, December) [Online]. Available: http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements
+  2. [fidler09_1] F. Fidler, C.P. Lai and K. Bergman, “Technical Note 1, GENI Requirements for Real-time Measurements” [Online]. Available: http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements
+  3. [lai09_1] C.P. Lai, F. Fidler, and K. Bergman, “Technical Note 2, Specifications and Networking Protocols” [Online]. Available: http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements
+  4. [fidler09_2] F. Fidler, C.P. Lai and K. Bergman, “Technical Note 3, Discrete-Event Network Simulations” [Online]. Available: http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements
+  5. [silo09_1] Net-Silos Team, “SILO Project – Services, Integration, controL and Optimization for the Future Internet” (2009, May) [Online]. Available: http://www.net-silos.net
+  6. [perfsonar09_1] perfSONAR (2009, May) [Online]. Available: http://www.perfsonar.net
+  7. [rutgers09_1] Rutgers University, “WINLAB – Wireless Information Network Laboratory” (2009, March) [Online]. Available: http://www.winlab.rutgers.edu/
+  8. [fidler09_3] F. Fidler, C.P. Lai and K. Bergman, “Technical Note 4, GENI Real-Time Measurements Software Architecture” [Online]. Available: http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements
+  9. [orcaben08_1] RENCI, Duke University, “A Prototype GENI Control Plane (ORCA) for a Metro-Scale Optical Testbed (BEN)” (2008, December) [Online]. Available: http://groups.geni.net/geni/wiki/ORCABEN
+  10. [gurkan08_1] D. Gurkan, “Data Plane Measurements” (2008, December) [Online]. Available: http://groups.geni.net/geni/wiki/Data%20Plane%20Measurements
+  11. [barford06_1] P. Barford (Eds), “GENI Instrumentation and Measurement Systems (GIMS) Specifications,” GENI Design Document 06-12, December 2006.
+  12. [netfpga09_1] NetFPGA (2009, June) [Online] Available: http://www.netfpga.org/
+  13. [gurkan09_1] D. Gurkan and R. Krishnappa, “Milestone 3, Identify External Measurement Equipment” [Online]. Available: http://groups.geni.net/geni/wiki/Data%20Plane%20Measurements
+  14. [barford09_1] P. Barford, “Requirements and Specifications for the Instrumentation and Measurement Systems for GENI” (2009, June) [Online]. Available: http://groups.geni.net/geni/wiki/MeasurementSystem