[[PageOutline]] == Project Number == 1718 == Project Title == GENI IMF: Integrated Measurement Framework and Tools for Cross Layer Experimentation [[BR]] a.k.a. IMF === Technical Contacts === PI: Rudra Dutta Department Of Computer Science North Carolina State University dutta@csc.ncsu.edu http://www.csc.ncsu.edu/directories/faculty_info.php?id=1665 [[BR]] Co-PI: George Rouskas Department of Computer Science North Carolina State University rouskas@ncsu.edu http://www.csc.ncsu.edu/directories/faculty_info.php?id=241 [[BR]] Subcontractor: Ilia Baldine Renaissance Computing Institute (RENCI) ibaldin@renci.org http://www.renci.org/about/leadership [[BR]] Subcontractor: Keren Bergman Department of Electrical Eng. Columbia University bergman@ee.columbia.edu http://www.ee.columbia.edu/fac-bios/bergman/faculty.html [[BR]] === Participating Organizations === [http://www.csc.ncsu.edu/ NCSU, CS Dept][[BR]] [http://www.ee.columbia.edu/ Columbia University, EE Dept][[BR]] [http://www.renci.org/ Renaissance Computing Insititute (RENCI), Chapel Hill, NC][[BR]] [https://ben.renci.org/ BEN: Breakable Experimental Network][[BR]] === Related Projects === [https://geni-imf.renci.org IMF Project Website] [[BR]] [wiki:ORCABEN ORCA/BEN project] and [wiki:ORCAAUG ORCA Augmentation][[BR]] [http://www.net-silos.net/ The SILO Project][[BR]] [https://ben.renci.org/ BEN: Breakable Experimental Network][[BR]] [http://www.cs.duke.edu/nicl/ New Internet Computing Lab (NICL)][[BR]] [http://nicl.cod.cs.duke.edu/orca/ Open Resource Control Architecture (ORCA)][[BR]] [http://www.cs.duke.edu/nicl/cereus/shirako.html Shirako][[BR]] [http://groups.geni.net/geni/wiki/Embedded%20Real-Time%20Measurements Embedded Real-Time Measurements][[BR]] [http://groups.geni.net/geni/wiki/LEARN LEARN: Programmable Measurements over Texas-based Research Network][[BR]] === GPO Liaison System Engineer === Harry Mussman hmussman@geni.net === I&M Group === This project is part of the GENI Instrumentation and Measurement group of projects. Relevant material is at: * [wiki:GeniInstrumentationandMeasurementsArchitecture GENI Instrumentation and Measurements Architecture] * [wiki:GENIIandMCAPCAT GENI Instrumentation and Measurement Capabilities Catalog] * [wiki:GeniInstMeas GENI Instrumentation and Measurement Work in Progress] and most recently at: * [wiki:GEC12InstMeasWorkingSession Instrumentation and Measurement (I&M) Working Session for GEC12] == Scope == This project will develop and integrate the GENI Integrated Measurement Framework (IMF) for optical communication substrates into the ORCA control framework prototype, and integrate the FIND SILO framework into the ORCA control framework prototype, and IMF and SILO with each other, to enable cross-layer experimentation involving the physical layer of an optical network. [[BR]] This will enable (a) measurements from an optical substrate to be passed to a measurement consumer inside a slice; this is a valuable capability since the optical substrate characteristics may be important to the experimenter in the slice although the substrate itself is not directly observable by the experimenter, and (b) enable the automated (in-stack) consumption of measurement data; this is important for an experimenter who does not merely want to see the optical substrate measurement data after the experiment, but would like to experiment with reactive protocols designed to run inside the stack and react in real-time to measurements. [[BR]] [[Image(IMF_Overview.png, 50%)]] [[BR]] Initial deployment will be in the Breakable Experimental Network (BEN), a regional optical network located in North Carolina. [[BR]] The IMF will utilize deliverables from the Embedded Real-time Measurements (ERM) project, including its Unified Measurements Framework (UMF).[[BR]] The IMF will provide a single point of access to various devices in the optical substrate. The IMF will include a software module that interfaces with the UMF to gather measurements from the optical substrate, and that communicates with SILO (and other software modules) over a specified communication protocol, also to be developed in this project. [[BR]] The interface between the IMF and the optical devices in BEN will be leveraged from the UMF of the ERM project. The IMF will gather measurements from devices with performance monitoring capabilities already incorporated into the BEN, including: the Infinera DTN, a remotely configurable optical add/drop multiplexer (ROADM); and the Polatis 24 port fiber switch. This will be guided by the output of the Data Plane Measurements project, that identified measurement capabilities in a wide range of equipment. In addition, the use of programmable measurement nodes and performance monitors will be assessed. [[BR]] The Services Integration, controL and Optimization (SILO) framework was previously developed and prototyped in the NSF Future Internet Design (FIND) project. It will be extended to provide the tools for the experimenter to enable cross-layer experimentation. New SILO services will be developed to interface with the IMF and to introduce optical substrate measurement capability into custom protocol stacks. [[BR]] Both the IMF and SILO will be integrated with ORCA, so that they can be assigned to the slice of a researcher, who can then configure and run them to conduct an experiment. This project will collaborate with the ORCA Augmentation project on an ontology for measurement capabilities.[[BR]] This project will collaborate with other GENI projects on the development of a common GENI instrumentation and measurement architecture. [[BR]] == Operating Capabilities == The overall architecture of the project has been articulated. The first diagram below shows the experimenter view of the proposed system; realization of this is part of our Year 2 goals. The second diagram below shows the component systems interaction architecture. [[BR]] January 2011 update: Upon consultation with GPO, the priorities of the project have been revised a little. Year 2 (Spiral 3) goals now focus on integrating the developed IMF capabilities for measurement, and transfer of measurement (from substrate to in-slice stack), with existing GENI Inst&Meas capabilities; specifically, to integrate and install perfSONAR (pS) MPs and archives to collect performance measurement data from Polatis switches and Infinera DWDM platforms on 4 BEN sites, and (subsequently) integrate pS MP polling into IMF to demonstrate ability of in-slice reactive protocols to consume these measurements and react accordingly. At GEC10, the first of these (collecting performance data from Polatis using pS) capability has been completed. This leverages Spiral 3 goals of the ERM project team, who also form the Columbia U part of the IMF team. === Experiment View (Feb 2010) === [[Image(IMF_exp_view.png, 50%)]] [[BR]] === Architecture (July 2010) === [[Image(IMF_Arch.png, 50%)]] [[BR]] At the end of Spiral 2, we are ahead of our goals with respect to the main vision of the project, and have demonstrated real-time reactive use of optical substrate measurements to allow in-slice protocols to react by tuning optical substrate parameters. The diagram below shows the essentials of the demo setup. The IMF project uses the BEN facility for its physical realization. See the [wiki:ORCABEN#CurrentCapabilities ORCA/BEN project] . [[Image(SILO.png, 40%)]] [[BR]] [[Image(QoS_Video_Streaming_Experiment_BEN.png, 40%)]] [[BR]] We experimentally demonstrate a video streaming application across BEN. We add cross-layer communication and control capabilities to BEN by integrating it with the SILO service-oriented architecture and a NetFPGA-based optical control plane. Dynamic optical power fluctuations introduced to BEN are either compensated using a semiconductor optical amplifier (SOA) or the lightpath is rerouted to ensure that a video requiring high QoS retains its fidelity. For comparison, a video requiring lower QoS is transmitted without impairment compensation or rerouting and suffers a noticeable degradation in quality. A detailed description is available in the GEC8 demo report. [[BR]] [[Image(Low_QoS_Video_Demo.png, 40%)]] [[BR]] [[Image(High_QoS_Video_Demo.png, 40%)]] [[BR]] === Architecture Update (March, 2011) === In line with the Spiral 3 goals of IMF, the architecture has been modified to include a pS MP to mediate the measurement from the Polatis switches, utilizing the ERM project capability which creates pS functionality into the switch, as shown in the diagram below. The pS MP can continue to serve other pS clients, and the IMF client can use other mechanisms (such as that developed in Spiral 2) to obtain measurements, in addition to this. This is shown in the diagram below. The substrate control part is currently not shown, since this will require newer pS functionality to be available for integration. [[Image(pS_blowup.pdf, 30%)]] [[BR]] === Milestones === == Spiral 4 == * [[MilestoneDate(IMF: S4.a Demonstration at GEC12 and Experimenter Outreach)]] * [[MilestoneDate(IMF: S4.b Documentation and Code Release)]] * [[MilestoneDate(IMF: S4.c Demonstration at GEC13 and Experimenter Outreach)]] * [[MilestoneDate(IMF: S4.d Documentation and Code Release)]] * [[MilestoneDate(IMF: S4.e Demonstration at GEC14 and Experimenter Outreach)]] * [[MilestoneDate(IMF: S4.f Final report and code release)]] === Status Reports and Demos === [http://groups.geni.net/geni/attachment/wiki/IMF/GEC6-ilia-imf-intro.pdf Presentation at GEC6 ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/GEC7_imf_cluster_d_meet.ppt Presentation at GEC7 Cluster D ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/GEC7_Measurment_Poster_v5.pdf Poster presented at GEC7 demo session][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/IMF-QSR_1Q10.pdf Status ending 1st Quarter, 2010 ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/IMF-QSR_2Q10.pdf Status ending 2nd Quarter, 2010 ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/GEC8_IMF_cluster_d_meet_w_backup.pptx Presentation at GEC8 Cluster D ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/GEC8_Poster_Demo_final_print.pdf Poster presented at GEC8 demo session][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/geni-imf_s2_review.pptx Spiral 2 Review presentation][[BR]] After GEC9 - No report, contract renewal still in negotiation[[BR]] [wiki:IMF-GEC10-QSR After-GEC10 Status Report] [[BR]] [wiki:IMF-GEC11-QSR After-GEC11 Status Report] [[BR]] [wiki:IMF-GEC13-QSR After-GEC13 Status Report] [[BR]] === Technical Documents === [http://groups.geni.net/geni/attachment/wiki/IMF/GEC6-ilia-imf-intro.pdf Initial architectural view - Presentation at GEC6 (also attached to Milestone S2.a ticket) ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/IMF%20Architecture.pdf First component architecture (Updated) ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/IMF-SILO.pdf Early draft IMF-SILO interface ][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/GEC8_demo_report.pdf Report describing GEC8 demo ][[BR]] [attachment:GEC10_IMF-ERM_Poster_v6.pdf GEC10 demo poster (joint poster of IMF and ERM) ][[BR]] [attachment:GEC10_imf_cluster-d_review.pptx Presentation at GEC10 Cluster D][[BR]] [attachment:GEC11_imf_cluster-d_review.pptx Presentation at GEC11 Cluster D][[BR]] [attachment:GEC12_demo_poster.pdf GEC12 demo poster][[BR]] [attachment:GEC12_IMF_overview.pptx Presentation at GEC12 I&M D][[BR]] [attachment:GEC13_IMF_demo_poster.pdf GEC13 demo poster][[BR]] [attachment:GEC13_IMF_overview.pptx Presentation at GEC13 I&M D][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/1569237337.pdf "Architectural Support for Internet Evolution and Innovation"][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/ants_silo.pdf "A Composition Algorithm for the SILO Cross-Layer Optimization Service Architecture"][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/ICC-SILO-2007.pdf "The SILO Architecture for Services Integration, controL, and Optimization for the Future Internet"][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/milcom2007%20final.pdf "ON THE SUITABILITY OF COMPOSABLE SERVICES FOR THE ASSURABLE FUTURE INTERNET"][[BR]] [http://groups.geni.net/geni/attachment/wiki/IMF/paper.pdf "A Unified Software Architecture to Enable Cross-Layer Design in the Future Internet"][[BR]] === Software Releases === === Connectivity === See the [wiki:ORCABEN#Spiral2Connectivity ORCA/BEN project]