Changes between Version 15 and Version 16 of IMF-GEC10-QSR


Ignore:
Timestamp:
05/12/11 09:27:28 (13 years ago)
Author:
Rudra Dutta
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • IMF-GEC10-QSR

    v15 v16  
    1515Upon consultation with GPO, the priorities of the project were revisited for Year 2 of this project. As a result, Year 2 (Spiral 3) goals were focused on integrating the developed IMF capabilities for measurement, and transfer of measurement (from substrate to in-slice stack), with existing GENI Inst&Meas capabilities; specifically, to integrate and install perfSONAR (pS) MPs and archives to collect performance measurement data from Polatis switches and Infinera DWDM platforms on 4 BEN sites, and (subsequently) integrate pS MP polling, or adopt other GENI I&M capabilities, into IMF to mediate the ability of in-slice reactive protocols to communicate back to actuators in the substrate.
    1616
    17 At GEC10, the first of these (collecting performance data from Polatis using pS) capability has been completed. This leverages Spiral 3 goals of the ERM project team, who also form the Columbia U part of the IMF team.
     17At GEC10, the first of these (collecting performance data from Polatis and Infinera using pS) capability has been completed. This leverages Spiral 3 goals of the ERM project team, who also form the Columbia U part of the IMF team.
    1818
    19 In order to achieve this, the main effort was to collaborate with the perfSONAR development team to define a modified form of the pS client, which we embed in the IMFRealTime service, to allow the measurement to be obtained by the in-slice software; additionally, to create a pS Measurement Archive in which measurements from the substrate optical switches could be made available for the client.  The latter part has also been successfully integrated into the universal ERM box by the Columbia team, as part of the ERM project.
     19In order to achieve this, the main effort was to collaborate with the perfSONAR development team to define a modified form of the pS MP service, which we embed in the IMFRealTime service, to allow the measurement to be obtained using the IMF MH from the substrate optical switches and other equipment; additionally, to create a lightweight and locally coupled pS Measurement Archive in which measurements could be made available for the client.  The lightweight pS code and architecture developed has also been successfully integrated into the universal ERM box by the Columbia team, as part of the ERM project.
    2020
    21 This is running on 6 of the 8 switches of the BEN testbed in North Carolina, and the IMF capability to query the pS archive serving those switches was demonstrated at GEC10.
     21This is running on 6 of the 8 optical equipments of the BEN testbed in North Carolina, and the IMF capability to query the pS archive serving those switches was demonstrated at GEC10.
    2222
    2323[[Image(wiki:IMF:perfsonar_in_imf.png, 50%)]]
    … …  
    5353As we mentioned above, the priorities of the IMF project in the three months previous to GEC10 has been on establishing the perfSONAR capability established in the Measurement Handler, and the capability demonstrated by querying it using a pS query.  In GEC10, we demonstrated this capability, and used the IMF Pub-Sub Module (PSM) to send a pS query to the MH and receive measurement data back.  The GEC10 snapshot of the IMF code (packaged below on this wiki) is shown below in terms of the system architecture at GEC8 in the diagram below.
    5454
    55 [[Image(GEC10_setup_on_arch.png, 90%)]] [[BR]][[BR]][[BR]]
     55[[Image(GEC10_setup_on_arch.png, 90%)]] [[BR]]
    5656
    5757The Java subscriber to the XMPP pub-sub service is not packaged in this release; the previously supplied code can, without change, be used (as part of a SILO service or standalone, as before), to extract the measurement data stored in the XMPP server.  The difference in this version is in how the IMF Pub-Sub Module acquires that data - it can now use a perfSONAR query to access the Measurement Handler (the IMF PSM can continue to reach the MH using an XML-RPC query as before as another alternative).
    … …  
    6161The main reason behind this solution is that we want to avoid the delay introduced by adding an extra layer of database between the users and the MP. Consequently, the IMFRealTime service also needs to provide a user interface for real-time user queries in the absence of a standard Perfsonar MA.  In essence, the IMFRealTime service acts as a pS MP service (that uses previously developed IMF Measurement Handler capability to gather measurement data), but also looks like a pS MA service to external users who issue queries to obtain measurement data from the Measurement Archive.  In fact, internally, it uses shared memory as inter-process communication to access the most fresh data in response to user queries.  The shared memory acts as the rendezvous instead of the usual MA database.
    6262
    63 '''About "!RealTime" Nomenclature: ''' This behavior is obviously not actually "real-time" in terms of the time when measurement is made.  In contrast, the original XML-RPC approach is "real-time" in that measurements are undertaken (by the MH) in response to incoming user queries.  Rather, the behavior is "real-time" in only the sense that the IMFRealTime service looks for the most recent available data in response to user queries.  To avoid this potential confusion, we are considering re-naming the IMFRealTime pS service to something else in future releases, and use the term "real-time" exclusively to refer to cases where actual fresh measurements are undertaken once user queries are received.
     63'''About "!RealTime" Nomenclature: ''' This behavior is obviously not actually "real-time" in terms of the time when measurement is made.  In contrast, the original XML-RPC approach is "real-time" in that measurements are "triggered", or undertaken (by the MH) in response to incoming user queries.  Rather, the behavior is "real-time" in only the sense that the IMFRealTime service looks for the most recent available data in response to user queries.  To avoid this potential confusion, we are considering re-naming the IMFRealTime pS service to something else in future releases, and use the term "real-time" exclusively to refer to cases where actual fresh measurements are undertaken once user queries are received.  In the future, we hope to investigate the possibility of triggered real-time measurements using the pS pathway as well; at this time, it appears this may not be a suitable match, since the pS architecture relies on static configuration file for measurements, and thus clearly separates the data query from collection.
    6464
    6565The developer notes, and other documentation included in the code below, provide more details regarding specifics.
    … …  
    103103The diagram above (in "Activities/Findings" section) shows where these tarballs fit in.  The pubsub_for_perfsonar.tar.gz file (!PubSub for perfSONAR) replaces the imf.tar.gz file of previous releases: the internal file heirarchy is the same, but the top-level name is different: "imf_pfsr" instead of "imf"; this is a kluge that will be fixed in the next release.  As a consequence, this version of the code can '''''only''''' use pS query to reach the MH, the previous XML-RPC option is not available in this cut (the MH is still capable of it).
    104104
    105 The following developer's notes provide more detail which should be helpful in using this code.  This file is also included as "doc/README.imf" in the perfsonar_imf_realtime.tar.gz tarball.  In addition, the following diagram of the code organization may be helpful.
     105The following developer's notes on the IMFRealTime pS service provide more detail which should be helpful in using this code.  This file is also included as "doc/README.imf" in the perfsonar_imf_realtime.tar.gz tarball.  In addition, the following diagram of the code organization for the PubSub module may be helpful.
    106106
    107107 * [attachment:release_notes.txt Developer Release Notes]