Changes between Version 14 and Version 15 of IMF-GEC10-QSR


Ignore:
Timestamp:
05/10/11 19:17:45 (8 years ago)
Author:
Rudra Dutta
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • IMF-GEC10-QSR

    v14 v15  
    1010
    1111 * Draft created by Rudra Dutta on March 22, 2011
    12  * Modified by Rudra Dutta March 22 - April ? 2011
    13  * Final version released April ? 2011
     12 * Modified by IMF team during March 22 - May 10, 2011
     13 * Final version released on May 10, 2011
    1414
    1515Upon consultation with GPO, the priorities of the project were revisited for Year 2 of this project. As a result, Year 2 (Spiral 3) goals were focused on integrating the developed IMF capabilities for measurement, and transfer of measurement (from substrate to in-slice stack), with existing GENI Inst&Meas capabilities; specifically, to integrate and install perfSONAR (pS) MPs and archives to collect performance measurement data from Polatis switches and Infinera DWDM platforms on 4 BEN sites, and (subsequently) integrate pS MP polling, or adopt other GENI I&M capabilities, into IMF to mediate the ability of in-slice reactive protocols to communicate back to actuators in the substrate.
     
    5151== Activities and Findings ==
    5252
     53As we mentioned above, the priorities of the IMF project in the three months previous to GEC10 has been on establishing the perfSONAR capability established in the Measurement Handler, and the capability demonstrated by querying it using a pS query.  In GEC10, we demonstrated this capability, and used the IMF Pub-Sub Module (PSM) to send a pS query to the MH and receive measurement data back.  The GEC10 snapshot of the IMF code (packaged below on this wiki) is shown below in terms of the system architecture at GEC8 in the diagram below.
     54
    5355[[Image(GEC10_setup_on_arch.png, 90%)]] [[BR]][[BR]][[BR]]
    5456
    55 The Java subscriber to the XMPP pub-sub service is not packaged in this release; the previously supplied code can, without change, be used (as part of a SILO service or standalone, as before), to extract the measurement data stored in the XMPP server.  The difference in this version is in how the IMF Pub-Sub Module acquires that data - it can now use a perfSONAR query to access the Measurement Handler (the IMF PSM can continue to reach the MH using an XML-RPC query as before as another alternative). 
     57The Java subscriber to the XMPP pub-sub service is not packaged in this release; the previously supplied code can, without change, be used (as part of a SILO service or standalone, as before), to extract the measurement data stored in the XMPP server.  The difference in this version is in how the IMF Pub-Sub Module acquires that data - it can now use a perfSONAR query to access the Measurement Handler (the IMF PSM can continue to reach the MH using an XML-RPC query as before as another alternative).
     58
     59In the MH, the pS capability is introduced by including a new pS service that we have called the "IMFRealTime" pS service.  (''Note:'' this name may be somewhat misleading and may change in subsequent releases; see discussion below.)  The IMFRealTime service is different from a normal Perfsonar service (MA or MP) in the sense that it implements a bit of both. It uses the mh Perl module and expands Perfsonar's configuration file to provide the MP service. However, instead of sending the results to an MA, the collected data are saved locally.
     60
     61The main reason behind this solution is that we want to avoid the delay introduced by adding an extra layer of database between the users and the MP. Consequently, the IMFRealTime service also needs to provide a user interface for real-time user queries in the absence of a standard Perfsonar MA.  In essence, the IMFRealTime service acts as a pS MP service (that uses previously developed IMF Measurement Handler capability to gather measurement data), but also looks like a pS MA service to external users who issue queries to obtain measurement data from the Measurement Archive.  In fact, internally, it uses shared memory as inter-process communication to access the most fresh data in response to user queries.  The shared memory acts as the rendezvous instead of the usual MA database.
     62
     63'''About "!RealTime" Nomenclature: ''' This behavior is obviously not actually "real-time" in terms of the time when measurement is made.  In contrast, the original XML-RPC approach is "real-time" in that measurements are undertaken (by the MH) in response to incoming user queries.  Rather, the behavior is "real-time" in only the sense that the IMFRealTime service looks for the most recent available data in response to user queries.  To avoid this potential confusion, we are considering re-naming the IMFRealTime pS service to something else in future releases, and use the term "real-time" exclusively to refer to cases where actual fresh measurements are undertaken once user queries are received.
     64
     65The developer notes, and other documentation included in the code below, provide more details regarding specifics.
    5666
    5767== Project Participants ==
     
    91101== Documentation and Release Notes ==
    92102
    93 The diagram above (in "Activities/Findings" section) shows where these tarballs fit in.  The pubsub_for_perfsonar.tar.gz file (PubSub for perfSONAR) replaces the imf.tar.gz file of previous releases: the internal file heirarchy is the same, but the top-level name is different: "imf_pfsr" instead of "imf"; this is a kluge that will be fixed in the next release.  As a consequence, this version of the code can '''''only''''' use pS query to reach the MH, the previous XML-RPC option is not available in this cut (the MH is still capable of it).
     103The diagram above (in "Activities/Findings" section) shows where these tarballs fit in.  The pubsub_for_perfsonar.tar.gz file (!PubSub for perfSONAR) replaces the imf.tar.gz file of previous releases: the internal file heirarchy is the same, but the top-level name is different: "imf_pfsr" instead of "imf"; this is a kluge that will be fixed in the next release.  As a consequence, this version of the code can '''''only''''' use pS query to reach the MH, the previous XML-RPC option is not available in this cut (the MH is still capable of it).
    94104
    95105The following developer's notes provide more detail which should be helpful in using this code.  This file is also included as "doc/README.imf" in the perfsonar_imf_realtime.tar.gz tarball.  In addition, the following diagram of the code organization may be helpful.