Changes between Version 11 and Version 12 of GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment


Ignore:
Timestamp:
10/22/14 09:23:17 (5 years ago)
Author:
mkeele@indiana.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment

    v11 v12  
    106106}}}
    107107 a. '''Task 3:''' Configure a !NetLogger probe to read the log file and graph the measurements.
    108 
    109 == __Exercise Task 2 -- Instrument existing application __ ==
    110 
    111 In this exercise, we will make use of a transparent !NetLogger wrapper than can intercept standard socket calls in existing applications.  This includes read()/write() and send()/recv().  When using the wrapper, you can specify an output file where the log messages are stored as well as the summary interval.  The wrapper uses a version of !NetLogger called NL-Calipers, which calculates and stores in-memory summary statistics over each event encountered.  The NL_INTERVAL variable determines how frequently those summary statistics are reported and the counters reset.
    112 
    113 The !NetLogger probe instructions above can be used to let BLiPP push the collected NL-Calipers measurements into the measurement store (MS) for graphing and archiving as we did in the previous exercise.  We will use 'iperf' as our test application to instrument with the wrapper library.  The iperf program should already be installed on the GEMINI intrumentized node.  If not, you can 'yum install iperf' or select another application to test with.  The following is an example of how to enable the wrapper using the Linux LD_PRELOAD mechanism (assumes default csh on default Fedora 15 images).
    114 
    115 {{{
    116 setenv LD_PRELOAD /usr/local/lib/nl_wrapper.so
    117 setenv NL_INTERVAL 1     # default is 1 second summaries
    118 setenv NL_FILE /tmp/iperf.log
    119 
    120 iperf -c VM-0 -t 120 -i 2
    121 ------------------------------------------------------------
    122 Client connecting to VM-0, TCP port 5001
    123 TCP window size: 49.7 KByte (default)
    124 ------------------------------------------------------------
    125 [  4] local 10.10.1.1 port 49730 connected with 10.10.1.1 port 5001
    126 [ ID] Interval       Transfer     Bandwidth
    127 [  4]  0.0- 2.0 sec  5.65 GBytes  24.3 Gbits/sec
    128 [  4]  2.0- 4.0 sec  6.48 GBytes  27.8 Gbits/sec
    129 [  4]  4.0- 6.0 sec  6.22 GBytes  26.7 Gbits/sec
    130 [  4]  6.0- 8.0 sec  6.74 GBytes  29.0 Gbits/sec
    131 ...
    132 
    133 tail -f /tmp/iperf.log
    134 ts=2014-03-12T02:44:40.734545Z event=wrapper.calipers.write v.sum=2496921600.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=63101.880412 r.min=0.000654 r.max=5.041231 r.mean=3.312435 r.sd=0.628077 g.sum=7558.227539 g.min=0.198364 g.max=1528.854370 g.mean=0.396757 g.sd=11.074916 count=19050 dur=1.000013 dur.i=0.990672
    135 ts=2014-03-12T02:44:41.734688Z event=wrapper.calipers.write v.sum=3468034048.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94498.685749 r.min=0.612486 r.max=5.041231 r.mean=3.571514 r.sd=0.395003 g.sum=7539.329529 g.min=0.198364 g.max=1.632690 g.mean=0.284944 g.sd=0.050552 count=26459 dur=1.000008 dur.i=0.988195
    136 ts=2014-03-12T02:44:42.734783Z event=wrapper.calipers.write v.sum=3484024832.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94586.291226 r.min=0.217728 r.max=4.854519 r.mean=3.558417 r.sd=0.267479 g.sum=7536.308289 g.min=0.205994 g.max=4.592896 g.mean=0.283522 g.sd=0.048784 count=26581 dur=1.000032 dur.i=0.987799
    137 ts=2014-03-12T02:44:43.734906Z event=wrapper.calipers.write v.sum=3365928960.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=89386.429279 r.min=0.422813 r.max=4.854519 r.mean=3.480780 r.sd=0.440131 g.sum=7541.175842 g.min=0.205994 g.max=2.365112 g.mean=0.293659 g.sd=0.055629 count=25680 dur=1.000017 dur.i=0.988437
    138 ...
    139 }}}
    140 
    141 Once a BLiPP test has been configured for the given node, the event type drop down will contain entries for each of the statistics recorded by the !NetLogger wrapper.  (You will need to wait a minute for the collection to start and then refresh the page.)  In the iperf case, the client performs a number of write() calls to send data to the server.
    142 
    143 Note that any additional applications started from a shell with the LD_PRELOAD environment variable set will invoke the intercept methods in the wrapper library.
    144 
    145 [[Image(wiki:GEMINI/Tutorial/Images:nl_wrapper_ets.png)]]
    146 
    147 Here is a summary of each statistic:
    148                                                                                                                                                                                                                                                        
    149  * - sum                                                                                                                                         
    150  * - mean                                                                                                                                       
    151  * - min                                                                                                                                         
    152  * - max                                                                                                                                         
    153  * - standard deviation.                                                                                                                         
    154  *                                                                                                                                               
    155  * These statistics are tracked for:                                                                                                             
    156  *   - 'v': the recorded value                                                                                                                 
    157  *   - 'g': the ratio of the duration(ns)/value (prefix=g for gap)                                                                                   
    158  *   - 'r': the ratio of the value/duration(ns) (prefix=r for rate)