Changes between Version 12 and Version 13 of GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment


Ignore:
Timestamp:
10/22/14 10:11:33 (5 years ago)
Author:
mkeele@indiana.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment

    v12 v13  
    106106}}}
    107107 a. '''Task 3:''' Configure a !NetLogger probe to read the log file and graph the measurements.
     108
     109== __Exercise Task 2 -- Instrument existing application __ ==
     110
     111In this exercise, we will make use of a transparent !NetLogger wrapper than can intercept standard socket calls in existing applications.  This includes read()/write() and send()/recv().  When using the wrapper, you can specify an output file where the log messages are stored as well as the summary interval.  The wrapper uses a version of !NetLogger called NL-Calipers, which calculates and stores in-memory summary statistics over each event encountered.  The NL_INTERVAL
     112variable determines how frequently those summary statistics are reported and the counters reset.
     113
     114The !NetLogger probe instructions above can be used to let BLiPP push the collected NL-Calipers measurements into the measurement store (MS) for graphing and archiving as we did in the previous exercise.  We will use 'iperf' as our test application to instrument with the wrapper library.  The iperf program should already be installed on the GEMINI intrumentized node.  If not, you can 'yum install iperf' or select another application to test with.  The following is an example of how to
     115enable the wrapper using the Linux LD_PRELOAD mechanism (assumes default csh on default Fedora 15 images).
     116
     117{{{
     118    setenv LD_PRELOAD /usr/local/lib/nl_wrapper.so
     119    setenv NL_INTERVAL 1     # default is 1 second summaries
     120    setenv NL_FILE /tmp/iperf.log
     121
     122    iperf -c VM-0 -t 120 -i 2
     123    ------------------------------------------------------------
     124    Client connecting to VM-0, TCP port 5001
     125    TCP window size: 49.7 KByte (default)
     126    ------------------------------------------------------------
     127    [  4] local 10.10.1.1 port 49730 connected with 10.10.1.1 port 5001
     128    [ ID] Interval       Transfer     Bandwidth
     129    [  4]  0.0- 2.0 sec  5.65 GBytes  24.3 Gbits/sec
     130    [  4]  2.0- 4.0 sec  6.48 GBytes  27.8 Gbits/sec
     131    [  4]  4.0- 6.0 sec  6.22 GBytes  26.7 Gbits/sec
     132    [  4]  6.0- 8.0 sec  6.74 GBytes  29.0 Gbits/sec
     133    ...
     134
     135    tail -f /tmp/iperf.log
     136    ts=2014-03-12T02:44:40.734545Z event=wrapper.calipers.write v.sum=2496921600.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=63101.880412 r.min=0.000654 r.max=5.041231 r.mean=3.312435 r.sd=0.628077 g.sum=7558.227539 g.min=0.198364 g.max=1528.854370 g.mean=0.396757 g.sd=11.074916 count=19050 dur=1.000013 dur.i=0.990672
     137    ts=2014-03-12T02:44:41.734688Z event=wrapper.calipers.write v.sum=3468034048.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94498.685749 r.min=0.612486 r.max=5.041231 r.mean=3.571514 r.sd=0.395003 g.sum=7539.329529 g.min=0.198364 g.max=1.632690 g.mean=0.284944 g.sd=0.050552 count=26459 dur=1.000008 dur.i=0.988195
     138    ts=2014-03-12T02:44:42.734783Z event=wrapper.calipers.write v.sum=3484024832.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94586.291226 r.min=0.217728 r.max=4.854519 r.mean=3.558417 r.sd=0.267479 g.sum=7536.308289 g.min=0.205994 g.max=4.592896 g.mean=0.283522 g.sd=0.048784 count=26581 dur=1.000032 dur.i=0.987799
     139    ts=2014-03-12T02:44:43.734906Z event=wrapper.calipers.write v.sum=3365928960.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=89386.429279 r.min=0.422813 r.max=4.854519 r.mean=3.480780 r.sd=0.440131 g.sum=7541.175842 g.min=0.205994 g.max=2.365112 g.mean=0.293659 g.sd=0.055629 count=25680 dur=1.000017 dur.i=0.988437
     140    ...
     141}}}
     142
     143Once a BLiPP test has been configured for the given node, the event type drop down will contain entries for each of the statistics recorded by the !NetLogger wrapper.  (You will need to wait a minute for the collection to start and then refresh the page.)  In the iperf case, the client performs a number of write() calls to send data to the server.
     144
     145Note that any additional applications started from a shell with the LD_PRELOAD environment variable set will invoke the intercept methods in the wrapper library.
     146
     147[[Image(wiki:GEMINI/Tutorial/Images:nl_wrapper_ets.png)]]
     148
     149Here is a summary of each statistic:
     150
     151* - sum
     152
     153* - mean
     154
     155* - min
     156
     157* - max
     158
     159* - standard deviation.
     160
     161*
     162
     163* These statistics are tracked for:
     164
     165*   - 'v': the recorded value
     166
     167*   - 'g': the ratio of the duration(ns)/value (prefix=g for gap)
     168
     169*   - 'r': the ratio of the value/duration(ns) (prefix=r for rate)