Changes between Version 12 and Version 13 of GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment

Timestamp:

10/22/14 10:11:33 (10 years ago)

Author:

mkeele@indiana.edu

Comment:

--

GEMINI/Tutorial/GEC21/GENI_Desktop_and_GEMINI_blipp/ExecuteExperiment

@@ -106 +106 @@
 }}}
  a. '''Task 3:''' Configure a !NetLogger probe to read the log file and graph the measurements.
+
+== __Exercise Task 2 -- Instrument existing application __ ==
+
+In this exercise, we will make use of a transparent !NetLogger wrapper than can intercept standard socket calls in existing applications.  This includes read()/write() and send()/recv().  When using the wrapper, you can specify an output file where the log messages are stored as well as the summary interval.  The wrapper uses a version of !NetLogger called NL-Calipers, which calculates and stores in-memory summary statistics over each event encountered.  The NL_INTERVAL
+variable determines how frequently those summary statistics are reported and the counters reset.
+
+The !NetLogger probe instructions above can be used to let BLiPP push the collected NL-Calipers measurements into the measurement store (MS) for graphing and archiving as we did in the previous exercise.  We will use 'iperf' as our test application to instrument with the wrapper library.  The iperf program should already be installed on the GEMINI intrumentized node.  If not, you can 'yum install iperf' or select another application to test with.  The following is an example of how to
+enable the wrapper using the Linux LD_PRELOAD mechanism (assumes default csh on default Fedora 15 images).
+
+{{{
+    setenv LD_PRELOAD /usr/local/lib/nl_wrapper.so
+    setenv NL_INTERVAL 1     # default is 1 second summaries
+    setenv NL_FILE /tmp/iperf.log
+
+    iperf -c VM-0 -t 120 -i 2
+    ------------------------------------------------------------
+    Client connecting to VM-0, TCP port 5001
+    TCP window size: 49.7 KByte (default)
+    ------------------------------------------------------------
+    [  4] local 10.10.1.1 port 49730 connected with 10.10.1.1 port 5001
+    [ ID] Interval       Transfer     Bandwidth
+    [  4]  0.0- 2.0 sec  5.65 GBytes  24.3 Gbits/sec
+    [  4]  2.0- 4.0 sec  6.48 GBytes  27.8 Gbits/sec
+    [  4]  4.0- 6.0 sec  6.22 GBytes  26.7 Gbits/sec
+    [  4]  6.0- 8.0 sec  6.74 GBytes  29.0 Gbits/sec
+    ...
+
+    tail -f /tmp/iperf.log
+    ts=2014-03-12T02:44:40.734545Z event=wrapper.calipers.write v.sum=2496921600.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=63101.880412 r.min=0.000654 r.max=5.041231 r.mean=3.312435 r.sd=0.628077 g.sum=7558.227539 g.min=0.198364 g.max=1528.854370 g.mean=0.396757 g.sd=11.074916 count=19050 dur=1.000013 dur.i=0.990672
+    ts=2014-03-12T02:44:41.734688Z event=wrapper.calipers.write v.sum=3468034048.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94498.685749 r.min=0.612486 r.max=5.041231 r.mean=3.571514 r.sd=0.395003 g.sum=7539.329529 g.min=0.198364 g.max=1.632690 g.mean=0.284944 g.sd=0.050552 count=26459 dur=1.000008 dur.i=0.988195
+    ts=2014-03-12T02:44:42.734783Z event=wrapper.calipers.write v.sum=3484024832.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=94586.291226 r.min=0.217728 r.max=4.854519 r.mean=3.558417 r.sd=0.267479 g.sum=7536.308289 g.min=0.205994 g.max=4.592896 g.mean=0.283522 g.sd=0.048784 count=26581 dur=1.000032 dur.i=0.987799
+    ts=2014-03-12T02:44:43.734906Z event=wrapper.calipers.write v.sum=3365928960.000000 v.min=131072.000000 v.max=131072.000000 v.mean=131072.000000 v.sd=0.000000 r.sum=89386.429279 r.min=0.422813 r.max=4.854519 r.mean=3.480780 r.sd=0.440131 g.sum=7541.175842 g.min=0.205994 g.max=2.365112 g.mean=0.293659 g.sd=0.055629 count=25680 dur=1.000017 dur.i=0.988437
+    ...
+}}}
+
+Once a BLiPP test has been configured for the given node, the event type drop down will contain entries for each of the statistics recorded by the !NetLogger wrapper.  (You will need to wait a minute for the collection to start and then refresh the page.)  In the iperf case, the client performs a number of write() calls to send data to the server. 
+
+Note that any additional applications started from a shell with the LD_PRELOAD environment variable set will invoke the intercept methods in the wrapper library.
+
+[[Image(wiki:GEMINI/Tutorial/Images:nl_wrapper_ets.png)]]
+
+Here is a summary of each statistic:
+
+* - sum 
+
+* - mean
+
+* - min
+
+* - max
+
+* - standard deviation.
+
+*
+
+* These statistics are tracked for: 
+
+*   - 'v': the recorded value
+
+*   - 'g': the ratio of the duration(ns)/value (prefix=g for gap)
+
+*   - 'r': the ratio of the value/duration(ns) (prefix=r for rate)