[pypy-commit] extradoc extradoc: again change the metric for explaining memory usage (not sure what's best)

[Raemi]

Author: Remi Meier <remi.meier at inf.ethz.ch>
Branch: extradoc
Changeset: r5275:082595a5aacc
Date: 2014-05-27 16:41 +0200
http://bitbucket.org/pypy/extradoc/changeset/082595a5aacc/

Log:	again change the metric for explaining memory usage (not sure what's
	best)

diff --git a/talk/dls2014/paper/paper.tex b/talk/dls2014/paper/paper.tex
--- a/talk/dls2014/paper/paper.tex
+++ b/talk/dls2014/paper/paper.tex
@@ -377,11 +377,9 @@
 propagated to all segments.
 
 \begin{figure*}[t]
-  \begin{centering}
-    \includegraphics[scale=0.8]{\string"segment addressing\string".pdf}
-    \par\end{centering}
-
-    \protect\caption{Segment Addressing\label{fig:Segment-Addressing}}
+  \centering
+  \includegraphics[scale=0.8]{\string"segment addressing\string".pdf}
+  \caption{Segment Addressing\label{fig:Segment-Addressing}}
 \end{figure*}
 
 
@@ -404,11 +402,9 @@
 require one page of physical memory for all of them.
 
 \begin{figure}[h]
-  \begin{centering}
-    \includegraphics[scale=0.8]{\string"mmap pages\string".pdf}
-    \par\end{centering}
-
-    \protect\caption{\texttt{mmap()} Page Mapping\label{fig:mmap()-Page-Mapping}}
+  \centering
+  \includegraphics[scale=0.8]{\string"mmap pages\string".pdf}
+  \caption{\texttt{mmap()} Page Mapping\label{fig:mmap()-Page-Mapping}}
 \end{figure}
 
 
@@ -445,13 +441,11 @@
 memory efficient again.
 
 \begin{figure}[h]
-  \begin{centering}
-    \includegraphics[width=1\columnwidth]{\string"page remapping\string".pdf}
-    \par\end{centering}
-
-    \protect\caption{Page Remapping: (I) after \texttt{mmap()}. (II) remap all pages to
-      segment~0, fully shared memory configuration. (III) privatise single
-      pages.\label{fig:Page-Remapping}}
+  \centering
+  \includegraphics[width=1\columnwidth]{\string"page remapping\string".pdf}
+  \caption{Page Remapping: (I) after \texttt{mmap()}. (II) remap all pages to
+    segment~0, fully shared memory configuration. (III) privatise single
+    pages.\label{fig:Page-Remapping}}
 \end{figure}
 
 
@@ -641,11 +635,9 @@
 
 
 \begin{figure*}[t]
-  \begin{centering}
-    \includegraphics[scale=0.8]{\string"segment layout\string".pdf}
-    \par\end{centering}
-
-    \protect\caption{Segment Layout\label{fig:Segment-Layout}}
+  \centering
+  \includegraphics[scale=0.8]{\string"segment layout\string".pdf}
+  \caption{Segment Layout\label{fig:Segment-Layout}}
 \end{figure*}
 
 
@@ -1003,12 +995,12 @@
 space. Thus, it can be at most the total size of memory used by
 old objects divided by 16.
 
-Third, since we do copy on write, if all objects were to be
-written to at the same time, all pages would need to be privatised
-for all objects in the old object space. In that case we would
-need the total amount of memory required by old objects multiplied
-by $N+1$ (incl. the sharing segment). Pages get re-shared during
-major collections if possible.
+Third, since we do copy on write, if all objects were to be written to
+at the same time, all pages would need to be privatised for all
+objects in the old object space. In that case we would need the total
+amount of memory required by old objects multiplied by $N+1$
+(incl. the sharing segment). During major collections we re-share the
+pages if possible.
 
 In figure \ref{fig:richards_mem} we look at the memory usage of one of
 our benchmarks called Richards\footnote{OS kernel simulation
@@ -1017,12 +1009,21 @@
 sharp drops in memory usage come from major collections that free old
 objects and re-share pages. The average memory usage stays around
 29~MiB and we see that in this benchmark we have around 1 major
-collection every second. The \emph{page privatisation}, which
-represents the percentage of used pages with at least one private
-copy, shows the same spikes as the memory usage. These come directly from
-re-sharing the pages. The maximum page privatisation is around $20\%$
-between major collections. Thus we can say that $~20\%$ of the old
-objects get modified between collections in this benchmark.
+collection every second.
+
+The \emph{page privatisation}, which represents the ratio between
+private pages and shared pages, gives us an idea about how many
+private copies exist for each shared page. If the ratio is $>1.0$, it
+means that on average we have more than one private copy for each
+shared page. In the worst case described above, this number would
+reach $N$ and means we need $N\times$ the memory for the private
+pages alone. In this benchmark, we see the same spikes as the memory
+usage. These come directly from re-sharing the pages. The maximum page
+privatisation is around $0.5$ between major collections, thus the
+private pages are responsible for a $50\%$ increase in the required
+memory. Since the spikes in the GC managed memory line actually
+show increases by $~80\%$, it means that the rest comes from actual
+garbage objects that were collected.
 
 For PyPy-STM the average memory requirement is 29~MiB and there are
 $\sim 11$ major collections during the runtime. Normal PyPy with a GIL
@@ -1044,6 +1045,7 @@
 \end{figure}
 
 
+
 \subsection{Overhead Breakdown}
 
 \remi{do it on a non-jit build (see reason above)}
diff --git a/talk/dls2014/paper/plots/plot_richards_mem.py b/talk/dls2014/paper/plots/plot_richards_mem.py
--- a/talk/dls2014/paper/plots/plot_richards_mem.py
+++ b/talk/dls2014/paper/plots/plot_richards_mem.py
@@ -34,7 +34,7 @@
         xs.append(float(time) - first_time)
         real_mem, max_rss, page_util = mems.split("/")
         y1s.append(int(real_mem) / 1024. / 1024)
-        y2s.append(float(page_util) * 100)
+        y2s.append(float(page_util))
 
 # RSS:
 # x2s = range(12)
@@ -48,8 +48,8 @@
 def plot_mems(ax, ax2):
     print sum(y1s) / len(xs)
     print sum(y2s) / len(xs)
-    a, = ax.plot(xs, y1s, 'b-')
-    b, = ax2.plot(xs, y2s, 'r-')
+    a, = ax.plot(xs, y1s, 'b-o', ms=3)
+    b, = ax2.plot(xs, y2s, 'r-x', ms=3)
     return ax.legend((a, b),
                      ('GC managed memory', 'Page privatisation'))
 
@@ -64,12 +64,12 @@
 
     ax.set_ylabel("Memory [MiB]", color='b')
     ax.set_xlabel("Runtime [s]")
-    ax.set_xlim(-0.5, 11.5)
     ax.set_ylim(0, 50)
 
     ax2 = ax.twinx()
-    ax2.set_ylim(0, 100)
-    ax2.set_ylabel("\% of pages with $>1$ private copy",
+    ax.set_xlim(-0.5, 11.8)
+    ax2.set_ylim(0, 1)
+    ax2.set_ylabel("Ratio = ${private~pages}\over{shared~pages}$",
                    color='r')
     legend = plot_mems(ax, ax2)
 
diff --git a/talk/dls2014/paper/plots/richards_mem.pdf b/talk/dls2014/paper/plots/richards_mem.pdf
index 17e9e4ae371aeb31f4e7578a35b7ab0ebb9a775c..c2e6b5c0924fe38ec35ab1a467cda4f6c3810450
GIT binary patch

[cut]