[pypy-commit] extradoc extradoc: more details
arigo
noreply at buildbot.pypy.org
Mon Mar 30 16:05:27 CEST 2015
Author: Armin Rigo <arigo at tunes.org>
Branch: extradoc
Changeset: r5520:eb519f972b4f
Date: 2015-03-30 16:05 +0200
http://bitbucket.org/pypy/extradoc/changeset/eb519f972b4f/
Log: more details
diff --git a/blog/draft/stm-mar2015.txt b/blog/draft/stm-mar2015.txt
--- a/blog/draft/stm-mar2015.txt
+++ b/blog/draft/stm-mar2015.txt
@@ -13,10 +13,10 @@
http://pypy.readthedocs.org/en/latest/stm.html
This is a special version of PyPy that contains the "Software
-Transactional Memory" (STM) core called stmgc-c7. It gives a
+Transactional Memory" (STM) plug-in called stmgc-c7. It gives a
replacement for Python's classical Global Interpreter Lock (GIL). The
current version scales only up to around 4 cores; the next version of
-the core, stmgc-c8, is in development and should address that
+the plug-in, stmgc-c8, is in development and should address that
limitation. Both versions only support 64-bit Linux for now
(contributions welcome).
@@ -46,24 +46,28 @@
get very far. It probably felt like a toy: on very small examples it
would nicely scale, but on any larger example it would not scale at
all. You didn't get any feedback about why, but the underlying reason
-for that is that, in any large example, there are a few unexpected STM
-conflicts that occur all the time. This prevents any parallelization.
+is that, in a typical large example, there are some STM conflicts that
+occur all the time and that won't be immediately found just by
+thinking. This prevents any parallelization.
Now PyPy-STM is no longer a black box: you have a way to learn about
-these conflicts and fix them. The tl;dr version is to run::
+these conflicts, fix them, and try again. The tl;dr version is to run::
- PYPYSTM=stmlog pypy-stm yourprogr.py
- print_stm_log.py stmlog
+ PYPYSTM=stmlog ./pypy-stm yourprogr.py
+ ./print_stm_log.py stmlog
+
+More details in `the STM user guide`_.
Performance
-----------
The performance is now more stable than it used to be. More
-precisely, the best case is "25%-40% single-core slow-down with
-very good scaling up to 4 threads", and the average performance seems
+precisely, the best case is still "25%-40% single-core slow-down with
+very good scaling up to 4 threads", but the average performance seems
not too far from that. There are still dark spots --- notably, the
-JIT is still slower to warm up, though it was improved a lot. Apart from
+JIT is still slower to warm up, though it was improved a lot. These
+are documented in the `current status`_ section. Apart from
that, we should not get more than 2x single-core slow-down in the
worst case. Please report such cases as bugs!
@@ -71,15 +75,15 @@
TransactionQueue
----------------
-PyPy-STM is more than "just" a Python without GIL. It is a Python in
-which you can do minor tweaks to your existing, *non-multithreaded*
-programs and get them to use multiple cores. The basic idea is to
-identify medium- or large-sized, likely-independent parts of the code
-and to ask PyPy-STM to run these parts in parallel. An example would be
-every iteration of some outermost loop over all items of a dictionary.
-This is done with a ``transaction.TransactionQueue`` instance. See
-``help(TransactionQueue)`` or read more about it in `the STM user
-guide`_.
+As explained before, PyPy-STM is more than "just" a Python without
+GIL. It is a Python in which you can do minor tweaks to your
+existing, *non-multithreaded* programs and get them to use multiple
+cores. You identify medium- or large-sized, likely-independent parts
+of the code and to ask PyPy-STM to run these parts in parallel. An
+example would be every iteration of some outermost loop over all items
+of a dictionary. This is done with a new API:
+``transaction.TransactionQueue()``. See ``help(TransactionQueue)`` or
+read more about it in `the STM user guide`_.
This is not a 100% mechanical change: very likely, you need to hunt
for and fix "STM conflicts" that prevent parallel execution (see
More information about the pypy-commit
mailing list