[pypy-dev] Pluggable HTM

[Taavi Burns]

My gut says that HTM would be something used to help make STM commits
faster (if that's possible), not to replace PyPy's STM machinery
entirely.

And maybe with something like CPython, one could replace the GIL
entirely with HTM, but you'd probably want to make the "ticks between
releases" a lot shorter to reduce the chance of conflicts.

Things to try in my Copious Amounts of Free Time. :/


On Tue, Dec 3, 2013 at 1:47 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Dimitri,
>
> On Wed, Nov 27, 2013 at 9:17 AM, Dimitri Vorona <alendit at googlemail.com> wrote:
>> the original STM proposal spoke of HTM as of a thing of a far future. Now,
>> Haswells are out and provide built-in HTM support in form of TSX. In the
>> near future I expect more and more systems to have it.
>>
>> Are there plan to make PyPy use HTM if it is available on the system?
>
> I don't know yet.  I've just started playing with an Intel Haswell,
> and getting slightly bad results in the form of too many random
> transaction aborts.
>
> This seems so for both "small" transactions that only access some 20KB
> of data, up to larger transaction of almost 768KB, which is
> impressively three times the size of the L2 cache; this seems to say
> that even the L3 cache can dedicate a part of its resources to storing
> the transaction cache lines.
>
> But a naive extrapolation of the single-threaded results shows that,
> if we had instead 8 threads running with the same results, even on
> completely independent data, they would still abort too many
> transactions each.  Whenever a transaction needs to be redone without
> HTM, it really needs to stop all other threads.  So "too many" is in
> this sense: even if it is only 10-20% on each core, it's enough to
> prevent any scaling beyond just a coupe of cores.
>
> It may be that I'm missing something, like a way to learn where
> conflicts occur.  But all in all it is unclear if this is good enough
> for PyPy (or CPython).  The next step, which I might do anyway, would
> be to extract from the pypy-stm branch the general logic (most notably
> the numerous conflict-avoiding small changes), and try to run that
> with HTM.  This probably requires writing a different GC, but it
> should be easy at this point to do, experimentally.
>
>
> A bientôt,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev



-- 
taa
/*eof*/