[pypy-dev] Pypy garbage collection

[Martin Koch]

Thanks, Carl.

This bit of code certainly exhibits the surprising property that some runs
unpredictably stall for a very long time. Further, it seems that this stall
time can be made arbitrarily large by increasing the number of nodes
generated (== more data in the old generation == more stuff to traverse if
lots of garbage is generated and survives the young generation?). As a user
of an incremental garbage collector, I would expect that there are pauses
due to GC, but that these are predictable and small.

I tried running

PYPY_GC_NURSERY=2000M pypy ./mem.py 10000000

but that seemed to have no effect.

I'm looking forward to the results of the Software Transactional Memory,
btw :)

/Martin


On Tue, Mar 18, 2014 at 9:47 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:

> On 17/03/14 20:04, Martin Koch wrote:
> > Well, it would appear that we have the problem because we're generating
> > a lot of garbage in the young generation, just like we're doing in the
> > example we've been studying here.
>
> No, I think it's because your generating a lot of garbage in the *old*
> generation. Meaning objects which survive one minor collection but then
> die.
>
> > I'm unsure how we can avoid that in
> > our real implementation. Can we force gc of the young generation? Either
> > by gc.collect() or implcitly somehow (does the gc e.g. kick in across
> > function calls?).
>
> That would make matters worse, because increasing the frequency of
> minor collects means *more* objects get moved to the old generation
> (where they cause problems). So indeed, maybe in your case making the
> new generation bigger might help. This can be done using
> PYPY_GC_NURSERY, I think (nursery is the space reserved for young
> objects). The risk is that minor collections become unreasonably slow.
>
> Anyway, if the example code you gave us also shows the problem I think
> we should eventually look into it. It's not really fair to say "but
> you're allocating too much!" to explain why the GC takes a lot of time.
>
> Cheers,
>
> Carl Friedrich
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20140318/9fe12e5f/attachment.html>