Tremendous slowdown due to garbage collection

[Dieter Maurer]

John Nagle <nagle at animats.com> writes on Mon, 28 Apr 2008 11:41:41 -0700:
> Dieter Maurer wrote:
> > Christian Heimes <lists at cheimes.de> writes on Sat, 12 Apr 2008 18:47:32 +0200:
> >> andreas.eisele at gmail.com schrieb:
> >>> which made me suggest to use these as defaults, but then
> 
> > We observed similar very bad behaviour -- in a Web application server.
> > Apparently, the standard behaviour is far from optimal when the
> > system contains a large number of objects and occationally, large
> > numbers of objects are created in a short time.
> > We have seen such behaviour during parsing of larger XML documents, for
> > example (in our Web application).
> 
>     Our solution to that was to modify BeautifulSoup to use weak pointers.
> All the pointers towards the root and towards previous parts of the
> document are "weak".  As a result, reference counting alone is sufficient
> to manage the tree.  We still keep GC enabled, but it doesn't find much
> to collect.

It will not help in our setup.

We, too, have almost no cycles -- but the GC does not know this:

  If a large number of objects are created temporarily and not released
  before the generation 1 threshoold is reached, then
  the garbage collector will start collections -- even, if there
  are no or very few cycles.
  A generation 2 garbage collection takes time proportional
  to the total number of (GC aware) objects -- independent of
  the number of cycles.

Dieter