python shutting down sloooooooowly/tuning dictionaries

[Till Plewe]

On Wed, Apr 21, 2004 at 06:55:24PM +0200, Fredrik Lundh wrote:
> > have you tried using
> >
> >     os._exit()
> >
> > instead of sys.exit()
> 
> better make that
> 
>     os._exit(0)
> 
> </F>
> 

and

On Wed, Apr 21, 2004 at 12:54:14PM -0400, Tim Peters wrote:
> [Till]
> > Thanks for the suggestions but unfortunately I already tried disabling
> > gc. It did not have any noticable effect (not that I was patient
> > enough to time it) nor did using sys.exit().
>
> sys.exit() doesn't skip cleanup.  os._exit() does, but you use that at your
> own risk (it's probably fine, but you've been warned).
>

Thanks. Using os._exit() works.

> > ...
> > To give an example of the actual time spans involved: 2-10 minutes for
> > running the program and > 30 minutes for shutdown. During that time
> > the amount of memory used by python does not seem to vary.
>
> This sounds like the platform C free() falling into quadratic-time behavior.
> You didn't say which version of Python you're using, or which OS + C
> runtime, and "stuff like that" varies according to both.  This kind of
> behavior is generally much rarer under Python 2.3 than under earlier
> Pythons -- but if your platform C malloc/free suck, there's not much more
> Python can do about that than 2.3 already does.
>

I am using Python 2.3 and 2.4, {Gentoo,Suse}Linux+gcc3.3,
{Free,Net}BSD + gcc3.3, gcc2.95, but I don't remember which combination
is the worst offender. Slow shutdowns did occur at least with both Suse9.0 
and FreeBSD5.2 on amd64. I will try some other combinations.

Thanks again.

- Till