automatic reclaiminig of limited resources (was Re: [Python-Dev] Product iteration)

[Peter Funk]

Hi all,

This rant is about automatically reclaimed resources like file descriptors.

First let me cite a few sentences from the ancient book "Programming Python"
written by Mark Lutz (p.122).  Since this was the very first available book 
on Python, it could be considered as some kind of Python Programmers Guide.
Much existing code today relies on the lessons teached there:

  """The short answer is that all files in this program are closed 
     automatically: there's no need to call the file object "close" method.
     [...]
     More specifically, Python uses a reference count garbage collection
     strategy to reclaim memory behind the scenes.
     [...]
     each time we set 'input' to a new file in the for loop above, its
     previous value can be reclaimed.
     [...]
     When a file object is reclaimed, the file is automatically closed
     [...]
     We usually don't need to close such files manually; 
     [...]
  """

A typical code example (simplified from a real application) :
>>> import os, rfc822
>>> posters = {}
>>> for arti in os.listdir('/var/spool/news/comp/lang/python'):
...     poster = rfc822.Message(open(arti)).get("From")    
...     posters[poster] = posters.get(poster, 0) + 1
... 
>>> for poster, howoften in posters.items():
...     print poster, "posted", howoften, "times to clp."
...

I'm in no way convinced to consider code like above as broken.  This simple
automatic and deterministic behaviour of C-Python is part of the
attractive elegance of Python and one of the major advantages over other
languages like Java, that IMO suck beans.

I've *NOT* tried scripts like the example above with JPython.  But if
JPython would run out of available file descriptors with such a simple
application, then I think JPython or the Java VM are broken.

Jeremy Hylton <jeremy@beopen.com>:
> The argument about finalization is specious.  You should not write
> code that depends on current finalization semantics to do things like
> closing files.  It's relying on an implementation-dependent feature
> that is not part of the language spec.  (Just try it on JPython.)

Eric S. Raymond" <esr@thyrsus.com>:
> I strongly agree.

Mark Hammond <mhammond@skippinet.com.au> (answering to Moshe):
> No - we will simply be pointing out their already broken code.  We arent
> breaking anything!
> 
> I have no problem with this at all.  The sooner we point out the broken
> code the better.  The last thing we want is for obviously and documented
> broken code to suddenly be considered non-broken simply by the volume of
> poor code out there...
> 
> Either way, I think you are overstating the problem.  The main scenario is
> closing resources, and for most users these are unlikely to be so scarce
> that delaying their finalization will cause problems.  If someone is
> writing a web-server, for example, and assumes that files or sockets are
> automatically closed, and they open them at such a rate that true GC causes
> them a problem, then IMO they have got what they deserve ;-)

no no no.

This time I have to agree with Moshe and disagree with Mark, Jeremy, Eric.
Or I've missed the point.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)