[Python-Dev] __del__ and tp_dealloc in the IO lib

[Guido van Rossum]

On Sun, Jan 18, 2009 at 5:38 PM, Gregory P. Smith <greg at krypto.org> wrote:
> +1 on getting rid of the IOBase __del__ in the C rewrite in favor of
> tp_dealloc.
>
> On Sun, Jan 18, 2009 at 11:53 PM, Christian Heimes <lists at cheimes.de> wrote:
>>
>> Brett Cannon schrieb:
>> > Fine by me. People should be using the context manager for guaranteed
>> > file closure anyway IMO.
>
> Yes they should.  (how I really really wish i didn't have to use 2.4 anymore
> ;)

Come on, the open-try-use-finally-close idiom isn't *that* bad...

> But lets at least be clear that is never acceptable for a python
> implementation to leak file descriptors/handles (or other system resources),
> they should be closed and released whenever the particular GC implementation
> gets around to it.

I would like to make a stronger promise. I think that for files open
for *writing*, all data written to the file should be flushed to disk
before the fd is closed. This is the real reason for having the
__del__: closing the fd is done by the C implementation of FileIO, but
since (until the rewrite in C) the buffer management is all in Python
(both the binary I/O buffer and the additional text I/O buffer), I
felt the downside of having a __del__ method was preferable over the
possibility of output files not being flushed (which is always
nightmarish to debug).

Of course, once both layers of buffering are implemented in C, the
need for __del__ to do this goes away, and I would be fine with doing
it all in tp_alloc.

>> You make a very good point! Perhaps we should stop promising that files
>> get closed as soon as possible and encourage people in using the with
>> statement.
>>
>> Christian
>
> eegads, do we actually -promise- that somewhere?  If so I'll happily go
> update those docs with a caveat.

I don't think we've promised that ever since the days when JPython
(with a P!) was young...

> I regularly point out in code reviews that the very convenient and common
> idiom of open(name, 'w').write(data) doesn't guarantee when the file will be
> closed; its up to the GC implementation details.  Good code should never
> depend on the GC for a timely release of scarce external resources (file
> descriptors/handles).

And buffer flushing. While I don't want to guarantee that the buffer
is flushed ASAP, I do want to continue promising that it is flushed
before the object is GC'ed and before the fd is closed.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)