[Python-Dev] __del__ and tp_dealloc in the IO lib

[Guido van Rossum]

On Sun, Jan 18, 2009 at 11:49 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Sun, Jan 18, 2009 at 9:32 PM, Guido van Rossum <guido at python.org> wrote:
>> On Sun, Jan 18, 2009 at 5:38 PM, Gregory P. Smith <greg at krypto.org> wrote:
>>> +1 on getting rid of the IOBase __del__ in the C rewrite in favor of
>>> tp_dealloc.
>>>
>>> On Sun, Jan 18, 2009 at 11:53 PM, Christian Heimes <lists at cheimes.de> wrote:
>>>>
>>>> Brett Cannon schrieb:
>>>> > Fine by me. People should be using the context manager for guaranteed
>>>> > file closure anyway IMO.
>>>
>>> Yes they should.  (how I really really wish i didn't have to use 2.4 anymore
>>> ;)
>>
>> Come on, the open-try-use-finally-close idiom isn't *that* bad...
>>
>>> But lets at least be clear that is never acceptable for a python
>>> implementation to leak file descriptors/handles (or other system resources),
>>> they should be closed and released whenever the particular GC implementation
>>> gets around to it.
>>
>> I would like to make a stronger promise. I think that for files open
>> for *writing*, all data written to the file should be flushed to disk
>> before the fd is closed. This is the real reason for having the
>> __del__: closing the fd is done by the C implementation of FileIO, but
>> since (until the rewrite in C) the buffer management is all in Python
>> (both the binary I/O buffer and the additional text I/O buffer), I
>> felt the downside of having a __del__ method was preferable over the
>> possibility of output files not being flushed (which is always
>> nightmarish to debug).
>>
>> Of course, once both layers of buffering are implemented in C, the
>> need for __del__ to do this goes away, and I would be fine with doing
>> it all in tp_alloc.
>>
>>>> You make a very good point! Perhaps we should stop promising that files
>>>> get closed as soon as possible and encourage people in using the with
>>>> statement.
>>>>
>>>> Christian
>>>
>>> eegads, do we actually -promise- that somewhere?  If so I'll happily go
>>> update those docs with a caveat.
>>
>> I don't think we've promised that ever since the days when JPython
>> (with a P!) was young...
>>
>>> I regularly point out in code reviews that the very convenient and common
>>> idiom of open(name, 'w').write(data) doesn't guarantee when the file will be
>>> closed; its up to the GC implementation details.  Good code should never
>>> depend on the GC for a timely release of scarce external resources (file
>>> descriptors/handles).
>>
>> And buffer flushing. While I don't want to guarantee that the buffer
>> is flushed ASAP, I do want to continue promising that it is flushed
>> before the object is GC'ed and before the fd is closed.
>
> Could we add a warning if the file has not been explicitly flushed?
> Consider removing the implicit flush later, if there's a sufficient
> implementation benefit to it.

No, I really want to keep the implicit flush, even if it's hard for
the implementation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)