[Python-Dev] __del__ and tp_dealloc in the IO lib

[Adam Olsen]

On Sun, Jan 18, 2009 at 9:32 PM, Guido van Rossum <guido at python.org> wrote:
> On Sun, Jan 18, 2009 at 5:38 PM, Gregory P. Smith <greg at krypto.org> wrote:
>> +1 on getting rid of the IOBase __del__ in the C rewrite in favor of
>> tp_dealloc.
>>
>> On Sun, Jan 18, 2009 at 11:53 PM, Christian Heimes <lists at cheimes.de> wrote:
>>>
>>> Brett Cannon schrieb:
>>> > Fine by me. People should be using the context manager for guaranteed
>>> > file closure anyway IMO.
>>
>> Yes they should.  (how I really really wish i didn't have to use 2.4 anymore
>> ;)
>
> Come on, the open-try-use-finally-close idiom isn't *that* bad...
>
>> But lets at least be clear that is never acceptable for a python
>> implementation to leak file descriptors/handles (or other system resources),
>> they should be closed and released whenever the particular GC implementation
>> gets around to it.
>
> I would like to make a stronger promise. I think that for files open
> for *writing*, all data written to the file should be flushed to disk
> before the fd is closed. This is the real reason for having the
> __del__: closing the fd is done by the C implementation of FileIO, but
> since (until the rewrite in C) the buffer management is all in Python
> (both the binary I/O buffer and the additional text I/O buffer), I
> felt the downside of having a __del__ method was preferable over the
> possibility of output files not being flushed (which is always
> nightmarish to debug).
>
> Of course, once both layers of buffering are implemented in C, the
> need for __del__ to do this goes away, and I would be fine with doing
> it all in tp_alloc.
>
>>> You make a very good point! Perhaps we should stop promising that files
>>> get closed as soon as possible and encourage people in using the with
>>> statement.
>>>
>>> Christian
>>
>> eegads, do we actually -promise- that somewhere?  If so I'll happily go
>> update those docs with a caveat.
>
> I don't think we've promised that ever since the days when JPython
> (with a P!) was young...
>
>> I regularly point out in code reviews that the very convenient and common
>> idiom of open(name, 'w').write(data) doesn't guarantee when the file will be
>> closed; its up to the GC implementation details.  Good code should never
>> depend on the GC for a timely release of scarce external resources (file
>> descriptors/handles).
>
> And buffer flushing. While I don't want to guarantee that the buffer
> is flushed ASAP, I do want to continue promising that it is flushed
> before the object is GC'ed and before the fd is closed.

Could we add a warning if the file has not been explicitly flushed?
Consider removing the implicit flush later, if there's a sufficient
implementation benefit to it.


-- 
Adam Olsen, aka Rhamphoryncus