[issue3531] file read preallocs 'size' bytes which can cause memory problems

[Andrew Dalke]

Andrew Dalke <dalke at dalkescientific.com> added the comment:

I'm still undecided on if this is a bug or not.  The problem occurs even 
when I'm not reading "data from a file of an unknown size."  My example 
causes a MemoryError on my machine even though the file I'm reading 
contains 0 bytes.

The problem is Python's implementation is "alloc the requested bytes and 
truncate if needed" vs what I expected "read chunks at a time up to the 
requested number of bytes."  There's nothing in the documentation which 
states the implementation, although "Note that this method may call the 
underlying C function fread more than once in an effort to acquire as 
close to size bytes as possible." leans slightly towards my 
interpretation.

I looked a little for real-world cases that could cause a denial-of-
service attack but didn't find one.

If there is a problem, it will occur very rarely.  Go ahead an mark it 
as "will not fix" or something similar.  I don't think the change in the 
code is justifiable.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3531>
_______________________________________