quopri module with large files
Menno Smits
menno at netbox.biz
Thu Apr 3 01:35:50 EST 2003
decode() in the standard quopri module loads the entire file being
decoded into memory and sends the result to binascii.a2b_qp(). This
can potentially lead to memory exhaustion problems if the file being
decoded is large.
Can anyone see a problem with replacing the offending code in decode()
which looks like this:
if a2b_qp is not None:
data = input.read()
odata = a2b_qp(data, header = header)
output.write(odata)
return
with this? :
if a2b_qp is not None:
while 1:
line = input.readline()
if not line:
break
output.write(a2b_qp(line, header=header))
This would be more efficient memory-wise when it comes to large email
attachments. The base64 module takes a similar approach.
I've compared the two implementations using a few test cases and they
both appear to give identical output. However, maybe someone can see a
problem with the line-by-line approach that I'm not seeing. A brief
check of RFC 2045 indicates there shouldn't be a problem.
I'm looking at the Python 2.2.2 implementation BTW.
--
Menno Smits
Senior Development Engineer
NetBox | Oxcoda
E-mail: menno at netbox.biz
Web: netbox.biz
More information about the Python-list
mailing list