quopri module with large files

Menno Smits menno at netbox.biz
Thu Apr 3 01:35:50 EST 2003


decode() in the standard quopri module loads the entire file being
decoded into memory and sends the result to binascii.a2b_qp().  This
can potentially lead to memory exhaustion problems if the file being
decoded is large. 

Can anyone see a problem with replacing the offending code in decode()
which looks like this:

    if a2b_qp is not None:
        data = input.read()
        odata = a2b_qp(data, header = header)
        output.write(odata)
        return

with this? :

    if a2b_qp is not None:
        while 1:
            line = input.readline()
            if not line:
                break
            output.write(a2b_qp(line, header=header))

This would be more efficient memory-wise when it comes to large email
attachments. The base64 module takes a similar approach.

I've compared the two implementations using a few test cases and they
both appear to give identical output. However, maybe someone can see a
problem with the line-by-line approach that I'm not seeing. A brief
check of RFC 2045 indicates there shouldn't be a problem.

I'm looking at the Python 2.2.2 implementation BTW.

-- 
Menno Smits
Senior Development Engineer
NetBox | Oxcoda
E-mail: menno at netbox.biz
Web: netbox.biz





More information about the Python-list mailing list