Still same bug even with email ver. 1.2

[Barry A. Warsaw]

Thanks Sheila and David for their private email heads-up; I sadly
don't have much time to follow c.l.py.  Also, thanks to Andrew for
filing the SF bug.

>>>>> "SK" == Sheila King <usenet at thinkspot.net> writes:

    SK> OK, I got the latest email module from the CVS. I installed
    SK> it. I ran the tests for the module. Everything tested out
    SK> fine.

    SK> I now eagerly get out my code from last night, that was
    SK> causing the 'int' has no attribute .lower error, and run it,
    SK> hoping for different results.

    SK> No good. Same results. :(

I now know what's going on, why I thought the bug was fixed, why
others disagreed <wink>, and I think I have a solution.  I'm cc'ing
mimelib-devel at lists.sf.net because my proposed solution has two parts,
one for Python 2.2.1 and one for Python 2.3.

Background: In Python 2.2, if you have a Message instance with a MIME
type of multipart/* but you have only one subpart, the Generator would
throw the exception that Sheila describes.  This is because the
payload of the outer part isn't a list, but the Generator expects a
list for multipart/* parts.  A multipart/* with one subpart, must have
a payload which is a list of length one, i.e. element zero would be
the single subpart.

What I fixed, and what is in Python 2.2.1, is the Parser class, which
takes a plain text representation of the message, and produces the
message object tree.  In the Parser, I always make sure that the
payload is a list for multipart/* MIME types by doing this:

                # Ensure that the container's payload is a list
                if not isinstance(container.get_payload(), ListType):
                    container.set_payload([msgobj])
                else:
                    container.add_payload(msgobj)

This works fine -- for object trees that are built by the parser from
flat text.  It doesn't work so well when you're building the object
tree from scratch, unless you employ the same kludge!  I gather from
the c.l.py thread that folks are going through the third example in
the Python 2.2 documentation

    http://www.python.org/doc/current/lib/node389.html

finding themselves with a directory with just a single file in it, and
are seeing Generator blow up when they end up with a multipart/*
object with a non-sequence payload.  Well, the example doesn't employ
this kludge, so it crashes.

Let me first outline what I think is the Right Fix and then what I
propose for Python 2.2.1.

The right fix is to clean up the messy semantics of
Message.add_payload(), where the first time you call it, it will set
the message's payload to the payload argument (a scalar), but the
/second/ time you call it, it transforms the payload into a list, then
appends the new payload to this list.

This is really pretty gross, and it rubbed me the wrong way when I
designed it, but I didn't have any better ideas at the time so there
you have it.  I believe this is the heart of the breakage.

What I think we ought to do is to deprecate Message.add_payload() but
leave its semantics unchanged, change Message.attach() to not be an
alias for add_payload(), and define the semantics of attach() to
always set the payload to a list.  Thus, the first time you call
attach().  I think attach() should retain the semantics that if the
MIME type isn't multipart/* it should throw an exception (i.e. you
can't attach() to a non-multipart; use set_type() first).

This means that if never set, a Message's payload would be None.
After the first attach() it would be a list of length one, and
subsequent attach()'ments would increase the length of the payload.

To get the old, initial semantics of add_payload(), you'd instead call
set_payload() which always just sets the message's payload to the
method's argument.  This would be used, for example, when you have a
message/rfc822 MIME type and want to set the payload to an instance of
a MIMEMessage, or if you have a image/gif and want to set the payload
to an instance of MIMEImage.

For Python 2.3 (and for the next version of the standalone email
package), I'd like to add a DeprecationWarning to add_payload(), then
make the semantic change to attach().  The downside is that this might
break code that relied on these semantics, however, I think if you had
such an object tree, you wouldn't be able to use the Generator to
flatten it anyway.  I'm open to suggestions here.

There's a problem though: this isn't a change that should be made in
Python 2.2.1!  It changes the API to the email package, and that
should only be done in Python 2.3, IMO.

For Python 2.2, I /think/ I can fix Generator._handle_multipart() to
watch out for subparts that are scalar[1].  If so, then the patch to
the Parser described above probably ought to be backed out.  As long
as the unit tests pass, I think we'd be okay.  This is probably a good
patch to keep for Python 2.3, just to be safe.

I'll try to work out a patch for Python 2.2.1 tomorrow.

Comments?
-Barry

[1] There's a related problem for when there's no payload at all for a
multipart/* message.  I think the same fix to
Generator._handle_multipart() can be made to handle this case, but
fixing the Parser will be a little more difficult.  I'll try though.