[ python-Bugs-1072623 ] FeedParser problem on end boundaries w/o newline

[SourceForge.net]

Bugs item #1072623, was opened at 2004-11-24 09:27
Message generated for change (Comment added) made by tlau
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1072623&group_id=5470

Category: Python Library
Group: Python 2.4
Status: Closed
Resolution: Wont Fix
Priority: 5
Submitted By: Tessa Lau (tlau)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: FeedParser problem on end boundaries w/o newline

Initial Comment:
(Python 2.3.4, Linux Debian unstable)

The email module's as_string() method generates
messages that do not include a trailing CRLF on the
last line.  This causes problems when Python-created
messages are piped to procmail and delivered to an mbox.

The attached test script illustrates this behavior. 
You must have a working procmail configured to deliver
mail to an mbox (the default configuration will work).
 If you then read the resulting mailbox with /bin/mail,
it appears as if there is only one message in the
mailbox, instead of two.  The second is concatenated on
to the end of the first.  The mbox does not contain a
blank line between the first message and the second. 
Pop servers require this blank line delimiter between
messages.

You could argue that this is procmail's responsibility,
but as far as I can tell from reading RFC 2822, each
line in an email message must terminate in CRLF, and
Python's email module is violating that spec.

----------------------------------------------------------------------

>Comment By: Tessa Lau (tlau)
Date: 2004-11-29 05:23

Message:
Logged In: YES 
user_id=112896

It must have been someone else on the email sig; I haven't
posted there recently.

Thanks for the workaround.  However, it only fixes the
problem for MIME messages, but not for non-MIME messages. 
The second message constructed in my test script still lacks
a trailing newline.

I can work around it after the message is generated by
checking for a final newline on the string and adding it if
it's missing, but that seems clunky.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2004-11-28 17:10

Message:
Logged In: YES 
user_id=12800

I must have been thinking about the message you posted to
the email sig, which uncovered the bug I commented on, and
fixed.  In the case of the original bug report, I don't
believe this is fixable.  There is, however a simple
workaround for you.  In your sample code, set the outer
message's epilogue to the empty string, e.g.:

msg1.epilogue = ''

...

msg2.epilogue = ''

This will cause the Generator to add a newline at the end of
the outer message.  We can't make that change in the Message
class because doing so would break inner message flattening.

However, if someone were to come up with a patch that fixes
this problem yet doesn't break any of the 217 tests in the
current test suite, I'd be happy to look at it.  As it is,
nothing will be changed for Python 2.4. final.

----------------------------------------------------------------------

Comment By: Tessa Lau (tlau)
Date: 2004-11-27 16:45

Message:
Logged In: YES 
user_id=112896

My original bugreport had to do with email generation, not
parsing.  Python seems to be generating email that is not
compliant with the RFC spec.  In my situation, parsing is
done by 3rd party programs (procmail and /bin/mail) which
also fail to deal gracefully with the lack of trailing newline.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2004-11-27 16:03

Message:
Logged In: YES 
user_id=12800

Changing the summary of this issue to reflect the real
problem.   The attachment 1072623.py illustrates that if the
end boundary of a string being parsed by the FeedParser does
not have a trailing newline, the parser doesn't recognize
this as an end boundary.  It just so happens that your
example produces a message string with that property (which
isn't a bug).

The fix is about as trivial as you can get: one character.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1072623&group_id=5470