[Mailman-Developers] Use of tabs when folding header lines--increasing subject length as a test

[Mark Sapiro]

Nathan Herring wrote:
>
>On 12/2/05 6:13:20 PM, "Mark Sapiro" <msapiro at value.net> wrote:
<snip>
>
>The other interesting one from an Outlook or Entourage perspective is the
>Thread-Topic, which ends up having a tab inserted into it.


Yes, and this is significant in that Thread-Topic is an MUA (in your
case Microsoft-Entourage) header that is completely untouched by
Mailman.

Therefore, it would appear simply instantiating an
email.Message.Message object and later writing it out is sufficient to
cause the header continuation white space to change from <space> to
<tab>.


>> Thus it will try to preserve the continuation-ws of an already folded
>> incoming subject, but will default to <tab>. Thus if the incoming
>> subject is not folded, but addition of the prefix lengthens it
>> sufficiently so it folds, it will be continued with a <tab>.
>
>It seems that for other headers, like Thread-Topic, even regularly folded
>items get its folding space turned into a folding tab.


Yes. The logic of trying to determine and preserve the continuation
white space of an already continued, incoming header is specific to
Mailman's manipulation of Subject: headers in CookHeaders. It is not
done elsewhere in Mailman.


>From my reading, it seems that email.Header doesn't preserve the FWS in the
>original header as it should. It would seem that the only time the
>continuation-ws parameter should be used is if there are sets of characters
>that need to be turned into multiple adjacent RFC 2047 encoded-words, as
>that FWS is not considered to be logically part of the header value, but
>merely an artifact of encoding. If email.Header did the correct
>preservation, then it would not matter whether you passed in <space> or
><tab>.


Actually, the email.Header.Header class is a constructor for Header
objects. It does not take a 'header' as an input argument. It makes a
class instance from a non-continued string or a list of (data,
charset) pairs to be RFC 2047 encoded. See the Python library
reference, sec. 12.2.5. The resulting Header instance is continued as
necessary using the continuation-ws argument as the leading white
space on continued lines. Within the class, continuation-ws defaults
to <space>, not <tab>.

There are really two issues here. The first is that CookHeaders
manipulates Subject: headers as Header class objects and unless the
incoming Subject: header is already continued with a <space>, it
specifies a <tab> for the continuation-ws character. This is strictly
a Mailman issue.

The other issue is that the Python email.Parser class API parses
message headers into strings, not Header class instances, and the
methods for flattening messages continue long header strings with
<tab>. This is a Python email library issue. To see it in action, just
do the following in an interactive Python session.

>>> import email
>>> from cStringIO import StringIO
>>> from email.Generator import Generator
>>> x = email.message_from_string('Header: a long string of words that will be ultimately continued because it\n is too long for one line')
>>> x['Subject'] = 'Some other long line which we build here and stick into a header to see what happens'
>>> fp = StringIO()
>>> g = Generator(fp, maxheaderlen=60)
>>> g.flatten(x)
>>> text = fp.getvalue()
>>> text
'Header: a long string of words that will be ultimately\n\tcontinued
because it is too long for one line\nSubject: Some other long line
which we build here and stick\n\tinto a header to see what happens\n\n'
>>>
(Lines 4, 5 and 11 above are each one long line in the original,
although they may get wrapped in emailing.)

Note that the original Header: was continued with a <space>. The
flatten method properly unfolds it but then refolds it with a <tab>.
Likewise it folds the Subject: with a <tab>.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan