[New-bugs-announce] [issue34220] Serialization of email message without header line length limit and a non-ASCII subject fails with TypeError

[Grigory Statsenko]

New submission from Grigory Statsenko <grisha100 at gmail.com>:

I have the following code that creates a simple email message with a) a pure-ASCII subject, b) non-ASCII subject
(I made it into a unittest):


import email.generator
import email.policy
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from unittest import TestCase


def create_message(subject, sender, recipients, body):
    msg = MIMEMultipart()
    msg.set_charset('utf-8')
    msg.policy = email.policy.SMTP
    msg.attach(MIMEText(body, 'html'))
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = ';'.join(recipients)
    return msg

class TestEmailMessage(TestCase):
    def _make_message(self, subject):
        return create_message(
            subject=subject, sender='me at site.com',
            recipients=['me at site.com'], body='Some text',
        )

    def test_ascii_message_no_len_limit(self):
        # very long subject consisting of a single word
        subject = 'Q' * 100
        msg = self._make_message(subject)
        self.assertTrue(str(msg))

    def test_non_ascii_message_no_len_limit(self):
        # very long subject consisting of a single word
        subject = 'Ц' * 100
        msg = self._make_message(subject)
        self.assertTrue(str(msg))


The ASCII one passes, while the non-ASCII version fails with the following exception:

Traceback (most recent call last):
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/home/grigory/PycharmProjects/smtptest/test_message.py", line 36, in test_non_ascii_message_no_len_limit
    self.assertTrue(str(msg))
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/message.py", line 135, in __str__
    return self.as_string()
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/message.py", line 158, in as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/generator.py", line 116, in flatten
    self._write(msg)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/generator.py", line 195, in _write
    self._write_headers(msg)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/generator.py", line 222, in _write_headers
    self.write(self.policy.fold(h, v))
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/policy.py", line 205, in _fold
    return value.fold(policy=self)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/headerregistry.py", line 258, in fold
    return header.fold(policy=policy)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/_header_value_parser.py", line 144, in fold
    return _refold_parse_tree(self, policy=policy)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/_header_value_parser.py", line 2645, in _refold_parse_tree
    part.ew_combine_allowed, charset)
  File "/home/grigory/.pyenv/versions/3.6.4/lib/python3.6/email/_header_value_parser.py", line 2722, in _fold_as_ew
    first_part = to_encode[:text_space]
TypeError: slice indices must be integers or None or have an __index__ method


The problem is that _fold_as_ew treats maxlen as an integer, but it can also have inf and None as valid values. In my case it's inf, but None can also get there if the HTTP email policy is used and its max_line_length value is not overridden when serializing.
I am supposing that the correct behavior in both of these cases should be no wrapping at all. And/or maybe one of these (inf & None) should be converted to the other at some point, so only one special case has to handled in the low-level code

This behavior is new in Python 3.6. It works in 3.5.
Also fails in 3.7 and 3.8

----------
components: email
messages: 322348
nosy: altvod, barry, r.david.murray
priority: normal
severity: normal
status: open
title: Serialization of email message without header line length limit and a non-ASCII subject fails with TypeError
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34220>
_______________________________________