Bug in email.generator.BytesGenerator() [was: Why does SMTP.send_message() do from mangling?]

[Grant Edwards]

On 2021-09-27, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> Why does SMTP.send_message(msg) do from mangling even though msg's
> policy has mangle_from_ set to False? The msg policy is
> email.policy.SMTP which has mangle_from_ disabled.
>
> One might expect that SMTP.send_message(msg) would use either msg's
> policy or email.policy.SMTP to send the message, but it does neither.

I've been looking at the smtplib.py sources, and the problem appears
to be in this section of send_message():


   912      def send_message(self, msg, from_addr=None, to_addrs=None,
   913                       mail_options=(), rcpt_options=()):
   914          """Converts message to a bytestring and passes it to sendmail.
   ...
   963          # Make a local copy so we can delete the bcc headers.
   964          msg_copy = copy.copy(msg)
   ...
   977          with io.BytesIO() as bytesmsg:
   978              if international:
   979                  g = email.generator.BytesGenerator(
   980                      bytesmsg, policy=msg.policy.clone(utf8=True))
   981                  mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME')
   982              else:
   983                  g = email.generator.BytesGenerator(bytesmsg)
   984              g.flatten(msg_copy, linesep='\r\n')
   985              flatmsg = bytesmsg.getvalue()

If 'international' is Frue, then the BytesGenerator uses msg.policy
with utf8 added, and I don't get the bogus from mangling: the
generator only does from mangling if the message policy has it
enabled.

If 'international' is False, then the generator always does from
mangling regardless of the message's policy.

According to
https://docs.python.org/3/library/email.generator.html#email.generator.BytesGenerator
the default from mangling behavior is _supposed_ to obey the message
policy if (as seen at 983) no policy or mangle_from_ value was
provided to the call to BytesGenerator(). In my tests, it doesn't
actually seem to work that way. AFAICT, the default behavior when no
policy or mangle_from_ value is passed to BytesGenerator() is to
enable from mangling regardless of the message's policy. I belive that
is a bug.

This can be worked around by changing

   983                  g = email.generator.BytesGenerator(bytesmsg)
to

   983                  g = email.generator.BytesGenerator(bytesmsg, policy=msg.policy)


Or BytesGenerator() could be fixed...

--
Grant