python-list at python.org

[MRAB]

On 2014-01-15 01:25, Florian Lindner wrote:
> Am Dienstag, 14. Januar 2014, 17:00:48 schrieb MRAB:
>> On 2014-01-14 16:37, Florian Lindner wrote:
>> > Hello!
>> >
>> > I'm using python 3.2.3 on debian wheezy. My script is called from my mail delivery agent (MDA) maildrop (like procmail) through it's xfilter directive.
>> >
>> > Script works fine when used interactively, e.g. ./script.py < testmail but when called from maildrop it's producing an infamous UnicodeDecodeError:
>> >
>> > File "/home/flindner/flofify.py", line 171, in main
>> >       mail = sys.stdin.read()
>> > File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
>> >       return codecs.ascii_decode(input, self.errors)[0]
>> >
>> > Exception for example is always like
>> >
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0x82 in position 869: ordinal not in range(128)
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1176: ordinal not in range(128)
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 846: ordinal not in range(128)
>> >
>> > I read mail from stdin "mail = sys.stdin.read()"
>> >
>> > Environment when called is:
>> >
>> > locale.getpreferredencoding(): ANSI_X3.4-1968
>> > environ["LANG"]: C
>> >
>> > System environment when using shell is:
>> >
>> > ~ % echo $LANG
>> > en_US.UTF-8
>> >
>> > As far as I know when reading from stdin I don't need an decode(...) call, since stdin has a decoding. I also tried some decoding/encoding stuff but changed nothing.
>> >
>> > Any ideas to help me?
>> >
>> When run from maildrop it thinks that the encoding of stdin is ASCII.
>
> Well, true. But what encoding does maildrop actually gives me? It obviously does not inherit LANG or is called from the MTA that way. I also tried:
>
locale.getpreferredencoding() said "ANSI_X3.4-1968", which is ASCII
(ask Wikipedia if you want to know why it's called that!).

>          inData = codecs.getreader('utf-8')(sys.stdin)
>          mail = inData.read()
>
> Failed also. But I'm not exactly an encoding expert.
>
Try:

sys.stdin = codecs.getreader('utf-8')(sys.stdin.detach())