[Mailman-Users] Broken pickle

[Mark Sapiro]

Grigory Batalov wrote:
>  Also, as you can see, I have no IncomingRunner:1:6 and it fails to start
>  every time I try to:
>
>$ sudo -u mailman /usr/bin/python /usr/share/mailman/bin/qrunner --runner=IncomingRunner:1:6


Note that the documentation states that the number of slices must be a
power of two. I don't think it really matters, and I'm sure it isn't
anything to do with the corrupt pickle, but that's what it says.


>Traceback (most recent call last):
>  File "/usr/share/mailman/bin/qrunner", line 278, in ?
>    main()
>  File "/usr/share/mailman/bin/qrunner", line 238, in main
>    qrunner.run()
>  File "/usr/share/mailman/Mailman/Queue/Runner.py", line 71, in run
>    filecnt = self._oneloop()
>  File "/usr/share/mailman/Mailman/Queue/Runner.py", line 100, in _oneloop
>    msg, msgdata = self._switchboard.dequeue(filebase)
>  File "/usr/share/mailman/Mailman/Queue/Switchboard.py", line 159, in dequeue
>    msg = cPickle.load(fp)
>ValueError: insecure string pickle
>
>  Using "strace" I have found which file it tries to load:
>
>$ python
>>>> file=open("/var/spool/mailman/in/1196682806.813381+4ffeef3dcbdc578279784fb47aa271ad8f6462f7.bak", "r")


Actually, I hope it was actually trying to open
1196682806.813381+4ffeef3dcbdc578279784fb47aa271ad8f6462f7.pck
and had then renamed it to
1196682806.813381+4ffeef3dcbdc578279784fb47aa271ad8f6462f7.bak

dequeue should never be trying to open a .bak file.

In any case, the problem file would have been the first one in the
/var/spool/mailman/in/ directory in sequence by the part of the name
up to the '+'.


>>>> import cPickle
>>>> msg=cPickle.load(file)
>Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
>ValueError: insecure string pickle
>
>  This file in spool looks regular except "^@" symbol at the end of first
>  (long) line:
>
>...bYQlIS1Mprnphtkfp4Urlx28fbCEAVsvLFc9KCkIQgH//2Q==\n\n------=_NextPart_000_0003_01C8359F.078C8BD9--\n\n\n^@


This is the string containing the raw message text and somehow the
quote at the end of the string has been changed to a null byte (^@). I
haven't got a clue as to how this could happen unless it's a Python
cPickle bug of some sort.

I just tried to make a queue entry of this type from a messsage that
had a trailing null byte and, with Python 2.5.1 at least, the null
byte was properly escaped as '\x00' in the pickle.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan