[Mailman-Developers] Patch for HyperArch

[Mark Sapiro]

On 03/09/2016 05:26 AM, Sebastian Hagedorn wrote:
> 
> --On 8. März 2016 um 15:43:11 -0800 Mark Sapiro <mark at msapiro.net> wrote:
> 
>> I'm still having difficulty duplicating what you saw.
>>
>> For the above three messages, the first one just gets detected as an
>> invalid date and archived under the current date by the current bin/arch.
>>
>> The other two throw "ValueError: year out of range" at
>>
>>   File "/var/MM/21/Mailman/Archiver/HyperArch.py", line 984, in
>> dateToVolName
>>     return time.strftime("%Y-%B",datetuple)
>>
>> which is a problem, but not the one you saw. I would like to see
>> messages that cause this error
>>
>>   File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 601, in
>> _set_date
>>     self.fromdate = time.ctime(int(self.date))
>> ValueError: timestamp out of range for platform time_t
>>
>> that you reported. Can you find those bad messages in the .mbox input
>> file you used and send them to me?
> 
> We played around and found that the error is related to our version of
> Python. Here's a minimal test script that shows the issue:
> 
> from email.Utils import parseaddr, parsedate_tz, mktime_tz, formatdate
> print mktime_tz(parsedate_tz("Fri, 4 Feb 100 00:51:42 +0100 (MET)"));
> 
> That's the Date header from the single piece of legitimate mail. Python
> 2.4 throws the same exception you were seeing: "ValueError: year out of
> range". However, our Python 2.7 (which we use for Mailman) does this:
> 
> -59008522098
> 
> When that value is then passed to time.ctime(), you get "ValueError:
> timestamp out of range for platform time_t". We're on RHEL 5, and our
> version of Python 2.7 is from the IUSCommunity repo:
> python27-2.7.10-1.ius.el5. Which version of Python were you using?


The particular system on which I'm testing is Ubuntu 15.10 and has
python2.7 2.7.10-4ubuntu installed via apt-get. It produces the same

-59008522098

result from your test script, but when I run bin/arch --wipe, I get

Traceback (most recent call last):
  File "../../../bin/arch", line 201, in <module>
    main()
  File "../../../bin/arch", line 189, in main
    archiver.processUnixMailbox(fp, start, end)
  File "/var/MM/21/Mailman/Archiver/pipermail.py", line 586, in
processUnixMailbox
    self.add_article(a)
  File "/var/MM/21/Mailman/Archiver/pipermail.py", line 611, in add_article
    archives = self.get_archives(article)
  File "/var/MM/21/Mailman/Archiver/HyperArch.py", line 914, in get_archives
    res = self.dateToVolName(float(article.date))
  File "/var/MM/21/Mailman/Archiver/HyperArch.py", line 984, in
dateToVolName
    return time.strftime("%Y-%B",datetuple)
ValueError: year out of range

I notice a few things here. First, your error comes in processing

    a = self._makeArticle(m, self.sequence)

called from pipermail.processUnixMailbox

Mine comes from

    self.add_article(a)

which is called after _makeArticle has already made the article, so I
don't see an exception in _makeArticle. In fact, after running
_makeArticle, I see

>>> a.fromdate
'Wed Feb  3 15:58:44 100\n'

which is exactly what is returned by time.ctime(-59008522098)

The other curious thing is there are no differences between the 2.1.18
pipermail.py and mine yet your

    a = self._makeArticle(m, self.sequence)

is at line 587 in your traceback, and in my pipermail.py it is at line 584.

In any case, The Date: that threw the original ValueError: timestamp out
of range for platform time_t is apparently not the "Fri, 4 Feb 100
00:51:42 +0100 (MET)" one, and I'd still like to see it. If you can find
all four messages in the .mbox file that produced the "Kein Betreff"
messages in the current archive, I'd like to see them.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan