[Mailman-Developers] Patch for HyperArch

[Mark Sapiro]

On 03/07/2016 04:00 AM, Sebastian Hagedorn wrote:
> Hi,
> 
> we recently needed to rebuild a rather old list archive. The oldest
> mails are from 2001, and as far as I could tell the last complete
> rebuild happened in 2005. When we ran "arch --wipe" now, it failed:
> 
...
> ValueError: timestamp out of range for platform time_t
> 
> Obviously the mails that caused this error were broken, but a previous
> version of arch was able to build the archive regardless. I wrote the
> following patch to work around the problem (I wrote it for 2.1.18, but I
> checked that the code looks the same in 2.1.21):
> 
> --- /service/HyperArch.py    2014-07-16 13:01:11.000000000 +0200
> +++ HyperArch.py    2016-03-07 11:25:34.000000000 +0100
> @@ -598,7 +598,14 @@
> 
>     def _set_date(self, message):
>         self.__super_set_date(message)
> -        self.fromdate = time.ctime(int(self.date))
> +    try:
> +            self.fromdate = time.ctime(int(self.date))
> +    except ValueError:
> +                syslog('error',
> +                   'Archive error. Date %s is invalid.',
> +                   int(self.date))
> +        self.date = str(int(time.time()))
> +        self.fromdate = time.ctime(int(self.date))
> 
>     def loadbody_fromHTML(self,fileobj):
>         self.body = []
> 
> If an exception is caught, the date is simply set to the current time.

I understand the patch, but I'm not sure if setting the current time is
appropriate. In particular, the self.__super_set_date(message) method,
if it doesn't find a valid date: or x-list-received-date: header in the
message will set the time to that of the previous article + 1 second.

In any case, I'd like to understand more about why/how the issue occurs.
To that end, I'd like to see a copy of the offending message from the
mbox file. Also, I wonder if bypassing the error and setting a date
which will almost certainly archive the message in the wrong period is
better than fixing the message in the mbox.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20160307/b3711df7/attachment.sig>