[Mailman-Users] Wrong Indentation in MailMan archive (threads)

[Kristin Solbakken]

Hi, and thanks for the detailed explanation, this all makes sense now. I
now ran the arch script with wipe option, and the archive is back to
normal. As for the mails that did not go to the archive I did not see any
related message in the error.log , but my guess is that this was caused by
the qrunner (which stopped a while back). Well now it is back to normal,
and all mails seem to be present in the archive :)

Best regards,
Kristin Solbakken

On Tue, Jan 8, 2019 at 9:17 PM Mark Sapiro <mark at msapiro.net> wrote:

> On 1/8/19 2:03 AM, Kristin Solbakken wrote:
> > Hi. We had some mails missing from the archive, so I ran ./bin/arch
> > listname to update the archive. This fixed the missing mails part, but
> > somehow must have messed up the indentation.
>
>
> If you actually ran './bin/arch listname', the input was the cumulative
> archives/private/listname.mbox/listname.mbox file. This is an issue
> because you are processing messages that are already in the archive and
> adding them again even though they don't appear because of duplicate
> Message-IDs. This is what's screwing up your threading.
>
>
> > All mails got indented three times, and it looks like none of them are
> > connected to each other anymore.
>
>
> Three is the maximum indent, otherwise they would be indented more.
>
> If you look at the source of
> <https://opm-project.org/pipermail/opm/2019-January/thread.html>, you
> will see each index entry is preceded by an HTML comment
>
> The first two of which are> <!--3
>
> 01546606492.2242-01546606492.2656-01546606492.3076-01546606492.3496-01546606492.3918-01546606492.4340-01546606492.4762-01546606492.4762-
> -->
> > <!--3
> 01546606492.2242-01546606492.2656-01546606492.3076-01546606492.3496-01546606492.3918-01546606492.4340-01546606492.4762-01546606492.4762-01546846679.4765-
> -->
>
> The '3' is the thread depth and the long strings represent the threaded
> messages. The pieces are '<time_stamp>.<sequence>-' every time stamp is
> 01546606492 which is Fri Jan  4 04:54:52 2019
>
> The first message is actually 8 messages deep in the thread and the
> second is threaded below it at a depth of 9.
>
> You have multiple copies of the same message in the archive. Looking
> only at the first of the above, the initial message in the thread is
> <https://opm-project.org/pipermail/opm/2019-January/002242.html>
> followed by
> <https://opm-project.org/pipermail/opm/2019-January/002656.html>,
> <https://opm-project.org/pipermail/opm/2019-January/003076.html>,
> <https://opm-project.org/pipermail/opm/2019-January/003496.html>,
> <https://opm-project.org/pipermail/opm/2019-January/003918.html>,
> <https://opm-project.org/pipermail/opm/2019-January/004340.html>,
> <https://opm-project.org/pipermail/opm/2019-January/004762.html>, and
> finally <https://opm-project.org/pipermail/opm/2019-January/004762.html>
> again.
>
> This is all because of re-adding messages that are already in the archive.
>
> The bottom line here is if you are going to process the entire
> archives/private/listname.mbox/listname.mbox file, you need to run
>
> ./bin/arch --wipe listname
>
> Running that should fix your archive. But the question is if that fixes
> the "missing" messages, why are the messages in the listname.mbox file
> but not in the pipermail archive.
>
> What's in Mailman's error log and Mailman's qfiles/shunt directory to
> explain that?
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/kristin.solba%40gmail.com
>