[Mailman-Users] nightly_htdig

[Mark Sapiro]

On 7/29/17 2:28 AM, Odhiambo Washington wrote:
> I am facing a weird problem with nightly_htdig. It's been refusing to
> complete of late. I would find the process still running even in the
> morning.
> Using `top`, I see it's the topmost process. I then decide to kill it, and
> it dumps core, which I believe is okay, but what would cause this
> behaviour, especially started after I upgraded to 2.1.24
> 
> I have rebuilt the archives, but that hasn't changed things.
> 
> Maybe it's something not related to mailman....


The htdig search engine <http://www.htdig.org/> is not part of Mailman
per se. It has not been supported for a long time. Richard Barrett
developed patches to integrate htdig search with Mailman 2.1 archives. I
have since incorporated those patches into a branch at
<https://code.launchpad.net/~msapiro/mailman/htdig> which is kept in
sync with the Mailman 2.1 master branch at
<https://code.launchpad.net/~mailman-coders/mailman/2.1>.

You may be able to use tools like gdb to find out more about the running
process before you kill it or to inspect the core dump.

You can also look at the Note: on each list's archive index page to see
when the archive search index was last rebuilt. This will help you find
on which list the process is "stuck".

Other questions I would have are:
How did you install Mailman?
How did you install htdig, and which version?
How did you upgrade Mailman to 2.1.24?
What steps did you follow to rebuild the archives?

All I can tell you is I run what is for this purpose essentially the
head of the <https://code.launchpad.net/~msapiro/mailman/htdig> branch
and htdig 3.2.0b6 installed from source, and I don't see this issue.

I do know that depending on the actual server resources available,
rebuilding the index for a large archive (tens of thousands or more
posts) can take a long time.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan