[Mailman-Users] Really Fustrating - Searchable Archives
Peter Russell
prussell at mteliza.com.au
Fri Jul 11 00:28:33 CEST 2003
Richard, thanks for your guide - i followed it step by step and everything
appears to work now, first time. This wasnt even really that hard, just
needed the guide to follow.
Maybe you could get this on a webpage and have a link where ever your
patches live? Or is it just me who couldnt do it? :)
I think i will just swap the production machine for my test machine rather
than do all that remotely.
Many thanks
Pete
Richard Barrett <r.barrett at openinfo.co.uk>
10/07/2003 09:21 AM
To: "Peter Russell" <prussell at mteliza.com.au>
cc:
Subject: Re: [Mailman-Users] Really Fustrating - Searchable Archives
Peter
I did the notes below for somebody else wanting to upgrade Mailman 2.1.2
installed from source. If you have installed Mailman from source on your
test machine why not take a look at my notes and see what you think. Try
adding the htdig integration to your test machine to see how much of a
problem you find it.
If you like I will take a look at a procedure for de-installing the RPM
installed MM on your production machine and replacing it with MM from
source. One thing you could do is send me the output from the following
command:
rpm -ql mailman
I run Suse Linux rather than RedHat so I cannot readily check the exact
details of the files RedHat's mailman RPM package installs. The command
I've given provides that information.
Regards
Richard
-----------------------------------------------------------------
The following notes are applicable to MM 2.1.2 installed from source.
Phase 1 - Initial preparation of Mailman
=======
1. Download the following patch files and save them in the directory above
your Mailman build directory.
templates-2.1.2-0.1.patch from
http://sourceforge.net/tracker/index.php?func=detail&aid=728836&group_id=103&atid=100103
tcache-2.1.2-0.3.patch.gz from
http://sourceforge.net/tracker/index.php?func=detail&aid=730769&group_id=103&atid=100103
tz-2.1.2-0.2.patch from
http://sourceforge.net/tracker/index.php?func=detail&aid=732366&group_id=103&atid=100103
indexing-2.1.2-0.1.patch.gz from
http://sourceforge.net/tracker/index.php?func=detail&aid=444879&group_id=103&atid=300103
htdig-2.1.2-0.3.patch.gz from
http://sourceforge.net/tracker/index.php?func=detail&aid=444884&group_id=103&atid=300103
2. gunzip all of the patch files that are .gz suffix. For example use the
command:
gunzip indexing-2.1.2-0.1.patch.gz
3. With your MM build directory as current working directory apply the
patches:
patch -p1 < ../templates-2.1.2-0.1.patch
patch -p1 < ../tcache-2.1.2-0.3.patch
patch -p1 < ../tz-2.1.2-0.2.patch
patch -p1 < ../indexing-2.1.2-0.1.patch
patch -p1 < ../htdig-2.1.2-0.3.patch
4. Run the Mailman ./configure using exactly the same command as you ran
when you originally installed Mailman.
5. Use mailmanctl to stop Mailman's daemons. For example use the command:
$prefix/bin/mailmanctl stop
6. Remove Mailman's crontab. For example use the following command as
root:
crontab -r -u mailman
7. Stop your MTA. For example you might run, as root, the following
command
if you were running sendmail:
/etc/rc.d/init.d/sendmail stop
8. Stop your web server. For example you might run, as root, the following
command:
/etc/rc.d/init.d/apache stop
9. Run the Mailman make install:
make install
10. Run the Mailman checkperms with $prefix as the current working
director, re-running it with the -f option if any errors are reported:
$prefix/bin/check_perms
11. Restart your web server.
12. Reinstall the Mailman crontab, which should now have an extra entry
for
running nightly_htdig in it.
13. Use mailmanctl to restart Mailman's daemons.
Mailman should now be patched to support the use of Htdig for searching
but
nothing will be activated until Htdig is installed and some Mailman
configuration variables have been set up. Check Mailman is working
normally
in all other respects.
Phase 2 - Installing Htdig
=======
I strongly recommend using Htdig 3.1.6. I have not tested my mailman-htdig
patches with other than the latest stable release which is 3.1.6. RedHat
seems to distribute htdig 3.2b3 as if it were a stable release and I have
had several reports of people having problems of one sort or another with
it.
1. Getting the htdig 3.1.6 source via this page http://htdig.org/where.html
works for me.
2. Having unpacked the source package I then run commands, as root, to
configure and install it, basically following the instructions here
http://htdig.org/install.html, with the following commands:
./configure --with-cgi-bin-dir=/usr/local/httpd/cgi-bin
--with-image-dir=/usr/local/httpd/htdocs/htdig
make
make install
3. In my case this installs htdig into the default /opt/www/htdig/
directory with some of the web related stuff under the ServerRoot
directory
for my Apache server. If ServerRoot in your Apache server's
/etc/httpd/conf/httpd.conf is not /usr/local/httpd then change the
./configure line above to be what your system is setup for.
4. To overcome permissions problems when rundig is first used to build
search indexes you should change the group ownership of some of the htdig
installation directories to allow the mailman user and group write access
to those directories and the files in them. In my case I say:
chgrp -R mailman /opt/www/htdig/conf /opt/www/htdig/db
Phase 3 - Configuring Mailman to use Htdig
=======
1. Add the following lines to your $prefix/Mailman/mm-cfg.py file. Note
again that the paths/URL on three of these lines need to fit in with the
way you installed htdig in Phase 2.
# Mailman-htdig configuration
USE_HTDIG = 1
HTDIG_RUNDIG_PATH = '/opt/www/htdig/bin/rundig'
HTDIG_HTSEARCH_PATH = '/opt/www/htdig/bin/htsearch'
HTDIG_FILES_URL = '/htdig/'
2. Use mailmanctl to restart the Mailman daemons.
3. From now on, the first new message sent to each list will trigger the
creation of list-specific htdig.conf files and add a serach form to the
list's archive TOC page. Until such a message is received by a list no
search facility will be enabled for it.
4. Until the nightly_htdig cron script is run, after the first new message
to a list, the list will have no search indexes. You can run, as the
mailman user, the nightly_htdig script from the command line.
For detailed information see the INSTALL.htdig-mm file added to the MM
build directory by the patches.
With luck and a fair wind you should now be up and running with htdig
searchable list archives.
-----------------------------------------------------------------
At 23:18 09/07/2003, you wrote:
>MANY thanks for taking the time to try and help me - much appreciated.
>
>I suppose io want to emulate what list.org have on the mailman-users
>archive. I hope i dont sound unappreciative, i realise that this software
>(and all the patches and the support) are people investing thier own time
>for little/no financial return - sounds like my job), i really do think
>mailman is fantatsic, but these things are fustrating, especiually when
>you see how good it can work when you see the searchabkle archives at
>list.org - kind of expected to get that when i installed it.
>
>Again many thanks in advance for trying to help me - answers to your
>questiosn follow.
>Pete
>
>1. Both. Internet/international users and local 'staff'.
>
>2. All public - we only have a handlfull of lists in mind.
>
>3. Preferably i would like to keep the search specific to each list. But
i
>dont think this is essentail.
>
>4. I dont have anything like that installed - i have mailman, sendmail
and
>apache. Except for a little bit of mail relaying for machines on our
>private network, the mailman machine is dedicated to mailman
>
>5. We only have a handful of Linux machines, everything else in win32 -
my
>boss beleives that linux is not mature or robust enough for enterprise
use
>:( In fact he says same about win2k and insists on using NT4 - i am sure
>he doesnt even like that and would prefer we had %100 DOS environment. We
>dont have any other linux machines running any kind of search software.
>
>6. I dont completely understand this question - i want to be able to
>search on the contents of the archive - the body/subject of messages.
>
>7. Red Hat 9 and the mailman version that came with it, RPM - which is a
>little bit of a hassle, because i am not an expert i dont know where to
>begin in regards to recompile and apply tio RPM instalation - where as
now
>i know that with a little bit of effort compiling was easy enough on my
>test machine. (i have a test machine i am trying to get this all working
>on first)
>
>8. I typed python and it returns 2.2.2 - it came with red hat 9
>
>
>
>
>
>Richard Barrett <r.barrett at openinfo.co.uk>
>
>10/07/2003 12:40 AM
>
> To: "Peter Russell" <prussell at mteliza.com.au>
> cc: mailman-users at python.org
> Subject: Re: [Mailman-Users] Really Fustrating -
> Searchable Archives
>
>
>At 14:34 09/07/2003, Peter Russell wrote:
> >Not being a Linux guru, just getting mailman and sendmail or postfix
all
> >set up and working properly is a fair task. Attacking a working
installing
> >to get mhonarc and htdig working and MM patched is just an excercise in
> >fustration. (without a detailed howto, slightly beyond my linux skills)
>
>You do not have to install Mhonarc to get searchable archives. Mhonarc is
>an external archiver and some people prefer that to using the Mailman
>internal pipermail archiver. Adding Mhonarc to the equation just
>complicates the issues for you rather than simplifying adding search.
>
>With the following information available, more directed advice will be
>achievable:
>
>1. are you making the list archives available over the internet or only
>within your own domain?
>
>2. are all your list archives public or do you have some private archives
>whose privacy you want to preserve over search?
>
>3. do you want per-list search indexes that will only return search hits
>relevant to individual lists or do you want site-wide search indexes that
>may return hits from many lists?
>
>4. do you have any search engine installed on the server running Mailman?
>If so what is it?
>
>5. do you have any search engine installed on any other server in your
>network? If so what is it and can you change its configuration files so
>that it can be directed to also index your list archives?
>
>6. do you want search forms on the TOC pages of individual list archives?
>
>7. what OS and version are you running?
>
>8. what version of Python are you running?
>
>8. what version of MM are you running and how was it installed: from
source
>or was it packaged?
>
>With the above information it will be easier to produce a directed guide
to
>setting up search for your list archives.
>
> >Is there anywhere that a 1, 2, 3 - press this then type that type guide
> >exists to achieve threaded archiving and a good search feature?
>
>Answer the above questions and you may get your wish. btw: Mailman's
>internal archiver produces threaded archives out of the box.
>
> >OR is there any chance that these types of features are going to be
> >avilable in the MM release? Or is there a CVS i can download and
install?
>
>It is being considered.
>
> >Can anyone tell me why archive search (or even threaded archives) are
not
> >used out of the box with this product?
>
>A fair number of people are running htdig with or without using the
mailman
>integration patches. It really is not that difficult to install and
setup.
>Like a lot of Open Software, there may be a certain lack of polish but
the
>base technology is pretty good and comes at a remarkably good price.
>
>btw: did I mention that Mailman's internal archiver produces threaded
>archives out of the box.
>
>
> >Is there a product that does EVERYTHING that mailman does and does have
the
> >search and archives features too?
>
>Maybe, but a mailing list supporting no-cost Open Software is probably
not
>the place to look for it.
>
> >The punters who wanted mailman are starting to get mad as hell at me
for
> >not having these features going, i appreciate ANY input/suggestions.
>
>------------------------------------------------------------------------------
>Richard Barrett http://www.openinfo.co.uk
>
>
------------------------------------------------------------------------------
Richard Barrett http://www.openinfo.co.uk
More information about the Mailman-Users
mailing list