[Moin-user] Title Search Problem, 1.9.2

[Thomas Waldmann]

On Fri, 2010-10-15 at 18:22 -0500, Bobby Baucom wrote:
> Using MoinMoin 1.9.2 & Xapian and having a problem with searches. When
> we were using MoinMoin 1.5.x my users created lots of pages with
> underscores in the page name.

Well, back then, moin mapped blank to underscore. Initially we thought
that copying this idea from mediawiki was a good idea, but soon later we
found it was not.

> We converted to 1.9.2 and now they are complaining about searches not
> finding pages. When doing a title search on a word or phrase within a
> page name, it does not find “matching” pages. For example, a title
> search for "yield curves" or even a title search for “yield_curves"
> will not find a page titled "SandP_Invest_Yield_Curves".

That's likely because the tokenizer splits on blanks and on CamelCase
lower-upper transitions, but not on underscores.

> I guess I could have changed underscores to spaces during the
> conversion, but it is too late now. 

Yeah.

> It appears to me that Xapian indexes the entire phrase instead of the
> four words that make up the phrase.

It simply doesn't know that these are four words.

>  If the page had been named "SandP Invtools Yield Curves" or even
> "SandP InvtoolsYieldCurves" then the search term would be able to find
> it. I notice that the MoinMoin web site does not exhibit the same
> behavior. It searches the way I want our wiki to be able to search. So
> there’s hope.

Hmm, strange. How did you test that? I did a quick test and it didn't
find a single word if the title was made of words connected with
underscores.

> I looked in the 1.9.3 change list and can find no explanation for the
> difference between search or indexing in 1.9.2 and 1.9.3. Is this a
> setting issue? Or is there a change in 1.9.3 that will solve my
> problem? We are using Xapian 1.0.15 with stemming enabled. Apache
> running on Ubuntu server. Python 2.6.4rc2. 

I don't think it is a configuration issue.

I am currently trying to extend the tests with the underscore issue and
fix the code, so the tests don't fail.