trying to strip out non ascii.. or rather convert non ascii

[Steven D'Aprano]

On Mon, 28 Oct 2013 09:23:41 -0500, Tim Chase wrote:

> On 2013-10-28 07:01, wxjmfauth at gmail.com wrote:
>>> Simply ignoring diactrics won't get you very far.
>> 
>> Right. As an example, these four French words : cote, côte, coté, côté
>> .
> 
> Distinct words with distinct meanings, sure.
> 
> But when a naïve (naive? ☺) person or one without the easy ability to
> enter characters with diacritics searches for "cote", I want to return
> possible matches containing any of your 4 examples.  It's slightly
> fuzzier if they search for "coté", in which case they may mean "coté" or
> they might mean be unable to figure out how to add a hat and want to
> type "côté". Though I'd rather get more results, even if it has some
> that only match fuzzily.

The right solution to that is to treat it no differently from other fuzzy 
searches. A good search engine should be tolerant of spelling errors and 
alternative spellings for any letter, not just those with diacritics. 
Ideally, a good search engine would successfully match all three of 
"naïve", "naive" and "niave", and it shouldn't rely on special handling 
of diacritics.



-- 
Steven