[Python-Dev] Encoding detection in the standard library?

[Oleg Broytmann]

On Mon, Apr 21, 2008 at 06:37:20PM -0300, Rodrigo Bernardo Pimentel wrote:
> On Mon, Apr 21 2008 at 06:31:06PM BRT, "\"Martin v. L??wis\"" <martin at v.loewis.de> wrote:
> > > This is useful when you get a hunk of data which _should_ be some  
> > > sort of intelligible text from the Big Scary Internet (say, a posted  
> > > web form or email message), and you want to do something useful with  
> > > it (say, search the content).
> > 
> > I don't think that should be part of the standard library. People
> > will mistake what it tells them for certain.
> 
> Maybe call it "charguess", then?

   The famous chardet returns probablity of its guessing:

>>> import chardet
>>> chardet.detect("dabc")
{'confidence': 1.0, 'encoding': 'ascii'}
>>> chardet.detect("тест")
{'confidence': 0.98999999999999999, 'encoding': 'KOI8-R'}

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.