NLTK

[Rhodri James]

On 03/08/18 08:49, mausg at mail.com wrote:
> I like to analyse text. my method consisted of something like
> words=text.split(), which would split the text into space-seperated
> units. then I tried to use the Python NLTK library, which had alot
> of features I wanted, but using `word-tokenize' gives a different
>   answer.-
> 
> What gives?.

I don't know or use NLTK and you haven't provided any examples, so I 
can't say for certain.  However vague memories from the natural language 
work I did decades ago suggest that perhaps splitting text by spaces is 
a little too simplistic, and NLTK may be trying to do something more 
comprehensive.

-- 
Rhodri James *-* Kynesim Ltd