permuting letters and fairy tales

[Johannes Nix]

Hello,

yesterday I met a cute person (after my dance class) who told me about an
interesting experiment regarding cognition. People were told to read a
typed text; However, in every word in the text, only the first and the
last letter were in the right place. The other letters had their
positions changed in an arbitrary manner. The surprising result, I was
told, was that people can read this mixed-up text fairly well.

Because I am a somewhat sceptical guy, at times, and because I thought
that I deserved some play, I decided to code the rule above in a
scriptlet. The resulting 23 lines are below, and the outcome is quite
interesting, not only from the point of view of librarians ;-) :

--------------------------------------------------
#!/usr/bin/python
import sys
import locale
import string
import re
import Numeric
import RandomArray

locale.setlocale(locale.LC_CTYPE, '')
wordsep = re.compile('([^%s])' % string.letters)

for line in sys.stdin.xreadlines():
    for word in wordsep.split(line):
        if word and word[0] in string.letters:
            word = string.lower(word)
            wlen = len(word)
            if wlen > 3:
                wa = Numeric.array(word)
                perm = RandomArray.permutation(wlen-2)
                wa[1:wlen-1] = Numeric.take(wa[1:wlen-1],perm)
                word = wa.tostring()
        sys.stdout.write('%s' % word)

--------------------------------------------------
 

For the Uninitiated, Numeric is a package which deals with array data;
arrays are mutable sequences and Numeric.take() can reorder items in
them; RandomArray.permutation() delivers the randomized reordering we
need.

Now I have two innocent questions:

- Is it possible to make it a bit more concise ;-))) ?

- Can it coerced to run a little bit faster ?
  (on my oldish, 300 MHz-AMD K6 , run time looks like this
  for a famous, 2663-word-long fairy tale from the Grimm's brothers:

  nix at aster:~> time <HaenselundGretel.txt ./python/perlmutt.py  >v
  
  real    0m6.970s
  user    0m3.634s
  sys     0m0.120s
  

And two remarks what is interesting about it:

- It's a good example how powerful libraries, like
  Numeric, make one's life easier. (BTW, why is Numeric 
  and stuff like take() still not included in the standard
  Library ? Batteries included, but calculator not ?)

- Perhaps it's useful to protect messages in some
  regions with not-so-democratic forms of government
  against automatic scanning by making the message 
  machine-unreadable, causing some Orwellian Confusion ;-) ?
  Of course, texts from Pythonistas would remain suspicious,
  due to the large number of "y" occurring in them....

have a nice evening....

Johannes