Question about unreasonable slowness

[allenjo5 at mail.northgrum.com]

Łukasz Langa wrote:
> allenjo5 at mail.northgrum.com:
> > Ok, I built Python 2.5 (same AIX 5.1 machine).  With the "for line in
> > shellOut" loop in, it now takes "only" 7 secs instead of the 11 secs in
> > python 2.4.3.   So, that's better, but still unreasonably slow.  And to
> > answer another's question, I'm using the ksh builtin 'time' command to
> > time the overall script.
> >
> > BTW, I don't think the AIX /bin/sh (actually ksh) is inherently slow.
> > This naively translated pure shell version of my python test script
> > completes in .1 secs:
> >
> > i=1
> > while ((i<20))
> > do ((i+=1))
> >    print next
> >    print "$shellIn" | /bin/sh -c ':' |
> >            while read line
> >            do print $line
> >            done
> > done
> >
> > Has anyone tried this on a true unix box (AIX, HPUX, Solaris, Linux)?
> > It seems to be functioning differently (and faster) on Windows and OS X
> >  (though I guess at its heard, OS X is essentially unix).
> >
> > John.
> >
>
> Linux 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:57:02 EDT 2006 i686 i686
> i386 GNU/Linux
>
> # <code>
>
> import os
> import timeit
>
> def test():
>    for i in xrange(20):
>        (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")
>        print 'next'
>        for line in shellOut:
>            print line
>
> print timeit.Timer("test()", "from __main__ import test\nimport
> os").timeit(1)
>
> # </code>
>
>
> This returns in 0.4 seconds. If I time it to do 50 tests, it returns
> after 20.2 - 20.5 seconds. Even if I substitute the for i in xrange()
> construct to your sh-like while statement. And all that through a
> network, with print statements intact. Guess your true Unix box has some
> features unavailable on Fedora Core or MacOS X ;-)

Yeah, apparently this is an AIX specific issue - perhaps the python
implementation of popen4() needs to do something special for AIX?

I've since tested my script on SunOS 5.9 with Python 2.4.2, and it took
only about 1.5 sec with or without the second for loop, but without it,
there were no extra python processes running in the background when the
main one ends, unlike what I saw on AIX.   This might be a clue to
someone who knows more than I do... any Python gurus out there runnin
AIX?

John.