mysql -> record array

[John Hunter]

>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:

    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

In the use case that prompted this message, the pull from mysql took
almost 3 seconds, and the conversion from lists to numpy arrays took
more that 4 seconds.  We have a list of about 500000 2 tuples of
floats.

Digging in a little bit, we found that numpy is about 3x slower than
Numeric here

  peds-pc311:~> python test.py
  with dtype: 4.25 elapsed seconds
  w/o dtype 5.79 elapsed seconds
  Numeric  1.58 elapsed seconds
  24.0b2
  1.0.1.dev3432

Hmm... So maybe the question is -- is there some low hanging fruit
here to get numpy speeds up?

import time
import numpy
import numpy.random
rand = numpy.random.rand

x = [(rand(), rand()) for i in xrange(500000)]
tnow = time.time()
y = numpy.array(x, dtype=numpy.float_)
tdone = time.time()
print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)

tnow = time.time()
y = numpy.array(x)
tdone = time.time()
print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)

import Numeric
tnow = time.time()
y = Numeric.array(x, Numeric.Float)
tdone = time.time()
print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)

print Numeric.__version__
print numpy.__version__

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV