[Numpy-discussion] "import numpy" is slow

[Scott Ransom]

On Thu, Jul 31, 2008 at 07:46:20AM -0500, Nathan Bell wrote:
> On Thu, Jul 31, 2008 at 7:31 AM, Hanni Ali <hanni.ali at gmail.com> wrote:
> >
> > I would just to highlight an alternate use of numpy to interactive use. We
> > have a cluster of machines which process tasks on an individual basis where
> > a master tasks may spawn 600 slave tasks to be processed. These tasks are
> > spread across the cluster and processed as scripts in a individual python
> > thread. Although reducing the process time by 300 seconds for the master
> > task is only about a 1.5% speedup (total time can be i excess of 24000s). We
> > process large number of these tasks in any given year and every little
> > helps!
> >
> 
> There are other components of NumPy/SciPy that are more worthy of
> optimization.  Given that programmer time is a scarce resource, it's
> more sensible to direct our efforts towards making the other 98.5% of
> the computation faster.

This is true in general, but I have a different use case for one
of my programs that uses numpy on a cluster.  Basically, the
program gets called thousands of times per day and the runtime for
each is only a second or two.  In this case I am much more
dominated by numpy's import time.

Scott

PS: Yes, I could change the way that the routine works so that it
is called many fewer times, however, that would be very difficult
(although not impossible).  A "free" speedup due to faster numpy
import would be very nice.


-- 
Scott M. Ransom            Address:  NRAO
Phone:  (434) 296-0320               520 Edgemont Rd.
email:  sransom at nrao.edu             Charlottesville, VA 22903 USA
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989