Upper memory limit

[Fernando Pérez]

Siegfried Gonzi wrote:

> def f(list,list2):
>         erg = []
>         for k in range( len(list) ):
>                 erg.append( function_on_list2( list2[j] ) )
>         return erg
> 
> 
> I find the above very readable (and often better in style than Numeric's
> idiosyncrasy).
> 

You may find it 'very readable (and often better in style than Numeric's
idiosyncrasy)' but in many cases the above is about the worst possible way of 
dealing with a Numeric array you could come up with. What you call 'Numeric's 
idiosyncrasy' has a very good reason for being the way it is: when you work 
with Numeric arrays as single entities and apply functions to them as a 
whole, you gain the _massive_ advantage of looping being done in C instead of 
Python. This can make a difference of one or two orders of magnitude in 
execution time.

I personally can't stand windows and consider Linux a zillion times better as 
an environment for scientific computing, but in this particular case I can't 
guarantee that your memory problems will go away in Linux. I simply have no 
idea. On the other hand you'll have a real operating system with real tools 
to work on your problems, instead of the candy-wrapper colored toy that 
Windows is (even if XP doesn't crash every 5 minutes like the previous 
generations used to, it's still a toy).

But I would seriously recommend you go through your code with a fine tooth 
comb and replace any instances of code like the above with Numeric array 
operations whenever possible. Put your stylistic ideas in your back pocket 
while you work on this problem, there's a very good reason for Numeric 
working the way it does. Incidentally this idea of working over whole arrays 
reappears (for the same exact underlying reason) if you look at systems like 
IDL or Matlab (and is _heavily_ emphasized in their documentations) so this 
has nothing to do with Numeric only.

Best of luck,

f.