client-server parallellised number crunching

[Hans Georg Schaathun]

On Tue, Apr 26, 2011 at 1:20 PM, Chris Angelico <rosuav at gmail.com> wrote:
> But question: Why are you doing major number crunching in Python? On
> your quad-core machine, recode in C and see if you can do the whole
> job without bothering the unreliable boxen at all.

The reason is very simple.  I cannot afford the time to code it in C.
Furthermore, the work is research and the system is experimental, 
making the legibility of the code paramount.

On Tue, 26 Apr 2011 13:33:25 -0700, Dan Stromberg
  <drsalists at gmail.com> wrote:
:  I'd suggest that rewriting an entire software system in C because of
:  one inner loop, is overkill.  'better to rewrite just the inner loop
:  (if needed after profiling), and leave the rest in Python.

Well, that's the other reason.  The most intense number crunching in
this part of the project is basic vector and matrix operations done in 
numpy.  AFAIU that means I use exactly the same libraries underneath as
I would have done in C.  Please correct me if I am wrong.

I could run a profiler and squeeze out some performance by coding
another couple of components in C, but I cannot afford the programming
time whereas I can afford to waste the CPU cycles.  And once I get the
client/server parallellisation working, I shall be able to reuse it at
negligible cost on other subproblems, whereas the profiling and C
reimplementation would cost almost as much time for every subsystem.

Does that answer your question, Chris?


-- 
:-- Hans Georg