to py or not to py ?

[Carl J. Van Arsdall]

Serge Orlov wrote:
> On 6/27/06, Chandrashekhar kaushik <shekhar.kaushik at gmail.com> wrote:
>   
>> HI all
>> I have the following prob.
>> I am to write a parallel vis application .
>> I wud have by default used C++ for the same but somehow
>> thought if py cud help me ..
>> It does as in many things that i would otherwise have written down
>> already exists ... ( like built in classes for sockets , threading etc )
>>
>> I would be doin the following type of tasks ..
>>
>> 1. sending data across the network
>>     the data is going to be huge
>>
>> 2. once data has been sent i will run some vis
>>     algos parallely on them and get the results
>>
>> now one thing that i wud req. is serializing my data structures so that
>> they can be sent across the net.
>>
>> pyton does allow this using cPickle , but it bloats the data like anythin
>> !!!
>> for example a class containing 2 integers which i expect will be 8 bytes
>> long ..
>> cPickle.dumps returns a string thats 86 bytes wide !!!! ( this is the binary
>> version protocol 1 )
>>
>> anyway to improve serialization ??
>>     
>
> Do it yourself using struct module.
>
>   
>> also is it actually a good idea to write high perf applications in python ?
>>     
>
> Take a look at Mercurial <http://www.selenic.com/mercurial/> sources.
> It's a high performance python application. Or watch Bryan
> O'Sullivan's Mercurial presentation
> <http://video.google.com/videoplay?docid=-7724296011317502612> he
> talks briefly how they made it work fast.
>
> But writing high performance application in python requires
> self-discipline and attention to details, looking at the way you spell
> I think it will be a challenge ;)
>   
One more comment would be that running parallel applications on python 
can be problematic because of the way python does threading (and i'm not 
presently aware of other threading libraries).  Parallelizing in python 
doesn't do much good unless you spend a lot of time blocking on IO 
because of the GIL. 

-- 

Carl J. Van Arsdall
cvanarsdall at mvista.com
Build and Release
MontaVista Software