regarding threading

Diez B. Roggisch nospam-deets at web.de
Tue Oct 14 08:15:35 EDT 2003


akash shetty wrote:

> but this takes an awful amt of time.(abt 7 mins)
> is there anyway to speed this up.
> is use of threading feasible and what code do i
> thread( since all i do is process the database).there
> are no other concurrent tasks. so do i divide the
> database into parts and multithread the searching on
> these parts concurrently. is this feasible. or shud i
> be using some kind of multiprocessing running the
> parts(files) as diff processes.

Multiple threads/processes won't buy you anything unless you have a
multiprocessor-machine. In fact, they'll slow down things, as context
switches (which are considerably slower between processes than between
threads) take also their time.

Threads only buy you performance on single processor-machines if you have to
deal with asynchronus events like network packets or userinteraction.

For speeding up your search - if you search brute-force, you could try to go
for something like a shift-and algorithm. 

And it might help to use C and memory-map parts of the file - but I have to
admit that I have no expirience in that field.

Diez




More information about the Python-list mailing list