Newbie Threads question

[Gordon McMillan]

Michael "Fred" Fredericks writes:

> What I'd like to do is learn how to use threads by "parallelizing" a
> simple set of now currently serialized tasks.	No shared resources
> here.  I understand that in all likelihood I will not get any
> performance increase (since I'm on a uni-processor system) but I
> want to learn how to do this stuff.

If your processing is CPU-bound, this is correct. If not, you will 
almost certainly see a performance boost.
 
> hypothetically my code looks like this:
> 
> def func1(param):
>    result = []
>    # do a bunch of work, takes about 3 seconds of wall-clock time
>    return result
> 
> def func2(param):
>    result = []
>    # do a bunch of work, takes about 5 seconds of wall-clock time
>    return result
> 
> def func3(list1, list2):
>    result = []
>    # does work on the two lists passed in, takes 0.5 secs of
>    wall-clock time return result
> 
> 
> #my main loop simplified
> list1 = func1(x)
> list2 = func2(y)
> reallist = func3(list1,list2)
> #do something with reallist.
> 
> Since func1 and func2 are completely independent - IE do not use any
> of the same resources, to me this would be a great place to do the
> work in parallel.
>  What I want to do is something like this:
> 
> #my new main loop
> list1 = thread.start_new_thread(func1,(x))
> list2 = thread.start_new_thread(func2,(y))
> reallist = func3(list1,list2)
> 
> I have read the documentation a couple of times, but what I don't
> know:
> 
> A) Is the return value of start_new_thread the same as the return
> value of the function it calls?

If it were, threads wouldn't be of much use, would they? You'll have 
to stash your intermediate results some place for the main thread to 
pick them up.
 
> B) Do I need to do anything fancy to make the func3 call wait until
> the calls to func1 and func2 have both returned?	

If you're using the low-level thread module, yes. If you use the 
higher level threading module, Guido's done most of the fancy stuff 
for you. You *will* have to wait on the 2 parallel threads before 
going on to func3.

If you're determined to go the low-level route, look at the 
test_thread script. Then go the high-level route!

> For example, should
> I have the func1 and func2 calls aquire a lock at the start of the
> function and release it at the end, and then make the main loop try
> to acquire both locks before proceeding to the reallist = func3()
> call?  If I did this, could the main loop acquire the locks before
> the new threads do, causing deadlock?

If you're mucking about at this level, you'll find you need the 
occaisional "sleep(0.000001)". This will force the current thread to 
release the interpreter lock. If _anything_ else is ready to go, it 
will go.

> C) I'm assuming I have to recompile python (since I didn't compile
> it "with threads")

Assuming you're on *nix, yes (it's on by default in Win32). You'll 
also need to make clean and start over from configure. Depending on 
your *nix, you may or may not have some cursing to do...

- Gordon