Threads vs Processes

[bryanjugglercryptographer at yahoo.com]

Carl J. Van Arsdall wrote:
> Ah, alright, I think I understand, so threading works well for sharing
> python objects.  Would a scenario for this be something like a a job
> queue (say Queue.Queue) for example.  This is a situation in which each
> process/thread needs access to the Queue to get the next task it must
> work on.  Does that sound right?

That's a reasonable and popular technique. I'm not sure what "this"
refers to in your question, so I can't say if it solves the
problem of which you are thinking.

>  Would the same apply to multiple
> threads needed access to a dictionary? list?

The Queue class is popular with threads because it already has
locking around its basic methods. You'll need to serialize your
operations when sharing most kinds of objects.

> Now if you are just passing ints and strings around, use processes with
> some type of IPC, does that sound right as well?

Also reasonable and popular. You can even pass many Python objects
by value using pickle, though you lose some safety.

>  Or does the term
> "shared memory" mean something more low-level like some bits that don't
> necessarily mean anything to python but might mean something to your
> application?

Shared memory means the same memory appears in multiple processes,
possibly at different address ranges. What any of them writes to
the memory, they can all read. The standard Python distribution
now offers shared memory via os.mmap(), but lacks cross-process
locks.

Python doesn't support allocating objects in shared memory, and
doing so would be difficult. That's what the POSH project is
about, but it looks stuck in alpha.


-- 
--Bryan