[Tutor] running multiple concurrent processes

[Oscar Benjamin]

On 4 November 2012 01:03, richard kappler <richkappler at gmail.com> wrote:
> Oscar, that was positively brilliant! Now I get it, I understand how to do
> it, and I think this has rearranged my entire plan for the "MCP." If the MCP
> is basically just a program that calls several other programs(processes) and
> does that bit of coordination between each, then my life just got measurably
> easier. I think. I did some reading after my last post, and manager seems
> like it would be a good tool for a lot of this, or at least what we're
> calling the MCP for purposes of this discussion. Thoughts?

I didn't really understand the above. Is 'manager' some kind of library?

>
> I am reluctant to usurp any more of your time, but you are such a phenomenal
> teacher, might I ask for a quick example of threading like you did with the
> two multiprocessing snippets?

It looks almost exactly the same. The key difference is that you can
just share the value of an integer without needing to do explicitly do
anything. (This is also a big source of problems since sharing the
same data structures directly only works if all objects/operations are
thread-safe http://en.wikipedia.org/wiki/Thread_safety).

#!/usr/bin/env python

from threading import Thread
from time import sleep
from random import randint

def update_brain():
    global how_many_spams
    while True:
        sleep(1)
        n = randint(1, 5)
        print 'thinking...', n
        how_many_spams = n

def chat_with_friends():
    global how_many_spams
    while True:
        sleep(.3)
        print 'chatting:' + how_many_spams * ' spam'

how_many_spams = 1

t1 = Thread(target=update_brain)
t2 = Thread(target=chat_with_friends)

t1.start()
t2.start()

>
> Bill, I appreciate your comment and have given it much thought, Ramit made
> one much the same the other day. Here lies the potential problem, though it
> might not be one at all, I need to do some experimenting. While I am a fan
> of monolithic programming, I'm wondering if what I'm trying to do would work
> on, say an old netbook. That might be a requirement. I'd prefer it not to
> be, but it might. Also, thanks for reminding my addled old brain that event
> driven is called interrupts. I knew that at one point, but seem to have
> flushed it somehow.

Who's Bill? Alan was referring to Twisted that is an event driven
framework. Event driven or asynchronous processing is a third option
(after threads or processes). The Twisted library is also capable of
launching threads and (I think) processes for you so it could
accommodate for all of the possibilities you want in one framework.

>
> eryksun, I am using Linux, not a big fan of Windows, though have to use one
> at work, so thanks for breaking down forking on both platforms for me.
>
> This probably falls into the dumb question category, but I'd rather look
> dumb than be dumb. Other than as limited by CPU and memory (eg: hardware) is
> there any limit to the number of processes that can be run using processing?
> In other words, are there any constraints within Python?

As far as I know Python will allow you to come close to the
OS/hardware limit for both threads and processes. Unless I've
misunderstood what you're planning to do though these limits are high
enough for you to simply not worry about them.


Oscar