[omaha] Using Multiprocessing

[Aaron Keck]

I use this pattern all the time for a lot of my scripts, but I can never
quite get my logging to work out right.  Anyone have any luck with logging
and multiprocessing?

On Wed, Nov 18, 2015 at 10:18 PM, Burch Kealey via Omaha <omaha at python.org>
wrote:

> I apologize, I have one brain cell.  I do not remember the source of the
> question.  I was going to borrow Jeff's laptop to log in at home but since
> it was a Linux machine it was beyond my capabilities.
>
> To run a multi-threaded app can be quite trivial with Python's
> multiprocessing library
>
> from multiprocessing import pool
> from multiprocessing import cpu_count
>
>
> def my_function(input):
>     do something with the input
>     return something (or not)
>
>
> items = some_iterable <- I usually am processing a list of things
>
> p = Pool(cpu_count)
> results = p.map(my_function, items)
>
> There is a lot more you can do with it but at the most basic, the items
> list will be broken into chunks and my_function will be called on separate
> threads iteratively with an item from the list.
>
> from the docs:  The multiprocessing module also introduces APIs which do
> not have analogs in the threading module. A prime example of this is the
> Pool object which offers a convenient means of parallelizing the execution
> of a function across multiple input values, distributing the input data
> across processes (data parallelism).
>
> This is part of the magic of Python, batteries included and you don't
> really have to know what is under the hood - it just works.
>
> Cheers
>
> Burch
> ________________________________________
> From: Omaha <omaha-bounces+bkealey=mail.unomaha.edu at python.org> on behalf
> of Jeff Hinrichs - DM&T via Omaha <omaha at python.org>
> Sent: Wednesday, November 18, 2015 9:58 PM
> To: Omaha Python Users Group
> Cc: Jeff Hinrichs - DM&T
> Subject: [omaha] Link to tonight's slides
>
> I am just getting this out here quick.
>
> https://docs.google.com/presentation/d/1DRXQ36LRCC1_wj1vEm2jsFf3TMMU9UjnvhpG-0Lor2s/edit?usp=sharing
>
> It will of course be on the meeting notes on the site, once those are in
> place.
>
> Rob Townley: I'll get some instructions setup for you on how to get the
> robot set up.
>
> --
> Best,
>
> Jeff Hinrichs
> 402.218.1473
> _______________________________________________
> Omaha Python Users Group mailing list
> Omaha at python.org
> https://mail.python.org/mailman/listinfo/omaha
> http://www.OmahaPython.org
> _______________________________________________
> Omaha Python Users Group mailing list
> Omaha at python.org
> https://mail.python.org/mailman/listinfo/omaha
> http://www.OmahaPython.org
>