multiprocessing shows no benefit

Ian Kelly ian.g.kelly at gmail.com
Wed Oct 18 12:13:18 EDT 2017


On Wed, Oct 18, 2017 at 9:46 AM, Jason <jasonhihn at gmail.com> wrote:
> #When I change line19 to True to use the multiprocessing stuff it all slows down.
>
> from multiprocessing import Process, Manager, Pool, cpu_count
> from timeit import default_timer as timer
>
> def f(a,b):
>         return dict_words[a]-b

Since the computation is so simple my suspicion is that the run time
is dominated by IPC, in other words the cost of sending objects back
and forth outweighs the gains you get from parallelization.

What happens if you remove dict_words from the Manager and just pass
dict_words[a] across instead of just a? Also, I'm not sure why
dict_keys is a managed list to begin with since it only appears to be
handled by the main process.



More information about the Python-list mailing list