[Idle-dev] multiprocessing module and IDLE without subprocess

[Tal Einat]

On Fri, Nov 28, 2008 at 4:17 PM, Weeble <clockworksaint at gmail.com> wrote:
> It appears that any attempt to spawn processes using the new
> multiprocessing module in Python 2.6 will instead open up a new empty
> file in an IDLE window, *if* running without a subprocess. I don't
> particularly *want* to run without a subprocess, but the alternative
> on Windows is pretty bad: you can only have one instance of IDLE and
> trying to open a new one will sit waiting for ages before complaining
> that it can't connect to the subprocess. Is it possible to get this to
> work without a subprocess, and if not, what will it take to get the
> subprocess working properly on Windows? I believe a patch was written
> some time ago to do this, but it never made it in. Was it too much of
> a hack? Is there a better way to do this?
>
> http://bugs.python.org/issue1529142
>
> I'd like to help, but I'm not sure how to get started in IDLE
> development. Is there a guide somewhere? I looked at
>
> http://www.python.org/idle/
>
> but it's completely empty.

First of all, the multiprocessing module doesn't work fully in an
interactive shell.

>From the Python2.6 docs:
"Note Functionality within this package requires that the __main__
method be importable by the children. This is covered in Programming
guidelines however it is worth pointing out here. This means that some
examples, such as the multiprocessing.Pool examples will not work in
the interactive interpreter."

And later on that same page (under Programming Guidelines -> Windows):
"Make sure that the main module can be safely imported by a new Python
interpreter without causing unintended side effects (such a starting a
new process)."

This is probably why IDLE is acting up for you. I would be surprised
if using the multiprocessing module will work well in IDLE either with
or without a sub-process.


As per the patch, someone should just work up an updated one with all
of the insight gained in the last several years (!) regarding this.

It should just remove the use rof the SO_REUSEADDR flag in the
sub-process spawning code and pass zero as the argument for the port
number, which instructs the underlying socket library to select an
available port. See http://bugs.python.org/issue1201569 and
http://mail.python.org/pipermail/idle-dev/2008-June/002687.html for
details.

The other changes in my original patch, while they do work, are
unnecessarily complex and make it much harder to get the patch
accepted. They should also not be significant if the spawning delays
are fixed.

If you'd like to work up such a patch and get it reviewed and
committed, please be my guest! I will help you in any way I can if you
take this up. Otherwise I guess it will wait until I find the time and
energy...

- Tal