[Python-Dev] Improved thread switching
Jesse Noller
jnoller at gmail.com
Thu Mar 20 14:58:46 CET 2008
FYI: I shot an email to stdlib-sig about the fact I am proposing the
inclusion of the pyProcessing module into the stdlib. Comments and
thoughts regarding that would be welcome. I've got a rough outline of
the PEP, but I need to spend more time with the code examples.
-jesse
On Wed, Mar 19, 2008 at 9:52 PM, Alex Martelli <aleaxit at gmail.com> wrote:
> Hmmm, sorry if I'm missing something obvious, but, if the occasional
> background computations are sufficiently heavy -- why not fork, do
> said computations in the child thread, and return the results via any
> of the various available IPC approaches? I've recently (at Pycon,
> mostly) been playing devil's advocate (i.e., being PRO-threads, for
> once) on the subject of utilizing multiple cores effectively -- but
> the classic approach (using multiple _processes_ instead) actually
> works quite well in many cases, and this application server would
> appear to be one. (the pyProcessing package appears to offer an easy
> way to migrate threaded code to multiple-processes approaches,
> although I've only played around with it, not [yet] used it for
> production code).
>
>
> Alex
>
>
>
> On Wed, Mar 19, 2008 at 10:49 AM, Adam Olsen <rhamph at gmail.com> wrote:
> > On Wed, Mar 19, 2008 at 11:25 AM, Stefan Ring <s.r at visotech.at> wrote:
> > > Adam Olsen <rhamph <at> gmail.com> writes:
> > >
> > >
> > > > So you want responsiveness when idle but throughput when busy?
> > >
> > > Exactly ;)
> > >
> > >
> > > > Are those calculations primarily python code, or does a C library do
> > > > the grunt work? If it's a C library you shouldn't be affected by
> > > > safethread's increased overhead.
> > > >
> > >
> > > It's Python code all the way. Frankly, it's a huge mess, but it would be very
> > > very hard to come up with a scalable solution that would allow to optimize
> > > certain hotspots and redo them in C or C++. There isn't even anything left to
> > > optimize in particular because all those low hanging fruit have already been
> > > taken care of. So it's just ~30kloc Python code over which the total time spent
> > > is quite uniformly distributed :(.
> >
> > I see. Well, at this point I think the most you can do is file a bug
> > so the problem doesn't get forgotten. If nothing else, if my
> > safethread stuff goes in it'll very likely include a --with-gil
> > option, so I may put together a FIFO scheduler.
> >
> >
> > --
> > Adam Olsen, aka Rhamphoryncus
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>
>
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>
More information about the Python-Dev
mailing list