[Python-ideas] multiprocessing and physical CPU cores count

[Giampaolo Rodola']

On Fri, Sep 13, 2013 at 10:14 AM, M.-A. Lemburg <mal at python.org> wrote:

> On 12.09.2013 21:51, Giampaolo Rodola' wrote:
> > On Thu, Sep 12, 2013 at 9:32 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >
> >>> Then the question is whether having physical CPU cores count can be
> >> useful.
> >>
> >> I suppose it doesn't hurt :-) I don't think it belongs specifically in
> >> multiprocessing, though. Perhaps in the platform module?
> >>
> >
> > I'd be +0.5 for multiprocessing because:
> >
> > - cpu_count() is already there
> > - physical_cpu_count() will likely be used by multiprocessing users only
> >
> > ...but my main concern was first figuring out whether it might actually
> > make sense to distinguish between virtual and physical CPUs in a real
> world
> > app.
>
> I'm with Antoine here: both APIs would make more sense in the
> platform module.


In the end it appears the os module would probably be better as cpu_count()
already ended up there (http://bugs.python.org/issue17914) as pointed out
by Victor a couple of emails ago.
I have the impression no one is opposed so I can probably start working on
a patch and submit it on the bug tracker.


> Do you need C code for determining the physical count ?


Yes, except on Linux where you'll just read /proc/cpuinfo.


>  >> (unless you want to contribute psutil to the stdlib?)
> >
> >
> > That's something I'd be happy to do if there's general approval but I
> guess
> > that's for another thread.
>
> I'd love to see psutils in the stdlib, but also be warned: once
> the code lives in the stdlib,
>
> a) making changes is difficult and adding new features as well,
>
> b) you are bound by the Python release cycle.
>
> For a package such psutil, it may actually be better to keep it
> outside the stdlib, since the outside world changes regularly
> and doesn't adhere to the Python release cycle or feature
> for patch level releases ;-)
>

Yeah, you're probably right,and there's at least a couple of high priority
functionalities I'd like to add first (to say one: dragonfly/open/net BSD
support).


--- Giampaolo
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/psutil/
https://code.google.com/p/pysendfile/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130913/fbd2a213/attachment.html>