Python 2.6, multiprocessing module and BSD

[Philip Semanchuk]

On Oct 22, 2008, at 10:11 AM, Jesse Noller wrote:

> On Tue, Oct 21, 2008 at 6:45 PM,  <YouCanCallMeAl at gmail.com> wrote:
>> It seems that the multiprocessing module in 2.6 is broken for *BSD;
>> I've seen issue 3770 regarding this. I'm curious if there are more
>> details on this issue since the posts in 3770 were a bit unclear. For
>> example, one post claimed that the problem was that sem_open isn't
>> implemented in *BSD, but it is available on FreeBSD 7 (I checked).  
>> I'd
>> be willing to help get this working if someone could point me in the
>> right direction.
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
> The BSD issue was raised late in the cycle for 2.6. The problem is
> that FBSD's support is "very experimental" as Phillip points out - and
> OpenBSD doesn't even have them.
>
> Due to the lateness of the issue and a finite amount of time I have to
> work on things, I chose to disable support for this on the various
> *BSDs until I can cook up a stable patch or have one provided by
> someone more familiar with the inner workings of Free-BSD. OpenBSD
> support is a non-starter.

Hi Jesse,
I wasn't aware of the multiprocessing module. It looks slick! Well done.

I don't know if you clicked on the link I gave for my posix_ipc  
module, but it looks like we're duplicating effort to some degree. My  
module makes POSIX semaphore & shared memory primitives available to  
Python programs. Obviously, what you've done is much more sophisticated.

One oversight I noticed the multiprocessing module docs is that a  
semaphore's acquire() method shouldn't have a timeout on OS X as  
sem_timedwait() isn't supported on that platform. (You note OS X's  
lack of support for sem_getvalue() elsewhere.)

A question - how do you handle the difference in error messages on  
various platforms? For instance, sem_trywait() raises error 35,  
"Resource temporarily unavailable" under OS X but error 11 under  
Ubuntu. Right now I'm just passing these up to the (Python) caller as  
OSErrors. This makes it really hard for the Python programmer to write  
cross-platform code.

The only solution I can think of (which I haven't coded yet) is to  
compile & run a series of small C programs during setup.py that test  
things like sem_trywait() to see what errors occur, and provide those  
constants to my main .c module so that it can detect those errors  
exactly and wrap them into a specific, custom error for the Python  
caller.

Any thoughts on this?

Cheers
Philip