[Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD

[Jesse Noller]

On Wed, Dec 29, 2010 at 8:17 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> FreeBSD 7.2 3.x buildbot is red since some weeks (or months?) because of
> a concurrent.futures failure. The problem is that
> test_concurrent_futures uses many (multiprocessing) POSIX semaphores,
> whereas POSIX semaphores support in FreeBSD is recent and limited. We
> have to use SysV semaphores (ftok, semget, semop, semctl, ...) instead
> of POSIX semaphores (sem_open, sem_getvalue, sem_unlink, ...). See:
>
>  * http://bugs.python.org/issue10348
>  * "Too many open files" errors on "x86 FreeBSD 7.2 3.x" buildbot
>   ^-- thread in python-dev opened last month
>
> I would like to know if it should be considered as a release blocker.
> Georg Brandl said yes on IRC. Does anyone know SysV API? I tried to
> write a patch, but I never used semaphores (POSIX or SysV).
>
> There is a third party module which looks complete and stable:
> http://semanchuk.com/philip/sysv_ipc/
>
> It is released under the BSD license. It supports semaphores, but also
> shared memory and message queues. We don't need all of those, semaphores
> would be enough. I added its author (Philip Semanchuk) to this thread.
>
> I don't know how we should decide to use POSIX or SysV semaphores. It
> looks like SysV is preferred on FreeBSD and Darwin (and maybe all BSD
> based OSes), and POSIX is preferred on Linux.
>
> Victor

The concurrent.futures tests should (like the multiprocessing test
suite) detect the lack of support and skip the tests on the broken
platforms. I'm sort of surprised FreeBSD support is still broken in
this way though (echoed by Philip) although it could be an issue on
that particular buildbot.

Moving from POSIX IPC to SYSV should *not* be on the plate for a
release blocker - that's a much larger task.

jesse