Python + Shoutpy + Twisted Locks

[Jean-Paul Calderone]

On Mon, 08 Oct 2007 23:00:46 GMT, John Nagle <nagle at animats.com> wrote:
>Chris Mellon wrote:
>> On 10/7/07, Michel Albert <exhuma at gmail.com> wrote:
>>> On Oct 6, 4:21 am, "Gabriel Genellina" <gagsl-... at yahoo.com.ar> wrote:
>>>> En Fri, 05 Oct 2007 04:55:55 -0300, exhuma.twn <exh... at gmail.com> escribi?:
>>>>
>>>>> [...] What I found
>>>>> is that "libshout" is blocking, which should be fine as the whole
>>>>> thing runs in it's separate thread. But the application hangs
>>>>> nevertheless while streaming. This effectively blocks out the other
>>>>> thread that checks the player status, which then fails to append new
>>>>> songs to the queue. So only one song is played when streaming.
>>>>> The other threads in my application run fine and don't block the rest
>>>>> of the app. So I guess, that the main problem is that blocking occurs
>>>>> "outside" the python world and "inside" the libshout world.
>>>> Only one thread at a time may be executing Python code; the Global
>>>> Interpreter Lock (GIL) ensures the mutual exclusion. Extension modules
>>>> (written in C) may use the macros
>>>> Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS to release/acquire the GIL
>>>> before/after an external blocking call.
>>>> I don't know libshout, or how you are doing the binding python-libshout,
>>>> but if your analysis is correct it means that the code is not releasing
>>>> the GIL at the appropiate points.
>>>>
>>>> --
>>>> Gabriel Genellina
>>> Hmmm... ok. I suppose rewriting the whole thing using twisted's
>>> deferreds could then solve the problem. Which are basically nothing
>>> more than callbacks with a weird name ;) Unfortunately this means that
>>> I have to re-think a lot. But in the end I suppose it will pay off.
>>>
>>> Thanks for taking the time and reading my little essay Gabriel ;)
>>>
>>
>> Using Twisted won't help if the libshout calls are really blocking the
>> main thread.
>
>     Right.
>
>     The whole point of Twisted is that you can't do anything that takes
>very long in a Twisted thread, or the whole server will stall.  If you
>try to do something ongoing, like streaming, in a Twisted server, you
>have to have long-running threads of your own and a scheduling system
>to coordinate them with the Twisted threads.

Except, of course, that there isn't any such thing as a "Twisted thread". :)

Jean-Paul