PyGILState_Release called twice in embedded application

[Barry]

> On 23 Mar 2023, at 14:34, Arnaud Loonstra <arnaud at sphaero.org> wrote:
> 
> On 23-03-2023 13:33, Barry Scott wrote:
>>>> On 23 Mar 2023, at 08:46, Arnaud Loonstra <arnaud at sphaero.org> wrote:
>>> 
>>> Hi all,
>>> 
>>> I'm running in a crash due to a ResourceWarning (some socket is not closed in a used module) after calling PyGILState_Release.
>>> 
>>> I'm running Python in a native thread (so a thread created by C not Python). I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls and releasing when finished using PyGILState_Release.
>>> 
>>> This has worked fine. But now I'm using a python module (openai) which doesn't close its socket correctly which results in a ResourceWarning which triggers an assert.
>>> 
>>> In the backtrace  (below) I can see PyGILState_Release is called again. (7) while I've already called it (126).
>>> 
>>> I can make the crash go away by adding
>>> 
>>> import warnings
>>> warnings.simplefilter("ignore", ResourceWarning)
>>> 
>>> to my python code. But I'd rather prevent this from happening in the first place.
>>> 
>>> Any suggestion very welcomed cause I'm puzzled.
>> What 3rd party C extension are you running with?
>> I'd be surprised if the cpython code was the issue.
>> Barry
> 
> I'm not using any 3rd party extension myself. But the issue is caused by the openai module using the requests module which is not closing sockets:
> 
> https://github.com/openai/openai-python/issues/140
> 
> I'm not aware what C extensions they might use.

In gdb use info shared to see what is loaded.

Leaving a socket open will not cause a logic error as you are encountering.
I expect a bug in C/C++ cod3 that is not part of python itself.
You are using 3rd party code, requests and openai that you name.

Barry

> 
> Btw, I've tested this with python 3.8 and 3.11.2.
> 
> Rg,
> 
> Arnaud
>