PyGILState_Release called twice in embedded application

[Arnaud Loonstra]

On 23-03-2023 13:33, Barry Scott wrote:
> 
> 
>> On 23 Mar 2023, at 08:46, Arnaud Loonstra <arnaud at sphaero.org> wrote:
>>
>> Hi all,
>>
>> I'm running in a crash due to a ResourceWarning (some socket is not closed in a used module) after calling PyGILState_Release.
>>
>> I'm running Python in a native thread (so a thread created by C not Python). I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls and releasing when finished using PyGILState_Release.
>>
>> This has worked fine. But now I'm using a python module (openai) which doesn't close its socket correctly which results in a ResourceWarning which triggers an assert.
>>
>> In the backtrace  (below) I can see PyGILState_Release is called again. (7) while I've already called it (126).
>>
>> I can make the crash go away by adding
>>
>> import warnings
>> warnings.simplefilter("ignore", ResourceWarning)
>>
>> to my python code. But I'd rather prevent this from happening in the first place.
>>
>> Any suggestion very welcomed cause I'm puzzled.
> 
> What 3rd party C extension are you running with?
> I'd be surprised if the cpython code was the issue.
> 
> Barry

I'm not using any 3rd party extension myself. But the issue is caused by 
the openai module using the requests module which is not closing sockets:

https://github.com/openai/openai-python/issues/140

I'm not aware what C extensions they might use.

Btw, I've tested this with python 3.8 and 3.11.2.

Rg,

Arnaud