pytest segfault, not with -v

[MRAB]

On 2021-11-20 17:40, Marco Sulla wrote:
> Indeed I have introduced a command line parameter in my bench.py
> script that simply specifies the number of times the benchmarks are
> performed. This way I have a sort of segfault checker.
> 
> But I don't bench any part of the library. I suppose I have to create
> a separate script that does a simple loop for all the cases, and
> remove the optional parameter from bench. How boring.
> PS: is there a way to monitor the Python consumed memory inside Python
> itself? In this way I could also trap memory leaks.
> 
I'm on Windows 10, so I debug in Microsoft Visual Studio. I also have a 
look at the memory usage in Task Manager. If the program uses more 
memory when there are more iterations, then that's a sign of a memory 
leak. For some objects I'd look at the reference count to see if it's 
increasing or decreasing for each iteration when it should be constant 
over time.

> On Sat, 20 Nov 2021 at 01:46, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> On 2021-11-19 23:44, Marco Sulla wrote:
>> > On Fri, 19 Nov 2021 at 20:38, MRAB <python at mrabarnett.plus.com> wrote:
>> >>
>> >> On 2021-11-19 17:48, Marco Sulla wrote:
>> >> > I have a battery of tests done with pytest. My tests break with a
>> >> > segfault if I run them normally. If I run them using pytest -v, the
>> >> > segfault does not happen.
>> >> >
>> >> > What could cause this quantical phenomenon?
>> >> >
>> >> Are you testing an extension that you're compiling? That kind of problem
>> >> can occur if there's an uninitialised variable or incorrect reference
>> >> counting (Py_INCREF/Py_DECREF).
>> >
>> > Ok, I know. But why can't it be reproduced if I do pytest -v? This way
>> > I don't know which test fails.
>> > Furthermore I noticed that if I remove the __pycache__ dir of tests,
>> > pytest does not crash, until I re-ran it with the __pycache__ dir
>> > present.
>> > This way is very hard for me to understand what caused the segfault.
>> > I'm starting to think pytest is not good for testing C extensions.
>> >
>> If there are too few Py_INCREF or too many Py_DECREF, it'll free the
>> object too soon, and whether or when that will cause a segfault will
>> depend on whatever other code is running. That's the nature of the
>> beast: it's unpredictable!
>>
>> You could try running each of the tests in a loop to see which one
>> causes a segfault. (Trying several in a loop will let you narrow it down
>> more quickly.)
>>
>> pytest et al. are good for testing behaviour, but not for narrowing down
>> segfaults.
>