[issue36880] Returning None from a callback with restype py_object decrements None's refcount too much

[dgelessus]

New submission from dgelessus <dgelessus+bugs.python.org at me.com>:

This occurs when writing a ctypes callback in Python whose restype is ctypes.py_object. If the callback returns None, the refcount of None is decremented once too often. This happens every time the callback is called, and if done often enough, Python attempts to deallocate None and crashes.

To reproduce:

    $ bin/python3
    Python 3.8.0a4+ (heads/master:09532feeec, May 10 2019, 23:53:49) 
    [Clang 8.0.0 (clang-800.0.42.1)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> import ctypes
    >>> FUNTYPE = ctypes.CFUNCTYPE(ctypes.py_object)
    >>> @FUNTYPE
    ... def fun():
    ...     return None
    ... 
    >>> print(fun())
    None
    >>> sys.getrefcount(None)
    4329
    >>> print(fun())
    None
    >>> sys.getrefcount(None)
    4327
    >>> print(fun())
    None
    >>> sys.getrefcount(None)
    4326
    >>> while True:
    ...     fun()
    ... 
    Fatal Python error: deallocating None
    
    Current thread 0x00007fff7bf80000 (most recent call first):
      File "<stdin>", line 2 in <module>
    Abort trap: 6
    # exits with code 134 (SIGABRT)

I've attached the crash report generated by OS X. (It's a plain text file, despite the extension.)

Interestingly, this only happens with None. Other returned objects are refcounted correctly:

    >>> thing = object()
    >>> @FUNTYPE
    ... def otherfun():
    ...     return thing
    ... 
    >>> print(otherfun())
    <object object at 0x10d920220>
    >>> sys.getrefcount(thing)
    2
    >>> print(otherfun())
    <object object at 0x10d920220>
    >>> sys.getrefcount(thing)
    2
    >>> print(otherfun())
    <object object at 0x10d920220>
    >>> sys.getrefcount(thing)
    2

I could reproduce this on the following configurations:

* Python 3.8.0a4 (self-compiled from master branch: 09532feeece39d5ba68a0d47115ce1967bfbd58e) on OS X 10.11 (x86_64)
* Python 3.7.3 (python.org installer) on OS X 10.11 (x86_64)
* Python 3.6.8 (from package manager) on Kubuntu 18.10 (x86_64)
* Python 3.5.3 (from package manager) on Raspbian Stretch (armv6)

----------
components: ctypes
files: python3.8_2019-05-11-000251.crash
messages: 342141
nosy: dgelessus
priority: normal
severity: normal
status: open
title: Returning None from a callback with restype py_object decrements None's refcount too much
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48325/python3.8_2019-05-11-000251.crash

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36880>
_______________________________________