Can you mock a C function using ctypes?

[Eryk Sun]

On 9/15/22, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>
> Does the pointer have to be passed? Or can it be "poked" into a global
> variable?

If the library exports the function pointer as a global variable, it
can be set in ctypes using the in_dll() method. Unfortunately, as far
as I know, a ctypes function prototype can't be used directly for
this, at least not easily, since the function pointer it creates
doesn't have a `value` or `contents` attribute to set the target
address. Instead, the exported global can be accessed as a void
pointer. It's clunky, but it works.

For example, Python's REPL supports a callback for reading a line from
the terminal. Normally it's either hooked by a C extension, such as
the readline module, or set to the default function
PyOS_StdioReadline(). We can use a ctypes callback to hook into this
and chain to the previous function.

    import ctypes

    PyOS_RFP = ctypes.CFUNCTYPE(
        ctypes.c_void_p,
        ctypes.c_void_p, # stdin (FILE *)
        ctypes.c_void_p, # stdout (FILE *)
        ctypes.c_char_p, # prompt
    )

    @PyOS_RFP
    def readline_hook(stdin, stdout, prompt):
        print('HOOKED: ', end='', flush=True)
        return prev_rfp(stdin, stdout, prompt)

    rfp_vp = ctypes.c_void_p.in_dll(ctypes.pythonapi,
                'PyOS_ReadlineFunctionPointer')

    if rfp_vp:
        # there's a previous RFP that can be hooked
        prev_rfp = ctypes.cast(rfp_vp, PyOS_RFP)
        rfp_vp.value = ctypes.cast(readline_hook, ctypes.c_void_p).value

In this trivial example, "HOOKED: " is printed to the terminal before
each line is read:

    HOOKED: >>> def f():
    HOOKED: ...     pass
    HOOKED: ...
    HOOKED: >>>

Note that I defined the return type of PyOS_RFP to be c_void_p. Using
c_char_p as the return type would convert the result to a bytes
object, which is wrong in this case. If the setfunc of the callback's
return type has to keep a reference to a converted Python object, such
as a bytes object, then the callback has no choice but to leak the
reference in order to keep the object alive indefinitely, which
usually causes a memory leak.