[issue26799] gdb support fails with "Invalid cast."

[Thomas]

Thomas added the comment:

I have done a bit more digging, turns out it is actually no problem at all to debug python in gdb with gdb with python support (at least using a fixed python-gdb-py).

Turns out the type->length of the the globally initialized ptr types is wrong: It is 4 instead of 8, causing the cast to fail. I suspect the initialization is done before the executable is loaded and gdb is using some default. To verify, I have put two prints in the global initialization and in a command invocation:

GOBAL INITIALIZATION: gdb.lookup_type('char').pointer().sizeof == 4
COMMAND INVOKE: gdb.lookup_type('char').pointer().sizeof == 8

I guess to be fully portable those types need to be looked up at least whenever gdb changes it's binary, but I have no idea if there is a hook for that. But it seems reasonable to just replace those globals with on-demand lookup functions or properties.

If you are interested in the actual python/c stack traces for the error:

Thread 1 "gdb" hit Breakpoint 1, value_cast (type=type at entry=0x2ef91e0, arg2=arg2 at entry=0x32b13f0) at ../../gdb/valops.c:571
571	      error (_("Invalid cast."));
(gdb) py-bt
Traceback (most recent call first):
  <built-in method cast of gdb.Value object at remote 0x3260de0>
  File "[...]/python-gdb.py", line 1151, in proxyval
    field_str = field_str.cast(_type_unsigned_char_ptr)
  File "[...]/python-gdb.py", line 945, in print_traceback
    % (self.co_filename.proxyval(visited),
  File "[...]/python-gdb.py", line 1578, in print_traceback
    pyop.print_traceback()
  File "[...]/python-gdb.py", line 1761, in invoke
    frame.print_traceback()
(gdb) bt
#0  value_cast (type=type at entry=0x2ef91e0, arg2=arg2 at entry=0x32b13f0) at ../../gdb/valops.c:571
#1  0x000000000052261f in valpy_do_cast (self=<gdb.Value at remote 0x3260de0>, args=<optimized out>, op=UNOP_CAST) at ../../gdb/python/py-value.c:525
#2  0x00007fc7ce2141de in PyCFunction_Call (func=func at entry=<built-in method cast of gdb.Value object at remote 0x3260de0>, args=args at entry=(<gdb.Type at remote 0x2ed0fb0>,), kwds=kwds at entry=0x0)
    at ../../Python-3.5.1/Objects/methodobject.c:109
#3  0x00007fc7ce2c8887 in call_function (pp_stack=pp_stack at entry=0x7ffffc81cec8, oparg=oparg at entry=1) at ../../Python-3.5.1/Python/ceval.c:4655
#4  0x00007fc7ce2c57ac in PyEval_EvalFrameEx (

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26799>
_______________________________________