right list for SIGABRT python binary question ?

[Karsten Hilbert]

On Wed, Oct 18, 2017 at 02:07:46PM +0200, Thomas Jollans wrote:

> > When run under a debug build it sayeth right away (simulated
> > as a minimal working example):
> > 
> >         root at hermes:~/bin# python2.7-dbg
> >         Python 2.7.14 (default, Sep 17 2017, 18:50:44)
> >         [GCC 7.2.0] on linux2
> >         Type "help", "copyright", "credits" or "license" for more information.
> >         >>> import mx.DateTime
> >         *** You don't have the (right) mxDateTime binaries installed !
> >         Traceback (most recent call last):
> >           File "<stdin>", line 1, in <module>
> >           File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in <module>
> >             from DateTime import *
> >           File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in <module>
> >             from mxDateTime import *
> >           File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in <module>
> >             raise ImportError, why
> >         ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4
> 
> I don't really what exactly is going on here, but in general extension
> modules compiled for non-debug builds won't work with debug builds.

OK, that helps. I found -dbg builds of mx.DateTime in Debian.

> > For good measure I have filed a bug with Debian asking for
> > recompilation of python-egenix-mxdatetime.
> 
> Even if the maintainers do that, it won't help. Check that the module
> works on its own with the regular Python build and then close the bug if
> the maintainers don't beat you to it.

Done.

> > When run under "normal" py2.7 it runs all the way through but
> > upon shutdown (*after* sys.exit(0)) faulthandler shows a
> > problem (and returns 134 which made me think of SIGABRT):
> 
> We still don't know what "it" is.
> 
> Strip down your script as much as possible. It looks like you're using a
> lot of extension modules, and any one of them could (in theory) be
> causing problems.

I know. However, stripping down isn't quite so easy (it never
is, which is a rather lame excuse for not doing so yet :-)

The script itself is but a meager 1843 lines (but using lots
of Python modules, both stdlib and my own). What it does is
take a large number of SQL files (roughly a thousand) and
replaying them against PostgreSQL thereby bootstrapping a
database layout from scratch up to the current version (21).
The abort-on-exit only happens if "enough" SQL scripts have
been run (sounds like a resource leak) and the bootstrapper
script exits (having encountered a database problem or not
makes no difference).  Running various parts of the whole
procedure does not make a difference as to whether the fault
occurs when exiting, if only "enough" SQL scripts are run.

I am currently running the bootstrapper with mxdatetime as a
dbg build to see what gives. The only other C extension I am
aware of that is in use is psycopg2.

However, none of these two (nor any other modules) have been
added to the bootstrapper (or updated) recently (in fact,
this setup has been in use for, what, a decade?) so it is
quite unclear why they should be at fault now.

The one thing that did change is PostgreSQL going from 9.6 to
10, effecting a libpq5 recompiled against the newer PG
client. I wonder whether psycopg2 needs a recompile against
that libpq5 (despite there not having been changes advertised
by PG).

I guess I'll have to downgrade libpq5 to 9.6 and see what
happens.

I'll report what I find.

(oh, and it's all available on github)

Thanks,
Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346