This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: pybsddb build fails
Type: Stage:
Components: Build Versions: Python 2.3
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: barry, loewis, zooko
Priority: normal Keywords:

Created on 2003-07-22 20:21 by zooko, last changed 2022-04-10 16:10 by admin. This issue is now closed.

Messages (15)
msg17243 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-22 20:21
This is either a bug in the build, or a bug in the
build docs, or me being dumb.

I read Modules/Setup and it said Berkeley DB 4.1 wasn't
supported, but I later realized that this statement was
out of date.  Then I read somewhere -- I'm sorry I
can't find this doc now that I'm writing this report --
that Berkeley DB 4.1 would be detected by the setup.py
script after the interpreter was built during "make".

So I installed Berkeley DB 4.1 and ran "make".  It
*did* detect that I had Berkeley DB installed, and
attempted to build the wrapper, but failed like this:

building '_bsddb' extension
gcc -pthread -DNDEBUG -g -fPIC -fno-strict-aliasing -I.
-I/home/zooko/playground/python/python/dist/src/./Include
-I/usr/local/stow/python-g/include -I/usr/local/include
 -I/home/zooko/playground/python/python/dist/src/Include -I/home/zooko/playground/python/python/dist/src -c /home/zooko/playground/python/python/dist/src/Modules/_bsddb.
c -o build/temp.linux-i686-2.3/_bsddb.o
gcc -pthread -shared build/temp.linux-i686-2.3/_bsddb.o
-L/usr/local/stow/python-g/lib -L/usr/local/lib
-ldb-4.1 -o build/lib.linux-i686-2.3/_bsddb.so
*** WARNING: renaming "_bsddb" since importing it
failed: build/lib.linux-i686-2.3/_bsddb.so: undefined
symbol: db_create
msg17244 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-22 20:28
Logged In: YES 
user_id=52562

I should mention that I installed the current bsddb4.1 from
sleepycat: db-4.1.25.tar.gz, and I just cvs up'ed Python a
few minutes ago.
msg17245 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-22 20:40
Logged In: YES 
user_id=12800

I'm betting that _bsddb couldn't be imported because the
runtime location of the library couldn't be found.  I
thought at one time we were adding the proper -R flag to get
that compiled in, but it doesn't look like that's getting
added for you.

You could try adding /usr/local/BerkeleyDB.4.1/lib to your
/etc/ld.so.conf file and run ldconfig.  If that "fixes" it,
then we should try to figure out why setup.py isn't adding
-R/usr/local/BerkeleyDB.4.1/lib to the link line.

You can also try moving
build/lib.linux-i686-2.3/_bsddb.so.failed back to _bsddb.so
and doing an "import _bsddb".  You'll get more information
about why the import failed.
msg17246 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-07-22 20:41
Logged In: YES 
user_id=21627

Please collect the following information:
1. Invoke the linker line with -Wl,--verbose (manually), to
find out the path it uses for the -ldb-4.1 command line
option. In particular, find out whether it uses a shared or
static library (.so or .a)
2. Perform nm on that library|grep db_create
 The symbol really should be there; it is on  all copies of
-ldb-something that I can get hold of. If it is not in your
copy, you somehow messed up the BerkeleyDB build.
msg17247 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-22 21:02
Logged In: YES 
user_id=52562

Okay, I've tried these.  -Wl,verbose tells me, as I expected:

attempt to open /usr/local/stow/python-g/lib/libdb-4.1.so failed
attempt to open /usr/local/stow/python-g/lib/libdb-4.1.a failed
attempt to open /usr/local/lib/libdb-4.1.so succeeded
-ldb-4.1 (/usr/local/lib/libdb-4.1.so)

nm tells me, as expected, that db_create is present:
MAIL pion:~$ nm /usr/local/lib/libdb-4.1.so | grep db_create
0001a554 T __bam_db_create
000783cc T __ham_db_create
000a5fe0 T __qam_db_create
000492c8 T db_create

Adding /usr/local/lib to my /etc/ld.so.conf and running
ldconfig *did* fix it.
msg17248 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-07-22 21:05
Logged In: YES 
user_id=21627

Ok. Can you try to find out why distutils fails to use the
-R/-rpath option?
msg17249 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-23 12:59
Logged In: YES 
user_id=52562

Could you tell me how to find out why distutils doesn't use
the -R/-rpath option?

Could it be that it assumes /usr/local/lib is already going
to be in the ld path?

By the way, having manually added /usr/local/lib to my
/etc/ld.so.conf, it compiles with no errors, but make test
yields the following:

HACK pion:~/playground/python/python/dist/src$ grep -i -E
--regex="bsddb|berk|db[^A-Za-z]"
/tmp/zooko/logs/make.test.python
test_bsddb
test_bsddb185
test_bsddb185 skipped -- No module named bsddb185
test_bsddb3
test_bsddb3 skipped -- Use of the `bsddb' resource not enabled
    test_aepack test_al test_bsddb185 test_bsddb3 test_bz2
test_cd
test_bsddb
test_bsddb185
test_bsddb185 skipped -- No module named bsddb185
test_bsddb3
test_bsddb3 skipped -- Use of the `bsddb' resource not enabled
    test_aepack test_al test_bsddb185 test_bsddb3 test_bz2
test_cd


Is that right?
msg17250 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-25 12:57
Logged In: YES 
user_id=52562

I don't know if this is important, but I still can't build
the bsddb3 module on my system without manually editing my
/etc/ld.so.conf, and even after I do manually edit my
/etc/ld.so.conf the tests seem to fail.
msg17251 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-25 14:28
Logged In: YES 
user_id=52562

Here's some more details about my system:
 * Debian unstable/testing
 * Linux pion 2.4.21 #1 Sat Jul 19 10:21:24 EDT 2003 i686
unknown unknown GNU/Linux
 * gcc (GCC) 3.3.1 20030626 (Debian prerelease)
 * GNU Make 3.80
 * AMD Athlon XP 1600+
msg17252 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-07-25 21:46
Logged In: YES 
user_id=21627

I now see what is happening. setup.py indeed expects that no
additional configuration is needed for /usr/local/lib. If
you had configured Sleepycat BSDDB in its standard location
(/usr/local/BerkeleyDBx.y), setup.py would have added a -R
option.

Closing it as "won't fix".
msg17253 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-26 13:15
Logged In: YES 
user_id=52562

Hm.  I tried this an failed, then examining setup.py, I find
that it is looking for "/usr/local/BerkeleyDB.x.y".  So I
moved my bsddb4.1 into there and tried again, and now I get
the same sorts of errors that I got earlier.  I will
investigate them in the same ways I did last time and report
back:


gcc -pthread -DNDEBUG -g -fPIC -fno-strict-aliasing
-I/usr/local/BerkeleyDB.4.1/include -I.
-I/home/zooko/playground/python/python/dist/src/./Include
-I.
-I/home/zooko/playground/python/python/dist/src/./Include
-I/usr/local/stow/python-g/include -I/usr/local/include
-I/home/zooko/playground/python/python/dist/src/Include
-I/home/zooko/playground/python/python/dist/src -c
/home/zooko/playground/python/python/dist/src/Modules/_bsddb.c
-o build/temp.linux-i686-2.3/_bsddb.o
gcc -pthread -shared build/temp.linux-i686-2.3/_bsddb.o
-L/usr/local/stow/python-g/lib -L/usr/local/lib -ldb-4.1 -o
build/lib.linux-i686-2.3/_bsddb.so
*** WARNING: renaming "_bsddb" since importing it failed:
build/lib.linux-i686-2.3/_bsddb.so: undefined symbol: db_create
building 'dbm' extension
gcc -pthread -DNDEBUG -g -fPIC -fno-strict-aliasing
-DHAVE_BERKDB_H -DDB_DBM_HSEARCH
-I/usr/local/BerkeleyDB.4.1/include -I.
-I/home/zooko/playground/python/python/dist/src/./Include
-I.
-I/home/zooko/playground/python/python/dist/src/./Include
-I/usr/local/stow/python-g/include -I/usr/local/include
-I/home/zooko/playground/python/python/dist/src/Include
-I/home/zooko/playground/python/python/dist/src -c
/home/zooko/playground/python/python/dist/src/Modules/dbmmodule.c
-o build/temp.linux-i686-2.3/dbmmodule.o
3.202
gcc -pthread -shared build/temp.linux-i686-2.3/dbmmodule.o
-L/usr/local/stow/python-g/lib -L/usr/local/lib -ldb-4.1 -o
build/lib.linux-i686-2.3/dbm.so
*** WARNING: renaming "dbm" since importing it failed:
build/lib.linux-i686-2.3/dbm.so: undefined symbol:
__db_ndbm_open
msg17254 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-26 13:58
Logged In: YES 
user_id=52562

Okay, I installed bsddb in /usr/local/BerkeleyDB.4.1.  It
still didn't build.  Investigating, I saw that it was
linking against /usr/lib/libdb4.so which was provided by
Debian.  I uninstalled the debian package and tried again. 
Now it builds but fails unit tests.

Just to be clear, I now have BerkeleyDB.4.1 in the standard
location, and have no other libdb4's on my system, and it
finishes "make" without any errors, but doesn't pass the
test.  I will continue to investigate.  The result of the
test is:

test_bsddb
test_bsddb skipped -- No module named _bsddb
test_bsddb185
test_bsddb185 skipped -- No module named bsddb185
test_bsddb3
test_bsddb3 skipped -- Use of the `bsddb' resource not enabled
...
224 tests OK.
1 test failed:
    test_ioctl
30 tests skipped:
    test_aepack test_al test_bsddb185 test_bsddb3 test_bz2
test_cd
    test_cl test_curses test_email_codecs test_gdbm test_gl
    test_imgfile test_linuxaudiodev test_locale test_macfs
    test_macostools test_nis test_normalization test_ossaudiodev
    test_pep277 test_plistlib test_scriptpackages
test_socket_ssl
    test_socketserver test_sunaudiodev test_timeout
test_unicode_file
    test_urllibnet test_winreg test_winsound
3 skips unexpected on linux2:
    test_locale test_bz2 test_gdbm
msg17255 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-26 14:08
Logged In: YES 
user_id=52562

Oh, okay,

test_bsddb3 skipped -- Use of the `bsddb' resource not enabled

just means that regrtest doesn't run the bsddb tests by default.

Also, I'm a bit confused about whether "test_bsddb3" is just
for the legacy (separately installed bsddb v3) or if it also
applies to the modern (bundled bsddb v4).
msg17256 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-26 14:12
Logged In: YES 
user_id=12800

test_bsddb3 is the extensive test suite from the pybsddb project.  
It's a bit of a misnomer, but is definitely still appropriate for the 
bsddb module in Python 2.3.  regrtest's "-u bsddb" enables it.
msg17257 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2003-07-26 15:13
Logged In: YES 
user_id=52562

Say, can anyone explain how to run *just* the bsddb unit test?

I tried:

HACK pion:~/playground/python/python/dist/src$ ./python -u
./Lib/test/regrtest.py -s ./Lib/test/test_bsddb.py
./Lib/test/test_bsddb
test ./Lib/test/test_bsddb crashed -- exceptions.ValueError:
Empty module name
1 test failed:
    ./Lib/test/test_bsddb
Traceback (most recent call last):
  File "./Lib/test/regrtest.py", line 1005, in ?
    main()
  File "./Lib/test/regrtest.py", line 332, in main
    os.unlink(filename)
OSError: [Errno 2] No such file or directory: '/tmp/pynexttest'

History
Date User Action Args
2022-04-10 16:10:08adminsetgithub: 38905
2003-07-22 20:21:46zookocreate