From mcepl at redhat.com Sat Jan 19 20:28:51 2013 From: mcepl at redhat.com (Matej Cepl) Date: Sat, 19 Jan 2013 20:28:51 +0100 Subject: [python-crypto] porting of m2crypto on python 3 Message-ID: <50FAF3F3.7070509@redhat.com> Hi, I have tried to give another shot to https://bugzilla.osafoundation.org/show_bug.cgi?id=12853 ("Python 3.* support" for m2crypto). My current repo is http://luther.ceplovi.cz/git/m2crypto.git/ (branch "python3"). Build runs without errors (but with tons of warnings; see below the log of python3 setup.py build), but when I try to import created build I get undefined symbol: matej at wycliff: M2Crypto (python3 %)$ python3 Python 3.3.0 (default, Oct 2 2012, 09:27:11) [GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import __m2crypto Traceback (most recent call last): File "", line 1, in ImportError: ./__m2crypto.cpython-33m.so: undefined symbol: PyFile_Check >>> That's probably problem with SWIG, isn't (I don't have PyFile_Check anywhere in my source files)? Does anybody have any idea how to work around it? Best, Mat?j ----------------------------------------------------------------- matej at wycliff: m2crypto (python3)$ ./fedora_setup.sh build running build running build_py creating build creating build/lib.linux-x86_64-3.3 creating build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/util.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/threading.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/m2xmlrpclib.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/m2urllib2.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/m2urllib.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/m2.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/httpslib.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/ftpslib.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/callback.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/__init__.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/X509.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/SMIME.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/Rand.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/RSA.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/RC4.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/Err.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/Engine.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/EVP.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/EC.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/DSA.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/DH.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/BN.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/BIO.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/AuthCookie.py -> build/lib.linux-x86_64-3.3/M2Crypto copying M2Crypto/ASN1.py -> build/lib.linux-x86_64-3.3/M2Crypto creating build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/timeout.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/ssl_dispatcher.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/cb.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/__init__.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/TwistedProtocolWrapper.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/Session.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/SSLServer.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/Context.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/Connection.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/Cipher.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL copying M2Crypto/SSL/Checker.py -> build/lib.linux-x86_64-3.3/M2Crypto/SSL creating build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/packet.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/RSA.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/PublicKeyRing.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/PublicKey.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/constants.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP copying M2Crypto/PGP/__init__.py -> build/lib.linux-x86_64-3.3/M2Crypto/PGP running build_ext building 'M2Crypto.__m2crypto' extension swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c swig -python -I/usr/include/python3.3m -I/usr/include -includeall -o SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i /usr/include/openssl/opensslconf-x86_64.h:5: Warning 205: CPP #error ""Don't include this file directly, use instead!"". SWIG/_bio.i:64: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_rand.i:21: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_evp.i:162: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_dh.i:36: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_rsa.i:43: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_dsa.i:31: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_ssl.i:214: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_x509.i:332: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_pkcs7.i:44: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_pkcs7.i:44: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_util.i:11: Warning 454: Setting a pointer/reference variable may leak memory. SWIG/_engine.i:168: Warning 454: Setting a pointer/reference variable may leak memory. creating build/temp.linux-x86_64-3.3 creating build/temp.linux-x86_64-3.3/SWIG gcc -pthread -Wno-unused-result -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.3m -I/usr/include -I/tmp/m2crypto/SWIG -c SWIG/_m2crypto_wrap.c -o build/temp.linux-x86_64-3.3/SWIG/_m2crypto_wrap.o -DTHREADING SWIG/_m2crypto_wrap.c: In function ?ssl_verify_callback?: SWIG/_m2crypto_wrap.c:3432:9: warning: passing argument 1 of ?PyType_GenericNew? from incompatible pointer type [enabled by default] In file included from /usr/include/python3.3m/pytime.h:6:0, from /usr/include/python3.3m/Python.h:65, from SWIG/_m2crypto_wrap.c:126: /usr/include/python3.3m/object.h:481:12: note: expected ?struct PyTypeObject *? but argument is of type ?struct PyObject *? SWIG/_m2crypto_wrap.c: In function ?bn_rand_range?: SWIG/_m2crypto_wrap.c:4138:14: warning: assignment from incompatible pointer type [enabled by default] SWIG/_m2crypto_wrap.c: In function ?pkey_get_modulus?: SWIG/_m2crypto_wrap.c:4758:13: warning: value computed is not used [-Wunused-value] SWIG/_m2crypto_wrap.c:4789:13: warning: value computed is not used [-Wunused-value] SWIG/_m2crypto_wrap.c: In function ?rsa_private_decrypt?: SWIG/_m2crypto_wrap.c:5373:5: warning: implicit declaration of function ?PyString_FromStringAndSize? [-Wimplicit-function-declaration] SWIG/_m2crypto_wrap.c:5373:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?rsa_padding_add_pkcs1_pss?: SWIG/_m2crypto_wrap.c:5449:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?x509v3_lhash?: SWIG/_m2crypto_wrap.c:6659:5: warning: return from incompatible pointer type [enabled by default] SWIG/_m2crypto_wrap.c: In function ?asn1_integer_set?: SWIG/_m2crypto_wrap.c:6873:9: warning: implicit declaration of function ?PyInt_AS_LONG? [-Wimplicit-function-declaration] SWIG/_m2crypto_wrap.c: In function ?util_string_to_hex?: SWIG/_m2crypto_wrap.c:7176:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?_wrap_err_print_errors_fp?: SWIG/_m2crypto_wrap.c:8545:5: warning: implicit declaration of function ?PyFile_Check? [-Wimplicit-function-declaration] SWIG/_m2crypto_wrap.c:8549:5: warning: implicit declaration of function ?PyFile_AsFile? [-Wimplicit-function-declaration] SWIG/_m2crypto_wrap.c:8549:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?_wrap_bio_new_fp?: SWIG/_m2crypto_wrap.c:8864:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?_wrap_x509_write_pem_file?: SWIG/_m2crypto_wrap.c:19303:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?_wrap_x509_name_print_ex_fp?: SWIG/_m2crypto_wrap.c:20086:9: warning: assignment makes pointer from integer without a cast [enabled by default] SWIG/_m2crypto_wrap.c: In function ?SWIG_InitializeModule?: SWIG/_m2crypto_wrap.c:26550:3: warning: statement with no effect [-Wunused-value] gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-3.3/SWIG/_m2crypto_wrap.o -L/usr/lib64 -L/usr/lib -lpython3.3m -lssl -lcrypto -o build/lib.linux-x86_64-3.3/M2Crypto/__m2crypto.cpython-33m.so matej at wycliff: m2crypto (python3 %)$ From solipsis at pitrou.net Sun Jan 20 00:10:06 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Jan 2013 00:10:06 +0100 Subject: [python-crypto] porting of m2crypto on python 3 In-Reply-To: <50FAF3F3.7070509@redhat.com> References: <50FAF3F3.7070509@redhat.com> Message-ID: <1358637006.3619.8.camel@localhost.localdomain> Le samedi 19 janvier 2013 ? 20:28 +0100, Matej Cepl a ?crit : > matej at wycliff: M2Crypto (python3 %)$ python3 > Python 3.3.0 (default, Oct 2 2012, 09:27:11) > [GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import __m2crypto > Traceback (most recent call last): > File "", line 1, in > ImportError: ./__m2crypto.cpython-33m.so: undefined symbol: PyFile_Check > >>> > > That's probably problem with SWIG, isn't (I don't have PyFile_Check > anywhere in my source files)? Does anybody have any idea how to work > around it? I'm not sure SWIG is Python 3-enabled. In any case, PyFile_Check doesn't exist in Python 3. The reason is that Python 2 file objects, which were wrappers around a FILE pointer, don't exist anymore. Instead, Python 3 I/O objects implement their own buffering layer around a file descriptor. It means that if m2crypto relies on FILE pointer-accepting OpenSSL functions, some amount of rewriting is needed. Regards Antoine. From mcepl at redhat.com Sun Jan 20 08:23:58 2013 From: mcepl at redhat.com (Matej Cepl) Date: Sun, 20 Jan 2013 08:23:58 +0100 Subject: [python-crypto] porting of m2crypto on python 3 In-Reply-To: <1358637006.3619.8.camel@localhost.localdomain> References: <50FAF3F3.7070509@redhat.com> <1358637006.3619.8.camel@localhost.localdomain> Message-ID: <50FB9B8E.6040409@redhat.com> On 20/01/13 00:10, Antoine Pitrou wrote: > I'm not sure SWIG is Python 3-enabled. I will investigate that but there are some rumours it is. > It means that if m2crypto relies on FILE pointer-accepting OpenSSL > functions, some amount of rewriting is needed. That object is added to the generated C code by swig, it is not in m2crypto?s *.i files. Mat?j From solipsis at pitrou.net Sun Jan 20 13:11:55 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Jan 2013 13:11:55 +0100 Subject: [python-crypto] porting of m2crypto on python 3 In-Reply-To: <50FB9B8E.6040409@redhat.com> References: <50FAF3F3.7070509@redhat.com> <1358637006.3619.8.camel@localhost.localdomain> <50FB9B8E.6040409@redhat.com> Message-ID: <1358683915.3429.3.camel@localhost.localdomain> Le dimanche 20 janvier 2013 ? 08:23 +0100, Matej Cepl a ?crit : > On 20/01/13 00:10, Antoine Pitrou wrote: > > I'm not sure SWIG is Python 3-enabled. > > I will investigate that but there are some rumours it is. > > > It means that if m2crypto relies on FILE pointer-accepting OpenSSL > > functions, some amount of rewriting is needed. > > That object is added to the generated C code by swig, it is not in > m2crypto?s *.i files. Well, look at http://www.swig.org/Doc2.0/SWIGDocumentation.html#Python_python3support There is a list of known-to-be-broken features in Python 3: * No more support for FILE* typemaps, because PyFile_AsFile has been dropped in Python 3. [...] ;-) Regards Antoine. From mcepl at redhat.com Sun Jan 20 21:03:45 2013 From: mcepl at redhat.com (=?UTF-8?B?TWF0xJtqIENlcGw=?=) Date: Sun, 20 Jan 2013 21:03:45 +0100 Subject: [python-crypto] porting of m2crypto on python 3 In-Reply-To: <1358683915.3429.3.camel@localhost.localdomain> References: <50FAF3F3.7070509@redhat.com> <1358637006.3619.8.camel@localhost.localdomain> <50FB9B8E.6040409@redhat.com> <1358683915.3429.3.camel@localhost.localdomain> Message-ID: <50FC4DA1.6020808@redhat.com> On 20/01/13 13:11, Antoine Pitrou wrote: > Well, look at > http://www.swig.org/Doc2.0/SWIGDocumentation.html#Python_python3support > > There is a list of known-to-be-broken features in Python 3: > > * No more support for FILE* typemaps, because PyFile_AsFile has > been dropped in Python 3. > [...] OK, I will continue my investigation on the swig list. Thanks for help, Mat?j