From jcea at jcea.es  Thu Dec  3 21:46:27 2015
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 4 Dec 2015 03:46:27 +0100
Subject: [Cryptography-dev] Minimal Test Case,
 bad interaction between "cryptography" and "wsgi" (Re: [modwsgi]
 mod_wsgi and creating new SSL connections inside the "body" iterator)
In-Reply-To: <11C98826-EA79-4B50-8FB4-87A1F1CE59F5@gmail.com>
References: <56606F68.5030403@jcea.es>
 <11C98826-EA79-4B50-8FB4-87A1F1CE59F5@gmail.com>
Message-ID: <5660FE83.8050504@jcea.es>

Context for "Cryptography" people:

Using "cryptography" and then trying to create a "ssl.SSLContext()"
hangs the progress when running under mod_wsgi. I can not reproduce this
issue with a standalone script, it must be run inside mod_wsgi. May be
useful to know that, in daemon mode, mod_wsgi uses several threads to
monitor health and do maintenance, beside the main wsgi application.

Mail thread in the mod_wsgi mailing list in
<https://groups.google.com/forum/#!topic/modwsgi/WFLao7kOqxE>.


On 04/12/15 01:00, Graham Dumpleton wrote:
> You don?t happen to be using PostgreSQL in the same web application
> process are you?

No. I have quite a few modules in my apache, but I am running WSGI in
daemon mode.

I have a minimal testcase. I can not reproduce this issue OUTSIDE of
mod_wsgi, so it seems to be a bad interaction between mod_wsgi and
"cryptography" library.

The minimal test case, mod_wsgi compiled to use Python 3.5.0, is:

"""
from cryptography.hazmat.primitives.ciphers import Cipher
from cryptography.hazmat.primitives.ciphers import algorithms as \
                                                    Cipher_algorithms
from cryptography.hazmat.primitives.ciphers import modes as \
                                                    Cipher_modes
from cryptography.hazmat.backends import default_backend as \
                                                    Cipher_backend

import ssl

def application(environ, start_response) :
    cipher = Cipher(
            Cipher_algorithms.AES(b'e'*16),
            Cipher_modes.ECB(),
            backend = Cipher_backend())

    ssl.SSLContext(ssl.PROTOCOL_SSLv23)  # <- HANGS HERE!!!

    crypto_iv = cipher.encryptor()

    start_response('200 OK', [])
    return [b'SUCCESS!\r\n']
"""

This code hangs the HTTP request in my machine.

I am using Cryptography 1.1.1 and mod_wsgi 4.4.21. This server has
OpenSSL 1.0.1. It is an Ubuntu 12.04 fully patched.

A "thread apply all bt" in GDB shows this:

"""
Thread 19 (Thread 0x7fa947867700 (LWP 11497)):
#0  0x00007fa952a3a6f3 in select () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fa952f4d815 in apr_sleep (t=<optimized out>) at
time/unix/time.c:246
#2  0x00007fa94cb78ee0 in wsgi_monitor_thread (thd=<optimized out>,
data=<optimized out>) at src/server/mod_wsgi.c:8658
#3  0x00007fa952d13e9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#4  0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x0000000000000000 in ?? ()

Thread 18 (Thread 0x7fa9477e6700 (LWP 11498)):
#0  0x00007fa952d180fe in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007fa94c792aff in take_gil () at Python/condvar.h:103
#2  0x00007fa94c793099 in PyEval_RestoreThread () at Python/ceval.c:450
#3  0x00007fa94c7ccb2b in PyGILState_Ensure () from
/usr/local/lib/libpython3.5m.so.1.0
#4  0x00007fa94cb7ac9a in wsgi_deadlock_thread (thd=<optimized out>,
data=<optimized out>) at src/server/mod_wsgi.c:8468
#5  0x00007fa952d13e9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x0000000000000000 in ?? ()

Thread 17 (Thread 0x7fa947765700 (LWP 11499)):
#0  0x00007fa952d17d84 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007fa94cb8325d in wsgi_worker_acquire (id=0) at
src/server/mod_wsgi.c:8044
#2  wsgi_daemon_worker (thread=<optimized out>, p=0x1788588) at
src/server/mod_wsgi.c:8166
#3  wsgi_daemon_thread (thd=0x15cae30, data=<optimized out>) at
src/server/mod_wsgi.c:8425
#4  0x00007fa952d13e9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5  0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#6  0x0000000000000000 in ?? ()

[QUITE A FEW THREADS WITH THE SAME STACK]

Thread 3 (Thread 0x7fa947057700 (LWP 11513)):
#0  0x00007fa952d180fe in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007fa94c792aff in take_gil () at Python/condvar.h:103
#2  0x00007fa94c793099 in PyEval_RestoreThread () at Python/ceval.c:450
#3  0x00007fa94c7ccb2b in PyGILState_Ensure () from
/usr/local/lib/libpython3.5m.so.1.0
#4  0x00007fa9363c51d1 in invoke_callback ()
   from
/usr/local/lib/python3.5/site-packages/_cffi_backend.cpython-35m-x86_64-linux-gnu.so
#5  0x00007fa9361aa819 in ffi_closure_unix64_inner () from
/usr/lib/x86_64-linux-gnu/libffi.so.6
#6  0x00007fa9361aab7c in ffi_closure_unix64 () from
/usr/lib/x86_64-linux-gnu/libffi.so.6
#7  0x00007fa94b7585e5 in ?? () from
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#8  0x00007fa94b7583c8 in ?? () from
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#9  0x00007fa94b76efbb in EC_KEY_generate_key () from
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#10 0x00007fa94bab3998 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0
#11 0x00007fa94474d6d5 in _ssl__SSLContext () from
/usr/local/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so
#12 0x00007fa94c7190a4 in tp_new_wrapper () from
/usr/local/lib/libpython3.5m.so.1.0
#13 0x00007fa94c7023e9 in PyCFunction_Call () from
/usr/local/lib/libpython3.5m.so.1.0
#14 0x00007fa94c79d04d in PyEval_EvalFrameEx () at Python/ceval.c:4651
#15 0x00007fa94c79dab6 in _PyEval_EvalCodeWithName () at Python/ceval.c:3962
#16 0x00007fa94c79dba8 in PyEval_EvalCodeEx () at Python/ceval.c:3983
#17 0x00007fa94c6df076 in function_call () from
/usr/local/lib/libpython3.5m.so.1.0
#18 0x00007fa94c6ade0a in PyObject_Call () from
/usr/local/lib/libpython3.5m.so.1.0
#19 0x00007fa94c714ab5 in slot_tp_new () from
/usr/local/lib/libpython3.5m.so.1.0
#20 0x00007fa94c7191e3 in type_call () from
/usr/local/lib/libpython3.5m.so.1.0
#21 0x00007fa94c6ade0a in PyObject_Call () from
/usr/local/lib/libpython3.5m.so.1.0
#22 0x00007fa94c79711e in PyEval_EvalFrameEx () at Python/ceval.c:4883
#23 0x00007fa94c79dab6 in _PyEval_EvalCodeWithName () at Python/ceval.c:3962
#24 0x00007fa94c79dba8 in PyEval_EvalCodeEx () at Python/ceval.c:3983
#25 0x00007fa94c6df076 in function_call () from
/usr/local/lib/libpython3.5m.so.1.0
#26 0x00007fa94c6ade0a in PyObject_Call () from
/usr/local/lib/libpython3.5m.so.1.0
#27 0x00007fa94c793787 in PyEval_CallObjectWithKeywords () at
Python/ceval.c:4526
#28 0x00007fa94cb81822 in Adapter_run (object=0x7fa946f16ea0,
self=0x7fa944723210) at src/server/mod_wsgi.c:2897
#29 wsgi_execute_script (r=0x7fa940004980) at src/server/mod_wsgi.c:3726
#30 0x00007fa94cb83c69 in wsgi_hook_daemon_handler (c=<optimized out>)
at src/server/mod_wsgi.c:12236
#31 wsgi_process_socket (bucket_alloc=0x7fa940006e40, sock=<optimized
out>, p=0x7fa940005e90, daemon=<optimized out>)
    at src/server/mod_wsgi.c:8007
#32 wsgi_daemon_worker (thread=<optimized out>, p=0x17a7638) at
src/server/mod_wsgi.c:8336
#33 wsgi_daemon_thread (thd=0x1795368, data=<optimized out>) at
src/server/mod_wsgi.c:8425
#34 0x00007fa952d13e9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#35 0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#36 0x0000000000000000 in ?? ()

[MORE BORING THREADS]

Thread 1 (Thread 0x7fa9537c2700 (LWP 11094)):
#0  0x00007fa952a359d3 in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fa952f49563 in apr_poll (aprset=0x7fffca6fd500, num=1,
nsds=0x7fffca6fd55c, timeout=<optimized out>)
    at poll/unix/poll.c:120
#2  0x00007fa94cb859ab in wsgi_daemon_main (daemon=0x15ca3d8,
p=0x159a138) at src/server/mod_wsgi.c:8944
#3  wsgi_start_process (p=0x159a138, daemon=0x15ca3d8) at
src/server/mod_wsgi.c:9665
#4  0x00007fa94cb87494 in wsgi_start_daemons (p=0x159a138) at
src/server/mod_wsgi.c:9877
#5  0x00000000004370f0 in ap_run_pre_mpm ()
#6  0x000000000048454c in worker_run ()
#7  0x0000000000433618 in ap_run_mpm ()
#8  0x000000000042b340 in main ()
"""

It is looks like some kind of callback is registered by "cryptography"
and when calling "ssl.SSLContext()" that callback tries to get the GIL
when it already has it, hanging. My speculation, I am not familiar with
the inner working of OpenSSL neither "cryptography", but the traceback
of "Thread 3" looks quite suspicious.

In recent Python releases, the GIL is a NOP if not threads are created.
Lets try something... Lets create another idle thread.

"""
from cryptography.hazmat.primitives.ciphers import Cipher
from cryptography.hazmat.primitives.ciphers import algorithms as \
                                                    Cipher_algorithms
from cryptography.hazmat.primitives.ciphers import modes as \
                                                    Cipher_modes
from cryptography.hazmat.backends import default_backend as \
                                                    Cipher_backend

import ssl, threading, time

def dummy() :
    cipher = Cipher(
            Cipher_algorithms.AES(b'e'*16),
            Cipher_modes.ECB(),
            backend = Cipher_backend())

    ssl.SSLContext(ssl.PROTOCOL_SSLv23)

    crypto_iv = cipher.encryptor()

    while True :
        pass

t = threading.Thread(target = dummy)
t.setDaemon(True)
t.start()
print("SSSS")

time.sleep(2)

print("OK!")
"""

This code works right. I can only reproduce the issue inside mod_wsgi.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151204/01e07589/attachment.sig>

From paul.l.kehrer at gmail.com  Thu Dec  3 21:58:02 2015
From: paul.l.kehrer at gmail.com (Paul Kehrer)
Date: Thu, 3 Dec 2015 20:58:02 -0600
Subject: [Cryptography-dev] Minimal Test Case,
 bad interaction between "cryptography" and "wsgi" (Re: [modwsgi]
 mod_wsgi and creating new SSL connections inside the "body" iterator)
In-Reply-To: <5660FE83.8050504@jcea.es>
References: <56606F68.5030403@jcea.es>
 <11C98826-EA79-4B50-8FB4-87A1F1CE59F5@gmail.com>
 <5660FE83.8050504@jcea.es>
Message-ID: <etPan.5661013a.5cf0eb61.5b88@atlas.local>

Can you reproduce this if you set:

WSGIApplicationGroup %{GLOBAL}

?

We're aware of a current issue with lock and library initialization related to Python subinterpreters (which mod_wsgi uses). Fixing this is...very hard, but?https://github.com/pyca/cryptography/issues/2299?covers most of what we know right now.

-Paul (reaperhulk)

On December 3, 2015 at 8:54:23 PM, Jesus Cea (jcea at jcea.es) wrote:

Context for "Cryptography" people:  

Using "cryptography" and then trying to create a "ssl.SSLContext()"  
hangs the progress when running under mod_wsgi. I can not reproduce this  
issue with a standalone script, it must be run inside mod_wsgi. May be  
useful to know that, in daemon mode, mod_wsgi uses several threads to  
monitor health and do maintenance, beside the main wsgi application.  

Mail thread in the mod_wsgi mailing list in  
<https://groups.google.com/forum/#!topic/modwsgi/WFLao7kOqxE>.  


On 04/12/15 01:00, Graham Dumpleton wrote:  
> You don?t happen to be using PostgreSQL in the same web application  
> process are you?  

No. I have quite a few modules in my apache, but I am running WSGI in  
daemon mode.  

I have a minimal testcase. I can not reproduce this issue OUTSIDE of  
mod_wsgi, so it seems to be a bad interaction between mod_wsgi and  
"cryptography" library.  

The minimal test case, mod_wsgi compiled to use Python 3.5.0, is:  

"""  
from cryptography.hazmat.primitives.ciphers import Cipher  
from cryptography.hazmat.primitives.ciphers import algorithms as \  
Cipher_algorithms  
from cryptography.hazmat.primitives.ciphers import modes as \  
Cipher_modes  
from cryptography.hazmat.backends import default_backend as \  
Cipher_backend  

import ssl  

def application(environ, start_response) :  
cipher = Cipher(  
Cipher_algorithms.AES(b'e'*16),  
Cipher_modes.ECB(),  
backend = Cipher_backend())  

ssl.SSLContext(ssl.PROTOCOL_SSLv23) # <- HANGS HERE!!!  

crypto_iv = cipher.encryptor()  

start_response('200 OK', [])  
return [b'SUCCESS!\r\n']  
"""  

This code hangs the HTTP request in my machine.  

I am using Cryptography 1.1.1 and mod_wsgi 4.4.21. This server has  
OpenSSL 1.0.1. It is an Ubuntu 12.04 fully patched.  

A "thread apply all bt" in GDB shows this:  

"""  
Thread 19 (Thread 0x7fa947867700 (LWP 11497)):  
#0 0x00007fa952a3a6f3 in select () from /lib/x86_64-linux-gnu/libc.so.6  
#1 0x00007fa952f4d815 in apr_sleep (t=<optimized out>) at  
time/unix/time.c:246  
#2 0x00007fa94cb78ee0 in wsgi_monitor_thread (thd=<optimized out>,  
data=<optimized out>) at src/server/mod_wsgi.c:8658  
#3 0x00007fa952d13e9a in start_thread () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#4 0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6  
#5 0x0000000000000000 in ?? ()  

Thread 18 (Thread 0x7fa9477e6700 (LWP 11498)):  
#0 0x00007fa952d180fe in pthread_cond_timedwait@@GLIBC_2.3.2 () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#1 0x00007fa94c792aff in take_gil () at Python/condvar.h:103  
#2 0x00007fa94c793099 in PyEval_RestoreThread () at Python/ceval.c:450  
#3 0x00007fa94c7ccb2b in PyGILState_Ensure () from  
/usr/local/lib/libpython3.5m.so.1.0  
#4 0x00007fa94cb7ac9a in wsgi_deadlock_thread (thd=<optimized out>,  
data=<optimized out>) at src/server/mod_wsgi.c:8468  
#5 0x00007fa952d13e9a in start_thread () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#6 0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6  
#7 0x0000000000000000 in ?? ()  

Thread 17 (Thread 0x7fa947765700 (LWP 11499)):  
#0 0x00007fa952d17d84 in pthread_cond_wait@@GLIBC_2.3.2 () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#1 0x00007fa94cb8325d in wsgi_worker_acquire (id=0) at  
src/server/mod_wsgi.c:8044  
#2 wsgi_daemon_worker (thread=<optimized out>, p=0x1788588) at  
src/server/mod_wsgi.c:8166  
#3 wsgi_daemon_thread (thd=0x15cae30, data=<optimized out>) at  
src/server/mod_wsgi.c:8425  
#4 0x00007fa952d13e9a in start_thread () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#5 0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6  
#6 0x0000000000000000 in ?? ()  

[QUITE A FEW THREADS WITH THE SAME STACK]  

Thread 3 (Thread 0x7fa947057700 (LWP 11513)):  
#0 0x00007fa952d180fe in pthread_cond_timedwait@@GLIBC_2.3.2 () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#1 0x00007fa94c792aff in take_gil () at Python/condvar.h:103  
#2 0x00007fa94c793099 in PyEval_RestoreThread () at Python/ceval.c:450  
#3 0x00007fa94c7ccb2b in PyGILState_Ensure () from  
/usr/local/lib/libpython3.5m.so.1.0  
#4 0x00007fa9363c51d1 in invoke_callback ()  
from  
/usr/local/lib/python3.5/site-packages/_cffi_backend.cpython-35m-x86_64-linux-gnu.so  
#5 0x00007fa9361aa819 in ffi_closure_unix64_inner () from  
/usr/lib/x86_64-linux-gnu/libffi.so.6  
#6 0x00007fa9361aab7c in ffi_closure_unix64 () from  
/usr/lib/x86_64-linux-gnu/libffi.so.6  
#7 0x00007fa94b7585e5 in ?? () from  
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0  
#8 0x00007fa94b7583c8 in ?? () from  
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0  
#9 0x00007fa94b76efbb in EC_KEY_generate_key () from  
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0  
#10 0x00007fa94bab3998 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0  
#11 0x00007fa94474d6d5 in _ssl__SSLContext () from  
/usr/local/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so  
#12 0x00007fa94c7190a4 in tp_new_wrapper () from  
/usr/local/lib/libpython3.5m.so.1.0  
#13 0x00007fa94c7023e9 in PyCFunction_Call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#14 0x00007fa94c79d04d in PyEval_EvalFrameEx () at Python/ceval.c:4651  
#15 0x00007fa94c79dab6 in _PyEval_EvalCodeWithName () at Python/ceval.c:3962  
#16 0x00007fa94c79dba8 in PyEval_EvalCodeEx () at Python/ceval.c:3983  
#17 0x00007fa94c6df076 in function_call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#18 0x00007fa94c6ade0a in PyObject_Call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#19 0x00007fa94c714ab5 in slot_tp_new () from  
/usr/local/lib/libpython3.5m.so.1.0  
#20 0x00007fa94c7191e3 in type_call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#21 0x00007fa94c6ade0a in PyObject_Call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#22 0x00007fa94c79711e in PyEval_EvalFrameEx () at Python/ceval.c:4883  
#23 0x00007fa94c79dab6 in _PyEval_EvalCodeWithName () at Python/ceval.c:3962  
#24 0x00007fa94c79dba8 in PyEval_EvalCodeEx () at Python/ceval.c:3983  
#25 0x00007fa94c6df076 in function_call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#26 0x00007fa94c6ade0a in PyObject_Call () from  
/usr/local/lib/libpython3.5m.so.1.0  
#27 0x00007fa94c793787 in PyEval_CallObjectWithKeywords () at  
Python/ceval.c:4526  
#28 0x00007fa94cb81822 in Adapter_run (object=0x7fa946f16ea0,  
self=0x7fa944723210) at src/server/mod_wsgi.c:2897  
#29 wsgi_execute_script (r=0x7fa940004980) at src/server/mod_wsgi.c:3726  
#30 0x00007fa94cb83c69 in wsgi_hook_daemon_handler (c=<optimized out>)  
at src/server/mod_wsgi.c:12236  
#31 wsgi_process_socket (bucket_alloc=0x7fa940006e40, sock=<optimized  
out>, p=0x7fa940005e90, daemon=<optimized out>)  
at src/server/mod_wsgi.c:8007  
#32 wsgi_daemon_worker (thread=<optimized out>, p=0x17a7638) at  
src/server/mod_wsgi.c:8336  
#33 wsgi_daemon_thread (thd=0x1795368, data=<optimized out>) at  
src/server/mod_wsgi.c:8425  
#34 0x00007fa952d13e9a in start_thread () from  
/lib/x86_64-linux-gnu/libpthread.so.0  
#35 0x00007fa952a4138d in clone () from /lib/x86_64-linux-gnu/libc.so.6  
#36 0x0000000000000000 in ?? ()  

[MORE BORING THREADS]  

Thread 1 (Thread 0x7fa9537c2700 (LWP 11094)):  
#0 0x00007fa952a359d3 in poll () from /lib/x86_64-linux-gnu/libc.so.6  
#1 0x00007fa952f49563 in apr_poll (aprset=0x7fffca6fd500, num=1,  
nsds=0x7fffca6fd55c, timeout=<optimized out>)  
at poll/unix/poll.c:120  
#2 0x00007fa94cb859ab in wsgi_daemon_main (daemon=0x15ca3d8,  
p=0x159a138) at src/server/mod_wsgi.c:8944  
#3 wsgi_start_process (p=0x159a138, daemon=0x15ca3d8) at  
src/server/mod_wsgi.c:9665  
#4 0x00007fa94cb87494 in wsgi_start_daemons (p=0x159a138) at  
src/server/mod_wsgi.c:9877  
#5 0x00000000004370f0 in ap_run_pre_mpm ()  
#6 0x000000000048454c in worker_run ()  
#7 0x0000000000433618 in ap_run_mpm ()  
#8 0x000000000042b340 in main ()  
"""  

It is looks like some kind of callback is registered by "cryptography"  
and when calling "ssl.SSLContext()" that callback tries to get the GIL  
when it already has it, hanging. My speculation, I am not familiar with  
the inner working of OpenSSL neither "cryptography", but the traceback  
of "Thread 3" looks quite suspicious.  

In recent Python releases, the GIL is a NOP if not threads are created.  
Lets try something... Lets create another idle thread.  

"""  
from cryptography.hazmat.primitives.ciphers import Cipher  
from cryptography.hazmat.primitives.ciphers import algorithms as \  
Cipher_algorithms  
from cryptography.hazmat.primitives.ciphers import modes as \  
Cipher_modes  
from cryptography.hazmat.backends import default_backend as \  
Cipher_backend  

import ssl, threading, time  

def dummy() :  
cipher = Cipher(  
Cipher_algorithms.AES(b'e'*16),  
Cipher_modes.ECB(),  
backend = Cipher_backend())  

ssl.SSLContext(ssl.PROTOCOL_SSLv23)  

crypto_iv = cipher.encryptor()  

while True :  
pass  

t = threading.Thread(target = dummy)  
t.setDaemon(True)  
t.start()  
print("SSSS")  

time.sleep(2)  

print("OK!")  
"""  

This code works right. I can only reproduce the issue inside mod_wsgi.  

--  
Jes?s Cea Avi?n _/_/ _/_/_/ _/_/_/  
jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/  
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/  
jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/ _/_/ _/_/  
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/  
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/  
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz  

_______________________________________________  
Cryptography-dev mailing list  
Cryptography-dev at python.org  
https://mail.python.org/mailman/listinfo/cryptography-dev  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151203/0c4c4939/attachment-0001.html>

From jcea at jcea.es  Thu Dec  3 22:34:05 2015
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 4 Dec 2015 04:34:05 +0100
Subject: [Cryptography-dev] Minimal Test Case,
 bad interaction between "cryptography" and "wsgi" (Re: [modwsgi]
 mod_wsgi and creating new SSL connections inside the "body" iterator)
In-Reply-To: <etPan.5661013a.5cf0eb61.5b88@atlas.local>
References: <56606F68.5030403@jcea.es>
 <11C98826-EA79-4B50-8FB4-87A1F1CE59F5@gmail.com> <5660FE83.8050504@jcea.es>
 <etPan.5661013a.5cf0eb61.5b88@atlas.local>
Message-ID: <566109AD.9030409@jcea.es>

Sending copy to the mod_wsgi mailing list, to summarize the thread.



On 04/12/15 03:58, Paul Kehrer wrote:
> Can you reproduce this if you set:
> 
> WSGIApplicationGroup %{GLOBAL}
> 
> ?
> 
> We're aware of a current issue with lock and library initialization
> related to Python subinterpreters (which mod_wsgi uses). Fixing this
> is...very hard,
> but https://github.com/pyca/cryptography/issues/2299 covers most of what
> we know right now.

OH, IT WORKED!!!!. Thanks.

Subscribing to that issue. Thanks.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151204/4de593f7/attachment.sig>

From paul.l.kehrer at gmail.com  Thu Dec 10 15:13:54 2015
From: paul.l.kehrer at gmail.com (Paul Kehrer)
Date: Thu, 10 Dec 2015 14:13:54 -0600
Subject: [Cryptography-dev] PyCA cryptography 1.1.2 released
Message-ID: <etPan.5669dd02.7c04b29f.326@atlas.local>

PyCA/cryptography (https://github.com/pyca/cryptography) 1.1.2 has been released. cryptography is a package which provides cryptographic recipes and primitives to Python developers. Our goal is for it to be your "cryptographic standard library". We support Python 2.6-2.7, Python 3.3+, and PyPy.

Changelog:

* Fixed a SIGBUS crash with the OS X wheels caused by redefinition of a method.
* Fixed a runtime error "undefined symbol EC_GFp_nistp224_method" that occurred with some OpenSSL installations.
* Updated Windows and OS X wheels to be compiled against OpenSSL 1.0.2e.

-Paul Kehrer (reaperhulk)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151210/a626490c/attachment.html>

From jcea at jcea.es  Sun Dec 13 11:55:55 2015
From: jcea at jcea.es (Jesus Cea)
Date: Sun, 13 Dec 2015 17:55:55 +0100
Subject: [Cryptography-dev] Supporting "memory view" and stream output
Message-ID: <566DA31B.3020708@jcea.es>

When calling Cryptography with a memoryview I get this error:

"""
TypeError: initializer for ctype 'unsigned char *' must be a cdata
pointer, not memoryview
"""

Memory view are useful to avoid copies, eating memory and CPU for no reason.

In my personal situation, I have 4 MB blocks, each divided in 16 pieces
of 256Kbytes independently encrypted. I must extract those 256Kbytes
fragments to call cryptography, instead of just passing a memory view
with the appropriate slice.

A secondary request would be to provide a different option for output,
for the same reason: provide a target for output: a bytearray/writable
memory view, or a file-like object. For the same reason: avoiding copies
and ease concatenation.

Hope you find these suggestions useful.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151213/b760fef0/attachment.sig>

From paul.l.kehrer at gmail.com  Sun Dec 13 12:44:37 2015
From: paul.l.kehrer at gmail.com (Paul Kehrer)
Date: Sun, 13 Dec 2015 10:44:37 -0700
Subject: [Cryptography-dev] Supporting "memory view" and stream output
In-Reply-To: <566DA31B.3020708@jcea.es>
References: <566DA31B.3020708@jcea.es>
Message-ID: <etPan.566dae85.591b8bf8.326@atlas>

cffi's architecture unfortunately imposes some limitations on support for memoryviews (specifically, no support for them with bytes/bytearray) when passing data. You can get some information about the challenges from these pages:
*?https://bitbucket.org/cffi/cffi/issues/47/creating-a-cdata-from-a-buffer
*?https://bitbucket.org/cffi/cffi/issues/65/support-memoryview-to-char-conversion

Here's an example of how we currently encrypt/decrypt in the OpenSSL backend:

? ? def update(self, data):
? ? ? ? <snip>
? ? ? ? buf = self._backend._ffi.new("unsigned char[]",
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?len(data) + self._block_size - 1)
? ? ? ? outlen = self._backend._ffi.new("int *")
? ? ? ? res = self._backend._lib.EVP_CipherUpdate(self._ctx, buf, outlen, data,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? len(data))
? ? ? ? self._backend.openssl_assert(res != 0)
? ? ? ? return self._backend._ffi.buffer(buf)[:outlen[0]]

There are a few potential wins that are available here (see:?https://github.com/pyca/cryptography/pull/2163#issuecomment-123878138?and the resulting branch?https://github.com/reaperhulk/cryptography/blob/cipher-optimizations-that-are-probably-a-bad-idea/src/cryptography/hazmat/backends/openssl/ciphers.py#L118), but without support for bytearray/bytes in memoryviews it's unclear to me how we can improve above and beyond what's in that branch. If you have some ideas we'd be happy to hear them!

-Paul Kehrer (reaperhulk)
On December 13, 2015 at 9:56:09 AM, Jesus Cea (jcea at jcea.es) wrote:

When calling Cryptography with a memoryview I get this error:  

"""  
TypeError: initializer for ctype 'unsigned char *' must be a cdata  
pointer, not memoryview  
"""  

Memory view are useful to avoid copies, eating memory and CPU for no reason.  

In my personal situation, I have 4 MB blocks, each divided in 16 pieces  
of 256Kbytes independently encrypted. I must extract those 256Kbytes  
fragments to call cryptography, instead of just passing a memory view  
with the appropriate slice.  

A secondary request would be to provide a different option for output,  
for the same reason: provide a target for output: a bytearray/writable  
memory view, or a file-like object. For the same reason: avoiding copies  
and ease concatenation.  

Hope you find these suggestions useful.  

--  
Jes?s Cea Avi?n _/_/ _/_/_/ _/_/_/  
jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/  
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/  
jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/ _/_/ _/_/  
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/  
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/  
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz  

_______________________________________________  
Cryptography-dev mailing list  
Cryptography-dev at python.org  
https://mail.python.org/mailman/listinfo/cryptography-dev  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151213/40eb5c48/attachment.html>

From alex.gaynor at gmail.com  Tue Dec 22 22:43:59 2015
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Tue, 22 Dec 2015 22:43:59 -0500
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
Message-ID: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>

Hi all,

I'd like to propose we formally deprecate (in our next release) and remove
(in the release thereafter) support for Python2.6.

It is no longer under support from its developers (which makes it a
security risk), its use is more or less confined to users of CentOS and
RHEL (which means SCL is available to them, and they have a corporate
benefactor to complain to if it doesn't work), and its seriously funking
with our development workflow because it's so abysmally slow in CI (which
means it's annoying me enough to want to solve this).

There is genuinely no justifiable reason to use Python 2.6 anymore, both
Django and Twisted have dropped it.

If this would _seriously_ impact you, please reply to this thread. If
possible I'd also like to hear from our downstreams.

Cheers,
Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151222/788f7836/attachment.html>

From paul.l.kehrer at gmail.com  Tue Dec 22 23:10:51 2015
From: paul.l.kehrer at gmail.com (Paul Kehrer)
Date: Tue, 22 Dec 2015 22:10:51 -0600
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
In-Reply-To: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
References: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
Message-ID: <etPan.567a1ecb.44c38d6f.1da@atlas.local>

Some numbers to inform this conversation. Here are the download counts for cryptography 1.1.2 right now (12 days of downloads):

? ? ? ? ? ? ? ? ? ? ? ? ? ? cryptography-1.1.2.tar.gz ? ?2015-12-10 ? ? ?261,761
?cryptography-1.1.2-pp27-none-macosx_10_10_x86_64.whl ? ?2015-12-10 ? ? ? ? ? 86
? ? ? ? ? ?cryptography-1.1.2-cp35-none-win_amd64.whl ? ?2015-12-10 ? ? ? ? ?292
? ? ? ? ? ? ? ?cryptography-1.1.2-cp35-none-win32.whl ? ?2015-12-10 ? ? ? ? ?325
? cryptography-1.1.2-cp35-cp35m-macosx_10_6_intel.whl ? ?2015-12-10 ? ? ? ? ?481
cryptography-1.1.2-cp35-cp35m-macosx_10_10_x86_64.whl ? ?2015-12-10 ? ? ? ? ? 50
? ? ? ? ? ?cryptography-1.1.2-cp34-none-win_amd64.whl ? ?2015-12-10 ? ? ? ? ?289
? ? ? ? ? ? ? ?cryptography-1.1.2-cp34-none-win32.whl ? ?2015-12-10 ? ? ? ? ?347
? cryptography-1.1.2-cp34-cp34m-macosx_10_6_intel.whl ? ?2015-12-10 ? ? ? ? ?254
cryptography-1.1.2-cp34-cp34m-macosx_10_10_x86_64.whl ? ?2015-12-10 ? ? ? ? ? 63
? ? ? ? ? ?cryptography-1.1.2-cp33-none-win_amd64.whl ? ?2015-12-10 ? ? ? ? ?108
? ? ? ? ? ? ? ?cryptography-1.1.2-cp33-none-win32.whl ? ?2015-12-10 ? ? ? ? ? 97
? cryptography-1.1.2-cp33-cp33m-macosx_10_6_intel.whl ? ?2015-12-10 ? ? ? ? ? 60
cryptography-1.1.2-cp33-cp33m-macosx_10_10_x86_64.whl ? ?2015-12-10 ? ? ? ? ? 49
? ? ? ? ? ?cryptography-1.1.2-cp27-none-win_amd64.whl ? ?2015-12-10 ? ? ? ?3,941
? ? ? ? ? ? ? ?cryptography-1.1.2-cp27-none-win32.whl ? ?2015-12-10 ? ? ? ?2,544
? ?cryptography-1.1.2-cp27-none-macosx_10_6_intel.whl ? ?2015-12-10 ? ? ? ?5,834
? cryptography-1.1.2-cp27-none-macosx_10_10_intel.whl ? ?2015-12-10 ? ? ? ? ?188
? ? ? ? ? ?cryptography-1.1.2-cp26-none-win_amd64.whl ? ?2015-12-10 ? ? ? ? ? 86
? ? ? ? ? ? ? ?cryptography-1.1.2-cp26-none-win32.whl ? ?2015-12-10 ? ? ? ? ? 63
? cryptography-1.1.2-cp26-none-macosx_10_10_intel.whl ? ?2015-12-10 ? ? ? ? ? 93

PyPI mirrors account for ~50-60 downloads, so you can see that the 2.6 wheel downloads are essentially insignificant. Of course, Python 2.6 users are disproportionately likely to be using a pip that predates wheel support or a tool like easy_install and without significant work parsing the PyPI logs we can't easily see the breakdown from the sdist (which obviously constitutes the vast, vast majority of all downloads).

In the absence of solid numbers on our actual download counts I'd propose fully dropping 2.6 support in a more graduated fashion. At the termination of support (2 releases by this proposal, which would be between March-May of 2016 at our current release rates) we would do the following:

* Remove the 2.6 trove classifier
* Changelog entry announcing that we no longer support 2.6 going forward.
* Remove 2.6 from our jenkins config
* Remove all but one 2.6 builder from our Travis config

Retaining one builder would not be unduly burdensome from a maintenance perspective, but it would allow us to realize almost all of the benefits of removing 2.6 including faster CI and less wheels to upload.

This approach would allow us to remove support quickly, but retain compatibility at least until such time as we decide we want a 2.7+ specific feature (which could terminate the last of 2.6 support at any time). It is possible that a bug would creep in due to not having full support in our test matrix, but we aren't claiming 2.6 compatibility and bugs of that type have been exceedingly rare thus far in the project.

-Paul
On December 22, 2015 at 9:44:12 PM, Alex Gaynor (alex.gaynor at gmail.com) wrote:

Hi all,

I'd like to propose we formally deprecate (in our next release) and remove (in the release thereafter) support for Python2.6.

It is no longer under support from its developers (which makes it a security risk), its use is more or less confined to users of CentOS and RHEL (which means SCL is available to them, and they have a corporate benefactor to complain to if it doesn't work), and its seriously funking with our development workflow because it's so abysmally slow in CI (which means it's annoying me enough to want to solve this).

There is genuinely no justifiable reason to use Python 2.6 anymore, both Django and Twisted have dropped it.

If this would _seriously_ impact you, please reply to this thread. If possible I'd also like to hear from our downstreams.

Cheers,
Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint:?125F 5C67 DFE9 4084
_______________________________________________  
Cryptography-dev mailing list  
Cryptography-dev at python.org  
https://mail.python.org/mailman/listinfo/cryptography-dev  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151222/94367b84/attachment.html>

From terrycwk1994 at gmail.com  Wed Dec 23 05:53:51 2015
From: terrycwk1994 at gmail.com (Terry Chia)
Date: Wed, 23 Dec 2015 10:53:51 +0000
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
In-Reply-To: <etPan.567a1ecb.44c38d6f.1da@atlas.local>
References: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
 <etPan.567a1ecb.44c38d6f.1da@atlas.local>
Message-ID: <CAGqxZSW9Bb90U7JnC2SA2MrMeMj-g0TT95BEJM0zMg974O9PCQ@mail.gmail.com>

Sounds good to me but would it not make more sense to remove 2.6 from our
Travis config and instead retain the CentOS / 2.6 builder in Jenkins given
that it's the only real place where 2.6 is being used?

On Wed, Dec 23, 2015 at 12:11 PM Paul Kehrer <paul.l.kehrer at gmail.com>
wrote:

> Some numbers to inform this conversation. Here are the download counts for
> cryptography 1.1.2 right now (12 days of downloads):
>
>                             cryptography-1.1.2.tar.gz    2015-12-10
>  261,761
>  cryptography-1.1.2-pp27-none-macosx_10_10_x86_64.whl    2015-12-10
>     86
>            cryptography-1.1.2-cp35-none-win_amd64.whl    2015-12-10
>    292
>                cryptography-1.1.2-cp35-none-win32.whl    2015-12-10
>    325
>   cryptography-1.1.2-cp35-cp35m-macosx_10_6_intel.whl    2015-12-10
>    481
> cryptography-1.1.2-cp35-cp35m-macosx_10_10_x86_64.whl    2015-12-10
>     50
>            cryptography-1.1.2-cp34-none-win_amd64.whl    2015-12-10
>    289
>                cryptography-1.1.2-cp34-none-win32.whl    2015-12-10
>    347
>   cryptography-1.1.2-cp34-cp34m-macosx_10_6_intel.whl    2015-12-10
>    254
> cryptography-1.1.2-cp34-cp34m-macosx_10_10_x86_64.whl    2015-12-10
>     63
>            cryptography-1.1.2-cp33-none-win_amd64.whl    2015-12-10
>    108
>                cryptography-1.1.2-cp33-none-win32.whl    2015-12-10
>     97
>   cryptography-1.1.2-cp33-cp33m-macosx_10_6_intel.whl    2015-12-10
>     60
> cryptography-1.1.2-cp33-cp33m-macosx_10_10_x86_64.whl    2015-12-10
>     49
>            cryptography-1.1.2-cp27-none-win_amd64.whl    2015-12-10
>  3,941
>                cryptography-1.1.2-cp27-none-win32.whl    2015-12-10
>  2,544
>    cryptography-1.1.2-cp27-none-macosx_10_6_intel.whl    2015-12-10
>  5,834
>   cryptography-1.1.2-cp27-none-macosx_10_10_intel.whl    2015-12-10
>    188
>            cryptography-1.1.2-cp26-none-win_amd64.whl    2015-12-10
>     86
>                cryptography-1.1.2-cp26-none-win32.whl    2015-12-10
>     63
>   cryptography-1.1.2-cp26-none-macosx_10_10_intel.whl    2015-12-10
>     93
>
> PyPI mirrors account for ~50-60 downloads, so you can see that the 2.6
> wheel downloads are essentially insignificant. Of course, Python 2.6 users
> are disproportionately likely to be using a pip that predates wheel support
> or a tool like easy_install and without significant work parsing the PyPI
> logs we can't easily see the breakdown from the sdist (which obviously
> constitutes the vast, vast majority of all downloads).
>
> In the absence of solid numbers on our actual download counts I'd propose
> fully dropping 2.6 support in a more graduated fashion. At the termination
> of support (2 releases by this proposal, which would be between March-May
> of 2016 at our current release rates) we would do the following:
>
> * Remove the 2.6 trove classifier
> * Changelog entry announcing that we no longer support 2.6 going forward.
> * Remove 2.6 from our jenkins config
> * Remove all but one 2.6 builder from our Travis config
>
> Retaining one builder would not be unduly burdensome from a maintenance
> perspective, but it would allow us to realize almost all of the benefits of
> removing 2.6 including faster CI and less wheels to upload.
>
> This approach would allow us to remove support quickly, but retain
> compatibility at least until such time as we decide we want a 2.7+ specific
> feature (which could terminate the last of 2.6 support at any time). It is
> possible that a bug would creep in due to not having full support in our
> test matrix, but we aren't claiming 2.6 compatibility and bugs of that type
> have been exceedingly rare thus far in the project.
>
> -Paul
>
> On December 22, 2015 at 9:44:12 PM, Alex Gaynor (alex.gaynor at gmail.com)
> wrote:
>
> Hi all,
>
> I'd like to propose we formally deprecate (in our next release) and remove
> (in the release thereafter) support for Python2.6.
>
> It is no longer under support from its developers (which makes it a
> security risk), its use is more or less confined to users of CentOS and
> RHEL (which means SCL is available to them, and they have a corporate
> benefactor to complain to if it doesn't work), and its seriously funking
> with our development workflow because it's so abysmally slow in CI (which
> means it's annoying me enough to want to solve this).
>
> There is genuinely no justifiable reason to use Python 2.6 anymore, both
> Django and Twisted have dropped it.
>
> If this would _seriously_ impact you, please reply to this thread. If
> possible I'd also like to hear from our downstreams.
>
> Cheers,
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151223/c1b7f068/attachment-0001.html>

From cory at lukasa.co.uk  Wed Dec 23 07:43:39 2015
From: cory at lukasa.co.uk (Cory Benfield)
Date: Wed, 23 Dec 2015 12:43:39 +0000
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
In-Reply-To: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
References: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
Message-ID: <ED0B2C4C-6AC6-49A5-820E-A0922C331EC4@lukasa.co.uk>


> On 23 Dec 2015, at 03:43, Alex Gaynor <alex.gaynor at gmail.com> wrote:
> 
> Hi all,
> 
> I'd like to propose we formally deprecate (in our next release) and remove (in the release thereafter) support for Python2.6.
> 
> It is no longer under support from its developers (which makes it a security risk), its use is more or less confined to users of CentOS and RHEL (which means SCL is available to them, and they have a corporate benefactor to complain to if it doesn't work), and its seriously funking with our development workflow because it's so abysmally slow in CI (which means it's annoying me enough to want to solve this).
> 
> There is genuinely no justifiable reason to use Python 2.6 anymore, both Django and Twisted have dropped it.
> 
> If this would _seriously_ impact you, please reply to this thread. If possible I'd also like to hear from our downstreams.

I strongly support this move. At this point I?ve received strong suggestions from Red Hat employees that would actively like Python OSS projects to stop supporting Python 2.6. If this is the case, I seen no reason for any PyCA project to continue to support Python 2.6 for their upcoming releases.

Cory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151223/f15b274e/attachment.sig>

From erik.trauschke at gmail.com  Wed Dec 23 12:20:12 2015
From: erik.trauschke at gmail.com (Erik Trauschke)
Date: Wed, 23 Dec 2015 09:20:12 -0800
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
In-Reply-To: <ED0B2C4C-6AC6-49A5-820E-A0922C331EC4@lukasa.co.uk>
References: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
 <ED0B2C4C-6AC6-49A5-820E-A0922C331EC4@lukasa.co.uk>
Message-ID: <CAHwug7ZpD36R75cK-yDA8YDiz_NQj+QH7xuMUkCiwK=WtGur4A@mail.gmail.com>

Same here, I see no issues with that.

On Wed, Dec 23, 2015 at 4:43 AM, Cory Benfield <cory at lukasa.co.uk> wrote:
>
>> On 23 Dec 2015, at 03:43, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>
>> Hi all,
>>
>> I'd like to propose we formally deprecate (in our next release) and remove (in the release thereafter) support for Python2.6.
>>
>> It is no longer under support from its developers (which makes it a security risk), its use is more or less confined to users of CentOS and RHEL (which means SCL is available to them, and they have a corporate benefactor to complain to if it doesn't work), and its seriously funking with our development workflow because it's so abysmally slow in CI (which means it's annoying me enough to want to solve this).
>>
>> There is genuinely no justifiable reason to use Python 2.6 anymore, both Django and Twisted have dropped it.
>>
>> If this would _seriously_ impact you, please reply to this thread. If possible I'd also like to hear from our downstreams.
>
> I strongly support this move. At this point I?ve received strong suggestions from Red Hat employees that would actively like Python OSS projects to stop supporting Python 2.6. If this is the case, I seen no reason for any PyCA project to continue to support Python 2.6 for their upcoming releases.
>
> Cory
>
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>

From simo at redhat.com  Wed Dec 23 12:52:41 2015
From: simo at redhat.com (Simo Sorce)
Date: Wed, 23 Dec 2015 12:52:41 -0500
Subject: [Cryptography-dev] [Proposal] Intent to deprecate and remove:
 Python 2.6
In-Reply-To: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
References: <CAFRnB2VDXA0dxthJnzF1hQcCqrH7EzFDACkuwXX5M-S6N9wawQ@mail.gmail.com>
Message-ID: <1450893161.3830.3.camel@redhat.com>

On Tue, 2015-12-22 at 22:43 -0500, Alex Gaynor wrote:
> Hi all,
> 
> I'd like to propose we formally deprecate (in our next release) and remove
> (in the release thereafter) support for Python2.6.
> 
> It is no longer under support from its developers (which makes it a
> security risk), its use is more or less confined to users of CentOS and
> RHEL (which means SCL is available to them, and they have a corporate
> benefactor to complain to if it doesn't work), and its seriously funking
> with our development workflow because it's so abysmally slow in CI (which
> means it's annoying me enough to want to solve this).
> 
> There is genuinely no justifiable reason to use Python 2.6 anymore, both
> Django and Twisted have dropped it.
> 
> If this would _seriously_ impact you, please reply to this thread. If
> possible I'd also like to hear from our downstreams.

We do not build python-cryptography for RHEL 6 so we do not have a
strong reason to ask for 2.6 support.
I can't speak for potential customers that may have a different opinion
and build it themselves.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York


From hs at ox.cx  Mon Dec 28 04:35:15 2015
From: hs at ox.cx (Hynek Schlawack)
Date: Mon, 28 Dec 2015 10:35:15 +0100
Subject: [Cryptography-dev] Keys/Certificates/CRLs/x509 in pyOpenSSL
Message-ID: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>

Hi,

we have quite a bit of pull requests on pyOpenSSL that revolve around improving the state of x509 objects in general as far as I understand it.

Since I already got reprimanded by Alex G for merging one because cryptography has routines for that, I wonder if we should close them all as WONTFIX and instead add methods akin to `PKey.from_cryptography()`, `key_instance.to_cryptography()`.

# Questions

- Am I misunderstanding something completely and this can?t happen for practical reasons?
- Does cryptography have everything in place to achieve this at all?

# Upsides

- Keeps pyOpenSSL from playing catch-up.
- Saves work in the long term.
- Gives users more power independently from pyOpenSSL releases.
- Risks of breaking backward compatibility are rather low.
- People would get used to do x509 work with cryptography in preparation for `cryptography.tls`.

# Downsides

- I would need *active* help from either Paul or Alex G or someone who really understands both x509 (I know, nobody actually understands that, please don?t go there) and the cryptography layer on both drafting this up and implementing it.
  Therefore this is absolutely *not my call* to make.
- It would further delay improvements to pyOpenSSL and the people on the bug tracker are already rightfully rather restless.  I?m afraid this might lead to months of nothing happening.

***

I welcome any feedback.  The current pyOpenSSL situation which is mostly a swamp of guilt is becoming unbearable to me.  When I took over maintainership I made it clear that I see myself mostly as a repo janitor and Bad Ideas Deflector?.  Sadly that?s not working out at all.  Getting rid of the burden of actually moving forward a whole sub-system might alleviate that a bit I guess (this is not meant as an ultimatum, I have no idea if it?d help).

Best,
?h
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151228/5564ef9e/attachment.sig>

From cory at lukasa.co.uk  Mon Dec 28 04:51:27 2015
From: cory at lukasa.co.uk (Cory Benfield)
Date: Mon, 28 Dec 2015 09:51:27 +0000
Subject: [Cryptography-dev] Keys/Certificates/CRLs/x509 in pyOpenSSL
In-Reply-To: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>
References: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>
Message-ID: <CE54E274-4F3E-4C1C-B785-190F8228A560@lukasa.co.uk>


> On 28 Dec 2015, at 09:35, Hynek Schlawack <hs at ox.cx> wrote:
> 
> Hi,
> 
> we have quite a bit of pull requests on pyOpenSSL that revolve around improving the state of x509 objects in general as far as I understand it.
> 
> Since I already got reprimanded by Alex G for merging one because cryptography has routines for that, I wonder if we should close them all as WONTFIX and instead add methods akin to `PKey.from_cryptography()`, `key_instance.to_cryptography()`.
> 
> I welcome any feedback.  The current pyOpenSSL situation which is mostly a swamp of guilt is becoming unbearable to me.  When I took over maintainership I made it clear that I see myself mostly as a repo janitor and Bad Ideas Deflector?.  Sadly that?s not working out at all.  Getting rid of the burden of actually moving forward a whole sub-system might alleviate that a bit I guess (this is not meant as an ultimatum, I have no idea if it?d help).

As official ?sometimes helps Hynek when he feels sad? person, I?m strongly in favour of deprecating whatever we can from PyOpenSSL if there is a good alternative available (i.e. cryptography). It?s frustrating and perplexing that installing PyOpenSSL gives you two interfaces for working with X509 certs, and where the top layer is arguably *less* helpful (and definitely more surprising) than the layer it uses to do the real work.

To make this kind of deprecation work I think we definitely need a to/from cryptography method to have been in place for a while, so I?m in favour of this plan. Long term, however, I want PyOpenSSL stripped down to be only what cryptography itself does not do.

Cory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151228/03f489a4/attachment.sig>

From paul.l.kehrer at gmail.com  Mon Dec 28 11:04:54 2015
From: paul.l.kehrer at gmail.com (Paul Kehrer)
Date: Mon, 28 Dec 2015 10:04:54 -0600
Subject: [Cryptography-dev] Keys/Certificates/CRLs/x509 in pyOpenSSL
In-Reply-To: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>
References: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>
Message-ID: <etPan.56815da7.4ab1508b.1da@atlas.local>

On December 28, 2015 at 3:43:02 AM, Hynek Schlawack (hs at ox.cx) wrote:
Hi,?

we have quite a bit of pull requests on pyOpenSSL that revolve around improving the state of x509 objects in general as far as I understand it.?

Since I already got reprimanded by Alex G for merging one because cryptography has routines for that, I wonder if we should close them all as WONTFIX and instead add methods akin to `PKey.from_cryptography()`, `key_instance.to_cryptography()`.?

# Questions?

- Am I misunderstanding something completely and this can?t happen for practical reasons??
- Does cryptography have everything in place to achieve this at all??
There's no reason why we can't do this technically. My past opposition has mostly been philosophical (if we do this we inextricably tie pyOpenSSL to cryptography), but perhaps I should just embrace it. We have everything we need to do it right now (x509 objects, RSA/EC/DSA objects), although it does require consuming some private classes/methods. Here's some untested code to show how we could do it:?https://github.com/reaperhulk/pyopenssl/commit/65d0658af74c892ca261d051ed50e37519eaff98


I welcome any feedback. The current pyOpenSSL situation which is mostly a swamp of guilt is becoming unbearable to me. When I took over maintainership I made it clear that I see myself mostly as a repo janitor and Bad Ideas Deflector?. Sadly that?s not working out at all. Getting rid of the burden of actually moving forward a whole sub-system might alleviate that a bit I guess (this is not meant as an ultimatum, I have no idea if it?d help).?
This is definitely a problem and I don't want you to get burned out due to guilt. Please don't hesitate to reach out (as you've done here) whenever you need/want some assistance in moving things forward.



Personally, I don't have a problem with features landing in pyOpenSSL. Yes, they're frequently duplicating features in cryptography, but until cryptography is actually a full replacement I don't feel comfortable telling people to use both APIs. Once pyOpenSSL's use case vanishes outside of "legacy compatibility shim" then I would support being more aggressive about rejecting new features. Of course, reviewing/landing feature PRs is hard work and I don't have a good answer for how to deal with that. Person-hours are a precious commodity :(



-Paul Kehrer (reaperhulk)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151228/a669dd4e/attachment.html>

From glyph at twistedmatrix.com  Tue Dec 29 05:45:30 2015
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 29 Dec 2015 02:45:30 -0800
Subject: [Cryptography-dev] Keys/Certificates/CRLs/x509 in pyOpenSSL
In-Reply-To: <CE54E274-4F3E-4C1C-B785-190F8228A560@lukasa.co.uk>
References: <DCA321D7-9702-4D9E-9E54-1105A1ABDC9A@ox.cx>
 <CE54E274-4F3E-4C1C-B785-190F8228A560@lukasa.co.uk>
Message-ID: <A320E762-2132-41E4-9E2D-30E83AE92B9F@twistedmatrix.com>


> On Dec 28, 2015, at 1:51 AM, Cory Benfield <cory at lukasa.co.uk <mailto:cory at lukasa.co.uk>> wrote:
> 
> 
>> On 28 Dec 2015, at 09:35, Hynek Schlawack <hs at ox.cx <mailto:hs at ox.cx>> wrote:
>> 
>> Hi,
>> 
>> we have quite a bit of pull requests on pyOpenSSL that revolve around improving the state of x509 objects in general as far as I understand it.
>> 
>> Since I already got reprimanded by Alex G for merging one because cryptography has routines for that, I wonder if we should close them all as WONTFIX and instead add methods akin to `PKey.from_cryptography()`, `key_instance.to_cryptography()`.
>> 
>> I welcome any feedback.  The current pyOpenSSL situation which is mostly a swamp of guilt is becoming unbearable to me.  When I took over maintainership I made it clear that I see myself mostly as a repo janitor and Bad Ideas Deflector?.  Sadly that?s not working out at all.  Getting rid of the burden of actually moving forward a whole sub-system might alleviate that a bit I guess (this is not meant as an ultimatum, I have no idea if it?d help).
> 
> As official ?sometimes helps Hynek when he feels sad? person, I?m strongly in favour of deprecating whatever we can from PyOpenSSL if there is a good alternative available (i.e. cryptography). It?s frustrating and perplexing that installing PyOpenSSL gives you two interfaces for working with X509 certs, and where the top layer is arguably *less* helpful (and definitely more surprising) than the layer it uses to do the real work.
> 
> To make this kind of deprecation work I think we definitely need a to/from cryptography method to have been in place for a while, so I?m in favour of this plan. Long term, however, I want PyOpenSSL stripped down to be only what cryptography itself does not do.

Long term, shouldn't pyOpenSSL be removed entirely, and Cryptography just does everything?

In the meanwhile though, I think that from_cryptography/to_cryptography are a good idea, and should contain a clear explanation of this plan: in other words, everyone using pyOpenSSL should start rewriting their applications to use the 'cryptography' objects as much as possible.

Right now, things are in a bit of a muddled state; pyOpenSSL is still the only package available to do many things (in particular: TLS) but pull requests are being refused on the grounds that Cryptography has superseded it, when there still isn't a clear interop strategy.  These methods (especially if properly documented) could really improve this situation.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20151229/6c30742f/attachment.html>