[Python-Dev] Infinie recursion in Pickle

[jepler@unpythonic.net]

Is this posted as an SF bug report yet?

I have a small program which can hit the recursion limit in pickle and
cause sig11 in cPickle.  It's a very deep nested tuple in this case.

If the first 'pickle.dump()' call is not commented out, the program dies
with "RuntimeError: maximum recursion depth exceeded".  If the second bit
of code is executed, it dies with segmentation violation.

On my system, redhat 7.2, the stack in the main thread is very large, but
the stack in other threads is very small.  On systems where the main stack
is smaller, just running
    cPickle.dump(x, open("/dev/null", "w"))
should show the problem, no threads needed.

Probably some sort of stack check should be present in cPickle, but there's
nothing much that can be done about data structures that are so deeply
recursive that they fill the stack.  Well, pickle could be rewritten to be
iterative, but that's a tall order.

(traceback from cPickle:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1026 (LWP 1198)]
0x4004912c in __new_sem_post (sem=0x8154358) at semaphore.c:137
137     semaphore.c: No such file or directory.
        in semaphore.c
(gdb) where
#0  0x4004912c in __new_sem_post (sem=0x8154358) at semaphore.c:137
#1  0x08095ec2 in PyThread_release_lock (lock=0x8154358)
    at Python/thread_pthread.h:412
#2  0x08079a4d in PyEval_SaveThread () at Python/ceval.c:329
#3  0x4025b09d in write_file (self=0x80ff3e8, s=0x4025db54 "(", n=1)
    at /home/jepler/cvs/python/dist/src/Modules/cPickle.c:414
#4  0x4025396a in save_tuple (self=0x80ff3e8, args=0x4062a66c)
    at /home/jepler/cvs/python/dist/src/Modules/cPickle.c:1340
#5  0x40254f18 in save (self=0x80ff3e8, args=0x4062a66c, pers_save=0)
    at /home/jepler/cvs/python/dist/src/Modules/cPickle.c:1944
#6  0x402539b1 in save_tuple (self=0x80ff3e8, args=0x4062a68c)
    at /home/jepler/cvs/python/dist/src/Modules/cPickle.c:1350
...)

import pickle, cPickle

x = ()
for i in range(100000):
    x = (x,)


#pickle.dump(x, open("/dev/null", "w"))


import thread, time
thread.start_new_thread(cPickle.dump, (x, open("/dev/null", "w")))
time.sleep(1000)