[Cython] Generators not closed on dealloc (Cython 0.15.1+, current git master)

[Matt Hagy]

Hi,

Cython generators aren’t closed on dealloc and this can lead to cleanup code not being ran (see example below). I couldn’t find any reference to this issue on the trac or the wiki and therefore wanted to report it. If this is a know issue please disregard this post.

I believe this can easily be corrected by adding the following line to the top of each generator dealloc function:

    if (p->resume_label != -1) __Pyx_Generator_Close(o);

This explicitly closes the generator unless it’s already finished. For testing I manually added this code to the generated C file and it worked as expected.

-Matt

Example:

I found this issue in the following code that relies upon a finally block to free memory. If not explicitly closed, the cleanup code is never executed and memory is leaked.

############################################
# generate_primes.pyx

from libc.stdlib cimport malloc, realloc, free

def generate_primes(unsigned int initial_buffer_size=64):
    cdef unsigned int test_number, i, N_primes, N_allocated
    cdef unsigned int *primes

    N_primes = 0
    N_allocated = initial_buffer_size
    primes = <unsigned int*>malloc(N_allocated * sizeof(unsigned int))
    if not primes: raise MemoryError

    test_number = 2

    try:
        while 1:
            # is the current value of test_number prime?
            # i.e. is it not divisible by any of the already seen primes
            i = 0
            while i<N_primes and test_number % primes[i] != 0:
                i += 1

            if i == N_primes:
                yield test_number

                # grow primes array if needed
                if N_primes == N_allocated:
                    N_allocated = N_allocated + 64
                    primes = <unsigned int*>realloc(primes, N_allocated * sizeof(unsigned int))
                    if not primes: raise MemoryError

                primes[N_primes] = test_number
                N_primes += 1

            test_number += 1
    finally:
        print 'clean up'
        free(primes)

###############################################