[Cython] Hudson pyregr testing takes too long

[Vitja Makarov]

2011/4/25 Stefan Behnel <stefan_ml at behnel.de>:
> Vitja Makarov, 25.04.2011 11:04:
>>
>> 2011/4/25 Stefan Behnel<stefan_ml at behnel.de>:
>>>
>>> Vitja Makarov, 25.04.2011 08:19:
>>>>
>>>> 2011/4/25 Stefan Behnel:
>>>>>
>>>>> Stefan Behnel, 07.04.2011 13:52:
>>>>>>
>>>>>> Stefan Behnel, 07.04.2011 13:46:
>>>>>>>
>>>>>>> I just noticed that the CPython pyregr tests have jumped up from ~14
>>>>>>> minutes for a run to ~4 hours when we added generator support.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-pyregr-py26-c/buildTimeTrend
>>>>>>>
>>>>>>> I currently have no idea why that is (well, it's likely because we
>>>>>>> compile
>>>>>>> more tests now, but Vitja's branch ran the tests in ~30 minutes). It
>>>>>>> would
>>>>>>> be great if someone could find the time to analyse this problem. The
>>>>>>> current run time makes it basically impossible to keep these tests
>>>>>>> enabled.
>>>>>>
>>>>>> Ok, it looks like this is mostly an issue with the Py2.6 tests. The
>>>>>> Py2.7
>>>>>> tests take 30-45 minutes, which is very long, but not completely out
>>>>>> of
>>>>>> bounds. I've disabled the Py2.6 pyregr tests for now.
>>>>>
>>>>> There seems to be a huge memory leak which almost certainly accounts
>>>>> for
>>>>> this. The Python process that runs the pyregr suite ends up with about
>>>>> 50GB
>>>>> of memory at the end, also in the latest Py3k builds.
>>>>>
>>>>> I have no idea where it may be, but it started to show when we merged
>>>>> the
>>>>> generator support. That's where I noticed the instant jump in the
>>>>> runtime.
>>>>
>>>> That's very strange for my branch it takes about 30 minutes that is ok.
>>>
>>> There's also a second path that's worth investigating. As part of the
>>> merge,
>>> there was another change that came in: the CythonPyregrTestCase
>>> implementation. This means that the regression tests are now being run
>>> differently than before. The massive memory consumption may simply be due
>>> to
>>> the mass of unit tests being loaded into memory.
>>
>>    def run_test():
>> ..................................
>>         try:
>>             module = __import__(self.module)
>>             if hasattr(module, 'test_main'):
>>                 module.test_main()
>>         except (unittest.SkipTest, support.ResourceDenied):
>>             result.addSkip(self, 'ok')
>>
>>
>> It seems that all the modules stay loaded so may be they should be
>> unloaded with del sys.modules[module_name]?
>
> (Binary) module unloading isn't really supported in CPython. There's PEP
> 3121 that has the potential to change it, but it's not completely
> implemented, neither in CPython nor in Cython. A major problem is that
> unloading a module deletes its globals but not necessarily the code that
> uses them. For example, instances of types defined in the module can still
> be alive at that point.
>
> The way runtests.py deals with this is forking before loading a module.
> However, this does not currently work with the "xmlrunner" which we use on
> Hudson, so we let all tests run in a single process there.
>


Btw when running plain python code with generators total ref counter
doesn't get back to initial value.
I tried to trace scope and generator destructors and they are run as
expected. So I'm not sure about leaks in generators.

-- 
vitja.