Program uses twice as much memory in Python 3.6 than in Python 3.5

[Pavol Lisy]

On 3/29/17, Jan Gosmann <jan at hyper-world.de> wrote:
> On 28 Mar 2017, at 14:21, INADA Naoki wrote:
>
>> On Wed, Mar 29, 2017 at 12:29 AM, Jan Gosmann <jan at hyper-world.de>
>> wrote:
>>
>> I suppose smaller and faster benchmark is better to others looking for
>> it.
>> I already stopped the azure instance.
>> [...]
>> There are no maxrss difference in "smaller existing examples"?
>> [...]
>> I want to investigate RAM usage, without any swapping.
>
> Running further trials indicate that the problem actually is related to
> swapping. If I reduce the model size in the benchmark slightly so that
> everything fits into the main memory, the problem disappears. Only when
> the memory usage exceeds the 32GB that I have, Python 3.6 will acquire
> way more memory (from the swap) than Python 3.5.
>
> Jan
> --
> https://mail.python.org/mailman/listinfo/python-list

Could you add table comparing time benchmarks when memory is bigger?
(if your hypothesis is true and memory measurement tools are right
than time difference has to be huge)

Did you compare "pip list" results? There could be more differences in
your environments (not only python version). For example different
numpy versions or some missing packages could change game.

I tried to search "except.*ImportError" in your repository, but I am
not sure that it could change it significantly...

( https://github.com/ctn-archive/gosmann-frontiers2017/search?utf8=%E2%9C%93&q=ImportError&type=

This one seems suspitious - sparse matrix class could be game changer

            from scipy.sparse import bsr_matrix
            assert bsr_matrix
        except (ValueError, ImportError):
            return False

)


This one doesn't seems suspicious to me (but who knows?):

    try:
        import faulthandler
        faulthandler.enable()
    except:
        pass

PL.