[Python.NET] Memory leak problem in function "AsManagedObject"
Tony Roberts
tony at pyxll.com
Wed Jan 20 03:53:14 EST 2016
Hi Eliana,
great, glad you were able to find the problem. I'll make that change in
github.
thanks,
Tony
On Tue, Jan 19, 2016 at 8:46 PM Eliana Mendes <eliana.mendesp at gmail.com>
wrote:
> Thank you all for the answers!
>
>
> I got the source code from the githup and built it myself, as you
> suggested. Unfortunately the memory leak problem remains even with this
> latest branch. But this time I was able to figure out where the problem is!
>
>
>
> The problem is not inside the “AsMangedObject”” as I thought before, but
> instead it is inside the “GetItem(int index)” method which is inside the "
> *pyobject*" class. This method creates a new instance of *PyInt *and a
> new instance of *PyObject*. Those are exactly the objects that my memory
> profiler was showing. In order to fix my problem I had to first edit this
> method to be like this:
>
>
> public virtual PyObject GetItem(int index) {
>
> PyInt key = new PyInt(index);
>
> PyObject item = GetItem((PyObject)key);
>
> key.Dispose();
>
> return item;
>
> }
>
>
>
> After doing that, in my code, I had to add a dispose for myTuple[i]. So I
> changed my code to look something similar to this:
>
>
>
> PyObject tupleResult = myTuple[0];
>
> results[0] =
> (double)tupleResult.AsManagedObject(typeof(double));
>
> tupleResult.Dispose();
>
>
>
> By doing those changes the problem was entirely fixed!
>
>
>
> I’m not sure who to contact exactly about changing the source code, but I
> would highly recommend in making those changes in the “GetItem” method in
> order to avoid someone else having the same memory problem.
>
>
>
> Thanks a lot for all your help!
>
>
> Best regards,
>
>
> Eliana
>
> 2016-01-18 16:54 GMT-06:00 Denis Akhiyarov <denis.akhiyarov at gmail.com>:
>
>> Your binaries are rather outdated..
>>
>> Here is the latest branch:
>>
>> https://github.com/pythonnet/pythonnet/tree/develop
>>
>> And binaries here:
>>
>> https://pypi.python.org/pypi/pythonnet/2.1.0.dev1
>>
>> http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonnet
>>
>> On Mon, Jan 18, 2016, 4:47 PM Eliana Mendes <eliana.mendesp at gmail.com>
>> wrote:
>>
>>> Hello Tony,
>>>
>>>
>>>
>>> Thanks a lot for your answer! I actually just got directly the
>>> “Python.Runtime.dll” from the download link in sourceforge:
>>> http://sourceforge.net/projects/pythonnet/files/. But from the
>>> description I see that the last update of that one was done in 2013. Would
>>> there be a newer version somewhere else that I could get? Perhaps this
>>> version I’m using is out of date and you have a newer one. Let me know if
>>> that’s the case.
>>>
>>>
>>>
>>> I tried your suggestion of putting the “using” statement for the tuple
>>> objects, and also for “myTuple”, and the memory is still increasing a lot.
>>> I’ve made a memory profile to see which objects are being retained and
>>> after I ran that statement 5 thousand times I got this:
>>>
>>>
>>> [image: Imagem inline 1]
>>>
>>>
>>> As you can see more that 12 thousand instances of “PyInt” and more than
>>> 6 thousand of “PyObject” are created . Apparently it is creating these
>>> “PyInt” and “PyObjects” somewhere inside the “AsManagedObject”, because I
>>> don’t have PyInt objects anywhere in my code and this just happens when I
>>> call “AsManagedObject”.
>>>
>>>
>>>
>>> Let me know if there is a newer version that could probably have that
>>> problem fixed which I am not aware of.
>>>
>>>
>>> I appreciate a lot your help!
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Eliana
>>>
>>> 2016-01-17 6:05 GMT-06:00 Tony Roberts <tony at pyxll.com>:
>>>
>>>> Hi Eliana,
>>>>
>>>> which version of pythonnet are you using? when you say you're using the
>>>> latest are you building it yourself from the develop branch on github?
>>>>
>>>> I had a quick look at the code that converts from a python object to a
>>>> double, and I don't see any obvious memory leaks there. Perhaps the leak is
>>>> the objects resulting from indexing into the tuple are never getting
>>>> disposed? Try something like this instead and see if it helps (and report
>>>> back as it may help improve the code if we know exactly what the problem
>>>> is).
>>>>
>>>> PyTuple myTuple = PyTuple.AsTuple(result);
>>>>
>>>> double result0;
>>>> using (var item0 = myTuple[0])
>>>> result0 = (double)item0.AsManagedObject(typeof(double));
>>>>
>>>> double result1;
>>>> using (var item1 = myTuple[1])
>>>> result1 = (double)item1.AsManagedObject(typeof(double));
>>>>
>>>> myTuple.Dispose();
>>>>
>>>> I would also use a using statement for the myTuple as well, just to be
>>>> sure dispose is called in the case an exception is thrown somewhere.
>>>>
>>>> Regards,
>>>> Tony
>>>>
>>>>
>>>> On Fri, Jan 15, 2016 at 8:06 PM Eliana Mendes <eliana.mendesp at gmail.com>
>>>> wrote:
>>>>
>>>>> Hello experts,
>>>>>
>>>>>
>>>>>
>>>>> I'm having a memory leak problem when using the
>>>>> function AsManagedObject(typeof(double)). Basically I have something like
>>>>> this:
>>>>>
>>>>>
>>>>>
>>>>> PyTuple myTuple = PyTuple.AsTuple(result);
>>>>>
>>>>> double result0 = (double)myTuple[0].AsManagedObject(typeof(double));
>>>>>
>>>>> double result1 = (double)myTuple[1].AsManagedObject(typeof(double));
>>>>>
>>>>> myTuple.Dispose();
>>>>>
>>>>>
>>>>>
>>>>> where "result" is just a PyObject that returned from a python
>>>>> function. I simplified the code above just so you can understand better,
>>>>> but the thing is that the line that calls "AsManagedObject” is executed
>>>>> thousands of times and it is increasing significantly the memory heap (it
>>>>> goes over 3 GB of memory in my scenario and it’s not released after
>>>>> execution). If I don't call just this specific function the memory remains
>>>>> stable. But I don’t know any other way to convert the PyObject to "double"
>>>>> unless using the “AsManagedObject” function.
>>>>>
>>>>> It sounds to me that some objects are allocated inside the
>>>>> "AsManagedObject" method and they are not being released. Maybe it’s a bug
>>>>> there. Any ideas? I'm using latest version of python for .NET.
>>>>>
>>>>>
>>>>>
>>>>> Thank you!
>>>>>
>>>>>
>>>>> Eliana Mendes
>>>>>
>>>>> Software Engineer
>>>>>
>>>>> _________________________________________________
>>>>> Python.NET mailing list - PythonDotNet at python.org
>>>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>>
>>>>
>>>> _________________________________________________
>>>> Python.NET mailing list - PythonDotNet at python.org
>>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>>
>>>
>>> _________________________________________________
>>> Python.NET mailing list - PythonDotNet at python.org
>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>
>>
>> _________________________________________________
>> Python.NET mailing list - PythonDotNet at python.org
>> https://mail.python.org/mailman/listinfo/pythondotnet
>>
>
> _________________________________________________
> Python.NET mailing list - PythonDotNet at python.org
> https://mail.python.org/mailman/listinfo/pythondotnet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20160120/2b3cf642/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 70883 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20160120/2b3cf642/attachment-0001.png>
More information about the PythonDotNet
mailing list