[Tutor] How to debug a memory leak in a wsgi application?

[Etienne Robillard]

Hi again James,


Le 2017-12-14 à 04:44, James Chapman a écrit :
> No, I'm saying you shouldn't need to make any kind of malloc calls 
> manually. Python handles memory allocation and deallocation on your 
> behalf.
All I did is installing a precompiled Python 2.7 build from the Debian 
repository. I believe it was built with pymalloc and debugging.
> Why do you need to call pymalloc?
I have not yet took the time to manually compile Python without pymalloc.
> Are you using ctypes?
No.

> And if you are I presume this is then to make C-calls into a shared 
> library?
I use Cython instead of ctypes. I'm guessing the memory leak was not 
caused by the Cython-generated C code, but from the uWSGI backend.

Cheers,

Etienne

>
>
>
> --
> James
>
> On 13 December 2017 at 18:30, Etienne Robillard <tkadm30 at yandex.com 
> <mailto:tkadm30 at yandex.com>> wrote:
>
>     Hi James,
>
>     Thank for your reply. Are you suggesting that under Linux the
>     malloc() glibc library call is more memory efficient than using
>     pymalloc?
>
>     Best regards,
>
>     Etienne
>
>
>     Le 2017-12-13 à 12:27, James Chapman a écrit :
>>     Why pymalloc? I presume this means you're using ctypes which
>>     means I have more questions.
>>
>>     If you're allocating your own blocks of memory then you need to
>>     free them too. IE, does each call to pymalloc have a
>>     corresponding call to pyfree?
>>
>>     Is the overhead of pythons built in malloc really a problem?
>>
>>     Are you changing pointers before you've freed the corresponding
>>     block of memory?
>>
>>     There are many ways to create a memory leak, all of them
>>     eliminated by letting python handle your memory allocations.
>>
>>     But, back to your original question, check out "valgrind".
>>
>>     HTH
>>
>>     --
>>     James
>>
>>     On 6 December 2017 at 16:23, Etienne Robillard
>>     <tkadm30 at yandex.com <mailto:tkadm30 at yandex.com>> wrote:
>>
>>         Hi Alan,
>>
>>         Thanks for the reply. I use Debian 9 with 2G of RAM and
>>         precompiled Python 2.7 with pymalloc. I don't know if
>>         debugging was enabled for this build and whether I should
>>         enable it to allow memory profiling with guppy... My problem
>>         is that guppy won't show the heap stats for the uWSGI master
>>         process. However I have partially resolved this issue by
>>         enabling --reload-on-rss 200 for the uwsgi process. 
>>         Previously, the htop utility indicated a 42.7% rss memory
>>         usage for 2 uWSGI processes. I have restarted the worker
>>         processes with SIGINT signal. Now my uwsgi command line looks
>>         like:
>>
>>         % uwsgi --reload-on-rss 200 --gevent 100 --socket
>>         localhost:8000 --with-file /path/to/file.uwsgi --threads 2
>>         --processes 4 --master --daemonize /var/log/uwsgi.log
>>
>>         My framework is Django with django-hotsauce 0.8.2 and
>>         werkzeug. The web server is nginx using uWSGI with the gevent
>>         pooling handler.
>>
>>         Etienne
>>
>>         Le 2017-12-06 à 10:00, Alan Gauld via Tutor a écrit :
>>
>>             On 06/12/17 09:21, Etienne Robillard wrote:
>>
>>                 Hi
>>
>>                 I think my wsgi application is leaking and I would
>>                 like to debug it.
>>
>>                 What is the best way to profile memory usage in a
>>                 running wsgi app?
>>
>>             This is probably a bit advanced for the tutor list, you might
>>             get a better response on the main Python list.
>>
>>             But to get a sensible answer you need to provide more data:
>>             What OS and Python version?
>>             What toolset/framework are you using?
>>             What measurements lead you to suspect a memory leak?
>>
>>
>>
>>         -- 
>>         Etienne Robillard
>>         tkadm30 at yandex.com <mailto:tkadm30 at yandex.com>
>>         https://www.isotopesoftware.ca/ <https://www.isotopesoftware.ca/>
>>
>>         _______________________________________________
>>         Tutor maillist  - Tutor at python.org <mailto:Tutor at python.org>
>>         To unsubscribe or change subscription options:
>>         https://mail.python.org/mailman/listinfo/tutor
>>         <https://mail.python.org/mailman/listinfo/tutor>
>>
>>
>
>     -- 
>     Etienne Robillard
>     tkadm30 at yandex.com <mailto:tkadm30 at yandex.com>
>     https://www.isotopesoftware.ca/ <https://www.isotopesoftware.ca/>
>
>

-- 
Etienne Robillard
tkadm30 at yandex.com
https://www.isotopesoftware.ca/