Memory usage problem of twisted server

[Dieter Maurer]

Victor Lin <bornstub at gmail.com> writes on Wed, 20 Jan 2010 02:52:25 -0800 (PST):
> Hi,
> 
> I encountered an increasing memory usage problem of my twisted server.
> I have posted a question on stackoverflow:
> http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server
> 
> I have read the article "Improving Python's Memory Allocator" (
> http://evanjones.ca/memoryallocator/ ) and Python Memory Management
> ( http://evanjones.ca/python-memory.html ). And I now know little
> about how Python manages memory. I am wondering, is that the
> increasing memory usage problem of my audio broadcasting caused by the
> how python manage memory?

Your careful reading has already told you that Python delegates
memory allocation for larger blocks (>= 256 bytes) to the underlying
C runtime library ("malloc" and friends).

The C runtime library does not use memory compaction, i.e.
it does not relocate used memory blocks in order to free space
in few large chunks. Therefore, it is sensible to memory fragmentation:
the free space gets scattered around in a large number of rather small
blocks.
The fragmentation rate is especially high when the memory request sizes
have a high variance.
> ....
> Is that my guessing correct? How can I monitor the memory allocation
> of Python?

Look at "http://guppy-pe.sourceforge.net/"

--
Dieter