Python for embedded systems with memory constraints

[vishnu]

Hi,
Thanks Cameron for your suggestions.
In fact I am using custom memory sub-allocator where I preallocate a
pool of memory during initialization of my application and ensure that
Python doesn't make any system mallocs later . With this arrangement,
python seems to run out of preallocated memory (of 10MB) after running
few simple scripts due to huge external fragmentation. My memory
sub-allocator got a good design which uses the best-fit algorithm and
coaelescing the adjacent blocks during each free call.
If anybody out there used their own memory manager and ran Python
without fragmentation , could provide some inputs on this.

Thanks in advance.

On 6/7/07, Cameron Laird <claird at lairds.us> wrote:
> In article <mailman.8686.1180981155.32031.python-list at python.org>,
> vishnu  <gkkvishnu at gmail.com> wrote:
> >Hi there,
> >
> >I am embedding python 2.5 on embedded system running on RTOS where I
> >had strict memory constraints.
> >As python is a huge malloc intensive application, I observed huge
> >memory fragmentation in my system which is leading to out of memory
> >after running few scripts.
> >So I decided to re-initialise the python with out restarting the whole python.
> >I tried to use Py_Finalise() after completion of each script , then
> >call Py_Initialise as is done in below link.
> >http://mail.python.org/pipermail/python-list/2001-November/114253.html
> >Which in every loop it causes a leak of 10K and after some iterations
> >it leaks of 200K etc. After few more runs this crashes. I read some
> >where this leak was solved in 2.5, but with 2.5 also I am having
> >problems....
> >And also I found Py_Finalise does not completely cleanup the memory,
> >So how do I re-initialise my memory pool?
> >
> >Does anybody faced this problem earlier and got any solution hack to
> >run the python for a embedded system within own managed memory pool of
> >say 10MB?
> >
> >Any help/ideas are greatly appreciated!. Thanks in advance.
>
> Your report is interesting and important--and surprising!  I thought
> Python memory allocation is "cleaner" than you seem to be observing.
>
> I hope one of the core Python maintainers can address this.  I haven't
> worked at this level recently enough to speculate on why it's happen-
> ing, nor will I soon be in a position to volunteer to research it on
> my own (although I'd eagerly contract to do so on a modestly paid
> basis).
>
> Depending on your schedule and technology, there are lots of technical
> fixes that might apply:
> A.  quick-starting Python variations that encourage you
>     to manage memory on a whole-process level;
> B.  use of one of the many Python variants (even PyPy?)
>     that might give you a more favorable memory profile;
> C.  switch to Lua or Tcl as more easily embeddable
>     alternative languages;
> D.  custom memory allocator;
> ...
> --
> http://mail.python.org/mailman/listinfo/python-list
>