embedded python?

[Peter Hansen]

Alexander May wrote:

> We are developing a distributed application running on approximately six
> thousand nodes with somewhat limited hardware resources.  Our hardware
> target is 66 MHz ARM style processor with 16 Mb ram.  We haven't selected
> specific hardware yet; the hardware target is what we are trying to fit into
> based on price constraints.  Each device needs to be able to handle about 2
> kbs (yes kilo, not mega) worth of network traffic.

You mention network, so presumably you have Ethernet (?) interfaces, and
plan on TCP or UDP, in which case you probably first need to consider
operating systems... Python does not automatically give you support for
networking, it just uses the underlying OS support.

> I intend to a least prototype the system in Python.  It would be great if we
> could we could use Python in production by embedding it in the hardware.  My
> question is does anyone have any real world experience using python in an
> embedded system?  

Yes, there are a few people around here who have worked on embedded
Python systems, including me (at a past employer).

> 1) Are there any embedded Pythons out there?  The nodes will likely be
> running some form of Linux, but I don't particularly feel like devoting
> resrouces to porting python.  Any embedded Linuxes supporting Python?
> Thoughts in general?

I doubt you'll need to do much "porting".  Unless the Linices you
are considering are quite unusual, they ought to look a lot like any
other Linux, and Python may well work largely unchanged.  Any required
changes could be in areas you don't need, anyway.  Again, make sure
you consider right away exactly what OS resources, other than
networking, that you actually need.  Multitasking?  Signals?  Audio?
GUI?  etc...

> 2) What are the resource requirements of Python?  How much overhead do the
> network related modules add?  

_socket.pyd is 49212 on Win32, and 124040 (probably with debugging
symbol table included?) on Linux, for Python 2.3.4.  This includes
lots of doc-strings, though, so you can probably shrink it pretty
easily with a configurable setting during compilation.

> In short, is embedding python realistic?

Definitely.  We did it first on a 33MHz 386 (or 286?) compatible
board with only 1MB of flash and 1MB of RAM, and a tiny embedded
OS with no networking.  That took a little slashing, such as removing
floating point support, but for the most part such things were
supported well by #defines during compilation.  I think we based
that one on Python 1.5.2.  (It was a few years ago.)  Performance,
however, turned out to be inadequate and we decided we'd get
farther faster by keeping Python and switching to faster hardware
(than by ditching Python and doing it all in C).

The next version was a 100MHz 486 compatible with 32MB RAM and
Compact Flash as a hard drive.  We picked 32MB flash as the target,
but used 64MB for prototyping and kept using it thereafter because
CF cards kept getting cheaper.  This version ran vanilla Linux
with everything non-essential removed, and stock Python recompiled
from sources with, I believe, no changes.  We stripped a few library
files from the resulting binary (e.g. unicodedata.so) to reduce the
footprint.

I know others have done similar things on hardware in this range,
but I don't know of anything freely available yet.  Not sure that
would make sense, either, as everybody uses different hardware in
this sort of thing...

The main advice I can give you from this experience is that if
your hardware doesn't quite cut it, considering putting the money
into better hardware rather than immediately abandoning any thought
of using Python.  On the other hand, you have 6K nodes, and we
had less than a few hundred, so the "economies of scale" just
weren't there as much in our case and development costs were
the largest consideration.  In your case, saving $100 on each
unit might be more important than saving a few thousand hours
of development time...  your call.

-Peter