any issues with long running python apps?

[Terry Reedy]

On 7/9/2010 3:13 PM, Les Schaffer wrote:
> i have been asked to guarantee that a proposed Python application will
> run continuously under MS Windows for two months time. And i am looking
> to know what i don't know.
>
> The app would read instrument data from a serial port, store the data in
> file, and display in matplotlib. typical sampling times are about once
> per hour. our design would be to read the serial port once a second
> (long story, but it worked for ten days straight so far) and just store
> and display the values once an hour. so basically we'd have a long
> running for-loop. maybe running in a separate thread.

Is this a dedicated machine, so you do not have anything else going that 
can delay for more than a second?
>
> i have thought through the issues on our end: we have to properly handle
> the serial port and make sure our GUI doesn't hang easily. we'd have to
> be sure we don't have any memory leaks in numpy or matplotlib and we're
> not storing data in memory stupidly. i've never looked into Windows
> serial port driver reliability over the long term. But we think if the
> serial port hangs, we can detect that and react accordingly.

I read the ibmpc serial port bios code in the early 80s. It was pretty 
simple then. I would be very surprised if it has been messed up since 
and not fixed.

> but none of this has anything to do with Python itself. i am sure python
> servers have been running reliably for long periods of time, but i've
> never had to deal with a two-month guarantee before. is there something
> else i am missing here that i should be concerned about on the
> pure-Python side of things? something under the hood of the python
> interpreter that could be problematic when run for a long time?
>
> or need we only concern ourselves with the nuts behind the wheel:that
> is, we the developers?

Python has been used for long-running background processes, at least on 
*nix, so the Python devs are sensitive to memory leak issues and respond 
to leak reports. That still leaves memory fragmentation. To try to avoid 
that, I would allocate all the needed data arrays immediately on startup 
(with dummy None pointers) and reuse them instead of deleting and 
regrowing them. Keeping explicit pointers is, of course more tedious and 
slightly error prone.

I hope someone with actual experience also answers.
-- 
Terry Jan Reedy