any issues with long running python apps?

[geremy condra]

On Fri, Jul 9, 2010 at 3:13 PM, Les Schaffer <schaffer at optonline.net> wrote:
> i have been asked to guarantee that a proposed Python application will run
> continuously under MS Windows for two months time. And i am looking to know
> what i don't know.

I normally use Linux for this sort of thing, so YMMV on the following advice.

> The app would read instrument data from a serial port, store the data in
> file, and display in matplotlib.  typical sampling times are about once per
> hour. our design would be to read the serial port once a second (long story,
> but it worked for ten days straight so far) and just store and display the
> values once an hour. so basically we'd have a long running for-loop. maybe
> running in a separate thread.

I'd keep the two timers, the process that actually checks and logs the data,
and any postprocessing code completely separate. I'd also use something
like the logging module to double up on where your data is stored- one on
the local machine, another physically separated in case you lose a hard
drive. It will also give you more information about where a failure might
have occurred if/when it does. I'd also handle output/display on a separate
machine.

> i have thought through the issues on our end:  we have to properly handle
> the serial port and make sure our GUI doesn't hang easily. we'd have to be
> sure we don't have any memory leaks in numpy or matplotlib and we're not
> storing data in memory stupidly. i've never looked into Windows serial port
> driver reliability over the long term. But we think if the serial port
> hangs, we can detect that and react accordingly.

I would launch this as a subprocess and log so that even if you miss a
measurement you still get what you need further on in the process.
Just make sure that you can detect it at the time and that you also
log an error when it occurs.

This also minimizes the amount of memory your application directly
handles and the susceptibility of your code to non-fatal problems with
the serial port.

> but none of this has anything to do with Python itself. i am sure python
> servers have been running reliably for long periods of time, but i've never
> had to deal with a two-month guarantee before.  is there something else i am
> missing here that i should be concerned about on the pure-Python side of
> things? something under the hood of the python interpreter that could be
> problematic when run for a long time?

Just ask all the what-ifs and you'll probably be ok.

Geremy Condra