[python-win32] optimizing code to read serial port

[Tony Cappellini]

<python-win32 at python.org>

Regarding an in-house python program we use for dumping diagnostic data
over a serial port on Windows 7 ...


I'm trying to determine if it's possible optimize the python code so that
dumping the data will occur faster. Currently, it takes ~25 minutes at 9600
baud to dump 8-15 MB of data. Going to higher baud rates does reduce this
time, but depending on which PC is running the program, going to higher
baud rates often results in ill-formed data. The serial port on the device
we are dumping data from has a 2-wire interface (no handshaking). Sometimes
115200 baud works fine, other times the data gets all jumbled. We've
decided to default to 9600 to avoid getting jumbled data.

Compared to an openly available terminal program (Tera Term Pro) written in
a compiled language, our Python version is much slower, but easier to
program. Tera Term's scripting language isn't as easy to script with as
Python.

At the lowest level of the Python app, the last call in python is
win32file.ReadFile( comport, bytes_, None ). I believe execution goes to a
PyWin32 extension module at this point.

By default, 1 is passed in for the number of bytes to read. I thought this
was very inefficient,
so I've experimented with values like 2,4,8,16. I didn't want to go higher
than 16 because the serial port FIFO on many PCs isn't any larger than 16.

While using cProfile, I could see that the number of function calls
executed was cut in half,
each time the # of bytes read doubled. Normally this would be a win, but
the total time to dump the same amount of data over the serial port would
almost double also.

While this may not be strictly a Python-specific issue, the performance
issue may be related to the Python-Win32 interface. For example, are there
more efficient ways to read data from a serial port using Python, than
calling win32file.ReadFile() ?

I would also expect reading more 8-16 bytes at a time to be more efficient
than reading 1 byte.
I do expect that buffering weighs in heavily here, but I have no control
over what the serial hardware does nor how windows interacts with it.

Having read several of the Python optimization tips, I've already
eliminated string concatenation, replaced globals variables with locals,
and eliminated several dot references that are inside of loops.There
doesn't seem to be any places I can use calls to map() or replace for loops
with list/gen comps.

There is too much code in the app to post here. I know it's difficult to
comment on code that you can't see.

The Python app is a single-threaded console app running on Python 2.7, but
we've seen the same performance issues as far back as Python 2.3 on Windows
XP.  I've finally decided to investigate this because users of the program
are complaining about the time to dump the data.

I would appreciate your thoughts on this issue.

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-win32/attachments/20140214/1b548184/attachment.html>