[python-win32] optimizing code to read serial port

[Vernon D. Cole]

Tony:

  Don't do it!

Do not attempt to make your data transfer faster while using this
 defective hardware design!  Trust me on this.  I have been doing serial
port data transfer since 300 baud was fast.

The problem with your data transfer speed is not your program optimization.
It is contained in the sentence:  "The serial port on the device we are
dumping data from has a 2-wire interface (no handshaking)."  This means
that the program servicing the serial port must be scheduled at time
intervals less than the time needed to fill your FIFO buffer.  Using
Windows, you cannot guarantee that with any degree of certainty --
particularly not at higher data rates -- because you have no control over
the hardware-level interface drivers.

If you really want higher data rates, you must use a dedicated device to
receive the data. May I humbly suggest that an hour of your programming
time would purchase a single board computer with a Python engine? Look at
http://www.raspberrypi.org/.  Let the SBC collect the data.  Transfer
blocks from it to the Windows environment using a more reliable protocol.

Consider that none of the new Windows boxes I saw on my last shopping trip
had serial ports.  You will soon need to be using another solution anyway.







On Sat, Feb 15, 2014 at 3:01 AM, Tony Cappellini <cappy2112 at gmail.com>wrote:

>
> <python-win32 at python.org>
>
> Regarding an in-house python program we use for dumping diagnostic data
> over a serial port on Windows 7 ...
>
>
> I'm trying to determine if it's possible optimize the python code so that
> dumping the data will occur faster. Currently, it takes ~25 minutes at 9600
> baud to dump 8-15 MB of data. Going to higher baud rates does reduce this
> time, but depending on which PC is running the program, going to higher
> baud rates often results in ill-formed data. The serial port on the device
> we are dumping data from has a 2-wire interface (no handshaking). Sometimes
> 115200 baud works fine, other times the data gets all jumbled. We've
> decided to default to 9600 to avoid getting jumbled data.
>
> Compared to an openly available terminal program (Tera Term Pro) written
> in a compiled language, our Python version is much slower, but easier to
> program. Tera Term's scripting language isn't as easy to script with as
> Python.
>
> At the lowest level of the Python app, the last call in python is
> win32file.ReadFile( comport, bytes_, None ). I believe execution goes to a
> PyWin32 extension module at this point.
>
> By default, 1 is passed in for the number of bytes to read. I thought this
> was very inefficient,
> so I've experimented with values like 2,4,8,16. I didn't want to go higher
> than 16 because the serial port FIFO on many PCs isn't any larger than 16.
>
> While using cProfile, I could see that the number of function calls
> executed was cut in half,
> each time the # of bytes read doubled. Normally this would be a win, but
> the total time to dump the same amount of data over the serial port would
> almost double also.
>
> While this may not be strictly a Python-specific issue, the performance
> issue may be related to the Python-Win32 interface. For example, are there
> more efficient ways to read data from a serial port using Python, than
> calling win32file.ReadFile() ?
>
> I would also expect reading more 8-16 bytes at a time to be more efficient
> than reading 1 byte.
> I do expect that buffering weighs in heavily here, but I have no control
> over what the serial hardware does nor how windows interacts with it.
>
> Having read several of the Python optimization tips, I've already
> eliminated string concatenation, replaced globals variables with locals,
> and eliminated several dot references that are inside of loops.There
> doesn't seem to be any places I can use calls to map() or replace for loops
> with list/gen comps.
>
> There is too much code in the app to post here. I know it's difficult to
> comment on code that you can't see.
>
> The Python app is a single-threaded console app running on Python 2.7, but
> we've seen the same performance issues as far back as Python 2.3 on Windows
> XP.  I've finally decided to investigate this because users of the program
> are complaining about the time to dump the data.
>
> I would appreciate your thoughts on this issue.
>
> thanks
>
> _______________________________________________
> python-win32 mailing list
> python-win32 at python.org
> https://mail.python.org/mailman/listinfo/python-win32
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-win32/attachments/20140215/63010928/attachment.html>