[issue23324] Nonblocking serial io using Arduino and Ubuntu 14.10 (Python 3.4.2) performance slowdown

[Carol Willing]

Carol Willing added the comment:

Alan, Thanks for reporting your issue. I have spent some time looking at it, and I have triaged it as best as I am able for the other developers and you. I am editing the issue title for clarity.

The issue is related to nonblocking i/o between a 'linux' (Ubuntu 14.10) system and an Arduino and the performance difference of the i/o channel noticed by the user on Python 3.4.2 and 2.7.9.

Third party packages (PyMata and pySerial) as well as threading module of CPython are used by the application that was reported in the original issue. Due to hardware constraints, I have not duplicated the issue.

I have done some research on the sources used by the application.

At the lowest level, i/o between the Arduino and the system is dependent on operating system implementation differences. There are differences between how i/o is implemented by pySerial for different operating systems. Comments in the pySerial source and docs state that nonblocking io and read differ and some functions (such as the nonblocking() method are highlighted as not portable.) There are also comments in the pySerial source code that not all systems support polling properly (presumably at their lower level os code).

Though perhaps not directly related (but at least tangentially),
this blog post explains some nonblocking differences between Python
3 and Python 2 (the last few paragraphs are most relevant).
http://ballingt.com/2014/03/01/nonblocking-stdin-in-python-3.html

---------------------------------------------------
Notes from a reverse walkthrough of the source code
---------------------------------------------------

In PyMata/examples/digital_analog_io/polling_digital_analog_io.py,
    analog = board.analog_read(POTENTIOMETER)

In PyMata/pymata.py, Python's threading module is imported, and
the PyMata class contains the API:
    data_lock = threading.Lock() is set
    analog_read(self, pin) method uses the data_lock and returns
        data value from pymata's command_handler's analog_response_table

Pymata's command_handler PyMata/pymata_command_handler.py has the
following information in its docstring:
    There is no blocking in either communications direction.

    There is blocking when accessing the data tables through the _data_lock

The get_analog_response_table method uses the data_lock.

PyMata's pymata_serial.py uses pySerial's class serial.Serial to
open and initialize a serial port. The following lines are in the
__init__ for PyMataSerial class:

        # without this, running python 3.4 is extremely sluggish
        if sys.platform == 'linux':
            # noinspection PyUnresolvedReferences
            self.arduino.nonblocking()

It should be noted that programs using pySerial's nonblocking()
method are not portable.

The following is the run method for PyMataSerial:

def run(self):
        """
        This method continually runs. If an incoming character is available on the serial port
        it is read and placed on the _command_deque
        @return: Never Returns
        """
        while not self.is_stopped():
            # we can get an OSError: [Errno9] Bad file descriptor when shutting down
            # just ignore it
            try:
                if self.arduino.inWaiting():
                    c = self.arduino.read()
                    self.command_deque.append(ord(c))
            except OSError:
                pass
            except IOError:
                self.stop()
        self.close()

The pySerial inWaiting() method is used to determine whether there
are bytes to be read by the read() method.
The source for pySerial http://svn.code.sf.net/p/pyserial/code/trunk/pyserial/serial/serialutil.py points out variations between Python 2
and 3 for string vs byte reading.

Interestingly, pySerial's Serial class can be assembled in two ways: 1) using pySerial's file-like emulation or 2) using io.RawIOBase.
>From http://svn.code.sf.net/p/pyserial/code/trunk/pyserial/serial/serialposix.py,
# assemble Serial class with the platform specific implementation and the base
# for file-like behavior. for Python 2.6 and newer, that provide the new I/O
# library, derive from io.RawIOBase
try:
    import io
except ImportError:
    # classic version with our own file-like emulation
    class Serial(PosixSerial, FileLike):
        pass
else:
    # io library present
    class Serial(PosixSerial, io.RawIOBase):
        pass

class PosixPollSerial(Serial):
    """\
    Poll based read implementation. Not all systems support poll properly.
    However this one has better handling of errors, such as a device
    disconnecting while it's in use (e.g. USB-serial unplugged).
    """

----------
components: +IO
nosy: +willingc
title: Python 3.4.2 running slow on Ubuntu 14.10 -> Nonblocking serial io using Arduino and Ubuntu 14.10 (Python 3.4.2) performance slowdown
type:  -> performance

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23324>
_______________________________________