Is print thread safe?

[Steven D'Aprano]

Cameron Simpson wrote:

> On 12Aug2014 02:07, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
> wrote:
>>INADA Naoki wrote:
>>
>>> On Python 3, print is thread safe.
>>> But Python 2 has broken scenario:
>>
>>Is this documented somewhere?
> 
> In python/2.7.6/reference/simple_stmts.html#index-22, "print" is described
> in terms of a "write" for each object, and a "write" for the separators.
> There is no mention of locking.

Ah, thanks!

> On that basis, I would find the interleaving described normal and
> expected. And certainly not "broken".

I personally didn't describe it as "broken", but it is, despite the
documentation. I just ran a couple of trials where I collected the output
of sys.stdout while 50 threads blasted "Spam ABCD EFGH" (plus the implicit
newline) to stdout as fast as possible using print. The result was that out
of 248165 lines[1], 595 were mangled. Many of the mangled lines were the
expected simple run-ons:

    Spam ABCD EFGHSpam ABCD EFGH\n\n

which makes sense given the documentation, but there were lots of anomalies.

Mysterious spaces appearing in the strings:

    Spam ABCD EFGH Spam ABCD EFGH\n\n
    Spam ABCD EFGH Spam ABCD EFGH\n Spam ABCD EFGH\n

occasional collisions mid-string:

    Spam ABSpam ABCD EFGH\nCD EFGH\n

letters disappearing:

    Spam AB\nD EFGH\n

and at least one utterly perplexing (to me) block of ASCII NULs appearing in
the middle of the output:

    \x00\x00\x00...\x00\x00\n


This is with Python 2.7.2 on Linux. 


> Just use a lock! And rebind "print"! Or use the logging system!

Personally, I believe that print ought to do its own locking. And print is a
statement, although in this case there's no need to support anything older
than 2.6, so something like this ought to work:


from __future__ import print_function

_print = print
_rlock = threading.RLock()
def print(*args, **kwargs):
    with _rlock:
        _print(*args, **kwargs)


Sadly, using print as a function alone isn't enough to fix this problem, but
in my quick tests, using locking as above does fix it, and with no
appreciable slowdown.


[1] Even the number of lines of output demonstrates a bug. I had fifty
threads printing 5000 times each, which makes 250000 lines, not 248165.

-- 
Steven