Is print thread safe?

[Cameron Simpson]

On 12Aug2014 09:56, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>Cameron Simpson wrote:
>> On 12Aug2014 02:07, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>>>Is this documented somewhere?
>>
>> In python/2.7.6/reference/simple_stmts.html#index-22, "print" is described
>> in terms of a "write" for each object, and a "write" for the separators.
>> There is no mention of locking.
>
>Ah, thanks!
>
>> On that basis, I would find the interleaving described normal and
>> expected. And certainly not "broken".
>
>I personally didn't describe it as "broken",

Yes, sorry.

>but it is, despite the
>documentation. I just ran a couple of trials where I collected the output
>of sys.stdout while 50 threads blasted "Spam ABCD EFGH" (plus the implicit
>newline) to stdout as fast as possible using print. The result was that out
>of 248165 lines[1], 595 were mangled. Many of the mangled lines were the
>expected simple run-ons:
>
>    Spam ABCD EFGHSpam ABCD EFGH\n\n
>
>which makes sense given the documentation, but there were lots of anomalies.
>
>Mysterious spaces appearing in the strings:
>
>    Spam ABCD EFGH Spam ABCD EFGH\n\n
>    Spam ABCD EFGH Spam ABCD EFGH\n Spam ABCD EFGH\n
>
>occasional collisions mid-string:
>
>    Spam ABSpam ABCD EFGH\nCD EFGH\n
>
>letters disappearing:
>
>    Spam AB\nD EFGH\n
>
>and at least one utterly perplexing (to me) block of ASCII NULs appearing in
>the middle of the output:
>
>    \x00\x00\x00...\x00\x00\n
>
>This is with Python 2.7.2 on Linux.

Sounds like print is not thread safe. Which it does not promise to be. But I 
would normally expect most file.write methods to be thread safe. Naively.

>> Just use a lock! And rebind "print"! Or use the logging system!
>
>Personally, I believe that print ought to do its own locking.

I don't, but I kind of believe "file"s should have thread safe write calls.  
Again, not guarrenteed AFAIR.

>And print is a
>statement, although in this case there's no need to support anything older
>than 2.6, so something like this ought to work:
>
>from __future__ import print_function
>
>_print = print
>_rlock = threading.RLock()
>def print(*args, **kwargs):
>    with _rlock:
>        _print(*args, **kwargs)
>
>Sadly, using print as a function alone isn't enough to fix this problem, but
>in my quick tests, using locking as above does fix it, and with no
>appreciable slowdown.

I would expect file.write to be fast enough that the lock would usually be 
free. With no evidence, just personal expectation. Taking a free lock should be 
almost instant.

>[1] Even the number of lines of output demonstrates a bug. I had fifty
>threads printing 5000 times each, which makes 250000 lines, not 248165.

Sounds like the file internals are unsafe. Ugh.

Cheers,
Cameron Simpson <cs at zip.com.au>

If it ain't broken, keep playing with it.