Using pytest, sometimes does not capture stderr

[Cameron Simpson]

On 05Apr2021 13:56, David <bouncingcats at gmail.com> wrote:
>On Mon, 5 Apr 2021 at 13:44, Cameron Simpson <cs at cskk.id.au> wrote:
>> On 05Apr2021 13:28, David <bouncingcats at gmail.com> wrote:
>> >Can anyone explain why the module_2.py test fails?
>> >Is it because stderr during module import is not the same as during test?
>> >Is it something to do with mutable defaults?
>> >How to investigate this?
>> >And how can I get the test to pass without changing module_2?
>
>> The code in module_2.py runs at different times.
>
>> When it is imported, sys.stderr is the OS-provided stderr. That
>> reference is kept in MSG_DESTINATION.
>
>> Then your test code runs, and changes sys.stderr. It then runs msg(),
>> which writes to the _original_ sys.stderr as preserved by
>> MSG_DESTINATION. Thus not captured.
>
>> By contrast, module_1.py looks up sys.stderr inside msg(), and finds the
>> new one the code harness put at sys.stderr. So it writes to the thing
>> that captures stuff.
>
>Thanks for confirming my suspicions so quickly. What you wrote
>makes sense, but there are two points that still puzzle me.
>1) The final line of the pytest failure output seems to shows that
>    pytest did capture (or is at least aware of) the stderr message
>    from module_2.

Yes. Unsure what's going on there. It could be timing. Suppose this 
happens:

- pytest pushes a capturing stderr onto sys.stderr
- pytest loads your module, which imports module_1 and module_2
- the test runner pushes a separate stderr capturer for the test?
- module_1 finds the per-test sys.stderr value
- module_2 finds pytest's outermost capturer (present when it was 
  imported), and doesn't look up sys.stderr at test time, instead using 
  the outer capturer

>2) My actual code that I would like to test does look like module_2.
>    Is there any way to test it with pytest?

I'd be inclined to give msg() an optional file= parameter:

    def msg(*args, file=None):
        if file is None:
            file = MSG_DESTINATION
        print(*args, file=file)

Then your test code can go:

    msg("a", "message", file=sys.stderr)

which looks up sys.stderr as it is inside the test itself, and passes it 
to msg(). Thus captured.

If you truly need to test msg() _without_ the file= parameter, you could 
monkey patch module_2:

    old_MSG_DESTINATION = module_2.MSG_DESTINATION
    module_2.MSG_DESTINATION = sys.stderr
    # now the module_2 module has an updated reference for sys.stderr
    ...
    msg("a", "message")
    ...
    module_2.MSG_DESTINATION = old_MSG_DESTINATION
    # normality restored

Cheers,
Cameron Simpson <cs at cskk.id.au>