[py-dev] issues with stdout redirection, ctypes and py.test

[holger krekel]

On Sat, Aug 08, 2009 at 16:24 +0200, holger krekel wrote:
> On Sat, Aug 08, 2009 at 14:43 +1000, Gordon Wrigley wrote:
> > Since I have no clear concept of what is supposed to happen when a C
> > function called via ctypes does a printf, I have instead gone through
> > the C library and arranged to have all the printing routed through a
> > function pointer which I now setup as a python callback that does a
> > sys.stdout.write(). Problem solved.
> 
> if you can try to ensure open the file (look for "fdopen(2)" or so) 

probably not the right suggestion - i didn't do C for a long time - 
but fsync(1) after each printf (maybe done in a macro anyway?) could
show if it's a buffering issue.  

best,
holger


> in unbuffered mode that should work.  Would be good to know. 
> I have played on the command line and it appeared to me that
> this buffering mode is very likely the culprit. 
> 
> cheers,
> holger
> 
> 
> > On Fri, Aug 7, 2009 at 8:31 PM, Gordon Wrigley<py-dev at tolomea.com> wrote:
> > > I need to have a think about what you've said and look at producing a
> > > minimal reproduction. However I did forget to mention that the prints
> > > from the python code are fine, it's only the ones from C that are
> > > affected, in the good case the two sets are interleaved in the py.test
> > > capture, in the bad case the python prints are in the py.test capture
> > > and the C prints come out of py.test's stdout.
> > > Now that I think about it some more I'm inclined to wonder from where
> > > (and when) the C library acquires it's stdout handle.
> > >
> > > On Fri, Aug 7, 2009 at 8:02 PM, holger krekel<holger at merlinux.eu> wrote:
> > >> Hi Gordon,
> > >>
> > >> On Fri, Aug 07, 2009 at 18:53 +1000, Gordon Wrigley wrote:
> > >>> My python code talks to a C library via ctypes and it in turn prints
> > >>> debug to stdout.
> > >>> And we test this with py.test, which by itself works just fine.
> > >>
> > >> by this you mean that it looks roughly like this:
> > >>
> > >>    http://paste.pocoo.org/show/132982/
> > >>
> > >> right?
> > >>
> > >>> But if at the bash prompt we then redirect the output of py.test to a
> > >>> file suddenly the C debug starts propagating out of py.test.
> > >>
> > >> hum, redirecting the above example with "py.test x.py >log" works fine.
> > >>
> > >> The following may be interesting to observe (i just did):
> > >>
> > >>    def test_hello():
> > >>        myfile = os.fdopen(1, "w")
> > >>        myfile.write("hello\n")
> > >>        myfile.flush()
> > >>        assert 0
> > >>
> > >> This works fine but if i leave out the flush() the write
> > >> will *not* be captured appropriately.  I guess that it is because
> > >> of the missing buffering ("os.fdopen(1, 'w', 1)" and no flush works)
> > >>
> > >> FYI the pytest_capture plugin resumes capturing for calling
> > >> into test methods, setup/teardown code and collection
> > >> respectively and otherwise suspends capturing.  Even if this
> > >> was modified (i.e. capturing would always happen) i imagine
> > >> that the missing flush/close could associate the actual write
> > >> into a different test.
> > >>
> > >> Maybe your behaviour is still a different issue or even a bug, though.
> > >>
> > >>> We generally run py.test with -v and -x, but removing those didn't
> > >>> change the behavior at all.
> > >>
> > >> that shouldn't change anything related to capturing, indeed.
> > >>
> > >>> I'm at a bit of loss as to where to begin with this one, any suggestions?
> > >>
> > >> maybe my above considerations help a bit and you can maybe
> > >> come up with a breaking example or fix things on your side?
> > >>
> > >> I want to do a 1.0.1 soon with some capturing fixes (related
> > >> to unicode handling, though) and i'd like to include any other
> > >> related improvements or add to the docs.
> > >>
> > >> best,
> > >> holger
> > >>
> > >>> gordonw at gohma:~$ uname -a
> > >>> Linux gohma 2.6.28-14-generic #47-Ubuntu SMP Sat Jul 25 00:28:35 UTC
> > >>> 2009 i686 GNU/Linux
> > >>> gordonw at gohma:~$ python
> > >>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
> > >>> [GCC 4.3.3] on linux2
> > >>> Type "help", "copyright", "credits" or "license" for more information.
> > >>> >>> import py
> > >>> >>> py.version
> > >>> '1.0.0'
> > >>>
> > >>> Regards,
> > >>> Gordon
> > >>> _______________________________________________
> > >>> py-dev mailing list
> > >>> py-dev at codespeak.net
> > >>> http://codespeak.net/mailman/listinfo/py-dev
> > >>>
> > >>
> > >> --
> > >> Metaprogramming, Python, Testing: http://tetamap.wordpress.com
> > >> Python, PyPy, pytest contracting: http://merlinux.eu
> > >>
> > >
> > 
> 
> -- 
> Metaprogramming, Python, Testing: http://tetamap.wordpress.com
> Python, PyPy, pytest contracting: http://merlinux.eu 
> 

-- 
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu