[py-dev] intermittent bugs in pytest/python on osx lion ?
holger krekel
holger at merlinux.eu
Sun Jun 17 10:03:08 CEST 2012
Thanks for your persistence. I recommend also compiling Python-3.3 (not
released yet - you need to grab the repository i think) and throwing
that in the mix. FYI you could use tox (http://tox.testrun.org) to
automate the various test runs.
Your guess of something corrupting bytecode makes sense.
You might want to use "export PYTHONDONTWRITECODE=1" to inhibit
pyc file writing (this also disables it for pytest's test module
assertion rewriting, so you may return to using the default
assert mode).
good luck,
holger
On Sat, Jun 16, 2012 at 21:39 -0400, Ian Miers wrote:
> Well, I just got the __import__ error but with a different file and module
> using assert=plain and not running the doctest module.
>
> The subsequent runs could not find a certain c extension function. I
> removed all the __pytcache__ and .pyc files with git clean -fxd -e "**.so"
> and it claimed it can't import charm.core despite the fact that it exists
> and can by imported via python3.2 -c "import charm.core" (which is
> imported from the expected location even ). The next test pass worked.
>
> No I am not using any plugins.
>
> Interestingly, I just got the error on a run with the unittest module.
> Though it maybe because something was corrupted since clearing the pyc
> files fixed it.
> I am now completely baffled.
> It looks like something is , for lack of a better term, corrupting the
> running copy of the interpreted code. Do you know anything that could
> possibly do that? Poorly written c extensions perhaps ?
>
> I think I am going to set up some script and just see if I can collect
> errors with pytest and unittest
>
> Ian
>
>
>
>
>
> ython3.2 -m pytest --assert=plain
> =============================================== test session starts
> ================================================
> platform darwin -- Python 3.2.3 -- pytest-2.2.4
> collected 98 items / 1 errors
>
> schemes/test/chamhash_test.py ..
> schemes/test/commit_test.py ..
> schemes/test/dabenc_test.py ..
> schemes/test/encap_bchk05_test.py .
> schemes/test/grpsig_test.py ..
> schemes/test/hibenc_test.py .
> schemes/test/ibenc_test.py .........
> schemes/test/pk_vrf_test.py .
> schemes/test/pkenc_test.py ...........
> schemes/test/pksig_test.py .................
> schemes/test/rsa_alg_test.py ....
> charm/test/toolbox/conversion_test.py ...
> charm/test/toolbox/paddingschemes_test.py s..........
> charm/test/toolbox/secretshare_test.py .
> charm/test/toolbox/symcrypto_test.py ..........
> charm/toolbox/paddingschemes_test.py s..........
> charm/toolbox/symcrypto_test.py ..........
>
> ====================================================== ERRORS
> ======================================================
> ___________________________________ ERROR collecting
> schemes/test/abenc_test.py ____________________________________
> schemes/test/abenc_test.py:1: in <module>
> > from schemes.abenc.abenc_adapt_hybrid import HybridABEnc as HybridABEnc
> schemes/abenc/abenc_adapt_hybrid.py:6: in <module>
> > from charm.toolbox.symcrypto import AuthenticatedCryptoAbstraction
> charm/toolbox/symcrypto.py:9: in <module>
> > import hmac
> E TypeError: __import__() argument 1 must be str without null bytes, not
> str
>
> On Sat, Jun 16, 2012 at 12:30 PM, holger krekel <holger at merlinux.eu> wrote:
>
> > On Sat, Jun 16, 2012 at 11:59 -0400, Ian Miers wrote:
> > > So these issues only happen on OSX and as I said,they come and go and
> > > change,which is why I am skeptical it's our code. Actually if I had to
> > > guess I'd say its something with python3 on OSX Lion, but so far we've
> > only
> > > seen the issues in test runs, not when attempting to manually recreate
> > the
> > > issue.
> > >
> > > Regarding import os.path error, I got it again. Trying to import the
> > module
> > > containing that import after that error caused a bus error. However all
> > > subsequent imports worked fine. The strange thing is that the pyc file
> > > didn't change ( or at least its md5 sum didn't) and the modification date
> > > for all of the pyc files for that package remain unchanged.
> > >
> > >
> > > python3.2 -m pytest --assert=plain seemed to work for a little bit, but
> > > then we started getting the same intermittent changing errors.
> >
> > With --assert=plain pytest should not interfere with anything related
> > to importing. Did you get the strange error with "import os.path"
> > in this configuration? Are you using any plugins (see output of
> > py.test --version)?
> >
> > The below error may in principle be a doctest/pytest bug with
> > python3.2 although it would be strange if it only happened sometimes.
> >
> > Holger
> >
> > > Allmost all runs with assert=reinterp , including after removing all
> > > __pychache__ and recompiling the c extensions produce the INTERNALERROR
> > > below.
> > >
> > >
> > >
> > > Thanks again,
> > > Ian
> > >
> > > python3.2 -m pytest --assert=reinterp
> > > ============================= test session starts
> > > ==============================
> > > platform darwin -- Python 3.2.3 -- pytest-2.2.4
> > > collected 232 items
> > >
> > > schemes/__init__.py .
> > > schemes/chamhash_adm05.py .
> > > schemes/chamhash_rsa_hw09.py .
> > > schemes/encap_bchk05.py .
> > > schemes/pk_fre_ccv11.py .
> > > schemes/pk_vrf.py .
> > > schemes/protocol_cns07.py .
> > > schemes/protocol_schnorr91.py .
> > > schemes/sigma1.py .
> > > schemes/sigma2.py .
> > > schemes/sigma3.py .
> > > schemes/abenc/__init__.py .
> > > schemes/abenc/abenc_adapt_hybrid.py F
> > > schemes/abenc/abenc_bsw07.py .
> > > schemes/abenc/abenc_lsw08.py .
> > > schemes/abenc/abenc_waters09.py .
> > > schemes/abenc/kpabenc_adapt_hybrid.py F
> > > schemes/commit/__init__.py .
> > > schemes/commit/commit_gs08.py .
> > > schemes/commit/commit_pedersen92.py .
> > > schemes/dabenc/__init__.py .
> > > schemes/dabenc/dabe_aw11.py
> > > INTERNALERROR> Traceback (most recent call last):
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > > line 74, in wrap_session
> > > INTERNALERROR> doit(config, session)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > > line 106, in _main
> > > INTERNALERROR> config.hook.pytest_runtestloop(session=session)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 421, in __call__
> > > INTERNALERROR> return self._docall(methods, kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 432, in _docall
> > > INTERNALERROR> res = mc.execute()
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 350, in execute
> > > INTERNALERROR> res = method(**kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > > line 119, in pytest_runtestloop
> > > INTERNALERROR> item.config.hook.pytest_runtest_protocol(item=item,
> > > nextitem=nextitem)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 421, in __call__
> > > INTERNALERROR> return self._docall(methods, kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 432, in _docall
> > > INTERNALERROR> res = mc.execute()
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 350, in execute
> > > INTERNALERROR> res = method(**kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > > line 61, in pytest_runtest_protocol
> > > INTERNALERROR> runtestprotocol(item, nextitem=nextitem)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > > line 68, in runtestprotocol
> > > INTERNALERROR> reports.append(call_and_report(item, "call", log))
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > > line 99, in call_and_report
> > > INTERNALERROR> report = hook.pytest_runtest_makereport(item=item,
> > > call=call)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > > line 141, in call_matching_hooks
> > > INTERNALERROR> return hookmethod.pcall(plugins, **kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 425, in pcall
> > > INTERNALERROR> return self._docall(methods, kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 432, in _docall
> > > INTERNALERROR> res = mc.execute()
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 350, in execute
> > > INTERNALERROR> res = method(**kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/capture.py",
> > > line 173, in pytest_runtest_makereport
> > > INTERNALERROR> rep = __multicall__.execute()
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > > line 350, in execute
> > > INTERNALERROR> res = method(**kwargs)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > > line 187, in pytest_runtest_makereport
> > > INTERNALERROR> longrepr = item.repr_failure(excinfo)
> > > INTERNALERROR> File
> > >
> > "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/doctest.py",
> > > line 44, in repr_failure
> > > INTERNALERROR> lineno = test.lineno + example.lineno + 1
> > > INTERNALERROR> TypeError: unsupported operand type(s) for +: 'NoneType'
> > and
> > > 'int'
> > >
> > > ===================== 2 failed, 19 passed in 3.42 seconds
> > > ======================
> > >
> > >
> > >
> > >
> > > On Sat, Jun 16, 2012 at 4:19 AM, holger krekel <holger at merlinux.eu>
> > wrote:
> > >
> > > > Hi Ian,
> > > >
> > > > On Fri, Jun 15, 2012 at 20:32 -0400, Ian Miers wrote:
> > > > > Hi, I just started using pytest. It's lovely.
> > > > > The TLDR on this is we are getting intermittent non reproducible
> > errors
> > > > > that change and sometimes disappear between test runs like the
> > > > following :
> > > > > > import os.path
> > > > > E TypeError: __import__() argument 1 must be str without null
> > bytes,
> > > > not
> > > > > str
> > > >
> > > > Looks odd. pytest does not override __import__ but it does by default
> > > > use a PEP302 compliant module loader. If you use
> > > >
> > > > py.test --assert=reinterpret # or --assert=plain
> > > >
> > > > do the errors go away? If so that points to a problem in pytest's
> > module
> > > > loader or some interaction problem with python3.2.
> > > >
> > > > If changing the assertion mode still leads to errors then i strongly
> > > > suspect it's other parts of the code you are running. I'd then suggest
> > > > to check if something in your environment modifies __import__
> > > > e.g. by writing a test that checks/prints out __import__?
> > > >
> > > > best,
> > > > holger
> > > >
> > > > > Longer version:
> > > > > We've been getting buggy results out of test runs on OSX Lion with
> > python
> > > > > 3.2.3 and py.test version 2.2.4. Specifically we've been getting
> > what
> > > > > appear to be false-positive test failures that change from run to
> > run
> > > > and
> > > > > cannot be reproduced by running some code ourselves or in the case of
> > > > > doctest manually running python3.2 -m doctest file.
> > > > >
> > > > > Some of these errors will stay around for multiple test runs even
> > after
> > > > > make clean, etc. Some will change from run to run. All of them
> > > > > eventually disappeared temporarily and we got a clean test pass,
> > though
> > > > > how I don't know. Morever, the problems cropped up again. All of
> > this was
> > > > > with no code changes and code known to work on ubuntu and partially
> > > > > manually tested on OSX.
> > > > >
> > > > > Ordinarly I'd say there was something wrong with our code. However,
> > some
> > > > of
> > > > > the errors are vanishingly unlikely. Claims that modules don't exist
> > when
> > > > > they do and are importable via python3.2 -c "from foo.bar.baz import
> > > > narf"
> > > > > and such.
> > > > >
> > > > > The most glaringly, however, is this gem:
> > > > > charm/toolbox/pairinggroup.py:3: in <module>
> > > > > > import os.path
> > > > > E TypeError: __import__() argument 1 must be str without null
> > bytes,
> > > > not
> > > > > str
> > > > >
> > > > > We also got a lovely bug where it appeared __pycache__ was corrupted
> > > > > during test runs. On an initial run, we could import a function
> > from a
> > > > > python c extension. On subsequent runs, it didn't exist. The
> > function was
> > > > > still in the .so file, as shown by nm, however help(module) returned
> > > > > function_name#$@^%#$% Function description. Deleting __pychache__
> > folders
> > > > > resolved it for the next test run but then it came back. It too
> > > > > disappeared after a couple of test runs never to be seen since.
> > > > >
> > > > > Has anyone seen anyhting like this? Are their known issues on OSX
> > with
> > > > > python ? With pytest? Does anyone have any idea how I might get a
> > better
> > > > > idea whats going on?
> > > > >
> > > > > As an addendum, the latest error I am getting is now :
> > > > > > test = [hashFn(struct.pack(">%dsI" % (len(seed)), seed, i)) for
> > i in
> > > > > ran]
> > > > > E UnicodeEncodeError: 'ascii' codec can't encode character '\x9e'
> > in
> > > > > position 0: ordinal not in range(128
> > > > > This actually might be in our code,though again it works on ubuntu
> > and at
> > > > > one point on OSX. Given that its a pattern that points to python or
> > > > pytest
> > > > > doing something to binaries, I'm including it anyway.
> > > > >
> > > > > The project is charm, you can see the code on this branch here via
> > > > > https://github.com/JHUISI/charm/tree/dev. Python was installed via
> > fenc
> > > > ( I
> > > > > think, its not my box). The errors happened both with python3.2 -m
> > pytest
> > > > > and with python3.2 setup.py test. Though it appears more so with
> > the
> > > > > later.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Ian
> > > >
> > > > > _______________________________________________
> > > > > py-dev mailing list
> > > > > py-dev at codespeak.net
> > > > > http://codespeak.net/mailman/listinfo/py-dev
> > > >
> > > >
> >
More information about the Pytest-dev
mailing list