From commits-noreply at bitbucket.org Wed Dec 2 14:37:22 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Wed, 02 Dec 2015 19:37:22 -0000 Subject: [Pytest-commit] commit/tox: 2 new changesets Message-ID: <20151202193722.15207.743@celery-worker-101.ash1.bb-inf.net> 2 new commits in tox: https://bitbucket.org/hpk42/tox/commits/488c8f286041/ Changeset: 488c8f286041 Branch: document-py-env User: pmoore Date: 2015-12-02 15:01:38+00:00 Summary: Document the 'py' environment Affected #: 1 file diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 488c8f286041e4067dcbf86342a5efe1083b7fe3 doc/example/basic.txt --- a/doc/example/basic.txt +++ b/doc/example/basic.txt @@ -30,6 +30,7 @@ Available "default" test environments names are:: + py py24 py25 py26 @@ -43,6 +44,8 @@ pypy pypy3 +The environment ``py`` uses the version of Python used to invoke tox. + However, you can also create your own test environment names, see some of the examples in :doc:`examples <../examples>`. https://bitbucket.org/hpk42/tox/commits/7c1263d2fd42/ Changeset: 7c1263d2fd42 User: hpk42 Date: 2015-12-02 18:25:47+00:00 Summary: Merged in pmoore/tox/document-py-env (pull request #181) Document the 'py' environment Affected #: 1 file diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 7c1263d2fd42791dc909bf40d08d28142059137b doc/example/basic.txt --- a/doc/example/basic.txt +++ b/doc/example/basic.txt @@ -30,6 +30,7 @@ Available "default" test environments names are:: + py py24 py25 py26 @@ -43,6 +44,8 @@ pypy pypy3 +The environment ``py`` uses the version of Python used to invoke tox. + However, you can also create your own test environment names, see some of the examples in :doc:`examples <../examples>`. Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Wed Dec 2 14:37:22 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Wed, 02 Dec 2015 19:37:22 -0000 Subject: [Pytest-commit] commit/tox: hpk42: Merged in pmoore/tox/document-py-env (pull request #181) Message-ID: <20151202193722.46186.72865@celery-worker-102.ash1.bb-inf.net> 1 new commit in tox: https://bitbucket.org/hpk42/tox/commits/7c1263d2fd42/ Changeset: 7c1263d2fd42 User: hpk42 Date: 2015-12-02 18:25:47+00:00 Summary: Merged in pmoore/tox/document-py-env (pull request #181) Document the 'py' environment Affected #: 1 file diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 7c1263d2fd42791dc909bf40d08d28142059137b doc/example/basic.txt --- a/doc/example/basic.txt +++ b/doc/example/basic.txt @@ -30,6 +30,7 @@ Available "default" test environments names are:: + py py24 py25 py26 @@ -43,6 +44,8 @@ pypy pypy3 +The environment ``py`` uses the version of Python used to invoke tox. + However, you can also create your own test environment names, see some of the examples in :doc:`examples <../examples>`. Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From issues-reply at bitbucket.org Fri Dec 4 08:45:54 2015 From: issues-reply at bitbucket.org (Tai Lee) Date: Fri, 04 Dec 2015 13:45:54 -0000 Subject: [Pytest-commit] Issue #292: Negated factor conditions. (hpk42/tox) Message-ID: <20151204134554.19845.40438@celery-worker-101.ash1.bb-inf.net> New issue 292: Negated factor conditions. https://bitbucket.org/hpk42/tox/issues/292/negated-factor-conditions Tai Lee: I'd like to do something like: ``` [testenv] commands = cov: coverage run ... runtests.py cov: coveralls !cov: python runtests.py ``` Currently I have to declare factor conditions for `cov` and `nocov` instead, which means users must always specific either `cov` or `nocov` with `tox -e`. If they forget, `tox` just reports success without running any command: ``` [testenv] commands = cov: coverage run ... runtests.py cov: coveralls nocov: python runtests.py ``` From commits-noreply at bitbucket.org Sun Dec 6 09:42:59 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 06 Dec 2015 14:42:59 -0000 Subject: [Pytest-commit] commit/tox: 3 new changesets Message-ID: <20151206144259.22105.58611@celery-worker-101.ash1.bb-inf.net> 3 new commits in tox: https://bitbucket.org/hpk42/tox/commits/3c2945f51c9b/ Changeset: 3c2945f51c9b Branch: fix-minor-typos-in-config-doc User: cdunklau Date: 2015-12-04 12:34:31+00:00 Summary: Created new branch fix-minor-typos-in-config-doc Affected #: 0 files https://bitbucket.org/hpk42/tox/commits/805d6eec10c6/ Changeset: 805d6eec10c6 Branch: fix-minor-typos-in-config-doc User: cdunklau Date: 2015-12-04 12:35:43+00:00 Summary: Fix two minor typos in config doc Affected #: 1 file diff -r 3c2945f51c9bd19876098b57bb0355303c40a993 -r 805d6eec10c6449ae1086c63a005fab85f7d70a9 doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -441,7 +441,7 @@ {[sectionname]valuename} which you can use to avoid repetition of config values. -You can put default values in one section and reference them in others to avoid repeting the same values:: +You can put default values in one section and reference them in others to avoid repeating the same values:: [base] deps = @@ -455,7 +455,7 @@ {[base]deps} [testenv:mercurial] - dep = + deps = mercurial {[base]deps} https://bitbucket.org/hpk42/tox/commits/522075b4a033/ Changeset: 522075b4a033 User: hpk42 Date: 2015-12-06 14:42:58+00:00 Summary: Merged in cdunklau/tox/fix-minor-typos-in-config-doc (pull request #182) Fix minor typos in config doc Affected #: 1 file diff -r 7c1263d2fd42791dc909bf40d08d28142059137b -r 522075b4a033c76783f7f34cb313272913457ea3 doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -441,7 +441,7 @@ {[sectionname]valuename} which you can use to avoid repetition of config values. -You can put default values in one section and reference them in others to avoid repeting the same values:: +You can put default values in one section and reference them in others to avoid repeating the same values:: [base] deps = @@ -455,7 +455,7 @@ {[base]deps} [testenv:mercurial] - dep = + deps = mercurial {[base]deps} Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Sun Dec 6 09:42:59 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 06 Dec 2015 14:42:59 -0000 Subject: [Pytest-commit] commit/tox: hpk42: Merged in cdunklau/tox/fix-minor-typos-in-config-doc (pull request #182) Message-ID: <20151206144259.17420.35310@celery-worker-104.ash1.bb-inf.net> 1 new commit in tox: https://bitbucket.org/hpk42/tox/commits/522075b4a033/ Changeset: 522075b4a033 User: hpk42 Date: 2015-12-06 14:42:58+00:00 Summary: Merged in cdunklau/tox/fix-minor-typos-in-config-doc (pull request #182) Fix minor typos in config doc Affected #: 1 file diff -r 7c1263d2fd42791dc909bf40d08d28142059137b -r 522075b4a033c76783f7f34cb313272913457ea3 doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -441,7 +441,7 @@ {[sectionname]valuename} which you can use to avoid repetition of config values. -You can put default values in one section and reference them in others to avoid repeting the same values:: +You can put default values in one section and reference them in others to avoid repeating the same values:: [base] deps = @@ -455,7 +455,7 @@ {[base]deps} [testenv:mercurial] - dep = + deps = mercurial {[base]deps} Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Sun Dec 6 10:02:18 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 06 Dec 2015 15:02:18 -0000 Subject: [Pytest-commit] commit/tox: 2 new changesets Message-ID: <20151206150218.21022.85361@celery-worker-101.ash1.bb-inf.net> 2 new commits in tox: https://bitbucket.org/hpk42/tox/commits/96bb07754ed7/ Changeset: 96bb07754ed7 User: pmezard Date: 2015-12-05 13:35:26+00:00 Summary: Fix minor typo in --help message Affected #: 1 file diff -r 7c1263d2fd42791dc909bf40d08d28142059137b -r 96bb07754ed7ac6fb83235978826be6cb9d12930 tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -51,7 +51,7 @@ "(overridable by '-e')") tw.line("TOX_TESTENV_PASSENV: space-separated list of extra " "environment variables to be passed into test command " - "environemnts") + "environments") def show_help_ini(config): https://bitbucket.org/hpk42/tox/commits/0072ceea89b0/ Changeset: 0072ceea89b0 User: hpk42 Date: 2015-12-06 15:02:17+00:00 Summary: Merged in pmezard/tox (pull request #183) Fix minor typo in --help message Affected #: 1 file diff -r 522075b4a033c76783f7f34cb313272913457ea3 -r 0072ceea89b02edb5618f7083ff1458e55dc86fd tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -51,7 +51,7 @@ "(overridable by '-e')") tw.line("TOX_TESTENV_PASSENV: space-separated list of extra " "environment variables to be passed into test command " - "environemnts") + "environments") def show_help_ini(config): Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Sun Dec 6 10:02:18 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 06 Dec 2015 15:02:18 -0000 Subject: [Pytest-commit] commit/tox: hpk42: Merged in pmezard/tox (pull request #183) Message-ID: <20151206150218.75057.69437@celery-worker-102.ash1.bb-inf.net> 1 new commit in tox: https://bitbucket.org/hpk42/tox/commits/0072ceea89b0/ Changeset: 0072ceea89b0 User: hpk42 Date: 2015-12-06 15:02:17+00:00 Summary: Merged in pmezard/tox (pull request #183) Fix minor typo in --help message Affected #: 1 file diff -r 522075b4a033c76783f7f34cb313272913457ea3 -r 0072ceea89b02edb5618f7083ff1458e55dc86fd tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -51,7 +51,7 @@ "(overridable by '-e')") tw.line("TOX_TESTENV_PASSENV: space-separated list of extra " "environment variables to be passed into test command " - "environemnts") + "environments") def show_help_ini(config): Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From issues-reply at bitbucket.org Sun Dec 6 10:53:43 2015 From: issues-reply at bitbucket.org (Luke Plant) Date: Sun, 06 Dec 2015 15:53:43 -0000 Subject: [Pytest-commit] Issue #293: tox startup is very slow due to setup.py and .tox virtualenvs (hpk42/tox) Message-ID: <20151206155343.4901.13476@celery-worker-103.ash1.bb-inf.net> New issue 293: tox startup is very slow due to setup.py and .tox virtualenvs https://bitbucket.org/hpk42/tox/issues/293/tox-startup-is-very-slow-due-to-setuppy Luke Plant: The issue is described here: https://bitbucket.org/pypa/setuptools/issues/450/egg_info-command-is-very-slow-if-there-are It could be considered a performance bug in setuptools, but fixing that is looking quite hard. I'm wondering if fixing it via tox would be better - after all, tox is triggering the really poor performance of setup.py by putting thousands of files in the working directory, and then suffering from it. Solution 1: specify a toxworkdir outside the package directory - http://tox.readthedocs.org/en/latest/config.html#tox-global-settings The problem with this is that it is difficult to find a place which would be appropriate when you consider multiple people working on a project, possibly on different platforms. Also, it needs to be applied in every project that notices the problem. Solution 2: have a different default value for toxworkdir, something like ~/.cache/tox/virtualenvs/{project} , where {project} gets substituted by something appropriate, like a mangled version of the working directory, perhaps combined with a platform name or something. This might also address issue #44 https://bitbucket.org/hpk42/tox/issues/44/cant-share-a-tox-directory-between-os-x This is a significant pain point for me on multiple projects, I'd be willing to work on a solution if at acceptable approach were suggested. From commits-noreply at bitbucket.org Mon Dec 7 06:49:34 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Mon, 07 Dec 2015 11:49:34 -0000 Subject: [Pytest-commit] commit/tox: 4 new changesets Message-ID: <20151207114934.8145.74566@celery-worker-101.ash1.bb-inf.net> 4 new commits in tox: https://bitbucket.org/hpk42/tox/commits/128a5ad7a9a6/ Changeset: 128a5ad7a9a6 Branch: issue285 User: hpk42 Date: 2015-12-07 11:38:55+00:00 Summary: refactor setenv processing into its own class so that we can cleanly implement lazyness and get rid of all kinds of ordering problems. Affected #: 4 files diff -r 0f1846de3af1d21c3d33099c47ef7d961620f362 -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -1,7 +1,10 @@ 2.3.0 (unreleased) ----- -- fix issue285 (WIP) setenv processing with self-references +- fix issue285 make setenv processing fully lazy to fix regressions + of tox-2.2.X and so that we can now have testenv attributes like + "basepython" depend on environment variables that are set in + a setenv section. - allow "#" in commands. This is slightly incompatible with commands sections that used a comment after a "\" line continuation. diff -r 0f1846de3af1d21c3d33099c47ef7d961620f362 -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 tests/test_config.py --- a/tests/test_config.py +++ b/tests/test_config.py @@ -644,7 +644,7 @@ assert envconfig.usedevelop is False assert envconfig.ignore_errors is False assert envconfig.envlogdir == envconfig.envdir.join("log") - assert list(envconfig.setenv.keys()) == ['PYTHONHASHSEED'] + assert list(envconfig.setenv.definitions.keys()) == ['PYTHONHASHSEED'] hashseed = envconfig.setenv['PYTHONHASHSEED'] assert isinstance(hashseed, str) # The following line checks that hashseed parses to an integer. @@ -1516,7 +1516,7 @@ return envconfigs["python"] def _check_hashseed(self, envconfig, expected): - assert envconfig.setenv == {'PYTHONHASHSEED': expected} + assert envconfig.setenv['PYTHONHASHSEED'] == expected def _check_testenv(self, newconfig, expected, args=None, tox_ini=None): envconfig = self._get_envconfig(newconfig, args=args, tox_ini=tox_ini) @@ -1565,7 +1565,7 @@ def test_noset(self, tmpdir, newconfig): args = ['--hashseed', 'noset'] envconfig = self._get_envconfig(newconfig, args=args) - assert envconfig.setenv == {} + assert not envconfig.setenv.definitions def test_noset_with_setenv(self, tmpdir, newconfig): tox_ini = """ @@ -1610,18 +1610,32 @@ class TestSetenv: - def test_getdict_lazy(self, tmpdir, newconfig): + def test_getdict_lazy(self, tmpdir, newconfig, monkeypatch): + monkeypatch.setenv("X", "2") config = newconfig(""" [testenv:X] key0 = key1 = {env:X} - key2 = {env:X:1} + key2 = {env:Y:1} """) envconfig = config.envconfigs["X"] - val = envconfig._reader.getdict_lazy("key0") - assert val == {"key1": "{env:X}", - "key2": "{env:X:1}"} + val = envconfig._reader.getdict_setenv("key0") + assert val["key1"] == "2" + assert val["key2"] == "1" + def test_getdict_lazy_update(self, tmpdir, newconfig, monkeypatch): + monkeypatch.setenv("X", "2") + config = newconfig(""" + [testenv:X] + key0 = + key1 = {env:X} + key2 = {env:Y:1} + """) + envconfig = config.envconfigs["X"] + val = envconfig._reader.getdict_setenv("key0") + d = {} + d.update(val) + assert d == {"key1": "2", "key2": "1"} def test_setenv_uses_os_environ(self, tmpdir, newconfig, monkeypatch): monkeypatch.setenv("X", "1") diff -r 0f1846de3af1d21c3d33099c47ef7d961620f362 -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 tox.ini --- a/tox.ini +++ b/tox.ini @@ -5,7 +5,7 @@ commands=echo {posargs} [testenv] -commands= py.test --timeout=180 {posargs} +commands= py.test --timeout=180 {posargs:tests} deps=pytest>=2.3.5 pytest-timeout diff -r 0f1846de3af1d21c3d33099c47ef7d961620f362 -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -26,6 +26,8 @@ hookimpl = pluggy.HookimplMarker("tox") +_dummy = object() + def get_plugin_manager(): # initialize plugin manager @@ -253,6 +255,47 @@ setattr(namespace, self.dest, 0) +class SetenvDict: + def __init__(self, dict, reader): + self.reader = reader + self.definitions = dict + self.resolved = {} + self._lookupstack = [] + + def __contains__(self, name): + return name in self.definitions + + def get(self, name, default=None): + try: + return self.resolved[name] + except KeyError: + try: + if name in self._lookupstack: + raise KeyError("recursion") + val = self.definitions[name] + except KeyError: + return os.environ.get(name, default) + self._lookupstack.append(name) + try: + self.resolved[name] = res = self.reader._replace(val) + finally: + self._lookupstack.pop() + return res + + def __getitem__(self, name): + x = self.get(name, _dummy) + if x is _dummy: + raise KeyError(name) + return x + + def keys(self): + return self.definitions.keys() + + def __setitem__(self, name, value): + self.definitions[name] = value + self.resolved[name] = value + + @hookimpl def tox_addoption(parser): # formatter_class=argparse.ArgumentDefaultsHelpFormatter) @@ -330,32 +373,15 @@ # add various core venv interpreter attributes def setenv(testenv_config, value): setenv = value - reader = testenv_config._reader - - # we need to resolve environment variable substitution - - replacing = [] # for detecting direct recursion - def setenv_reader(name): - if name in setenv and name not in replacing: - return setenv[name] - return os.environ.get(name) - reader.set_envreader(setenv_reader) - - for name, value in setenv.items(): - replacing.append(name) - setenv[name] = reader._replace(value) - replacing.pop() - config = testenv_config.config if "PYTHONHASHSEED" not in setenv and config.hashseed is not None: setenv['PYTHONHASHSEED'] = config.hashseed return setenv parser.add_testenv_attribute( - name="setenv", type="dict_lazy", postprocess=setenv, + name="setenv", type="dict_setenv", postprocess=setenv, help="list of X=Y lines with environment variable settings") - def basepython_default(testenv_config, value): if value is None: for f in testenv_config.factors: @@ -532,21 +558,33 @@ self.factors = factors self._reader = reader - @property - def envbindir(self): + def get_envbindir(self): """ path to directory where scripts/binaries reside. """ - if sys.platform == "win32": + if (sys.platform == "win32" + and "jython" not in self.basepython + and "pypy" not in self.basepython): return self.envdir.join("Scripts") else: return self.envdir.join("bin") @property + def envbindir(self): + return self.get_envbindir() + + @property def envpython(self): """ path to python executable. """ - return self.envbindir.join(self.basepython) + return self.get_envpython() - # no @property to avoid early calling (see callable(subst[key]) checks) - def envsitepackagesdir(self): + def get_envpython(self): + """ path to python/jython executable. """ + if "jython" in str(self.basepython): + name = "jython" + else: + name = "python" + return self.envbindir.join(name) + + def get_envsitepackagesdir(self): """ return sitepackagesdir of the virtualenv environment. (only available during execution, not parsing) """ @@ -707,10 +745,13 @@ vc = TestenvConfig(config=config, envname=name, factors=factors, reader=reader) reader.addsubstitutions(**subs) reader.addsubstitutions(envname=name) + reader.addsubstitutions(envbindir=vc.get_envbindir, + envsitepackagesdir=vc.get_envsitepackagesdir, + envpython=vc.get_envpython) for env_attr in config._testenv_attr: atype = env_attr.type - if atype in ("bool", "path", "string", "dict", "dict_lazy", "argv", "argvlist"): + if atype in ("bool", "path", "string", "dict", "dict_setenv", "argv", "argvlist"): meth = getattr(reader, "get" + atype) res = meth(env_attr.name, env_attr.default) elif atype == "space-separated-list": @@ -727,9 +768,6 @@ if atype == "path": reader.addsubstitutions(**{env_attr.name: res}) - if env_attr.name == "basepython": - reader.addsubstitutions(envbindir=vc.envbindir, envpython=vc.envpython, - envsitepackagesdir=vc.envsitepackagesdir) return vc def _getenvdata(self, reader): @@ -818,13 +856,12 @@ self.factors = factors self._subs = {} self._subststack = [] - self._envreader = os.environ.get - - def set_envreader(self, envreader): - self._envreader = envreader + self._setenv = None def get_environ_value(self, name): - return self._envreader(name) + if self._setenv is None: + return os.environ.get(name) + return self._setenv.get(name) def addsubstitutions(self, _posargs=None, **kw): self._subs.update(kw) @@ -847,9 +884,11 @@ value = self.getstring(name, None) return self._getdict(value, default=default, sep=sep) - def getdict_lazy(self, name, default=None, sep="\n"): - value = self.getstring(name, None, replace="noenv") - return self._getdict(value, default=default, sep=sep) + def getdict_setenv(self, name, default=None, sep="\n"): + value = self.getstring(name, None, replace=False) + definitions = self._getdict(value, default=default, sep=sep) + self._setenv = SetenvDict(definitions, reader=self) + return self._setenv def _getdict(self, value, default, sep): if value is None: @@ -903,7 +942,7 @@ x = self._apply_factors(x) if replace and x and hasattr(x, 'replace'): - x = self._replace(x, name=name, opt_replace_env=(replace!="noenv")) + x = self._replace(x, name=name) # print "getstring", self.section_name, name, "returned", repr(x) return x @@ -920,14 +959,14 @@ lines = s.strip().splitlines() return '\n'.join(filter(None, map(factor_line, lines))) - def _replace(self, value, name=None, section_name=None, opt_replace_env=True): + def _replace(self, value, name=None, section_name=None): if '{' not in value: return value section_name = section_name if section_name else self.section_name self._subststack.append((section_name, name)) try: - return Replacer(self, opt_replace_env=opt_replace_env).do_replace(value) + return Replacer(self).do_replace(value) finally: assert self._subststack.pop() == (section_name, name) @@ -942,10 +981,8 @@ ''', re.VERBOSE) - - def __init__(self, reader, opt_replace_env): + def __init__(self, reader): self.reader = reader - self.opt_replace_env = opt_replace_env def do_replace(self, x): return self.RE_ITEM_REF.sub(self._replace_match, x) @@ -967,11 +1004,10 @@ "Malformed substitution; no substitution type provided") if sub_type == "env": - if self.opt_replace_env: - return self._replace_env(match) - return "{env:%s}" %(g["substitution_value"]) - if sub_type != None: - raise tox.exception.ConfigError("No support for the %s substitution type" % sub_type) + return self._replace_env(match) + if sub_type is not None: + raise tox.exception.ConfigError( + "No support for the %s substitution type" % sub_type) return self._replace_substitution(match) def _replace_env(self, match): @@ -1007,8 +1043,7 @@ raise ValueError('%s already in %s' % ( (section, item), self.reader._subststack)) x = str(cfg[section][item]) - return self.reader._replace(x, name=item, section_name=section, - opt_replace_env=self.opt_replace_env) + return self.reader._replace(x, name=item, section_name=section) raise tox.exception.ConfigError( "substitution key %r not found" % key) @@ -1023,7 +1058,6 @@ return str(val) - class _ArgvlistReader: @classmethod def getargvlist(cls, reader, value): https://bitbucket.org/hpk42/tox/commits/1bf39138fe84/ Changeset: 1bf39138fe84 Branch: issue285 User: hpk42 Date: 2015-12-07 11:39:34+00:00 Summary: reshuffle tests related to setenv processing and integrate nelfin's cross-section test but mark it as xfailing because i am not sure we need to go to the trouble Affected #: 5 files diff -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -1,10 +1,11 @@ 2.3.0 (unreleased) ----- -- fix issue285 make setenv processing fully lazy to fix regressions +- fix issue285: make setenv processing fully lazy to fix regressions of tox-2.2.X and so that we can now have testenv attributes like "basepython" depend on environment variables that are set in - a setenv section. + a setenv section. Thanks Nelfin for some tests and initial + work on a PR. - allow "#" in commands. This is slightly incompatible with commands sections that used a comment after a "\" line continuation. diff -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 setup.py --- a/setup.py +++ b/setup.py @@ -48,7 +48,7 @@ description='virtualenv-based automation of test activities', long_description=open("README.rst").read(), url='http://tox.testrun.org/', - version='2.3.0.dev1', + version='2.3.0.dev2', license='http://opensource.org/licenses/MIT', platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'], author='holger krekel', diff -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 tests/test_config.py --- a/tests/test_config.py +++ b/tests/test_config.py @@ -735,46 +735,6 @@ if bp == "jython": assert envconfig.envpython == envconfig.envbindir.join(bp) - def test_setenv_overrides(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv = - PYTHONPATH = something - ANOTHER_VAL=else - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'PYTHONPATH' in envconfig.setenv - assert 'ANOTHER_VAL' in envconfig.setenv - assert envconfig.setenv['PYTHONPATH'] == 'something' - assert envconfig.setenv['ANOTHER_VAL'] == 'else' - - def test_setenv_with_envdir_and_basepython(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv = - VAL = {envdir} - basepython = {env:VAL} - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'VAL' in envconfig.setenv - assert envconfig.setenv['VAL'] == envconfig.envdir - assert envconfig.basepython == envconfig.envdir - - def test_setenv_ordering_1(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv= - VAL={envdir} - commands=echo {env:VAL} - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'VAL' in envconfig.setenv - assert envconfig.setenv['VAL'] == envconfig.envdir - assert str(envconfig.envdir) in envconfig.commands[0] - @pytest.mark.parametrize("plat", ["win32", "linux2"]) def test_passenv_as_multiline_list(self, tmpdir, newconfig, monkeypatch, plat): monkeypatch.setattr(sys, "platform", plat) @@ -1672,6 +1632,61 @@ """) assert config.envconfigs["env1"].setenv["X"] == "3" + def test_setenv_overrides(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv = + PYTHONPATH = something + ANOTHER_VAL=else + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'PYTHONPATH' in envconfig.setenv + assert 'ANOTHER_VAL' in envconfig.setenv + assert envconfig.setenv['PYTHONPATH'] == 'something' + assert envconfig.setenv['ANOTHER_VAL'] == 'else' + + def test_setenv_with_envdir_and_basepython(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv = + VAL = {envdir} + basepython = {env:VAL} + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'VAL' in envconfig.setenv + assert envconfig.setenv['VAL'] == envconfig.envdir + assert envconfig.basepython == envconfig.envdir + + def test_setenv_ordering_1(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv= + VAL={envdir} + commands=echo {env:VAL} + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'VAL' in envconfig.setenv + assert envconfig.setenv['VAL'] == envconfig.envdir + assert str(envconfig.envdir) in envconfig.commands[0] + + @pytest.mark.xfail(reason="we don't implement cross-section substitution for setenv") + def test_setenv_cross_section_subst(self, monkeypatch, newconfig): + """test that we can do cross-section substitution with setenv""" + monkeypatch.delenv('TEST', raising=False) + config = newconfig(""" + [section] + x = + NOT_TEST={env:TEST:defaultvalue} + + [testenv] + setenv = {[section]x} + """) + envconfig = config.envconfigs["python"] + assert envconfig.setenv["NOT_TEST"] == "defaultvalue" + class TestIndexServer: def test_indexserver(self, tmpdir, newconfig): diff -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 tox/__init__.py --- a/tox/__init__.py +++ b/tox/__init__.py @@ -1,5 +1,5 @@ # -__version__ = '2.3.0.dev1' +__version__ = '2.3.0.dev2' from .hookspecs import hookspec, hookimpl # noqa diff -r 128a5ad7a9a68825dc837b25e7594e2d92809c40 -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -271,7 +271,7 @@ except KeyError: try: if name in self._lookupstack: - raise KeyError("recursion") + raise KeyError(name) val = self.definitions[name] except KeyError: return os.environ.get(name, default) @@ -1028,7 +1028,8 @@ if envvalue is None: if default is None: raise tox.exception.ConfigError( - "substitution env:%r: unknown environment variable %r" % + "substitution env:%r: unknown environment variable %r " + " or recursive definition." % (envkey, envkey)) return default return envvalue https://bitbucket.org/hpk42/tox/commits/2565db0f6c44/ Changeset: 2565db0f6c44 Branch: issue285 User: hpk42 Date: 2015-12-07 11:41:10+00:00 Summary: merge default Affected #: 6 files diff -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 -r 2565db0f6c44a138226ee990b295c4cc8f53003d CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -13,6 +13,9 @@ - fix issue289: fix build_sphinx target, thanks Barry Warsaw. +- fix issue252: allow environment names with special characters. + Thanks Julien Castets for initial PR and patience. + 2.2.1 ----- diff -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 -r 2565db0f6c44a138226ee990b295c4cc8f53003d doc/config-v2.txt --- a/doc/config-v2.txt +++ b/doc/config-v2.txt @@ -143,7 +143,7 @@ A testenv can define a new ``platform`` setting. If its value is not contained in the string obtained from calling -``platform.platform()`` the environment will be skipped. +``sys.platform`` the environment will be skipped. Expanding the ``envlist`` setting ---------------------------------------------------------- diff -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 -r 2565db0f6c44a138226ee990b295c4cc8f53003d tests/test_z_cmdline.py --- a/tests/test_z_cmdline.py +++ b/tests/test_z_cmdline.py @@ -301,6 +301,23 @@ ]) +def test_venv_special_chars_issue252(cmd, initproj): + initproj("pkg123-0.7", filedefs={ + 'tests': {'test_hello.py': "def test_hello(): pass"}, + 'tox.ini': ''' + [tox] + envlist = special&&1 + [testenv:special&&1] + changedir=tests + ''' + }) + result = cmd.run("tox", ) + assert result.ret == 0 + result.stdout.fnmatch_lines([ + "*installed*pkg123*" + ]) + + def test_unknown_environment(cmd, initproj): initproj("env123-0.7", filedefs={ 'tox.ini': '' diff -r 1bf39138fe845f4a95092fc69dbfee4a7d854176 -r 2565db0f6c44a138226ee990b295c4cc8f53003d tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -533,11 +533,11 @@ action = self.newaction(venv, "envreport") with action: pip = venv.getcommandpath("pip") - # we can't really call internal helpers here easily :/ - # output = venv._pcall([str(pip), "freeze"], - # cwd=self.config.toxinidir, - # action=action) - output = py.process.cmdexec("%s freeze" % (pip)) + output = venv._pcall([str(pip), "freeze"], + cwd=self.config.toxinidir, + action=action) + # the output contains a mime-header, skip it + output = output.split("\n\n")[-1] packages = output.strip().split("\n") action.setactivity("installed", ",".join(packages)) envlog = self.resultlog.get_envlog(venv.name) https://bitbucket.org/hpk42/tox/commits/bcddfde43e80/ Changeset: bcddfde43e80 Branch: issue285 User: hpk42 Date: 2015-12-07 11:49:18+00:00 Summary: add py35 to default test envs and automize versions generated in doc Affected #: 4 files diff -r 2565db0f6c44a138226ee990b295c4cc8f53003d -r bcddfde43e804ea8284cfa2e29a47dcb31a8bc62 doc/Makefile --- a/doc/Makefile +++ b/doc/Makefile @@ -12,6 +12,8 @@ PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . +SITETARGET=$(shell ./_getdoctarget.py) + .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest help: @@ -36,8 +38,10 @@ clean: -rm -rf $(BUILDDIR)/* + install: clean html @rsync -avz $(BUILDDIR)/html/ testrun.org:/www/testrun.org/tox/latest + @rsync -avz $(BUILDDIR)/html/ testrun.org:/www/testrun.org/tox/$(SITETARGET) #dev #latexpdf #@scp $(BUILDDIR)/latex/*.pdf testrun.org:www-tox/latest diff -r 2565db0f6c44a138226ee990b295c4cc8f53003d -r bcddfde43e804ea8284cfa2e29a47dcb31a8bc62 doc/_getdoctarget.py --- /dev/null +++ b/doc/_getdoctarget.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python + +import py + +def get_version_string(): + fn = py.path.local(__file__).join("..", "..", + "tox", "__init__.py") + for line in fn.readlines(): + if "version" in line and not line.strip().startswith('#'): + return eval(line.split("=")[-1]) + +def get_minor_version_string(): + return ".".join(get_version_string().split(".")[:2]) + +if __name__ == "__main__": + print (get_minor_version_string()) diff -r 2565db0f6c44a138226ee990b295c4cc8f53003d -r bcddfde43e804ea8284cfa2e29a47dcb31a8bc62 doc/conf.py --- a/doc/conf.py +++ b/doc/conf.py @@ -13,6 +13,13 @@ import sys, os +# The short X.Y version. +sys.path.insert(0, os.path.dirname(__file__)) +import _getdoctarget + +version = _getdoctarget.get_minor_version_string() +release = _getdoctarget.get_version_string() + # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -47,9 +54,6 @@ # |version| and |release|, also used in various other places throughout the # built documents. # -# The short X.Y version. -release = "2.2" -version = "2.2.0" # The full version, including alpha/beta/rc tags. # The language for content autogenerated by Sphinx. Refer to documentation diff -r 2565db0f6c44a138226ee990b295c4cc8f53003d -r bcddfde43e804ea8284cfa2e29a47dcb31a8bc62 tox.ini --- a/tox.ini +++ b/tox.ini @@ -1,5 +1,5 @@ [tox] -envlist=py27,py26,py34,py33,pypy,flakes,py26-bare +envlist=py27,py26,py34,py33,py35,pypy,flakes,py26-bare [testenv:X] commands=echo {posargs} Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Mon Dec 7 08:07:53 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Mon, 07 Dec 2015 13:07:53 -0000 Subject: [Pytest-commit] commit/tox: 3 new changesets Message-ID: <20151207130753.14724.26129@celery-worker-102.ash1.bb-inf.net> 3 new commits in tox: https://bitbucket.org/hpk42/tox/commits/2b23f8d9e743/ Changeset: 2b23f8d9e743 User: hpk42 Date: 2015-12-07 11:57:20+00:00 Summary: merge branch issue285, bump versions Affected #: 9 files diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,12 @@ 2.3.0 (unreleased) ----- +- fix issue285: make setenv processing fully lazy to fix regressions + of tox-2.2.X and so that we can now have testenv attributes like + "basepython" depend on environment variables that are set in + a setenv section. Thanks Nelfin for some tests and initial + work on a PR. + - allow "#" in commands. This is slightly incompatible with commands sections that used a comment after a "\" line continuation. Thanks David Stanek for the PR. diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 doc/Makefile --- a/doc/Makefile +++ b/doc/Makefile @@ -12,6 +12,8 @@ PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . +SITETARGET=$(shell ./_getdoctarget.py) + .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest help: @@ -36,8 +38,10 @@ clean: -rm -rf $(BUILDDIR)/* + install: clean html @rsync -avz $(BUILDDIR)/html/ testrun.org:/www/testrun.org/tox/latest + @rsync -avz $(BUILDDIR)/html/ testrun.org:/www/testrun.org/tox/$(SITETARGET) #dev #latexpdf #@scp $(BUILDDIR)/latex/*.pdf testrun.org:www-tox/latest diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 doc/_getdoctarget.py --- /dev/null +++ b/doc/_getdoctarget.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python + +import py + +def get_version_string(): + fn = py.path.local(__file__).join("..", "..", + "tox", "__init__.py") + for line in fn.readlines(): + if "version" in line and not line.strip().startswith('#'): + return eval(line.split("=")[-1]) + +def get_minor_version_string(): + return ".".join(get_version_string().split(".")[:2]) + +if __name__ == "__main__": + print (get_minor_version_string()) diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 doc/conf.py --- a/doc/conf.py +++ b/doc/conf.py @@ -13,6 +13,13 @@ import sys, os +# The short X.Y version. +sys.path.insert(0, os.path.dirname(__file__)) +import _getdoctarget + +version = _getdoctarget.get_minor_version_string() +release = _getdoctarget.get_version_string() + # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -47,9 +54,6 @@ # |version| and |release|, also used in various other places throughout the # built documents. # -# The short X.Y version. -release = "2.2" -version = "2.2.0" # The full version, including alpha/beta/rc tags. # The language for content autogenerated by Sphinx. Refer to documentation diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 setup.py --- a/setup.py +++ b/setup.py @@ -48,7 +48,7 @@ description='virtualenv-based automation of test activities', long_description=open("README.rst").read(), url='http://tox.testrun.org/', - version='2.3.0.dev2', + version='2.3.0.dev3', license='http://opensource.org/licenses/MIT', platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'], author='holger krekel', diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 tests/test_config.py --- a/tests/test_config.py +++ b/tests/test_config.py @@ -283,18 +283,9 @@ commands = ls {env:TEST} """) - reader = SectionReader("testenv:py27", config._cfg) - x = reader.getargvlist("commands") - assert x == [ - "ls testvalue".split() - ] - assert x != [ - "ls {env:TEST}".split() - ] - y = reader.getargvlist("setenv") - assert y == [ - "TEST=testvalue".split() - ] + envconfig = config.envconfigs["py27"] + assert envconfig.commands == [["ls", "testvalue"]] + assert envconfig.setenv["TEST"] == "testvalue" class TestIniParser: @@ -653,7 +644,7 @@ assert envconfig.usedevelop is False assert envconfig.ignore_errors is False assert envconfig.envlogdir == envconfig.envdir.join("log") - assert list(envconfig.setenv.keys()) == ['PYTHONHASHSEED'] + assert list(envconfig.setenv.definitions.keys()) == ['PYTHONHASHSEED'] hashseed = envconfig.setenv['PYTHONHASHSEED'] assert isinstance(hashseed, str) # The following line checks that hashseed parses to an integer. @@ -744,46 +735,6 @@ if bp == "jython": assert envconfig.envpython == envconfig.envbindir.join(bp) - def test_setenv_overrides(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv = - PYTHONPATH = something - ANOTHER_VAL=else - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'PYTHONPATH' in envconfig.setenv - assert 'ANOTHER_VAL' in envconfig.setenv - assert envconfig.setenv['PYTHONPATH'] == 'something' - assert envconfig.setenv['ANOTHER_VAL'] == 'else' - - def test_setenv_with_envdir_and_basepython(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv = - VAL = {envdir} - basepython = {env:VAL} - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'VAL' in envconfig.setenv - assert envconfig.setenv['VAL'] == envconfig.envdir - assert envconfig.basepython == envconfig.envdir - - def test_setenv_ordering_1(self, tmpdir, newconfig): - config = newconfig(""" - [testenv] - setenv= - VAL={envdir} - commands=echo {env:VAL} - """) - assert len(config.envconfigs) == 1 - envconfig = config.envconfigs['python'] - assert 'VAL' in envconfig.setenv - assert envconfig.setenv['VAL'] == envconfig.envdir - assert str(envconfig.envdir) in envconfig.commands[0] - @pytest.mark.parametrize("plat", ["win32", "linux2"]) def test_passenv_as_multiline_list(self, tmpdir, newconfig, monkeypatch, plat): monkeypatch.setattr(sys, "platform", plat) @@ -1525,7 +1476,7 @@ return envconfigs["python"] def _check_hashseed(self, envconfig, expected): - assert envconfig.setenv == {'PYTHONHASHSEED': expected} + assert envconfig.setenv['PYTHONHASHSEED'] == expected def _check_testenv(self, newconfig, expected, args=None, tox_ini=None): envconfig = self._get_envconfig(newconfig, args=args, tox_ini=tox_ini) @@ -1574,7 +1525,7 @@ def test_noset(self, tmpdir, newconfig): args = ['--hashseed', 'noset'] envconfig = self._get_envconfig(newconfig, args=args) - assert envconfig.setenv == {} + assert not envconfig.setenv.definitions def test_noset_with_setenv(self, tmpdir, newconfig): tox_ini = """ @@ -1618,6 +1569,125 @@ self._check_hashseed(envconfigs["hash2"], '123456789') +class TestSetenv: + def test_getdict_lazy(self, tmpdir, newconfig, monkeypatch): + monkeypatch.setenv("X", "2") + config = newconfig(""" + [testenv:X] + key0 = + key1 = {env:X} + key2 = {env:Y:1} + """) + envconfig = config.envconfigs["X"] + val = envconfig._reader.getdict_setenv("key0") + assert val["key1"] == "2" + assert val["key2"] == "1" + + def test_getdict_lazy_update(self, tmpdir, newconfig, monkeypatch): + monkeypatch.setenv("X", "2") + config = newconfig(""" + [testenv:X] + key0 = + key1 = {env:X} + key2 = {env:Y:1} + """) + envconfig = config.envconfigs["X"] + val = envconfig._reader.getdict_setenv("key0") + d = {} + d.update(val) + assert d == {"key1": "2", "key2": "1"} + + def test_setenv_uses_os_environ(self, tmpdir, newconfig, monkeypatch): + monkeypatch.setenv("X", "1") + config = newconfig(""" + [testenv:env1] + setenv = + X = {env:X} + """) + assert config.envconfigs["env1"].setenv["X"] == "1" + + def test_setenv_default_os_environ(self, tmpdir, newconfig, monkeypatch): + monkeypatch.delenv("X", raising=False) + config = newconfig(""" + [testenv:env1] + setenv = + X = {env:X:2} + """) + assert config.envconfigs["env1"].setenv["X"] == "2" + + def test_setenv_uses_other_setenv(self, tmpdir, newconfig): + config = newconfig(""" + [testenv:env1] + setenv = + Y = 5 + X = {env:Y} + """) + assert config.envconfigs["env1"].setenv["X"] == "5" + + def test_setenv_recursive_direct(self, tmpdir, newconfig): + config = newconfig(""" + [testenv:env1] + setenv = + X = {env:X:3} + """) + assert config.envconfigs["env1"].setenv["X"] == "3" + + def test_setenv_overrides(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv = + PYTHONPATH = something + ANOTHER_VAL=else + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'PYTHONPATH' in envconfig.setenv + assert 'ANOTHER_VAL' in envconfig.setenv + assert envconfig.setenv['PYTHONPATH'] == 'something' + assert envconfig.setenv['ANOTHER_VAL'] == 'else' + + def test_setenv_with_envdir_and_basepython(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv = + VAL = {envdir} + basepython = {env:VAL} + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'VAL' in envconfig.setenv + assert envconfig.setenv['VAL'] == envconfig.envdir + assert envconfig.basepython == envconfig.envdir + + def test_setenv_ordering_1(self, tmpdir, newconfig): + config = newconfig(""" + [testenv] + setenv= + VAL={envdir} + commands=echo {env:VAL} + """) + assert len(config.envconfigs) == 1 + envconfig = config.envconfigs['python'] + assert 'VAL' in envconfig.setenv + assert envconfig.setenv['VAL'] == envconfig.envdir + assert str(envconfig.envdir) in envconfig.commands[0] + + @pytest.mark.xfail(reason="we don't implement cross-section substitution for setenv") + def test_setenv_cross_section_subst(self, monkeypatch, newconfig): + """test that we can do cross-section substitution with setenv""" + monkeypatch.delenv('TEST', raising=False) + config = newconfig(""" + [section] + x = + NOT_TEST={env:TEST:defaultvalue} + + [testenv] + setenv = {[section]x} + """) + envconfig = config.envconfigs["python"] + assert envconfig.setenv["NOT_TEST"] == "defaultvalue" + + class TestIndexServer: def test_indexserver(self, tmpdir, newconfig): config = newconfig(""" diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 tox.ini --- a/tox.ini +++ b/tox.ini @@ -1,11 +1,11 @@ [tox] -envlist=py27,py26,py34,py33,pypy,flakes,py26-bare +envlist=py27,py26,py34,py33,py35,pypy,flakes,py26-bare [testenv:X] commands=echo {posargs} [testenv] -commands= py.test --timeout=180 {posargs} +commands= py.test --timeout=180 {posargs:tests} deps=pytest>=2.3.5 pytest-timeout diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 tox/__init__.py --- a/tox/__init__.py +++ b/tox/__init__.py @@ -1,5 +1,5 @@ # -__version__ = '2.3.0.dev2' +__version__ = '2.3.0.dev3' from .hookspecs import hookspec, hookimpl # noqa diff -r af8d3ca4a53065d29c08ceb276040cb3bf42843f -r 2b23f8d9e743e5871403995347ef92e690c301a8 tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -26,6 +26,8 @@ hookimpl = pluggy.HookimplMarker("tox") +_dummy = object() + def get_plugin_manager(): # initialize plugin manager @@ -253,6 +255,47 @@ setattr(namespace, self.dest, 0) +class SetenvDict: + def __init__(self, dict, reader): + self.reader = reader + self.definitions = dict + self.resolved = {} + self._lookupstack = [] + + def __contains__(self, name): + return name in self.definitions + + def get(self, name, default=None): + try: + return self.resolved[name] + except KeyError: + try: + if name in self._lookupstack: + raise KeyError(name) + val = self.definitions[name] + except KeyError: + return os.environ.get(name, default) + self._lookupstack.append(name) + try: + self.resolved[name] = res = self.reader._replace(val) + finally: + self._lookupstack.pop() + return res + + def __getitem__(self, name): + x = self.get(name, _dummy) + if x is _dummy: + raise KeyError(name) + return x + + def keys(self): + return self.definitions.keys() + + def __setitem__(self, name, value): + self.definitions[name] = value + self.resolved[name] = value + + @hookimpl def tox_addoption(parser): # formatter_class=argparse.ArgumentDefaultsHelpFormatter) @@ -323,11 +366,22 @@ parser.add_argument("args", nargs="*", help="additional arguments available to command positional substitution") - # add various core venv interpreter attributes parser.add_testenv_attribute( name="envdir", type="path", default="{toxworkdir}/{envname}", help="venv directory") + # add various core venv interpreter attributes + def setenv(testenv_config, value): + setenv = value + config = testenv_config.config + if "PYTHONHASHSEED" not in setenv and config.hashseed is not None: + setenv['PYTHONHASHSEED'] = config.hashseed + return setenv + + parser.add_testenv_attribute( + name="setenv", type="dict_setenv", postprocess=setenv, + help="list of X=Y lines with environment variable settings") + def basepython_default(testenv_config, value): if value is None: for f in testenv_config.factors: @@ -385,17 +439,6 @@ name="recreate", type="bool", default=False, postprocess=recreate, help="always recreate this test environment.") - def setenv(testenv_config, value): - setenv = value - config = testenv_config.config - if "PYTHONHASHSEED" not in setenv and config.hashseed is not None: - setenv['PYTHONHASHSEED'] = config.hashseed - return setenv - - parser.add_testenv_attribute( - name="setenv", type="dict", postprocess=setenv, - help="list of X=Y lines with environment variable settings") - def passenv(testenv_config, value): # Flatten the list to deal with space-separated values. value = list( @@ -515,8 +558,7 @@ self.factors = factors self._reader = reader - @property - def envbindir(self): + def get_envbindir(self): """ path to directory where scripts/binaries reside. """ if (sys.platform == "win32" and "jython" not in self.basepython @@ -526,7 +568,15 @@ return self.envdir.join("bin") @property + def envbindir(self): + return self.get_envbindir() + + @property def envpython(self): + """ path to python executable. """ + return self.get_envpython() + + def get_envpython(self): """ path to python/jython executable. """ if "jython" in str(self.basepython): name = "jython" @@ -534,8 +584,7 @@ name = "python" return self.envbindir.join(name) - # no @property to avoid early calling (see callable(subst[key]) checks) - def envsitepackagesdir(self): + def get_envsitepackagesdir(self): """ return sitepackagesdir of the virtualenv environment. (only available during execution, not parsing) """ @@ -696,10 +745,13 @@ vc = TestenvConfig(config=config, envname=name, factors=factors, reader=reader) reader.addsubstitutions(**subs) reader.addsubstitutions(envname=name) + reader.addsubstitutions(envbindir=vc.get_envbindir, + envsitepackagesdir=vc.get_envsitepackagesdir, + envpython=vc.get_envpython) for env_attr in config._testenv_attr: atype = env_attr.type - if atype in ("bool", "path", "string", "dict", "argv", "argvlist"): + if atype in ("bool", "path", "string", "dict", "dict_setenv", "argv", "argvlist"): meth = getattr(reader, "get" + atype) res = meth(env_attr.name, env_attr.default) elif atype == "space-separated-list": @@ -716,9 +768,6 @@ if atype == "path": reader.addsubstitutions(**{env_attr.name: res}) - if env_attr.name == "basepython": - reader.addsubstitutions(envbindir=vc.envbindir, envpython=vc.envpython, - envsitepackagesdir=vc.envsitepackagesdir) return vc def _getenvdata(self, reader): @@ -799,16 +848,6 @@ is_section_substitution = re.compile("{\[[^{}\s]+\]\S+?}").match -RE_ITEM_REF = re.compile( - r''' - (?[^[:{}]+):)? # optional sub_type for special rules - (?P[^{}]*) # substitution key - [}] - ''', - re.VERBOSE) - - class SectionReader: def __init__(self, section_name, cfgparser, fallbacksections=None, factors=()): self.section_name = section_name @@ -817,6 +856,12 @@ self.factors = factors self._subs = {} self._subststack = [] + self._setenv = None + + def get_environ_value(self, name): + if self._setenv is None: + return os.environ.get(name) + return self._setenv.get(name) def addsubstitutions(self, _posargs=None, **kw): self._subs.update(kw) @@ -836,17 +881,26 @@ return [x.strip() for x in s.split(sep) if x.strip()] def getdict(self, name, default=None, sep="\n"): - s = self.getstring(name, None) - if s is None: + value = self.getstring(name, None) + return self._getdict(value, default=default, sep=sep) + + def getdict_setenv(self, name, default=None, sep="\n"): + value = self.getstring(name, None, replace=False) + definitions = self._getdict(value, default=default, sep=sep) + self._setenv = SetenvDict(definitions, reader=self) + return self._setenv + + def _getdict(self, value, default, sep): + if value is None: return default or {} - value = {} - for line in s.split(sep): + d = {} + for line in value.split(sep): if line.strip(): name, rest = line.split('=', 1) - value[name.strip()] = rest.strip() + d[name.strip()] = rest.strip() - return value + return d def getbool(self, name, default=None): s = self.getstring(name, default) @@ -888,11 +942,7 @@ x = self._apply_factors(x) if replace and x and hasattr(x, 'replace'): - self._subststack.append((self.section_name, name)) - try: - x = self._replace(x) - finally: - assert self._subststack.pop() == (self.section_name, name) + x = self._replace(x, name=name) # print "getstring", self.section_name, name, "returned", repr(x) return x @@ -909,8 +959,58 @@ lines = s.strip().splitlines() return '\n'.join(filter(None, map(factor_line, lines))) + def _replace(self, value, name=None, section_name=None): + if '{' not in value: + return value + + section_name = section_name if section_name else self.section_name + self._subststack.append((section_name, name)) + try: + return Replacer(self).do_replace(value) + finally: + assert self._subststack.pop() == (section_name, name) + + +class Replacer: + RE_ITEM_REF = re.compile( + r''' + (?[^[:{}]+):)? # optional sub_type for special rules + (?P[^{}]*) # substitution key + [}] + ''', + re.VERBOSE) + + def __init__(self, reader): + self.reader = reader + + def do_replace(self, x): + return self.RE_ITEM_REF.sub(self._replace_match, x) + + def _replace_match(self, match): + g = match.groupdict() + + # special case: opts and packages. Leave {opts} and + # {packages} intact, they are replaced manually in + # _venv.VirtualEnv.run_install_command. + sub_value = g['substitution_value'] + if sub_value in ('opts', 'packages'): + return '{%s}' % sub_value + + try: + sub_type = g['sub_type'] + except KeyError: + raise tox.exception.ConfigError( + "Malformed substitution; no substitution type provided") + + if sub_type == "env": + return self._replace_env(match) + if sub_type is not None: + raise tox.exception.ConfigError( + "No support for the %s substitution type" % sub_type) + return self._replace_substitution(match) + def _replace_env(self, match): - env_list = self.getdict('setenv') match_value = match.group('substitution_value') if not match_value: raise tox.exception.ConfigError( @@ -924,75 +1024,40 @@ else: envkey = match_value - if envkey not in os.environ and default is None: - if envkey not in env_list and default is None: + envvalue = self.reader.get_environ_value(envkey) + if envvalue is None: + if default is None: raise tox.exception.ConfigError( - "substitution env:%r: unknown environment variable %r" % + "substitution env:%r: unknown environment variable %r " + " or recursive definition." % (envkey, envkey)) - if envkey in os.environ: - return os.environ.get(envkey, default) - else: - return env_list.get(envkey, default) + return default + return envvalue def _substitute_from_other_section(self, key): if key.startswith("[") and "]" in key: i = key.find("]") section, item = key[1:i], key[i + 1:] - if section in self._cfg and item in self._cfg[section]: - if (section, item) in self._subststack: + cfg = self.reader._cfg + if section in cfg and item in cfg[section]: + if (section, item) in self.reader._subststack: raise ValueError('%s already in %s' % ( - (section, item), self._subststack)) - x = str(self._cfg[section][item]) - self._subststack.append((section, item)) - try: - return self._replace(x) - finally: - self._subststack.pop() + (section, item), self.reader._subststack)) + x = str(cfg[section][item]) + return self.reader._replace(x, name=item, section_name=section) raise tox.exception.ConfigError( "substitution key %r not found" % key) def _replace_substitution(self, match): sub_key = match.group('substitution_value') - val = self._subs.get(sub_key, None) + val = self.reader._subs.get(sub_key, None) if val is None: val = self._substitute_from_other_section(sub_key) if py.builtin.callable(val): val = val() return str(val) - def _replace_match(self, match): - g = match.groupdict() - - # special case: opts and packages. Leave {opts} and - # {packages} intact, they are replaced manually in - # _venv.VirtualEnv.run_install_command. - sub_value = g['substitution_value'] - if sub_value in ('opts', 'packages'): - return '{%s}' % sub_value - - handlers = { - 'env': self._replace_env, - None: self._replace_substitution, - } - try: - sub_type = g['sub_type'] - except KeyError: - raise tox.exception.ConfigError( - "Malformed substitution; no substitution type provided") - - try: - handler = handlers[sub_type] - except KeyError: - raise tox.exception.ConfigError("No support for the %s substitution type" % sub_type) - - return handler(match) - - def _replace(self, x): - if '{' in x: - return RE_ITEM_REF.sub(self._replace_match, x) - return x - class _ArgvlistReader: @classmethod https://bitbucket.org/hpk42/tox/commits/0eda603514fe/ Changeset: 0eda603514fe User: hpk42 Date: 2015-12-07 13:06:51+00:00 Summary: merge Affected #: 3 files diff -r 2b23f8d9e743e5871403995347ef92e690c301a8 -r 0eda603514fecf1bf174801d087ab634c96877d5 doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -441,7 +441,7 @@ {[sectionname]valuename} which you can use to avoid repetition of config values. -You can put default values in one section and reference them in others to avoid repeting the same values:: +You can put default values in one section and reference them in others to avoid repeating the same values:: [base] deps = @@ -455,7 +455,7 @@ {[base]deps} [testenv:mercurial] - dep = + deps = mercurial {[base]deps} diff -r 2b23f8d9e743e5871403995347ef92e690c301a8 -r 0eda603514fecf1bf174801d087ab634c96877d5 doc/example/basic.txt --- a/doc/example/basic.txt +++ b/doc/example/basic.txt @@ -30,6 +30,7 @@ Available "default" test environments names are:: + py py24 py25 py26 @@ -43,6 +44,8 @@ pypy pypy3 +The environment ``py`` uses the version of Python used to invoke tox. + However, you can also create your own test environment names, see some of the examples in :doc:`examples <../examples>`. diff -r 2b23f8d9e743e5871403995347ef92e690c301a8 -r 0eda603514fecf1bf174801d087ab634c96877d5 tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -51,7 +51,7 @@ "(overridable by '-e')") tw.line("TOX_TESTENV_PASSENV: space-separated list of extra " "environment variables to be passed into test command " - "environemnts") + "environments") def show_help_ini(config): https://bitbucket.org/hpk42/tox/commits/42e1c6cb4b79/ Changeset: 42e1c6cb4b79 User: hpk42 Date: 2015-12-07 12:00:35+00:00 Summary: fix flakes problem Affected #: 1 file diff -r 0eda603514fecf1bf174801d087ab634c96877d5 -r 42e1c6cb4b798b78b632915e1246ca5c83767963 tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -534,8 +534,8 @@ with action: pip = venv.getcommandpath("pip") output = venv._pcall([str(pip), "freeze"], - cwd=self.config.toxinidir, - action=action) + cwd=self.config.toxinidir, + action=action) # the output contains a mime-header, skip it output = output.split("\n\n")[-1] packages = output.strip().split("\n") Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Wed Dec 9 07:33:49 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Wed, 09 Dec 2015 12:33:49 -0000 Subject: [Pytest-commit] commit/tox: 4 new changesets Message-ID: <20151209123349.1945.11360@celery-worker-102.ash1.bb-inf.net> 4 new commits in tox: https://bitbucket.org/hpk42/tox/commits/2e58464b95ee/ Changeset: 2e58464b95ee User: hpk42 Date: 2015-12-09 10:53:22+00:00 Summary: internal: push some optional object creation into tests because tox core doesn't need it. Affected #: 4 files diff -r 42e1c6cb4b798b78b632915e1246ca5c83767963 -r 2e58464b95eed4d93ef05e2b396aed5219daa09a CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -16,6 +16,8 @@ - fix issue252: allow environment names with special characters. Thanks Julien Castets for initial PR and patience. +- internal: push some optional object creation into tests because + tox core doesn't need it. 2.2.1 ----- diff -r 42e1c6cb4b798b78b632915e1246ca5c83767963 -r 2e58464b95eed4d93ef05e2b396aed5219daa09a tests/test_venv.py --- a/tests/test_venv.py +++ b/tests/test_venv.py @@ -56,7 +56,8 @@ venv = VirtualEnv(envconfig, session=mocksession) assert venv.path == envconfig.envdir assert not venv.path.check() - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -96,7 +97,8 @@ """) envconfig = config.envconfigs['site'] venv = VirtualEnv(envconfig, session=mocksession) - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -105,7 +107,8 @@ envconfig = config.envconfigs['nosite'] venv = VirtualEnv(envconfig, session=mocksession) - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -122,14 +125,15 @@ {distshare}/dep1-* """) venv = mocksession.getenv("py123") - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) == 1 distshare = venv.session.config.distshare distshare.ensure("dep1-1.0.zip") distshare.ensure("dep1-1.1.zip") - venv.install_deps() + venv.install_deps(action) assert len(l) == 2 args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir @@ -154,11 +158,12 @@ dep2 """) venv = mocksession.getenv("py123") - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) == 1 - venv.install_deps() + venv.install_deps(action) assert len(l) == 2 args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir @@ -183,12 +188,13 @@ :abc2:dep3 """) venv = mocksession.getenv('py123') - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) == 1 l[:] = [] - venv.install_deps() + venv.install_deps(action) # two different index servers, two calls assert len(l) == 3 args = " ".join(l[0].args) @@ -211,12 +217,13 @@ dep1 """) venv = mocksession.getenv('python') - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) == 1 l[:] = [] - venv.install_deps() + venv.install_deps(action) assert len(l) == 1 args = " ".join(l[0].args) assert "--pre " in args @@ -246,10 +253,12 @@ deps=xyz """) venv = mocksession.getenv('python') - venv.update() + + action = mocksession.newaction(venv, "update") + venv.update(action) mocksession.installpkg(venv, pkg) mocksession.report.expect("verbosity0", "*create*") - venv.update() + venv.update(action) mocksession.report.expect("verbosity0", "*recreate*") @@ -263,7 +272,8 @@ finally: tox.config.make_hashseed = original_make_hashseed venv = mocksession.getenv('python') - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) venv.test() mocksession.report.expect("verbosity0", "python runtests: PYTHONHASHSEED='123456789'") @@ -274,7 +284,8 @@ commands = echo foo bar ''') venv = mocksession.getenv('python') - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) venv.test() mocksession.report.expect("verbosity0", "*runtests*commands?0? | echo foo bar") @@ -343,7 +354,8 @@ dep2 """) venv = mocksession.getenv('py123') - venv.create() + action = mocksession.newaction(venv, "getenv") + venv.create(action) l = mocksession._pcalls assert len(l) == 1 args = l[0].args @@ -429,7 +441,8 @@ envconfig = config.envconfigs['python'] venv = VirtualEnv(envconfig, session=mocksession) cconfig = venv._getliveconfig() - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) assert not venv.path_config.check() mocksession.installpkg(venv, pkg) assert venv.path_config.check() @@ -439,36 +452,42 @@ mocksession.report.expect("*", "*create*") # modify config and check that recreation happens mocksession._clearmocks() - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) mocksession.report.expect("*", "*reusing*") mocksession._clearmocks() + action = mocksession.newaction(venv, "update") cconfig.python = py.path.local("balla") cconfig.writeconfig(venv.path_config) - venv.update() + venv.update(action) mocksession.report.expect("verbosity0", "*recreate*") def test_dep_recreation(self, newconfig, mocksession): config = newconfig([], "") envconfig = config.envconfigs['python'] venv = VirtualEnv(envconfig, session=mocksession) - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) cconfig = venv._getliveconfig() cconfig.deps[:] = [("1" * 32, "xyz.zip")] cconfig.writeconfig(venv.path_config) mocksession._clearmocks() - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) mocksession.report.expect("*", "*recreate*") def test_develop_recreation(self, newconfig, mocksession): config = newconfig([], "") envconfig = config.envconfigs['python'] venv = VirtualEnv(envconfig, session=mocksession) - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) cconfig = venv._getliveconfig() cconfig.usedevelop = True cconfig.writeconfig(venv.path_config) mocksession._clearmocks() - venv.update() + action = mocksession.newaction(venv, "update") + venv.update(action) mocksession.report.expect("verbosity0", "*recreate*") @@ -481,21 +500,25 @@ commands=abc """) venv = mocksession.getenv("python") + action = mocksession.newaction(venv, "getenv") monkeypatch.setenv("PATH", "xyz") l = [] monkeypatch.setattr("py.path.local.sysfind", classmethod( lambda *args, **kwargs: l.append(kwargs) or 0 / 0)) - py.test.raises(ZeroDivisionError, "venv._install(list('123'))") + with pytest.raises(ZeroDivisionError): + venv._install(list('123'), action=action) assert l.pop()["paths"] == [venv.envconfig.envbindir] - py.test.raises(ZeroDivisionError, "venv.test()") + with pytest.raises(ZeroDivisionError): + venv.test(action) assert l.pop()["paths"] == [venv.envconfig.envbindir] - py.test.raises(ZeroDivisionError, "venv.run_install_command(['qwe'])") + with pytest.raises(ZeroDivisionError): + venv.run_install_command(['qwe'], action=action) assert l.pop()["paths"] == [venv.envconfig.envbindir] monkeypatch.setenv("PIP_RESPECT_VIRTUALENV", "1") monkeypatch.setenv("PIP_REQUIRE_VIRTUALENV", "1") monkeypatch.setenv("__PYVENV_LAUNCHER__", "1") - py.test.raises(ZeroDivisionError, "venv.run_install_command(['qwe'])") + py.test.raises(ZeroDivisionError, "venv.run_install_command(['qwe'], action=action)") assert 'PIP_RESPECT_VIRTUALENV' not in os.environ assert 'PIP_REQUIRE_VIRTUALENV' not in os.environ assert '__PYVENV_LAUNCHER__' not in os.environ diff -r 42e1c6cb4b798b78b632915e1246ca5c83767963 -r 2e58464b95eed4d93ef05e2b396aed5219daa09a tests/test_z_cmdline.py --- a/tests/test_z_cmdline.py +++ b/tests/test_z_cmdline.py @@ -34,7 +34,8 @@ report = session.report report.expect("using") venv = session.getvenv("mypython") - venv.update() + action = session.newaction(venv, "update") + venv.update(action) report.expect("logpopen") diff -r 42e1c6cb4b798b78b632915e1246ca5c83767963 -r 2e58464b95eed4d93ef05e2b396aed5219daa09a tox/venv.py --- a/tox/venv.py +++ b/tox/venv.py @@ -114,12 +114,10 @@ def _ispython3(self): return "python3" in str(self.envconfig.basepython) - def update(self, action=None): + def update(self, action): """ return status string for updating actual venv to match configuration. if status string is empty, all is ok. """ - if action is None: - action = self.session.newaction(self, "update") rconfig = CreationConfig.readconfig(self.path_config) if not self.envconfig.recreate and rconfig and \ rconfig.matches(self._getliveconfig()): @@ -172,12 +170,9 @@ def matching_platform(self): return re.match(self.envconfig.platform, sys.platform) - def create(self, action=None): + def create(self, action): # if self.getcommandpath("activate").dirpath().check(): # return - if action is None: - action = self.session.newaction(self, "create") - config_interpreter = self.getsupportedinterpreter() args = [sys.executable, '-m', 'virtualenv'] if self.envconfig.sitepackages: @@ -239,9 +234,7 @@ extraopts = ['-U', '--no-deps'] self._install([sdistpath], extraopts=extraopts, action=action) - def install_deps(self, action=None): - if action is None: - action = self.session.newaction(self, "install_deps") + def install_deps(self, action): deps = self._getresolvedeps() if deps: depinfo = ", ".join(map(str, deps)) @@ -259,7 +252,7 @@ l.append("--pre") return l - def run_install_command(self, packages, options=(), action=None): + def run_install_command(self, packages, action, options=()): argv = self.envconfig.install_command[:] # use pip-script on win32 to avoid the executable locking i = argv.index('{packages}') https://bitbucket.org/hpk42/tox/commits/0823f359a05b/ Changeset: 0823f359a05b User: hpk42 Date: 2015-12-09 11:05:51+00:00 Summary: some small refactorings Affected #: 2 files diff -r 2e58464b95eed4d93ef05e2b396aed5219daa09a -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -211,7 +211,7 @@ if sys.platform == "win32": ext = os.path.splitext(str(newargs[0]))[1].lower() if ext == '.py' and self.venv: - newargs = [str(self.venv.getcommandpath())] + newargs + newargs = [str(self.envconfig.envpython)] + newargs return newargs diff -r 2e58464b95eed4d93ef05e2b396aed5219daa09a -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 tox/venv.py --- a/tox/venv.py +++ b/tox/venv.py @@ -56,19 +56,29 @@ def __init__(self, envconfig=None, session=None): self.envconfig = envconfig self.session = session - self.path = envconfig.envdir - self.path_config = self.path.join(".tox-config1") + + @property + def path(self): + """ Path to environment base dir. """ + return self.envconfig.envdir + + @property + def path_config(self): + return self.path.join(".tox-config1") @property def name(self): + """ test environment name. """ return self.envconfig.envname def __repr__(self): return "" % (self.path) - def getcommandpath(self, name=None, venv=True, cwd=None): - if name is None: - return self.envconfig.envpython + def getcommandpath(self, name, venv=True, cwd=None): + """ return absolute path (str or localpath) for specified + command name. If venv is True we will check if the + command is coming from the venv or is whitelisted to come + from external. """ name = str(name) if os.path.isabs(name): return name https://bitbucket.org/hpk42/tox/commits/ba0ff401e037/ Changeset: ba0ff401e037 User: hpk42 Date: 2015-12-09 12:13:32+00:00 Summary: implement new experimental hooks for venv creation Affected #: 8 files diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -16,6 +16,13 @@ - fix issue252: allow environment names with special characters. Thanks Julien Castets for initial PR and patience. +- introduce experimental tox_testenv_create(venv, action) and + tox_testenv_install_deps(venv, action) hooks to allow + plugins to do additional work on creation or installing + deps. These hooks are experimental mainly because of + the involved "venv" object whose current public API is not + fully guranteed. + - internal: push some optional object creation into tests because tox core doesn't need it. diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df doc/plugins.txt --- a/doc/plugins.txt +++ b/doc/plugins.txt @@ -80,3 +80,9 @@ .. autoclass:: tox.config.TestenvConfig() :members: + +.. autoclass:: tox.venv.VirtualEnv() + :members: + +.. autoclass:: tox.session.Session() + :members: diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tests/test_venv.py --- a/tests/test_venv.py +++ b/tests/test_venv.py @@ -57,7 +57,7 @@ assert venv.path == envconfig.envdir assert not venv.path.check() action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -98,7 +98,7 @@ envconfig = config.envconfigs['site'] venv = VirtualEnv(envconfig, session=mocksession) action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -108,7 +108,7 @@ envconfig = config.envconfigs['nosite'] venv = VirtualEnv(envconfig, session=mocksession) action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) >= 1 args = l[0].args @@ -126,14 +126,14 @@ """) venv = mocksession.getenv("py123") action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) == 1 distshare = venv.session.config.distshare distshare.ensure("dep1-1.0.zip") distshare.ensure("dep1-1.1.zip") - venv.install_deps(action) + tox_testenv_install_deps(action=action, venv=venv) assert len(l) == 2 args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir @@ -159,11 +159,11 @@ """) venv = mocksession.getenv("py123") action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) == 1 - venv.install_deps(action) + tox_testenv_install_deps(action=action, venv=venv) assert len(l) == 2 args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir @@ -189,12 +189,12 @@ """) venv = mocksession.getenv('py123') action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) == 1 l[:] = [] - venv.install_deps(action) + tox_testenv_install_deps(action=action, venv=venv) # two different index servers, two calls assert len(l) == 3 args = " ".join(l[0].args) @@ -218,12 +218,12 @@ """) venv = mocksession.getenv('python') action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) == 1 l[:] = [] - venv.install_deps(action) + tox_testenv_install_deps(action=action, venv=venv) assert len(l) == 1 args = " ".join(l[0].args) assert "--pre " in args @@ -355,7 +355,7 @@ """) venv = mocksession.getenv('py123') action = mocksession.newaction(venv, "getenv") - venv.create(action) + tox_testenv_create(action=action, venv=venv) l = mocksession._pcalls assert len(l) == 1 args = l[0].args @@ -649,3 +649,26 @@ assert venv.status == "ignored failed command" mocksession.report.expect("warning", "*command failed but result from " "testenv is ignored*") + + +def test_tox_testenv_create(newmocksession): + l = [] + + class Plugin: + @hookimpl + def tox_testenv_create(self, action, venv): + l.append(1) + + @hookimpl + def tox_testenv_install_deps(self, action, venv): + l.append(2) + + mocksession = newmocksession([], """ + [testenv] + commands=testenv_fail + ignore_outcome=True + """, plugins=[Plugin()]) + + venv = mocksession.getenv('python') + venv.update(action=mocksession.newaction(venv, "getenv")) + assert l == [1, 2] diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tox/_pytestplugin.py --- a/tox/_pytestplugin.py +++ b/tox/_pytestplugin.py @@ -31,7 +31,7 @@ @pytest.fixture def newconfig(request, tmpdir): - def newconfig(args, source=None): + def newconfig(args, source=None, plugins=()): if source is None: source = args args = [] @@ -40,7 +40,7 @@ p.write(s) old = tmpdir.chdir() try: - return parseconfig(args) + return parseconfig(args, plugins=plugins) finally: old.chdir() return newconfig @@ -168,9 +168,8 @@ mocksession = request.getfuncargvalue("mocksession") newconfig = request.getfuncargvalue("newconfig") - def newmocksession(args, source): - config = newconfig(args, source) - mocksession.config = config + def newmocksession(args, source, plugins=()): + mocksession.config = newconfig(args, source, plugins=plugins) return mocksession return newmocksession diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -29,13 +29,17 @@ _dummy = object() -def get_plugin_manager(): +def get_plugin_manager(plugins=()): # initialize plugin manager + import tox.venv pm = pluggy.PluginManager("tox") pm.add_hookspecs(hookspecs) pm.register(tox.config) pm.register(tox.interpreters) + pm.register(tox.venv) pm.load_setuptools_entrypoints("tox") + for plugin in plugins: + pm.register(plugin) pm.check_pending() return pm @@ -186,7 +190,7 @@ return value -def parseconfig(args=None): +def parseconfig(args=None, plugins=()): """ :param list[str] args: Optional list of arguments. :type pkg: str @@ -194,7 +198,7 @@ :raise SystemExit: toxinit file is not found """ - pm = get_plugin_manager() + pm = get_plugin_manager(plugins) if args is None: args = sys.argv[1:] diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tox/hookspecs.py --- a/tox/hookspecs.py +++ b/tox/hookspecs.py @@ -30,3 +30,14 @@ per-testenv configuration, notably the ``.envname`` and ``.basepython`` setting. """ + + + at hookspec +def tox_testenv_create(venv, action): + """ [experimental] perform creation action for this venv. + """ + + + at hookspec +def tox_testenv_install_deps(venv, action): + """ [experimental] perform install dependencies action for this venv. """ diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tox/session.py --- a/tox/session.py +++ b/tox/session.py @@ -313,6 +313,8 @@ class Session: + """ (unstable API). the session object that ties + together configuration, reporting, venv creation, testing. """ def __init__(self, config, popen=subprocess.Popen, Report=Reporter): self.config = config diff -r 0823f359a05bde1d8f12f02e98b6b3a26d83c2b2 -r ba0ff401e0374506e64ca0edaf663eea226955df tox/venv.py --- a/tox/venv.py +++ b/tox/venv.py @@ -5,7 +5,7 @@ import codecs import py import tox -from .config import DepConfig +from .config import DepConfig, hookimpl class CreationConfig: @@ -58,6 +58,10 @@ self.session = session @property + def hook(self): + return self.envconfig.config.pluginmanager.hook + + @property def path(self): """ Path to environment base dir. """ return self.envconfig.envdir @@ -76,7 +80,8 @@ def getcommandpath(self, name, venv=True, cwd=None): """ return absolute path (str or localpath) for specified - command name. If venv is True we will check if the + command name. If it's a localpath we will rewrite it as + as a relative path. If venv is True we will check if the command is coming from the venv or is whitelisted to come from external. """ name = str(name) @@ -138,13 +143,14 @@ else: action.setactivity("recreate", self.envconfig.envdir) try: - self.create(action) + self.hook.tox_testenv_create(action=action, venv=self) + self.just_created = True except tox.exception.UnsupportedInterpreter: return sys.exc_info()[1] except tox.exception.InterpreterNotFound: return sys.exc_info()[1] try: - self.install_deps(action) + self.hook.tox_testenv_install_deps(action=action, venv=self) except tox.exception.InvocationError: v = sys.exc_info()[1] return "could not install deps %s; v = %r" % ( @@ -180,28 +186,6 @@ def matching_platform(self): return re.match(self.envconfig.platform, sys.platform) - def create(self, action): - # if self.getcommandpath("activate").dirpath().check(): - # return - config_interpreter = self.getsupportedinterpreter() - args = [sys.executable, '-m', 'virtualenv'] - if self.envconfig.sitepackages: - args.append('--system-site-packages') - # add interpreter explicitly, to prevent using - # default (virtualenv.ini) - args.extend(['--python', str(config_interpreter)]) - # if sys.platform == "win32": - # f, path, _ = py.std.imp.find_module("virtualenv") - # f.close() - # args[:1] = [str(config_interpreter), str(path)] - # else: - self.session.make_emptydir(self.path) - basepath = self.path.dirpath() - basepath.ensure(dir=1) - args.append(self.path.basename) - self._pcall(args, venv=False, action=action, cwd=basepath) - self.just_created = True - def finish(self): self._getliveconfig().writeconfig(self.path_config) @@ -244,13 +228,6 @@ extraopts = ['-U', '--no-deps'] self._install([sdistpath], extraopts=extraopts, action=action) - def install_deps(self, action): - deps = self._getresolvedeps() - if deps: - depinfo = ", ".join(map(str, deps)) - action.setactivity("installdeps", "%s" % depinfo) - self._install(deps, action=action) - def _installopts(self, indexserver): l = [] if indexserver: @@ -390,3 +367,35 @@ if not path.check(file=1): return "0" * 32 return path.computehash() + + + at hookimpl +def tox_testenv_create(venv, action): + # if self.getcommandpath("activate").dirpath().check(): + # return + config_interpreter = venv.getsupportedinterpreter() + args = [sys.executable, '-m', 'virtualenv'] + if venv.envconfig.sitepackages: + args.append('--system-site-packages') + # add interpreter explicitly, to prevent using + # default (virtualenv.ini) + args.extend(['--python', str(config_interpreter)]) + # if sys.platform == "win32": + # f, path, _ = py.std.imp.find_module("virtualenv") + # f.close() + # args[:1] = [str(config_interpreter), str(path)] + # else: + venv.session.make_emptydir(venv.path) + basepath = venv.path.dirpath() + basepath.ensure(dir=1) + args.append(venv.path.basename) + venv._pcall(args, venv=False, action=action, cwd=basepath) + + + at hookimpl +def tox_testenv_install_deps(venv, action): + deps = venv._getresolvedeps() + if deps: + depinfo = ", ".join(map(str, deps)) + action.setactivity("installdeps", "%s" % depinfo) + venv._install(deps, action=action) https://bitbucket.org/hpk42/tox/commits/4a8b2093838f/ Changeset: 4a8b2093838f User: hpk42 Date: 2015-12-09 12:24:10+00:00 Summary: deprecate indexserver options Affected #: 2 files diff -r ba0ff401e0374506e64ca0edaf663eea226955df -r 4a8b2093838fdaeea640a49be9b3df87190b4fdc CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,10 @@ 2.3.0 (unreleased) ----- +- DEPRECATE use of "indexservers" in tox.ini. It complicates + the internal code and it is recommended to rather use the + devpi system for managing indexes for pip. + - fix issue285: make setenv processing fully lazy to fix regressions of tox-2.2.X and so that we can now have testenv attributes like "basepython" depend on environment variables that are set in @@ -20,8 +24,8 @@ tox_testenv_install_deps(venv, action) hooks to allow plugins to do additional work on creation or installing deps. These hooks are experimental mainly because of - the involved "venv" object whose current public API is not - fully guranteed. + the involved "venv" and session objects whose current public + API is not fully guranteed. - internal: push some optional object creation into tests because tox core doesn't need it. diff -r ba0ff401e0374506e64ca0edaf663eea226955df -r 4a8b2093838fdaeea640a49be9b3df87190b4fdc doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -93,9 +93,6 @@ .. versionadded:: 1.6 - **WARNING**: This setting is **EXPERIMENTAL** so use with care - and be ready to adapt your tox.ini's with post-1.6 tox releases. - the ``install_command`` setting is used for installing packages into the virtual environment; both the package under test and any defined dependencies. Must contain the substitution key @@ -166,7 +163,8 @@ package installation. Each line defines a dependency, which will be passed to the installer command for processing. Each line specifies a file, a URL or a package name. You can additionally specify - an :confval:`indexserver` to use for installing this dependency. + an :confval:`indexserver` to use for installing this dependency + but this functionality is deprecated since tox-2.3. All derived dependencies (deps required by the dep) will then be retrieved from the specified indexserver:: @@ -259,8 +257,9 @@ .. versionadded:: 0.9 - Multi-line ``name = URL`` definitions of python package servers. - Dependencies can specify using a specified index server through the + (DEPRECATED, will be removed in a future version) Multi-line ``name = + URL`` definitions of python package servers. Dependencies can + specify using a specified index server through the ``:indexservername:depname`` pattern. The ``default`` indexserver definition determines where unscoped dependencies and the sdist install installs from. Example:: Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Fri Dec 11 05:19:22 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Fri, 11 Dec 2015 10:19:22 -0000 Subject: [Pytest-commit] commit/tox: hpk42: Added tag 2.3.0 for changeset 4a8b2093838f Message-ID: <20151211101922.66751.87768@celery-worker-102.ash1.bb-inf.net> 1 new commit in tox: https://bitbucket.org/hpk42/tox/commits/86e1a4f0772a/ Changeset: 86e1a4f0772a User: hpk42 Date: 2015-12-11 10:19:16+00:00 Summary: Added tag 2.3.0 for changeset 4a8b2093838f Affected #: 1 file diff -r 4a8b2093838fdaeea640a49be9b3df87190b4fdc -r 86e1a4f0772af335a022e32160b033bf2bbd9aba .hgtags --- a/.hgtags +++ b/.hgtags @@ -32,3 +32,4 @@ a674d923b2d8987db4425262c0df9ca259546020 2.2.0 49d4884aba78ec17864c5b1a444fb5305fd0d93e 2.2.0 bc147d78aa43853a0a7c825da1bb2b68ca034a2c 2.2.1 +4a8b2093838fdaeea640a49be9b3df87190b4fdc 2.3.0 Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From issues-reply at bitbucket.org Fri Dec 11 11:17:00 2015 From: issues-reply at bitbucket.org (Jordan Pittier) Date: Fri, 11 Dec 2015 16:17:00 -0000 Subject: [Pytest-commit] Issue #294: Tox 2.3 regression (hpk42/tox) Message-ID: <20151211161700.316.14009@celery-worker-103.ash1.bb-inf.net> New issue 294: Tox 2.3 regression https://bitbucket.org/hpk42/tox/issues/294/tox-23-regression Jordan Pittier: Hi, At least two openstack project (Tempest and Nova) are hit by a regression introduced with Tox 2.3 released a couple of hours ago on pypi. Here is a relevant stack trace: ``` #!shell (toxtox)jordan at jordan-XPS13:/opt/stack/tempest (master %=)$ tox --version 2.3.0 imported from /opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/__init__.pyc (toxtox)jordan at jordan-XPS13:/opt/stack/tempest (master %=)$ tox -epep8 Traceback (most recent call last): File "/opt/stack/tempest/toxtox/bin/tox", line 11, in sys.exit(cmdline()) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/session.py", line 38, in main config = prepare(args) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/session.py", line 26, in prepare config = parseconfig(args) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/config.py", line 229, in parseconfig parseini(config, inipath) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/config.py", line 729, in __init__ self.make_envconfig(name, section, reader._subs, config) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/config.py", line 760, in make_envconfig res = meth(env_attr.name, env_attr.default) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/config.py", line 893, in getdict_setenv definitions = self._getdict(value, default=default, sep=sep) File "/opt/stack/tempest/toxtox/local/lib/python2.7/site-packages/tox/config.py", line 904, in _getdict name, rest = line.split('=', 1) ValueError: need more than 1 value to unpack ``` I think Tox failed on these lines: ``` #!shell [testenv:all-plugin] sitepackages = True # 'all' includes slow tests setenv = {[tempestenv]setenv} OS_TEST_TIMEOUT=1200 deps = {[tempestenv]deps} ``` Full tox.ini here: https://github.com/openstack/tempest/blob/02ef4c2956318b09cc47506af2361ab3d8907a10/tox.ini From issues-reply at bitbucket.org Sun Dec 13 21:34:54 2015 From: issues-reply at bitbucket.org (Timothy Allen) Date: Mon, 14 Dec 2015 02:34:54 -0000 Subject: [Pytest-commit] Issue #295: Support loading environments from external files (hpk42/tox) Message-ID: <20151214023454.8356.63023@celery-worker-104.ash1.bb-inf.net> New issue 295: Support loading environments from external files https://bitbucket.org/hpk42/tox/issues/295/support-loading-environments-from-external Timothy Allen: Currently tox allows you to define a bunch of test environments in a single `tox.ini` file, which is a great way to help multiple contributors to a single project stay in sync. However, I have many small projects with differing release schedules, and I want them all to be consistent with each other. Copy/pasting `tox.ini` changes from project to project is error-prone and tedious, so I want a way to put environment configuration in a central location and make each project include it by reference, possibly overriding some settings. For example, let's say I always want to be warned about code with a McCabe complexity over 10. I could make a separate file containing: [testenv] commands= flake8 --max-complexity 10 ...then all my projects' `tox.ini` files could say something like: [tox] inherit_from=http://somehost/generic-tox-settings.ini ...and when tox saw that setting, it could download the file, parse it, and then update it with the contents of the local `tox.ini`. As an alternative to downloading a file every time tox is run, perhaps `inherit_from` could be an absolute or relative path (relative to the directory containing tox.ini) so that the extra settings could be distributed via SVN externals or git submodules. From commits-noreply at bitbucket.org Mon Dec 14 06:05:39 2015 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Mon, 14 Dec 2015 11:05:39 -0000 Subject: [Pytest-commit] commit/tox: 2 new changesets Message-ID: <20151214110539.34942.87551@celery-worker-101.ash1.bb-inf.net> 2 new commits in tox: https://bitbucket.org/hpk42/tox/commits/cfabce992c4c/ Changeset: cfabce992c4c User: hpk42 Date: 2015-12-14 11:01:15+00:00 Summary: fix issue294: re-allow cross-section substitution for setenv setting. Affected #: 5 files diff -r 86e1a4f0772af335a022e32160b033bf2bbd9aba -r cfabce992c4c2c05297633c935eceef00656efd9 CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -1,4 +1,9 @@ -2.3.0 (unreleased) +2.3.1 +----- + +- fix issue294: re-allow cross-section substitution for setenv. + +2.3.0 ----- - DEPRECATE use of "indexservers" in tox.ini. It complicates diff -r 86e1a4f0772af335a022e32160b033bf2bbd9aba -r cfabce992c4c2c05297633c935eceef00656efd9 setup.py --- a/setup.py +++ b/setup.py @@ -48,7 +48,7 @@ description='virtualenv-based automation of test activities', long_description=open("README.rst").read(), url='http://tox.testrun.org/', - version='2.3.0.dev3', + version='2.3.1', license='http://opensource.org/licenses/MIT', platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'], author='holger krekel', diff -r 86e1a4f0772af335a022e32160b033bf2bbd9aba -r cfabce992c4c2c05297633c935eceef00656efd9 tests/test_config.py --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1672,8 +1672,7 @@ assert envconfig.setenv['VAL'] == envconfig.envdir assert str(envconfig.envdir) in envconfig.commands[0] - @pytest.mark.xfail(reason="we don't implement cross-section substitution for setenv") - def test_setenv_cross_section_subst(self, monkeypatch, newconfig): + def test_setenv_cross_section_subst_issue294(self, monkeypatch, newconfig): """test that we can do cross-section substitution with setenv""" monkeypatch.delenv('TEST', raising=False) config = newconfig(""" @@ -1687,6 +1686,36 @@ envconfig = config.envconfigs["python"] assert envconfig.setenv["NOT_TEST"] == "defaultvalue" + def test_setenv_cross_section_subst_twice(self, monkeypatch, newconfig): + """test that we can do cross-section substitution with setenv""" + monkeypatch.delenv('TEST', raising=False) + config = newconfig(""" + [section] + x = NOT_TEST={env:TEST:defaultvalue} + [section1] + y = {[section]x} + + [testenv] + setenv = {[section1]y} + """) + envconfig = config.envconfigs["python"] + assert envconfig.setenv["NOT_TEST"] == "defaultvalue" + + def test_setenv_cross_section_mixed(self, monkeypatch, newconfig): + """test that we can do cross-section substitution with setenv""" + monkeypatch.delenv('TEST', raising=False) + config = newconfig(""" + [section] + x = NOT_TEST={env:TEST:defaultvalue} + + [testenv] + setenv = {[section]x} + y = 7 + """) + envconfig = config.envconfigs["python"] + assert envconfig.setenv["NOT_TEST"] == "defaultvalue" + assert envconfig.setenv["y"] == "7" + class TestIndexServer: def test_indexserver(self, tmpdir, newconfig): diff -r 86e1a4f0772af335a022e32160b033bf2bbd9aba -r cfabce992c4c2c05297633c935eceef00656efd9 tox/__init__.py --- a/tox/__init__.py +++ b/tox/__init__.py @@ -1,5 +1,5 @@ # -__version__ = '2.3.0.dev3' +__version__ = '2.3.1' from .hookspecs import hookspec, hookimpl # noqa diff -r 86e1a4f0772af335a022e32160b033bf2bbd9aba -r cfabce992c4c2c05297633c935eceef00656efd9 tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -889,7 +889,7 @@ return self._getdict(value, default=default, sep=sep) def getdict_setenv(self, name, default=None, sep="\n"): - value = self.getstring(name, None, replace=False) + value = self.getstring(name, None, replace=True, crossonly=True) definitions = self._getdict(value, default=default, sep=sep) self._setenv = SetenvDict(definitions, reader=self) return self._setenv @@ -931,7 +931,7 @@ def getargv(self, name, default=""): return self.getargvlist(name, default)[0] - def getstring(self, name, default=None, replace=True): + def getstring(self, name, default=None, replace=True, crossonly=False): x = None for s in [self.section_name] + self.fallbacksections: try: @@ -946,7 +946,7 @@ x = self._apply_factors(x) if replace and x and hasattr(x, 'replace'): - x = self._replace(x, name=name) + x = self._replace(x, name=name, crossonly=crossonly) # print "getstring", self.section_name, name, "returned", repr(x) return x @@ -963,14 +963,14 @@ lines = s.strip().splitlines() return '\n'.join(filter(None, map(factor_line, lines))) - def _replace(self, value, name=None, section_name=None): + def _replace(self, value, name=None, section_name=None, crossonly=False): if '{' not in value: return value section_name = section_name if section_name else self.section_name self._subststack.append((section_name, name)) try: - return Replacer(self).do_replace(value) + return Replacer(self, crossonly=crossonly).do_replace(value) finally: assert self._subststack.pop() == (section_name, name) @@ -982,22 +982,28 @@ (?:(?P[^[:{}]+):)? # optional sub_type for special rules (?P[^{}]*) # substitution key [}] - ''', - re.VERBOSE) + ''', re.VERBOSE) - def __init__(self, reader): + def __init__(self, reader, crossonly=False): self.reader = reader + self.crossonly = crossonly def do_replace(self, x): return self.RE_ITEM_REF.sub(self._replace_match, x) def _replace_match(self, match): g = match.groupdict() + sub_value = g['substitution_value'] + if self.crossonly: + if sub_value.startswith("["): + return self._substitute_from_other_section(sub_value) + # in crossonly we return all other hits verbatim + start, end = match.span() + return match.string[start:end] # special case: opts and packages. Leave {opts} and # {packages} intact, they are replaced manually in # _venv.VirtualEnv.run_install_command. - sub_value = g['substitution_value'] if sub_value in ('opts', 'packages'): return '{%s}' % sub_value @@ -1048,7 +1054,8 @@ raise ValueError('%s already in %s' % ( (section, item), self.reader._subststack)) x = str(cfg[section][item]) - return self.reader._replace(x, name=item, section_name=section) + return self.reader._replace(x, name=item, section_name=section, + crossonly=self.crossonly) raise tox.exception.ConfigError( "substitution key %r not found" % key) https://bitbucket.org/hpk42/tox/commits/2d5d0e7584cc/ Changeset: 2d5d0e7584cc User: hpk42 Date: 2015-12-14 11:05:34+00:00 Summary: Added tag 2.3.1 for changeset cfabce992c4c Affected #: 1 file diff -r cfabce992c4c2c05297633c935eceef00656efd9 -r 2d5d0e7584cc4cc35cc7e0519ce9610dd52b7a62 .hgtags --- a/.hgtags +++ b/.hgtags @@ -33,3 +33,4 @@ 49d4884aba78ec17864c5b1a444fb5305fd0d93e 2.2.0 bc147d78aa43853a0a7c825da1bb2b68ca034a2c 2.2.1 4a8b2093838fdaeea640a49be9b3df87190b4fdc 2.3.0 +cfabce992c4c2c05297633c935eceef00656efd9 2.3.1 Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From issues-reply at bitbucket.org Wed Dec 16 10:51:00 2015 From: issues-reply at bitbucket.org (Maxim Kovgan) Date: Wed, 16 Dec 2015 15:51:00 -0000 Subject: [Pytest-commit] Issue #296: support tracking changes in requirements.txt without full rebuild (hpk42/tox) Message-ID: <20151216155100.15509.68877@celery-worker-102.ash1.bb-inf.net> New issue 296: support tracking changes in requirements.txt without full rebuild https://bitbucket.org/hpk42/tox/issues/296/support-tracking-changes-in Maxim Kovgan: h4. Use case: during the development, you often switch between new and old requirements. This currently forces the user to run `tox -r` Sometimes it's not serious, but for automated tests in CI, using internet - it can be several minutes overhead of ~5 or even more minutes. This becomes problematic for short builds: test suite that runs 1 minutes is waiting for tox ~5 minutes. h4. scenarios I'd like to handle are: # addition of new packages # removal of currently installed packages # upgrade of existing package(s) # downgrade of existing package(s) h3. Scope only handle 4 situations, nothing else h3. implementation: Let's use `difflib` module's functionality: it already has `ndiff()` seemingly very useful for this purpose. based on that diff we should generate 2 lists: # packages_to_remove # packages_to_install Testing should be done first on the function that properly detects what needs to be removed and what needs to be installed. # Initially - just remove anything that is changed, and install it from scratch # later - we might want to optimize: ## packages to remove - to contain packages that are supposed to be removed or downgraded ## packages to install - should contain packages that are supposed to be installed or upgraded with `-I` option (i.e. older will be uninstalled) A very easy to implement code can take existing, current pip freeze output and compare it to requirements.txt to be used. As a result, if there are diffs (lines with "-" and lines with "+"), those with "-" can be uninstalled, and those with "+" - installed. This would automatically resolve problems of full virtualenv recreation of From issues-reply at bitbucket.org Wed Dec 16 11:01:01 2015 From: issues-reply at bitbucket.org (Peter Bittner) Date: Wed, 16 Dec 2015 16:01:01 -0000 Subject: [Pytest-commit] Issue #297: Allow `setup.cfg` as an alternative configuration file (hpk42/tox) Message-ID: <20151216160101.6706.19295@celery-worker-101.ash1.bb-inf.net> New issue 297: Allow `setup.cfg` as an alternative configuration file https://bitbucket.org/hpk42/tox/issues/297/allow-setupcfg-as-an-alternative Peter Bittner: (This issue comes from a comment in issue #185.) I'd love to see the tox configuration living in `setup.cfg` too in general, as an alternative. It would allow a more elegant Python package configuration for developers: You could put configuration of all your tools into a single file. The ini-style allows for separate sections. I've described some considerations in a [comment on issue 13](https://bitbucket.org/hpk42/tox/issues/13/tox-should-reuse-tests_require#comment-22015917). A few projects already look into `setup.cfg`. [flake8](http://flake8.readthedocs.org/en/latest/config.html), [behave](https://github.com/behave/behave/blob/master/docs/behave.rst#configuration-files), [py.test](http://pytest.org/latest/customize.html?highlight=setup.cfg#initialization-determining-rootdir-and-inifile), among others. Pylint is a prominent exception (its maintainer considers this idea harmful; [declined PR](https://bitbucket.org/logilab/pylint/pull-requests/276/allow-setupcfg-to-be-project-config-file/diff); [discussion](https://github.com/PyCQA/pylint/issues/617)). From issues-reply at bitbucket.org Tue Dec 22 03:51:36 2015 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Tue, 22 Dec 2015 08:51:36 -0000 Subject: [Pytest-commit] Issue #298: support having tox.ini in a configuration folder (hpk42/tox) Message-ID: <20151222085136.13262.32194@celery-worker-101.ash1.bb-inf.net> New issue 298: support having tox.ini in a configuration folder https://bitbucket.org/hpk42/tox/issues/298/support-having-toxini-in-a-configuration Ronny Pfannschmidt: this is a alternative to #297, the idea is to have project configuration files per tool inside a subfilder of the project From issues-reply at bitbucket.org Sat Dec 26 15:41:57 2015 From: issues-reply at bitbucket.org (Ned Batchelder) Date: Sat, 26 Dec 2015 20:41:57 -0000 Subject: [Pytest-commit] Issue #299: 2.3.1 fails with: AttributeError: 'Action' object has no attribute 'envconfig' (hpk42/tox) Message-ID: <20151226204157.32223.76314@celery-worker-101.ash1.bb-inf.net> New issue 299: 2.3.1 fails with: AttributeError: 'Action' object has no attribute 'envconfig' https://bitbucket.org/hpk42/tox/issues/299/231-fails-with-attributeerror-action Ned Batchelder: Windows 7, running tox for coverage.py's repo, using the "coverage-4.0.3" tag: ``` C:\Users\IEUser\coverage>pip install tox==2.2.1 Collecting tox==2.2.1 Downloading tox-2.2.1-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): virtualenv>=1.11.2 in c:\python27\lib\site-packages (from tox==2.2.1) Requirement already satisfied (use --upgrade to upgrade): py>=1.4.17 in c:\python27\lib\site-packages (from tox==2.2.1) Requirement already satisfied (use --upgrade to upgrade): pluggy<0.4.0,>=0.3.0 in c:\python27\lib\site-packages (from tox==2.2.1) Installing collected packages: tox Successfully installed tox-2.2.1 C:\Users\IEUser\coverage>tox -e doc doc recreate: C:\Users\IEUser\coverage\.tox\doc doc installdeps: -rdoc/requirements.pip doc develop-inst: C:\Users\IEUser\coverage doc installed: alabaster==0.7.7,Babel==2.1.1,colorama==0.3.5,-e hg+file:///N:/coverage/trunk at 1e245736f39c8968d7b437cfa47bdc7ccf7f8 ef7#egg=coverage-dev,docutils==0.12,Jinja2==2.8,MarkupSafe==0.23,pyenchant==1.6.6,Pygments==2.0.2,pytz==2015.7,six==1.10.0,snowbal lstemmer==1.2.1,Sphinx==1.3.3,sphinx-rtd-theme==0.1.9,sphinxcontrib-spelling==2.1.2,wheel==0.24.0 doc runtests: PYTHONHASHSEED='427' doc runtests: commands[0] | sphinx-build -aEnq doc doc/_build/html doc runtests: commands[1] | sphinx-build -aEnQW doc doc/_build/html doc runtests: commands[2] | rst2html.py --strict CHANGES.rst doc/_build/trash doc runtests: commands[3] | rst2html.py --strict README.rst doc/_build/trash ___________________________________________________________ summary ____________________________________________________________ doc: commands succeeded congratulations :) C:\Users\IEUser\coverage>pip install tox==2.3.1 Collecting tox==2.3.1 Using cached tox-2.3.1-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): virtualenv>=1.11.2 in c:\python27\lib\site-packages (from tox==2.3.1) Requirement already satisfied (use --upgrade to upgrade): py>=1.4.17 in c:\python27\lib\site-packages (from tox==2.3.1) Requirement already satisfied (use --upgrade to upgrade): pluggy<0.4.0,>=0.3.0 in c:\python27\lib\site-packages (from tox==2.3.1) Installing collected packages: tox Found existing installation: tox 2.2.1 Uninstalling tox-2.2.1: Successfully uninstalled tox-2.2.1 Successfully installed tox-2.3.1 C:\Users\IEUser\coverage>tox -e doc doc recreate: C:\Users\IEUser\coverage\.tox\doc doc installdeps: -rdoc/requirements.pip doc develop-inst: C:\Users\IEUser\coverage doc installed: alabaster==0.7.7,Babel==2.1.1,colorama==0.3.5,-e hg+file:///N:/coverage/trunk at 1e245736f39c8968d7b437cfa47bdc7ccf7f8 ef7#egg=coverage-dev,docutils==0.12,Jinja2==2.8,MarkupSafe==0.23,pyenchant==1.6.6,Pygments==2.0.2,pytz==2015.7,six==1.10.0,snowbal lstemmer==1.2.1,Sphinx==1.3.3,sphinx-rtd-theme==0.1.9,sphinxcontrib-spelling==2.1.2,wheel==0.24.0 doc runtests: PYTHONHASHSEED='17' doc runtests: commands[0] | sphinx-build -aEnq doc doc/_build/html doc runtests: commands[1] | sphinx-build -aEnQW doc doc/_build/html doc runtests: commands[2] | rst2html.py --strict CHANGES.rst doc/_build/trash Traceback (most recent call last): File "c:\python27\lib\runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "c:\python27\lib\runpy.py", line 72, in _run_code exec code in run_globals File "C:\Python27\Scripts\tox.exe\__main__.py", line 9, in File "c:\python27\lib\site-packages\tox\session.py", line 39, in main retcode = Session(config).runcommand() File "c:\python27\lib\site-packages\tox\session.py", line 375, in runcommand return self.subcommand_test() File "c:\python27\lib\site-packages\tox\session.py", line 548, in subcommand_test self.runtestenv(venv) File "c:\python27\lib\site-packages\tox\session.py", line 556, in runtestenv venv.test(redirect=redirect) File "c:\python27\lib\site-packages\tox\venv.py", line 332, in test ignore_ret=ignore_ret, testcommand=True) File "c:\python27\lib\site-packages\tox\venv.py", line 362, in _pcall redirect=redirect, ignore_ret=ignore_ret) File "c:\python27\lib\site-packages\tox\session.py", line 136, in popen stdout=stdout, stderr=STDOUT) File "c:\python27\lib\site-packages\tox\session.py", line 219, in _popen args = self._rewriteargs(cwd, args) File "c:\python27\lib\site-packages\tox\session.py", line 214, in _rewriteargs newargs = [str(self.envconfig.envpython)] + newargs AttributeError: 'Action' object has no attribute 'envconfig' C:\Users\IEUser\coverage> ```