From dirkjan at ochtman.nl Sun Jul 6 20:02:13 2014 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sun, 6 Jul 2014 20:02:13 +0200 Subject: [Python-porting] Doctests in single-source port In-Reply-To: References: Message-ID: On Tue, Apr 1, 2014 at 3:59 AM, Lennart Regebro wrote: > There are a few here: http://python3porting.com/problems.html#running-doctests > > But several of them actually reduce doctests value as documentation, > so it's not brilliant. Thanks for the pointer, that turned out to be useful. I just wrote a blog post on another solution that helped me solve my problems: http://dirkjan.ochtman.nl/writing/2014/07/06/single-source-python-23-doctests.html Cheers, Dirkjan From dirkjan at ochtman.nl Thu Jul 17 09:03:39 2014 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 17 Jul 2014 09:03:39 +0200 Subject: [Python-porting] Installing different files on Python 2/3 Message-ID: After porting CouchDB-Python and trying out an install on a system that has both, I ran into a minor issue. I searched around a bit, but didn't find any promising solution. With CouchDB-Python, I chose a single-source solution, with a util module that imports a util2 or util3 module based on the Python version at run time. One of the problems solved in these files is the exec incompatibility. util2 has this version of the function: def pyexec(code, gns, lns): exec code in gns, lns Whereas util3 has this version: pyexec = exec However, on installation, all modules are byte-compiled, and both of these cause errors at the syntax level in the version of Python they're not intended for. I'm using a MANIFEST.in to list out the SOURCES.txt; is there any nice way of manipulating that data dependent on the python version? Cheers, Dirkjan From guido at python.org Thu Jul 17 17:45:19 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2014 08:45:19 -0700 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: The Py2 version can just call exec(code, gns, lns). It is Py2 syntax too! On Jul 17, 2014 12:06 AM, "Dirkjan Ochtman" wrote: > After porting CouchDB-Python and trying out an install on a system > that has both, I ran into a minor issue. I searched around a bit, but > didn't find any promising solution. > > With CouchDB-Python, I chose a single-source solution, with a util > module that imports a util2 or util3 module based on the Python > version at run time. One of the problems solved in these files is the > exec incompatibility. util2 has this version of the function: > > def pyexec(code, gns, lns): > exec code in gns, lns > > Whereas util3 has this version: > > pyexec = exec > > However, on installation, all modules are byte-compiled, and both of > these cause errors at the syntax level in the version of Python > they're not intended for. I'm using a MANIFEST.in to list out the > SOURCES.txt; is there any nice way of manipulating that data dependent > on the python version? > > Cheers, > > Dirkjan > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Thu Jul 17 19:25:50 2014 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 17 Jul 2014 19:25:50 +0200 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: On Thu, Jul 17, 2014 at 5:45 PM, Guido van Rossum wrote: > The Py2 version can just call exec(code, gns, lns). It is Py2 syntax too! That sounds like an awesome solution, but it doesn't quite solve the problem: Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/djc/src/couchdb-python/couchdb/tests/__main__.py", line 11, in from couchdb.tests import client, couch_tests, design, couchhttp, \ File "couchdb/tests/view.py", line 12, in from couchdb import view File "couchdb/view.py", line 56 exec(string, {'log': _log}, globals_) SyntaxError: unqualified exec is not allowed in function 'add_fun' it is a nested function The code is here: https://github.com/djc/couchdb-python/blob/afdbe601/couchdb/view.py#L56 I read up a little on unqualified and qualified exec, but I couldn't really figure out why that error is triggering in this case. Cheers, Dirkjan From guido at python.org Thu Jul 17 19:41:23 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2014 10:41:23 -0700 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: The code you link to doesn't match the traceback. I recommend that you experiment a bit and Google around for the error message. On Thu, Jul 17, 2014 at 10:25 AM, Dirkjan Ochtman wrote: > On Thu, Jul 17, 2014 at 5:45 PM, Guido van Rossum > wrote: > > The Py2 version can just call exec(code, gns, lns). It is Py2 syntax too! > > That sounds like an awesome solution, but it doesn't quite solve the > problem: > > Traceback (most recent call last): > File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main > "__main__", fname, loader, pkg_name) > File "/usr/lib/python2.7/runpy.py", line 72, in _run_code > exec code in run_globals > File "/home/djc/src/couchdb-python/couchdb/tests/__main__.py", line > 11, in > from couchdb.tests import client, couch_tests, design, couchhttp, \ > File "couchdb/tests/view.py", line 12, in > from couchdb import view > File "couchdb/view.py", line 56 > exec(string, {'log': _log}, globals_) > SyntaxError: unqualified exec is not allowed in function 'add_fun' it > is a nested function > > The code is here: > > https://github.com/djc/couchdb-python/blob/afdbe601/couchdb/view.py#L56 > > I read up a little on unqualified and qualified exec, but I couldn't > really figure out why that error is triggering in this case. > > Cheers, > > Dirkjan > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Tue Jul 22 15:02:14 2014 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 22 Jul 2014 15:02:14 +0200 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: On Thu, Jul 17, 2014 at 7:41 PM, Guido van Rossum wrote: > The code you link to doesn't match the traceback. I recommend that you > experiment a bit and Google around for the error message. The only thing that doesn't match is that I replaced util.pyexec() by just exec(), as you suggested. And all the Google hits about unqualified exec wouldn't seem to apply to this code, as I'm supplying both globals and locals (though the empty locals dict is confusingly named globals_ here). Cheers, Dirkjan From drnlmuller at gmail.com Tue Jul 22 16:43:26 2014 From: drnlmuller at gmail.com (Neil Muller) Date: Tue, 22 Jul 2014 16:43:26 +0200 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: On 22 July 2014 15:02, Dirkjan Ochtman wrote: > > The only thing that doesn't match is that I replaced util.pyexec() by > just exec(), as you suggested. And all the Google hits about > unqualified exec wouldn't seem to apply to this code, as I'm supplying > both globals and locals (though the empty locals dict is confusingly > named globals_ here). This looks a lot like http://bugs.python.org/issue21591 , which suggests that exec(s, {}, {}) and "exec in {}, {}" aren't identical when nested functions are involved. -- Neil Muller drnlmuller at gmail.com From guido at python.org Tue Jul 22 20:27:58 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 22 Jul 2014 11:27:58 -0700 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: Good sleuthing, Neil! Dirkjan: it seems that you need some kind of minimal helper function that wraps the exec() call to avoid this bug. Essentially you should just keep your pyexec() helper but change its body to exec(...). On Tue, Jul 22, 2014 at 7:43 AM, Neil Muller wrote: > On 22 July 2014 15:02, Dirkjan Ochtman wrote: > > > > The only thing that doesn't match is that I replaced util.pyexec() by > > just exec(), as you suggested. And all the Google hits about > > unqualified exec wouldn't seem to apply to this code, as I'm supplying > > both globals and locals (though the empty locals dict is confusingly > > named globals_ here). > > This looks a lot like http://bugs.python.org/issue21591 , which > suggests that exec(s, {}, {}) and "exec in {}, {}" aren't identical > when nested functions are involved. > > -- > Neil Muller > drnlmuller at gmail.com > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcannon at gmail.com Fri Jul 25 20:42:07 2014 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 25 Jul 2014 18:42:07 +0000 Subject: [Python-porting] Pylint checkers for Python 2/3 compatibility Message-ID: I think I've reached beta quality for caniusepython3 3.0.0 where I have implemented custom Pylint checkers for various things that will break in Python 3 (e.g. referencing built-ins that are removed or have different semantics, etc.). The goal with the checkers is for projects to use them after running modernize(3k) or futurize on their code to prevent regressing their Python 2/3 source-compatibility. The code can be found at https://github.com/brettcannon/caniusepython3/tree/pylint . Feedback is welcome before I make an official release. -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Fri Jul 25 23:13:27 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 25 Jul 2014 16:13:27 -0500 Subject: [Python-porting] Pylint checkers for Python 2/3 compatibility In-Reply-To: References: Message-ID: How smart are the checkers, relative to, say, the sorts of things that 2to3 looks for (or modernize or futurize)? Aaron Meurer On Fri, Jul 25, 2014 at 1:42 PM, Brett Cannon wrote: > I think I've reached beta quality for caniusepython3 3.0.0 where I have > implemented custom Pylint checkers for various things that will break in > Python 3 (e.g. referencing built-ins that are removed or have different > semantics, etc.). The goal with the checkers is for projects to use them > after running modernize(3k) or futurize on their code to prevent regressing > their Python 2/3 source-compatibility. > > The code can be found at > https://github.com/brettcannon/caniusepython3/tree/pylint . Feedback is > welcome before I make an official release. > > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting > From ericsnowcurrently at gmail.com Thu Jul 17 15:21:16 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 17 Jul 2014 07:21:16 -0600 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: On Jul 17, 2014 1:05 AM, "Dirkjan Ochtman" wrote: > However, on installation, all modules are byte-compiled, and both of > these cause errors at the syntax level in the version of Python > they're not intended for. Isn't that what cache tags (PEP 3147) solve? -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From shai at platonix.com Thu Jul 17 22:23:02 2014 From: shai at platonix.com (Shai Berger) Date: Thu, 17 Jul 2014 23:23:02 +0300 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: Message-ID: <2031950.zGla8SkiLb@deblack> I think Dirkjan tried to replace the invocation of util.pyexec with an exec; keeping the invocation, and replacing the python2 implementation, should work. Shai. On Thursday 17 July 2014 10:41:23 Guido van Rossum wrote: > The code you link to doesn't match the traceback. I recommend that you > experiment a bit and Google around for the error message. > > > On Thu, Jul 17, 2014 at 10:25 AM, Dirkjan Ochtman > > wrote: > > On Thu, Jul 17, 2014 at 5:45 PM, Guido van Rossum > > > > wrote: > > > The Py2 version can just call exec(code, gns, lns). It is Py2 syntax > > > too! > > > > That sounds like an awesome solution, but it doesn't quite solve the > > problem: > > > > Traceback (most recent call last): > > File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main > > > > "__main__", fname, loader, pkg_name) > > > > File "/usr/lib/python2.7/runpy.py", line 72, in _run_code > > > > exec code in run_globals > > > > File "/home/djc/src/couchdb-python/couchdb/tests/__main__.py", line > > > > 11, in > > > > from couchdb.tests import client, couch_tests, design, couchhttp, \ > > > > File "couchdb/tests/view.py", line 12, in > > > > from couchdb import view > > > > File "couchdb/view.py", line 56 > > > > exec(string, {'log': _log}, globals_) > > > > SyntaxError: unqualified exec is not allowed in function 'add_fun' it > > is a nested function > > > > The code is here: > > > > https://github.com/djc/couchdb-python/blob/afdbe601/couchdb/view.py#L56 > > > > I read up a little on unqualified and qualified exec, but I couldn't > > really figure out why that error is triggering in this case. > > > > Cheers, > > > > Dirkjan From efiring.ocean at gmail.com Fri Jul 25 10:21:54 2014 From: efiring.ocean at gmail.com (Eric Firing) Date: Thu, 24 Jul 2014 22:21:54 -1000 Subject: [Python-porting] python-future: scripts generate unwanted output Message-ID: <53D213A2.5020205@gmail.com> I have used futurize plus some manual fixup in an ongoing attempt to port a package. Some of it is now working on both Python 2 and 3, but although scripts work correctly, they begin by spewing an enormous number of lines like: Detecting hooks ... Not detected. Uninstalling hooks ... Not removing future.types.newobject Not removing future.types.newdict Not removing future.types.math Not removing future.builtins.newnext ... and/or Generating grammar tables from /usr/lib/python3.4/lib2to3/Grammar.txt Generating grammar tables from /usr/lib/python3.4/lib2to3/PatternGrammar.txt Adding transformation: division Generating grammar tables from /usr/lib/python3.4/lib2to3/PatternGrammar.txt Generating grammar tables from /usr/lib/python3.4/lib2to3/PatternGrammar.txt Adding transformation: print_with_import Why is this happening, and how can I suppress it? Is it related to use of the logging module? Eric From dirkjan at ochtman.nl Sat Jul 26 16:24:01 2014 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 26 Jul 2014 16:24:01 +0200 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: <2031950.zGla8SkiLb@deblack> References: <2031950.zGla8SkiLb@deblack> Message-ID: On Thu, Jul 17, 2014 at 10:23 PM, Shai Berger wrote: > I think Dirkjan tried to replace the invocation of util.pyexec with an exec; > keeping the invocation, and replacing the python2 implementation, should work. Yes, that's what I ended up doing: https://github.com/djc/couchdb-python/commit/8fdba0f09df00d69618858c70d11ddbeecd30026 I also took a look at the Python bug Neil pointed at (thanks for finding that, Neil!) and came up with a patch, which is now waiting for review. Cheers, Dirkjan From guido at python.org Sat Jul 26 16:51:04 2014 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Jul 2014 07:51:04 -0700 Subject: [Python-porting] python-future: scripts generate unwanted output In-Reply-To: <53D213A2.5020205@gmail.com> References: <53D213A2.5020205@gmail.com> Message-ID: I suggest that you contact the authors and maintainers of futurize. From the message logged it looks to me as if the conversion is happening manually on each run rather than converting the source once and committing the converted source. (The second set of messages are logged by the parser generator that is part of lib2to3 in the standard library; the first set is apparently from the futurize package.) On Fri, Jul 25, 2014 at 1:21 AM, Eric Firing wrote: > I have used futurize plus some manual fixup in an ongoing attempt to port > a package. Some of it is now working on both Python 2 and 3, but although > scripts work correctly, they begin by spewing an enormous number of lines > like: > > Detecting hooks ... > Not detected. > Uninstalling hooks ... > Not removing future.types.newobject > Not removing future.types.newdict > Not removing future.types.math > Not removing future.builtins.newnext > > ... > > and/or > > Generating grammar tables from /usr/lib/python3.4/lib2to3/Grammar.txt > Generating grammar tables from /usr/lib/python3.4/lib2to3/ > PatternGrammar.txt > Adding transformation: division > Generating grammar tables from /usr/lib/python3.4/lib2to3/ > PatternGrammar.txt > Generating grammar tables from /usr/lib/python3.4/lib2to3/ > PatternGrammar.txt > Adding transformation: print_with_import > > Why is this happening, and how can I suppress it? Is it related to use of > the logging module? > > Eric > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcannon at gmail.com Sat Jul 26 16:59:39 2014 From: bcannon at gmail.com (Brett Cannon) Date: Sat, 26 Jul 2014 14:59:39 +0000 Subject: [Python-porting] Pylint checkers for Python 2/3 compatibility References: Message-ID: On Fri Jul 25 2014 at 5:13:47 PM Aaron Meurer wrote: > How smart are the checkers, relative to, say, the sorts of things that > 2to3 looks for (or modernize or futurize)? > They are smarter as the checkers have more information (e.g. is this a method and not a function, is this a built-in and not just something named 'map'). -Brett > > Aaron Meurer > > On Fri, Jul 25, 2014 at 1:42 PM, Brett Cannon wrote: > > I think I've reached beta quality for caniusepython3 3.0.0 where I have > > implemented custom Pylint checkers for various things that will break in > > Python 3 (e.g. referencing built-ins that are removed or have different > > semantics, etc.). The goal with the checkers is for projects to use them > > after running modernize(3k) or futurize on their code to prevent > regressing > > their Python 2/3 source-compatibility. > > > > The code can be found at > > https://github.com/brettcannon/caniusepython3/tree/pylint . Feedback is > > welcome before I make an official release. > > > > _______________________________________________ > > Python-porting mailing list > > Python-porting at python.org > > https://mail.python.org/mailman/listinfo/python-porting > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sat Jul 26 17:31:46 2014 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Jul 2014 08:31:46 -0700 Subject: [Python-porting] Pylint checkers for Python 2/3 compatibility In-Reply-To: References: Message-ID: Is there a write-up somewhere of the strengths and weaknesses of pylint's type checking? On Sat, Jul 26, 2014 at 7:59 AM, Brett Cannon wrote: > > > On Fri Jul 25 2014 at 5:13:47 PM Aaron Meurer wrote: > >> How smart are the checkers, relative to, say, the sorts of things that >> 2to3 looks for (or modernize or futurize)? >> > > They are smarter as the checkers have more information (e.g. is this a > method and not a function, is this a built-in and not just something named > 'map'). > > -Brett > > >> >> Aaron Meurer >> >> On Fri, Jul 25, 2014 at 1:42 PM, Brett Cannon wrote: >> > I think I've reached beta quality for caniusepython3 3.0.0 where I have >> > implemented custom Pylint checkers for various things that will break in >> > Python 3 (e.g. referencing built-ins that are removed or have different >> > semantics, etc.). The goal with the checkers is for projects to use them >> > after running modernize(3k) or futurize on their code to prevent >> regressing >> > their Python 2/3 source-compatibility. >> > >> > The code can be found at >> > https://github.com/brettcannon/caniusepython3/tree/pylint . Feedback is >> > welcome before I make an official release. >> > >> > _______________________________________________ >> > Python-porting mailing list >> > Python-porting at python.org >> > https://mail.python.org/mailman/listinfo/python-porting >> > >> > > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcannon at gmail.com Sat Jul 26 18:40:29 2014 From: bcannon at gmail.com (Brett Cannon) Date: Sat, 26 Jul 2014 16:40:29 +0000 Subject: [Python-porting] Pylint checkers for Python 2/3 compatibility References: Message-ID: Not that I know of. I think it's locally inferred. I have yet to use any type information, just scope details. On Sat, Jul 26, 2014, 11:32 Guido van Rossum wrote: > Is there a write-up somewhere of the strengths and weaknesses of pylint's > type checking? > > > On Sat, Jul 26, 2014 at 7:59 AM, Brett Cannon wrote: > >> >> >> On Fri Jul 25 2014 at 5:13:47 PM Aaron Meurer wrote: >> >>> How smart are the checkers, relative to, say, the sorts of things that >>> 2to3 looks for (or modernize or futurize)? >>> >> >> They are smarter as the checkers have more information (e.g. is this a >> method and not a function, is this a built-in and not just something named >> 'map'). >> >> -Brett >> >> >>> >>> Aaron Meurer >>> >>> On Fri, Jul 25, 2014 at 1:42 PM, Brett Cannon wrote: >>> > I think I've reached beta quality for caniusepython3 3.0.0 where I have >>> > implemented custom Pylint checkers for various things that will break >>> in >>> > Python 3 (e.g. referencing built-ins that are removed or have different >>> > semantics, etc.). The goal with the checkers is for projects to use >>> them >>> > after running modernize(3k) or futurize on their code to prevent >>> regressing >>> > their Python 2/3 source-compatibility. >>> > >>> > The code can be found at >>> > https://github.com/brettcannon/caniusepython3/tree/pylint . Feedback >>> is >>> > welcome before I make an official release. >>> > >>> > _______________________________________________ >>> > Python-porting mailing list >>> > Python-porting at python.org >>> > https://mail.python.org/mailman/listinfo/python-porting >>> > >>> >> >> _______________________________________________ >> Python-porting mailing list >> Python-porting at python.org >> https://mail.python.org/mailman/listinfo/python-porting >> >> > > > -- > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Sat Jul 26 19:21:03 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Sat, 26 Jul 2014 12:21:03 -0500 Subject: [Python-porting] Installing different files on Python 2/3 In-Reply-To: References: <2031950.zGla8SkiLb@deblack> Message-ID: In SymPy's compatibility file, we use def exec_(_code_, _globs_=None, _locs_=None): """Execute code in a namespace.""" if _globs_ is None: frame = sys._getframe(1) _globs_ = frame.f_globals if _locs_ is None: _locs_ = frame.f_locals del frame elif _locs_ is None: _locs_ = _globs_ exec("exec _code_ in _globs_, _locs_") which I think is either taken from six or based off of what is in six. Aaron Meurer On Sat, Jul 26, 2014 at 9:24 AM, Dirkjan Ochtman wrote: > On Thu, Jul 17, 2014 at 10:23 PM, Shai Berger wrote: >> I think Dirkjan tried to replace the invocation of util.pyexec with an exec; >> keeping the invocation, and replacing the python2 implementation, should work. > > Yes, that's what I ended up doing: > > https://github.com/djc/couchdb-python/commit/8fdba0f09df00d69618858c70d11ddbeecd30026 > > I also took a look at the Python bug Neil pointed at (thanks for > finding that, Neil!) and came up with a patch, which is now waiting > for review. > > Cheers, > > Dirkjan > _______________________________________________ > Python-porting mailing list > Python-porting at python.org > https://mail.python.org/mailman/listinfo/python-porting From heudyfeliz at gmail.com Sat Jul 26 21:03:22 2014 From: heudyfeliz at gmail.com (Heudy Enrique Feliz Castillo) Date: Sat, 26 Jul 2014 11:03:22 -0800 Subject: [Python-porting] (no subject) Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: