From pcmanticore at gmail.com Thu Jun 2 17:44:18 2016 From: pcmanticore at gmail.com (Claudiu Popa) Date: Thu, 2 Jun 2016 22:44:18 +0100 Subject: [code-quality] Patch to expose path of rcfile In-Reply-To: <2e95e1d9-e58b-86c9-f234-dce46ce4c59a@hudson-trading.com> References: <2e95e1d9-e58b-86c9-f234-dce46ce4c59a@hudson-trading.com> Message-ID: On Thu, May 26, 2016 at 8:57 PM, Tim Stumbaugh wrote: > Hello, > > We are in the early phases of rolling out pylint to our Python application. > We have a particularly strange setup where we use __path__ in a number of > "virtual packages" in order to set the correct location of the real package. > Activating astroid's behavior which scans sys.path (the from pkgutil import > extend_path trick) in combination with manipulating sys.path in an init-hook > works great and allows pylint to see the package the same way the > interpreter does. > > However, figuring out the correct path to insert onto sys.path is > unfortunately not trivial, since we support running pylint as part of our > pre-commit review tool, which runs it in a different directory but passes > --rcfile on the command-line. In order to get the correct directory, we need > to have the path of the current rcfile (be it one that came from > config.find_pylintrc or one that came from --rcfile). Unfortunately, it > seems that config.PYLINTRC does not get updated to include the path of the > file if --rcfile is given. > > Our current solution to this problem is to use inspect within the init-hook > to step back up to the frame of Run and look at the config_file member of > the linter: > > init-hook=import inspect; > sys.path.append(os.path.dirname(inspect.stack()[2][0].f_locals["linter"].config_file)) > > However, since this essentially couples us to the internals of pylint, it > would be nice to have something a little less invasive. I prepared a patch > to pylint/lint.py that would set config.PYLINTRC in cb_set_rcfile. That > allows the init-hook above to be replaced by: > > init-hook=sys.path.append(os.path.dirname(config.PYLINTRC)) > > And preserve the same behavior. > > My question is, would this be a change that the project is interested in? I > didn't just want to send a pull request out of the blue, since the actual > change is trivial. Is there some better way to accomplish what I am trying > to do? > > Thanks for your consideration, > tjs > -- > Tim Stumbaugh > Operations > Hudson River Trading > > _______________________________________________ > code-quality mailing list > code-quality at python.org > https://mail.python.org/mailman/listinfo/code-quality > Hi Tim, Sorry for the delay in the first place, I totally forgot about answering. Unfortunately, config.PYLINTRC doesn't sound to be the right place for this, since it is just a global constant (after computed), that we use to determine a potential rcfile. I would be happy with another one that is designed specifically for this purpose, but another idea that occurs to me would be to just use an environment variable, e.g PYLINTRC, which is already used by find_pylintrc function. Then your hook becomes just --init-hook="sys.path.append(os.path.dirname(os.environ['PYLINTRC']))" Would this be a solution you could use? Claudiu From pcmanticore at gmail.com Thu Jun 2 18:15:35 2016 From: pcmanticore at gmail.com (Claudiu Popa) Date: Thu, 2 Jun 2016 23:15:35 +0100 Subject: [code-quality] Patch to expose path of rcfile In-Reply-To: References: <2e95e1d9-e58b-86c9-f234-dce46ce4c59a@hudson-trading.com> Message-ID: On Thu, Jun 2, 2016 at 11:09 PM, Tim Stumbaugh wrote: > We run pylint two ways: > > Interactively, in the same location as the .pylintrc file. We don't > explicitly pass it to pylint, and the normal routines for finding a > .pylintrc file work well here. There won't be an environment variable set > here (since it's just running within a user's shell session), but > config.PYLINTRC is correct. > Via arcanist's "arc lint" command, which changes the working directory to > the incorrect directory (i.e. pylint can't find an rc file it without help), > but with --rcfile. Arcanist has a specific configuration setting just for > pylint's rc file which we use. > > So while it's certainly possible to use environment variables to solve this > problem, I think that would be more complicated (given the two different > cases) and somewhat difficult to express as an init-hook. > > Here's another proposal: would it be amenable to move the function that > executes the init-hook to be a method on the linter class, instead of just a > pure function in the module? That way, the init-hook could have closer > access to the linter (via the "self" name, presumably) and therefore the > pylintrc file? > > Of course, if you'd prefer not to support this workflow, I completely > understand, and our workaround is workable (even if it's a bit brittle). > tjs > Sure, that sounds like a solution that could work, feel free to send a pull request with it. Claudiu From stum at hudson-trading.com Thu Jun 2 18:09:50 2016 From: stum at hudson-trading.com (Tim Stumbaugh) Date: Thu, 2 Jun 2016 18:09:50 -0400 Subject: [code-quality] Patch to expose path of rcfile In-Reply-To: References: <2e95e1d9-e58b-86c9-f234-dce46ce4c59a@hudson-trading.com> Message-ID: We run pylint two ways: * Interactively, in the same location as the .pylintrc file. We don't explicitly pass it to pylint, and the normal routines for finding a .pylintrc file work well here. There won't be an environment variable set here (since it's just running within a user's shell session), but config.PYLINTRC is correct. * Via arcanist's "arc lint" command, which changes the working directory to the incorrect directory (i.e. pylint can't find an rc file it without help), but with --rcfile. Arcanist has a specific configuration setting just for pylint's rc file which we use. So while it's certainly possible to use environment variables to solve this problem, I think that would be more complicated (given the two different cases) and somewhat difficult to express as an init-hook. Here's another proposal: would it be amenable to move the function that executes the init-hook to be a method on the linter class, instead of just a pure function in the module? That way, the init-hook could have closer access to the linter (via the "self" name, presumably) and therefore the pylintrc file? Of course, if you'd prefer not to support this workflow, I completely understand, and our workaround is workable (even if it's a bit brittle). tjs On 2016-06-02 17:44, Claudiu Popa wrote: > On Thu, May 26, 2016 at 8:57 PM, Tim Stumbaugh wrote: >> Hello, >> >> We are in the early phases of rolling out pylint to our Python application. >> We have a particularly strange setup where we use __path__ in a number of >> "virtual packages" in order to set the correct location of the real package. >> Activating astroid's behavior which scans sys.path (the from pkgutil import >> extend_path trick) in combination with manipulating sys.path in an init-hook >> works great and allows pylint to see the package the same way the >> interpreter does. >> >> However, figuring out the correct path to insert onto sys.path is >> unfortunately not trivial, since we support running pylint as part of our >> pre-commit review tool, which runs it in a different directory but passes >> --rcfile on the command-line. In order to get the correct directory, we need >> to have the path of the current rcfile (be it one that came from >> config.find_pylintrc or one that came from --rcfile). Unfortunately, it >> seems that config.PYLINTRC does not get updated to include the path of the >> file if --rcfile is given. >> >> Our current solution to this problem is to use inspect within the init-hook >> to step back up to the frame of Run and look at the config_file member of >> the linter: >> >> init-hook=import inspect; >> sys.path.append(os.path.dirname(inspect.stack()[2][0].f_locals["linter"].config_file)) >> >> However, since this essentially couples us to the internals of pylint, it >> would be nice to have something a little less invasive. I prepared a patch >> to pylint/lint.py that would set config.PYLINTRC in cb_set_rcfile. That >> allows the init-hook above to be replaced by: >> >> init-hook=sys.path.append(os.path.dirname(config.PYLINTRC)) >> >> And preserve the same behavior. >> >> My question is, would this be a change that the project is interested in? I >> didn't just want to send a pull request out of the blue, since the actual >> change is trivial. Is there some better way to accomplish what I am trying >> to do? >> >> Thanks for your consideration, >> tjs >> -- >> Tim Stumbaugh >> Operations >> Hudson River Trading >> >> _______________________________________________ >> code-quality mailing list >> code-quality at python.org >> https://mail.python.org/mailman/listinfo/code-quality >> > > Hi Tim, > > Sorry for the delay in the first place, I totally forgot about answering. > > Unfortunately, config.PYLINTRC doesn't sound to be the right > place for this, since it is just a global constant (after computed), > that we use to determine a potential rcfile. I would be happy with > another one that is designed specifically for this purpose, but another > idea that occurs to me would be to just use an environment variable, > e.g PYLINTRC, which is already used by find_pylintrc function. Then > your hook becomes just > --init-hook="sys.path.append(os.path.dirname(os.environ['PYLINTRC']))" > Would this be a solution you could use? > > > Claudiu -- Tim Stumbaugh Operations Hudson River Trading -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Fri Jun 3 09:04:15 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Fri, 3 Jun 2016 08:04:15 -0500 Subject: [code-quality] PyCon 2016 Message-ID: Hi all! As some of you may have realized, PyCon 2016 took place this week (and sprints are still going on!). A bunch of us from this mailing list (and also those who are no on the list) met up a couple times throughout the week to discuss the Python Code Quality Authority, pycodestyle, Flake8, and other similar projects. Ian Lee, the maintainer of pycodestyle, did a lot of great work organizing a lightning talk, open space, breakfast meetup, and sprint on pycodestyle. Ian and I had a few chances to speak about the future of pycodestyle and Flake8. Those of you following along with the mailing list and some of my probably confusing messages might realize that I'm working on Flake8 3.0 (https://gitlab.com/pycqa/flake8/commits/proposed/3.0) to remove most of our inner dependencies on pycodestyle (while still using it for its expertise in check writing). In looking over the code a little, we've decided to try to turn as much of that work into libraries hosted in the PyCQA as possible so pycodestyle can begin to rely on them as well. Further, some of the responsibilities that Flake8 has taken away from pycodestyle mean that it could, ostensibly, simplify it's internal handling of checks if it wants to (e.g., by removing its handling of AST checks). In conclusion, I think that these meetings were extremely valuable. I recognize that most of us (including me) can't afford to go to too many of these a year, but I wonder if we should start doing virtual meet-ups (perhaps with Google Hangouts on Air) to reap some of the same benefits. I know PyLadies Remote has an account to do this with and I'm sure we could create a similar account for the PyCQA/this mailing list. Cheers, Ian (not Ian Lee... the other Ian ;-) ) From matt at itsmemattchung.com Fri Jun 3 09:56:27 2016 From: matt at itsmemattchung.com (Matt Chung) Date: Fri, 03 Jun 2016 06:56:27 -0700 Subject: [code-quality] PyCon 2016 In-Reply-To: References: Message-ID: <1464962187.43417.627000641.743F8BD3@webmail.messagingengine.com> It was really nice meeting you both over Pycon. I like the idea of the google hangouts and would be totally up for it. Cheers, Matt Chung blog[http://blog.itsmemattchung.com/]?| github[https://github.com/itsmemattchung]?| twitter[https://twitter.com/itsmemattchung]?| linkedin[https://www.linkedin.com/in/matt-chung-32792354] On Fri, Jun 3, 2016, at 06:04 AM, Ian Cordasco wrote: > Hi all! > > As some of you may have realized, PyCon 2016 took place this week (and > sprints are still going on!). A bunch of us from this mailing list > (and also those who are no on the list) met up a couple times > throughout the week to discuss the Python Code Quality Authority, > pycodestyle, Flake8, and other similar projects. > > Ian Lee, the maintainer of pycodestyle, did a lot of great work > organizing a lightning talk, open space, breakfast meetup, and sprint > on pycodestyle. > > Ian and I had a few chances to speak about the future of pycodestyle > and Flake8. Those of you following along with the mailing list and > some of my probably confusing messages might realize that I'm working > on Flake8 3.0 (https://gitlab.com/pycqa/flake8/commits/proposed/3.0) > to remove most of our inner dependencies on pycodestyle (while still > using it for its expertise in check writing). > > In looking over the code a little, we've decided to try to turn as > much of that work into libraries hosted in the PyCQA as possible so > pycodestyle can begin to rely on them as well. > > Further, some of the responsibilities that Flake8 has taken away from > pycodestyle mean that it could, ostensibly, simplify it's internal > handling of checks if it wants to (e.g., by removing its handling of > AST checks). > > In conclusion, I think that these meetings were extremely valuable. I > recognize that most of us (including me) can't afford to go to too > many of these a year, but I wonder if we should start doing virtual > meet-ups (perhaps with Google Hangouts on Air) to reap some of the > same benefits. I know PyLadies Remote has an account to do this with > and I'm sure we could create a similar account for the PyCQA/this > mailing list. > > Cheers, > Ian (not Ian Lee... the other Ian ;-) ) > _______________________________________________ > code-quality mailing list > code-quality at python.org > https://mail.python.org/mailman/listinfo/code-quality From me at the-compiler.org Sat Jun 4 06:07:16 2016 From: me at the-compiler.org (Florian Bruhin) Date: Sat, 4 Jun 2016 12:07:16 +0200 Subject: [code-quality] PyCon 2016 In-Reply-To: References: Message-ID: <20160604100716.rqsiwxfxmsswwluq@tonks> * Ian Cordasco [2016-06-03 08:04:15 -0500]: > In conclusion, I think that these meetings were extremely valuable. I > recognize that most of us (including me) can't afford to go to too > many of these a year, but I wonder if we should start doing virtual > meet-ups (perhaps with Google Hangouts on Air) to reap some of the > same benefits. I know PyLadies Remote has an account to do this with > and I'm sure we could create a similar account for the PyCQA/this > mailing list. FWIW the pytest maintainers have done such a hangout meeting a few months ago, and it turned out quite well too. It's definitely valuable at least having seen each other virtually once (and we'll soon see each other in reallife soon at the pytest sprint) :) Maybe https://talky.io/ would be an alternative if not everyone has a Google Hangout account - I've only tried it with two people so far and not a group though. Florian -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From ben+python at benfinney.id.au Sat Jun 4 06:16:42 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 04 Jun 2016 20:16:42 +1000 Subject: [code-quality] PyCon 2016 References: <20160604100716.rqsiwxfxmsswwluq@tonks> Message-ID: <85mvn145th.fsf@benfinney.id.au> Florian Bruhin writes: > Maybe https://talky.io/ would be an alternative if not everyone has a > Google Hangout account - I've only tried it with two people so far and > not a group though. To avoid vendor-locked solutions, Jitsi would be a better option . It's free software, anyone can run an instance of it, or the public server can be used. -- \ ?You can never entirely stop being what you once were. That's | `\ why it's important to be the right person today, and not put it | _o__) off until tomorrow.? ?Larry Wall | Ben Finney From edreamleo at gmail.com Fri Jun 3 18:27:56 2016 From: edreamleo at gmail.com (Edward K. Ream) Date: Fri, 3 Jun 2016 17:27:56 -0500 Subject: [code-quality] pyflakes-study group and code In-Reply-To: References: Message-ID: On Tue, May 31, 2016 at 9:34 AM, Phil Frost wrote: > Edward, thanks for doing this work. It looks really great. ? The project is complete, imo, with a concluding post here .? All comments and corrections welcome. Edward -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Sat Jun 4 08:36:17 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sat, 4 Jun 2016 07:36:17 -0500 Subject: [code-quality] PyCon 2016 In-Reply-To: References: <20160604100716.rqsiwxfxmsswwluq@tonks> <85mvn145th.fsf@benfinney.id.au> Message-ID: On Jun 4, 2016 5:17 AM, "Ben Finney" wrote: > > Florian Bruhin writes: > > > Maybe https://talky.io/ would be an alternative if not everyone has a > > Google Hangout account - I've only tried it with two people so far and > > not a group though. > > To avoid vendor-locked solutions, Jitsi would be a better option > . It's free software, anyone can run an > instance of it, or the public server can be used. My main reason for suggesting Google hangouts on air is that people can then watch the recording on YouTube. Do either of those allow me (or someone else) to record the meeting so it can be uploaded? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Sat Jun 4 09:48:24 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 04 Jun 2016 23:48:24 +1000 Subject: [code-quality] PyCon 2016 References: <20160604100716.rqsiwxfxmsswwluq@tonks> <85mvn145th.fsf@benfinney.id.au> Message-ID: <85a8j13w0n.fsf@benfinney.id.au> Ian Cordasco writes: > Do either of those allow me (or someone else) to record the meeting so > it can be uploaded? I have no experience with that. Someone else will need to investigate. -- \ ?Come on Milhouse, there?s no such thing as a soul! It?s just | `\ something they made up to scare kids, like the Boogie Man or | _o__) Michael Jackson.? ?Bart, _The Simpsons_ | Ben Finney From edreamleo at gmail.com Wed Jun 8 08:31:07 2016 From: edreamleo at gmail.com (Edward K. Ream) Date: Wed, 8 Jun 2016 07:31:07 -0500 Subject: [code-quality] Pull request for pyflakes-study code? Message-ID: Hello all, The pyflakes-study code project is complete. I am wondering whether a pull request for the actual pyflakes project would be appropriate. I would be happy to issue the pull request, but I'm not sure how to do that from one project (pyflakes_study) to another (pyflakes itself). Perhaps the pyflakes devs could investigate what I have done first. Notes: 1. Only checker.py should have to change. All other files would remain unchanged. 2. All unit tests pass within the pyflakes-study environment, with the exception of the api tests, which rely on the actual pyflakes directory structure. 3. The new code uses two top-level switches, aft (use the AstFullTraverser class) and new_scope (replace the scope property with a few other lines of code). Leaving in these switches makes it easy to see where the substantive changes are, but presumably they would be removed from the final code after your evaluation. 4. I changed _FieldsOrder._get_fields during my study, but this method disappears when aft is True. Several other important methods also disappear when aft is True. 5. It would be possible to evaluate checker.py using any editor, but the best view of the code is obtained by using Leo, that is, pyflakes_study.leo. Otoh, I am not suggesting that this .leo file be part of the official pyflakes distro. ?All comments and suggestions welcome. Edward? ------------------------------------------------------------------------------------------ Edward K. Ream: edreamleo at gmail.com Leo: http://leoeditor.com/ ------------------------------------------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From indigo at bitglue.com Thu Jun 9 13:20:40 2016 From: indigo at bitglue.com (Phil Frost) Date: Thu, 9 Jun 2016 13:20:40 -0400 Subject: [code-quality] Pull request for pyflakes-study code? In-Reply-To: References: Message-ID: <5759A568.7050606@bitglue.com> On 06/08/2016 08:31 AM, Edward K. Ream wrote: > The pyflakes-study code > project is complete. I am wondering whether a pull request for the actual > pyflakes project would be appropriate. > > I would be happy to issue the pull request, but I'm not sure how to do that > from one project (pyflakes_study) to another (pyflakes itself). Perhaps the > pyflakes devs could investigate what I have done first. Consider the pull requested. I'm slammed with personal commitments right now, but I'll be sure to take a look in a couple weeks after things have settled down. From what I've read from your changes they sound really great. Thank you for taking the time to do this, and I promise your work won't go unnoticed. From jbc.develop at gmail.com Sat Jun 11 12:07:46 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sat, 11 Jun 2016 16:07:46 +0000 Subject: [code-quality] Average number style errors on Python Message-ID: Hi guys i currently searching for some paper or note that has how is the average number of style errors on python. If someone know about this kind of work i will be pleased -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ianlee1521 at gmail.com Sat Jun 11 12:24:37 2016 From: ianlee1521 at gmail.com (Ian Lee) Date: Sat, 11 Jun 2016 09:24:37 -0700 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: I personally don't, but I'd be interested if you ended up finding something. On Saturday, June 11, 2016, Juan BC wrote: > Hi guys i currently searching for some paper or note that has how is the > average number of style errors on python. If someone know about this kind > of work i will be pleased > -- > > Juan BC (from phone) > -- ~ Ian Lee | IanLee1521 at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbc.develop at gmail.com Sat Jun 11 14:32:48 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sat, 11 Jun 2016 18:32:48 +0000 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: Thanks for the quick reply. I think i gonna retrieve a lots of public gists and store it into dataframe. Also i think to retrieve some uknow python packages. Mostly because the famous ones are really good in quality On Sat, 11 Jun 2016 at 13:24 Ian Lee wrote: > I personally don't, but I'd be interested if you ended up finding > something. > > On Saturday, June 11, 2016, Juan BC wrote: > >> Hi guys i currently searching for some paper or note that has how is the >> average number of style errors on python. If someone know about this kind >> of work i will be pleased >> -- >> >> Juan BC (from phone) >> > > > -- > > ~ Ian Lee | IanLee1521 at gmail.com > > -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Sat Jun 11 20:22:33 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sat, 11 Jun 2016 19:22:33 -0500 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: On Sat, Jun 11, 2016 at 1:32 PM, Juan BC wrote: > Thanks for the quick reply. > > I think i gonna retrieve a lots of public gists and store it into > dataframe. Also i think to retrieve some uknow python packages. Mostly > because the famous ones are really good in quality That's actually not entirely the case. I can think of a couple "famous" or popular packages that violate PEP-0008 quite a lot because the creator dislikes the style guide. From graffatcolmingov at gmail.com Sat Jun 11 21:07:19 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sat, 11 Jun 2016 20:07:19 -0500 Subject: [code-quality] Flake8 User Survey Message-ID: Hey all, Flake8 3.0 is close to having a beta release (and therefore a final release). Before that, I need some more information about the last three things we need to implement. As such, I've made a survey to find out what I can simplify for 3.0: http://goo.gl/forms/xiC7dt23RuGavESu1 Cheers, Ian From ianlee1521 at gmail.com Sat Jun 11 22:19:24 2016 From: ianlee1521 at gmail.com (Ian Lee) Date: Sat, 11 Jun 2016 19:19:24 -0700 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: Even the Cpython standard library has quite a few violations.. On Saturday, June 11, 2016, Ian Cordasco wrote: > On Sat, Jun 11, 2016 at 1:32 PM, Juan BC > wrote: > > Thanks for the quick reply. > > > > I think i gonna retrieve a lots of public gists and store it into > > dataframe. Also i think to retrieve some uknow python packages. Mostly > > because the famous ones are really good in quality > > That's actually not entirely the case. I can think of a couple > "famous" or popular packages that violate PEP-0008 quite a lot because > the creator dislikes the style guide. > -- ~ Ian Lee | IanLee1521 at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbc.develop at gmail.com Sun Jun 12 12:31:53 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sun, 12 Jun 2016 16:31:53 +0000 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: True. But if i measure a big project i gonna measure the tolerance of some community to the pep8 errors. for example yesterday i running flake8 into astropy repository and they has 11461 errors, but in the scikit-learn case they has only 5. On Sat, 11 Jun 2016 at 23:19 Ian Lee wrote: > Even the Cpython standard library has quite a few violations.. > > On Saturday, June 11, 2016, Ian Cordasco > wrote: > >> On Sat, Jun 11, 2016 at 1:32 PM, Juan BC wrote: >> > Thanks for the quick reply. >> > >> > I think i gonna retrieve a lots of public gists and store it into >> > dataframe. Also i think to retrieve some uknow python packages. Mostly >> > because the famous ones are really good in quality >> >> That's actually not entirely the case. I can think of a couple >> "famous" or popular packages that violate PEP-0008 quite a lot because >> the creator dislikes the style guide. >> > > > -- > > ~ Ian Lee | IanLee1521 at gmail.com > > -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbc.develop at gmail.com Sun Jun 12 12:32:27 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sun, 12 Jun 2016 16:32:27 +0000 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: When i publish the result i gonna free the dataset i currently creating On Sun, 12 Jun 2016 at 13:31 Juan BC wrote: > True. > > But if i measure a big project i gonna measure the tolerance of some > community to the pep8 errors. for example yesterday i running flake8 into > astropy repository and they has 11461 errors, but in the scikit-learn case > they has only 5. > > > > > > On Sat, 11 Jun 2016 at 23:19 Ian Lee wrote: > >> Even the Cpython standard library has quite a few violations.. >> >> On Saturday, June 11, 2016, Ian Cordasco >> wrote: >> >>> On Sat, Jun 11, 2016 at 1:32 PM, Juan BC wrote: >>> > Thanks for the quick reply. >>> > >>> > I think i gonna retrieve a lots of public gists and store it into >>> > dataframe. Also i think to retrieve some uknow python packages. Mostly >>> > because the famous ones are really good in quality >>> >>> That's actually not entirely the case. I can think of a couple >>> "famous" or popular packages that violate PEP-0008 quite a lot because >>> the creator dislikes the style guide. >>> >> >> >> -- >> >> ~ Ian Lee | IanLee1521 at gmail.com >> >> -- > > Juan BC (from phone) > -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Sun Jun 12 13:31:55 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sun, 12 Jun 2016 12:31:55 -0500 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: You should keep in mind that these projects don't all use Flake8. Astropy uses only pep8/pycodestyle and scikit-learn uses landscape.io. Simply running flake8 on projects won't be sufficient. You'll have to take into account what style means to each project if you're going to identify the number of style errors. On Sun, Jun 12, 2016 at 11:31 AM, Juan BC wrote: > True. > > But if i measure a big project i gonna measure the tolerance of some > community to the pep8 errors. for example yesterday i running flake8 into > astropy repository and they has 11461 errors, but in the scikit-learn case > they has only 5. > > > > > > On Sat, 11 Jun 2016 at 23:19 Ian Lee wrote: >> >> Even the Cpython standard library has quite a few violations.. >> >> On Saturday, June 11, 2016, Ian Cordasco >> wrote: >>> >>> On Sat, Jun 11, 2016 at 1:32 PM, Juan BC wrote: >>> > Thanks for the quick reply. >>> > >>> > I think i gonna retrieve a lots of public gists and store it into >>> > dataframe. Also i think to retrieve some uknow python packages. Mostly >>> > because the famous ones are really good in quality >>> >>> That's actually not entirely the case. I can think of a couple >>> "famous" or popular packages that violate PEP-0008 quite a lot because >>> the creator dislikes the style guide. >> >> >> >> -- >> >> ~ Ian Lee | IanLee1521 at gmail.com >> > -- > > Juan BC (from phone) From jbc.develop at gmail.com Sun Jun 12 15:00:23 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sun, 12 Jun 2016 19:00:23 +0000 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: as i say before, i am no interested in how is a quality of a project based on the tolerance of their own community. Mi interest is the quality based on a random sample of code from a random sample of programmers. I use flake because my index of quality is based also on flake8. On Sun, 12 Jun 2016 at 14:31 Ian Cordasco wrote: > You should keep in mind that these projects don't all use Flake8. > > Astropy uses only pep8/pycodestyle and scikit-learn uses landscape.io. > Simply running flake8 on projects won't be sufficient. You'll have to > take into account what style means to each project if you're going to > identify the number of style errors. > > On Sun, Jun 12, 2016 at 11:31 AM, Juan BC wrote: > > True. > > > > But if i measure a big project i gonna measure the tolerance of some > > community to the pep8 errors. for example yesterday i running flake8 into > > astropy repository and they has 11461 errors, but in the scikit-learn > case > > they has only 5. > > > > > > > > > > > > On Sat, 11 Jun 2016 at 23:19 Ian Lee wrote: > >> > >> Even the Cpython standard library has quite a few violations.. > >> > >> On Saturday, June 11, 2016, Ian Cordasco > >> wrote: > >>> > >>> On Sat, Jun 11, 2016 at 1:32 PM, Juan BC > wrote: > >>> > Thanks for the quick reply. > >>> > > >>> > I think i gonna retrieve a lots of public gists and store it into > >>> > dataframe. Also i think to retrieve some uknow python packages. > Mostly > >>> > because the famous ones are really good in quality > >>> > >>> That's actually not entirely the case. I can think of a couple > >>> "famous" or popular packages that violate PEP-0008 quite a lot because > >>> the creator dislikes the style guide. > >> > >> > >> > >> -- > >> > >> ~ Ian Lee | IanLee1521 at gmail.com > >> > > -- > > > > Juan BC (from phone) > -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Sun Jun 12 15:39:25 2016 From: me at the-compiler.org (Florian Bruhin) Date: Sun, 12 Jun 2016 21:39:25 +0200 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: <20160612193925.yq6roqsrsdzm67h6@tonks> * Juan BC [2016-06-12 19:00:23 +0000]: > as i say before, i am no interested in how is a quality of a project based > on the tolerance of their own community. > > Mi interest is the quality based on a random sample of code from a random > sample of programmers. I use flake because my index of quality is based > also on flake8. Using flake8 errors as a measurement of quality is flawed to begin with, in my opinion. As Ian said, coding style is subjective. If a project has their own style guide instead of following pep8 (maybe because it was created before pep8 was widespread, like Twisted I believe, or many parts of the stdlib), does it have a worse quality? I don't believe so. Even pep8 says: Many projects have their own coding style guidelines. In the event of any conflicts, such project-specific guides take precedence for that project. A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important. [...] Some other good reasons to ignore a particular guideline: [...] - Because the code in question predates the introduction of the guideline and there is no other reason to be modifying that code. Florian -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From graffatcolmingov at gmail.com Sun Jun 12 16:18:39 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sun, 12 Jun 2016 15:18:39 -0500 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: On Sun, Jun 12, 2016 at 2:00 PM, Juan BC wrote: > as i say before, i am no interested in how is a quality of a project based > on the tolerance of their own community. You didn't say that actually. > Mi interest is the quality based on a random sample of code from a random > sample of programmers. I use flake because my index of quality is based also > on flake8. I think you need to watch https://www.youtube.com/watch?v=wf-BqAjZb8M Following the letter of the document isn't following the spirit and the quality of a code base can be measured on far more than one axis. Quality of a code base should also be determined by whether it uses obvious security problems (as can be caught via bandit) and other logical errors (as can be caught by pylint). Quality can be further measured by the tests a project has. I urge you to reconsider how you're measuring "quality". Instead of calling it quality, please consider calling it "style conformance". They are very different things. From jbc.develop at gmail.com Sun Jun 12 16:25:44 2016 From: jbc.develop at gmail.com (Juan BC) Date: Sun, 12 Jun 2016 20:25:44 +0000 Subject: [code-quality] Average number style errors on Python In-Reply-To: References: Message-ID: sorry i think this is an steryl discussion. 1. I see this video before and i never say that my quality index is ONLY style. (and also all your recomendation are already inside the another parts of the equation) I am not offended or angry, but because how the (F%$$#%) scientific publication works i can't tell you everything about the research before the journal make their decision about it. tanks for the comments. PS: i gonna free the dataset in the near future. On Sun, 12 Jun 2016 at 17:18 Ian Cordasco wrote: > On Sun, Jun 12, 2016 at 2:00 PM, Juan BC wrote: > > as i say before, i am no interested in how is a quality of a project > based > > on the tolerance of their own community. > > You didn't say that actually. > > > Mi interest is the quality based on a random sample of code from a random > > sample of programmers. I use flake because my index of quality is based > also > > on flake8. > > I think you need to watch https://www.youtube.com/watch?v=wf-BqAjZb8M > > Following the letter of the document isn't following the spirit and > the quality of a code base can be measured on far more than one axis. > Quality of a code base should also be determined by whether it uses > obvious security problems (as can be caught via bandit) and other > logical errors (as can be caught by pylint). Quality can be further > measured by the tests a project has. > > I urge you to reconsider how you're measuring "quality". Instead of > calling it quality, please consider calling it "style conformance". > They are very different things. > -- Juan BC (from phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: From pcmanticore at gmail.com Fri Jun 17 12:56:00 2016 From: pcmanticore at gmail.com (Claudiu Popa) Date: Fri, 17 Jun 2016 17:56:00 +0100 Subject: [code-quality] Some news regarding Pylint's release process Message-ID: Hi folks, I want with this e-mail to bring a bit of insight into what is happening currently with Pylint, especially what are our plans regarding its releases, and whatnot, since I'm feeling that at least this aspect is a bit opaque to the outside world. The current version that we are working on is 2.0, which is scheduled for release somewhere at the beginning of July. The release date went back and forth for a couple of times, due to not having enough time to work on the planned features, but the approach that we will take after the new release would simplify the process of getting out faster a new version to the users. There are some interesting features already implemented for the next major release (implicit namespace package support, special object attributes understanding, lots of bug fixes and new checks), but the major improvement and also the reason why we are bumping the major number is represented by the new tiered analysis. Pylint is known for being a bit overwhelming by default, which is the reason we are planning to ease its default interaction. The plan can be found at the corresponding issue https://github.com/PyCQA/pylint/issues/746, but the main idea is that, with 2.0, a pylint interaction would like this: $ pylint myproject # core checkers enabled <...messages..> 10/10 - Congrats, you're clean on a core. You might try with "--pedantic" now. $ pylint myproject --pedantic # pedantic checkers enabled. <...messages..> 10/10 - Congrats, you're clean on pedantic. You might try with --refactoring now $ pylint myproject --refactoring # refactoring checkers enabled <...messages..> 10/10 - Congrats, you're clean on refactoring. Last up, try with --style now. $ pylint myproject --style <...messages..> 10/10 - Now you're pylint clean. or $ pylint myproject --everything <...messages..> Core: 10/10 Pedantic: 10/10 Refactoring: 10/10 Style: 10/10 AGGREGATE: 10/10 -- congrats, you're pylint clean. This feature is still in progress and you can see what is left for this milestone here: https://github.com/PyCQA/pylint/milestones/2.0 I mentioned earlier about a new approach for handling releases and the minor releases in particular. Since Pylint is a diverse beast, with various areas which can be worked on at a particular time (new features, fixing false positives, new checks, improving the inference engine and implementing major features, such as PEP 484 support and whatnot), the focus can be lost into one and another, leading to some areas to flourish in the detriment of another, which can also mean that users would have to wait even more for having a bug fix or a feature they are waiting for. The attemptive plan is to have a short release cycle, around 2 - 3 weeks for minor releases and around 4 - 5 months for major releases. Each minor release would have at most 10 issues that are assigned for it, although we might assign less than this number, just so that we could have time to introduce more urgent bug fixes into a following release. Between some minor releases, we will introduce gaps into the schedule for working on the next major release. You can see the current milestones plan here https://github.com/PyCQA/pylint/milestones, including the already mentioned gaps. If you feel like there is an issue that is worth adding into one milestone or another, please do not hesitate to ask. In the end, I want to mention that we are definitely in need for more contributors. If there is anyone who would like to do so, but does not know how to approach the codebase or anything like this, do not hesitate to ask, either on this mailing list or on IRC, I'm trying as much as I can to help anyone that is interested in doing so. All the best, Claudiu From saija.sorsa at intel.com Tue Jun 21 10:06:37 2016 From: saija.sorsa at intel.com (Sorsa, Saija) Date: Tue, 21 Jun 2016 14:06:37 +0000 Subject: [code-quality] Pylint output is not constant Message-ID: <3805D0A1E419744A9AB607B0EA52B2071E58A187@IRSMSX101.ger.corp.intel.com> Hi, pylint output is not constant and it reports different errors if you run it to one file in a directory compared when you run it for a multiple different files in a directory. For example: pylint *.py and pylint mypython.py gives different error messages for mypython.py-file. The error occurs when pylint processes multiple different files at the same time. Br, Saija -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pcmanticore at gmail.com Tue Jun 21 10:16:01 2016 From: pcmanticore at gmail.com (Claudiu Popa) Date: Tue, 21 Jun 2016 15:16:01 +0100 Subject: [code-quality] Pylint output is not constant In-Reply-To: <3805D0A1E419744A9AB607B0EA52B2071E58A187@IRSMSX101.ger.corp.intel.com> References: <3805D0A1E419744A9AB607B0EA52B2071E58A187@IRSMSX101.ger.corp.intel.com> Message-ID: On Tue, Jun 21, 2016 at 3:06 PM, Sorsa, Saija wrote: > Hi, > > > pylint output is not constant and it reports different errors if you run it > to one file in a directory compared when you run it for a multiple different > files in a directory. For example: > > > > pylint *.py > > and > > pylint mypython.py > > > > gives different error messages for mypython.py-file. The error occurs when > pylint processes multiple different files at the same time. > > > Hi, Can you give us some examples on which you get different output? The idea is that passing multiple files to pylint might result in a better understanding of what is happening in your code, which could lead to having different results than just running on a single file. But for better understanding what is happening, a sample or two would be useful. Thank you, Claudiu From vgr255 at live.ca Mon Jun 27 21:08:20 2016 From: vgr255 at live.ca (=?iso-8859-1?Q?=C9manuel_Barry?=) Date: Mon, 27 Jun 2016 21:08:20 -0400 Subject: [code-quality] Change in unicode escape sequences Message-ID: Hello Code Quality people, I'm writing to you today about a change that I'm proposing to CPython, which will warn on invalid Unicode (and bytes) escape sequences, starting 3.6. It will eventually become an error, although when is yet to be decided. See http://bugs.python.org/issue27364 for the patch and discussion. First message sums up pretty well what the end result will be (with maybe a few minor differences). As Victor Stinner and Guido both suggested, it would be good to introduce this in the linters, to help folks who are running e.g. 3.5 (or 2.7 with plans to migrate). So here I am, asking the maintainers of the linters to introduce this, hopefully before 3.6.0 hits the shelves in December. Thanks in advance! -Emanuel From me at the-compiler.org Tue Jun 28 06:52:20 2016 From: me at the-compiler.org (Florian Bruhin) Date: Tue, 28 Jun 2016 12:52:20 +0200 Subject: [code-quality] Change in unicode escape sequences In-Reply-To: References: Message-ID: <20160628105220.zkwpivbea7j756im@tonks> * ?manuel Barry [2016-06-27 21:08:20 -0400]: > Hello Code Quality people, > I'm writing to you today about a change that I'm proposing to CPython, which > will warn on invalid Unicode (and bytes) escape sequences, starting 3.6. It > will eventually become an error, although when is yet to be decided. See > http://bugs.python.org/issue27364 for the patch and discussion. First > message sums up pretty well what the end result will be (with maybe a few > minor differences). > > As Victor Stinner and Guido both suggested, it would be good to introduce > this in the linters, to help folks who are running e.g. 3.5 (or 2.7 with > plans to migrate). So here I am, asking the maintainers of the linters to > introduce this, hopefully before 3.6.0 hits the shelves in December. pylint already does: $ cat x.py print("hello \world") $ pylint x.py [...] W: 1, 0: Anomalous backslash in string: '\w'. String constant might be missing an r prefix. (anomalous-backslash-in-string) [...] Florian -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From vgr255 at live.ca Tue Jun 28 06:57:11 2016 From: vgr255 at live.ca (=?iso-8859-1?Q?=C9manuel_Barry?=) Date: Tue, 28 Jun 2016 06:57:11 -0400 Subject: [code-quality] Change in unicode escape sequences In-Reply-To: <20160628105220.zkwpivbea7j756im@tonks> References: <20160628105220.zkwpivbea7j756im@tonks> Message-ID: > From: Florian Bruhin > Sent: Tuesday, June 28, 2016 6:52 AM > Subject: Re: [code-quality] Change in unicode escape sequences > > pylint already does: > > $ cat x.py > print("hello \world") > > $ pylint x.py > [...] > W: 1, 0: Anomalous backslash in string: '\w'. String constant might > be missing an r prefix. (anomalous-backslash-in-string) > [...] > > Florian Great, that's one less thing to worry about :) What about the other ones? -Emanuel From graffatcolmingov at gmail.com Tue Jun 28 08:45:13 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Tue, 28 Jun 2016 07:45:13 -0500 Subject: [code-quality] Change in unicode escape sequences In-Reply-To: References: <20160628105220.zkwpivbea7j756im@tonks> Message-ID: On Tue, Jun 28, 2016 at 5:57 AM, ?manuel Barry wrote: >> From: Florian Bruhin >> Sent: Tuesday, June 28, 2016 6:52 AM >> Subject: Re: [code-quality] Change in unicode escape sequences >> >> pylint already does: >> >> $ cat x.py >> print("hello \world") >> >> $ pylint x.py >> [...] >> W: 1, 0: Anomalous backslash in string: '\w'. String constant might >> be missing an r prefix. (anomalous-backslash-in-string) >> [...] >> >> Florian > > Great, that's one less thing to worry about :) What about the other ones? > -Emanuel I think pydocstyle does this for docstrings by default. PyFlakes might also be a good candidate but they try to have as few checks that could cause false positives as possible. From amir at rachum.com Tue Jun 28 08:51:37 2016 From: amir at rachum.com (Amir Rachum) Date: Tue, 28 Jun 2016 15:51:37 +0300 Subject: [code-quality] Change in unicode escape sequences In-Reply-To: References: <20160628105220.zkwpivbea7j756im@tonks> Message-ID: Pydocstyle: $ cat x.py """hello \world.""" $ pydocstyle x.py x.py:1 at module level: D301: Use r""" if any backslashes in a docstring On Tue, Jun 28, 2016 at 3:45 PM, Ian Cordasco wrote: > On Tue, Jun 28, 2016 at 5:57 AM, ?manuel Barry wrote: > >> From: Florian Bruhin > >> Sent: Tuesday, June 28, 2016 6:52 AM > >> Subject: Re: [code-quality] Change in unicode escape sequences > >> > >> pylint already does: > >> > >> $ cat x.py > >> print("hello \world") > >> > >> $ pylint x.py > >> [...] > >> W: 1, 0: Anomalous backslash in string: '\w'. String constant might > >> be missing an r prefix. (anomalous-backslash-in-string) > >> [...] > >> > >> Florian > > > > Great, that's one less thing to worry about :) What about the other ones? > > -Emanuel > > I think pydocstyle does this for docstrings by default. PyFlakes might > also be a good candidate but they try to have as few checks that could > cause false positives as possible. > _______________________________________________ > code-quality mailing list > code-quality at python.org > https://mail.python.org/mailman/listinfo/code-quality > -------------- next part -------------- An HTML attachment was scrubbed... URL: From indigo at bitglue.com Wed Jun 29 00:42:15 2016 From: indigo at bitglue.com (Phil Frost) Date: Tue, 28 Jun 2016 21:42:15 -0700 Subject: [code-quality] Change in unicode escape sequences In-Reply-To: References: <20160628105220.zkwpivbea7j756im@tonks> Message-ID: On Tue, Jun 28, 2016 at 5:45 AM, Ian Cordasco wrote: > PyFlakes might > also be a good candidate but they try to have as few checks that could > cause false positives as possible. I'd be open to checking this in Pyflakes, since I can't really think of a useful reason to want invalid escapes in literals. Unused imports are in a similar boat: not an error per se, but there's very little reason you'd want one, while there are a lot of reasons you'd not want one.