From davidhalter88 at gmail.com  Fri Oct  3 02:56:42 2014
From: davidhalter88 at gmail.com (Dave Halter)
Date: Fri, 3 Oct 2014 02:56:42 +0200
Subject: [code-quality] Jedi gets a new parser
In-Reply-To: <CAMy=CLr7ZcftVL53wo+hqJ8B1LhjYOEd-4fEfEFQEwwwTj_Kyg@mail.gmail.com>
References: <CAA=HWYgNh0E=Vtqmcx7JZgrge35bvgbmzbC+3Dndx5Bj9jqbsQ@mail.gmail.com>
 <CAMy=CLr7ZcftVL53wo+hqJ8B1LhjYOEd-4fEfEFQEwwwTj_Kyg@mail.gmail.com>
Message-ID: <CAA=HWYiAYCVic2kJGfUPKZvzJ4ECsRgwv5JdCn8jJ=6UznEHeQ@mail.gmail.com>

Claudiu, Peter

Does anyone of you know why lib2to3 is using both a bottom-up parser and
pgen2? It "looks" like there are two parsers. I'm quite confused by the
parser design of lib2to3, because in older versions (python 2.6) there's
only pgen2.

~ Dave

2014-09-29 19:15 GMT+02:00 Claudiu Popa <pcmanticore at gmail.com>:

> On Mon, Sep 29, 2014 at 1:02 PM, Dave Halter <davidhalter88 at gmail.com>
> wrote:
> > Hi all!
> >
> > I will be writing a new parser for Jedi. This is necessary for a quite a
> few
> > reasons described in:
> > https://github.com/davidhalter/jedi/issues/480.
> >
> > I'm posting this here, because some of you may be interested in actually
> > using the new parser. It's going to be pure Python and contrary to the
> > Python's internal parser will have error recovery and indent positions.
> >
> > If you're interested, I'd be really glad to hear your feedback!
> >
> > ~ Dave
> >
>
>
> Hello, Dave.
>
> That's awesome, If your parser can generate a new AST, then we can
> definitely use it, although we have
> in mind to use lib2to3 to replace the standard ast module for our use
> cases, but that can change if your parser
> is finished until taking action from our side. Having always correct
> information, such as line and col offsets definitely will
> help us (we actually have a couple of problems regarding this, where
> the lineno for some nodes is different than reality).
> Regarding the parser approach, I'm not sure what's best in this situation.
> I'm working on a static analysis tool, in the vein of pylint, for the
> Powershell language
> (https://github.com/RoPython/wispy) where I'm using modgrammar, a
> recursive descent top down parser to parse the code
> and retrieve an AST from it. modgrammar is pretty good, easy to use,
> but slow sometimes and backtracks too much on ambiguities.
> But it can memorize whitespace tokens and such, as long as you
> explicitly define the whitespace token in the grammar, which
> can be a pretty daunting task.
>
> Anyway, if there is something I can help you with your parser, don't
> hesitate to send me an email.
>
>
> Claudiu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141003/2b95b657/attachment.html>

From mats.hallingstrom at gmail.com  Fri Oct  3 07:51:48 2014
From: mats.hallingstrom at gmail.com (=?UTF-8?Q?Mats_Hallingstr=C3=B6m?=)
Date: Fri, 3 Oct 2014 07:51:48 +0200
Subject: [code-quality] Pylint keeps memory of imports between files.
Message-ID: <CA+CquU+aiy4LZrjfKQRqkDq1YCjKFuEqV8iwLsTP721Z3VvU1g@mail.gmail.com>

Hi Code Quality

This may already be a reported bug, but then i need help in finding that
issue.


Situation:
I invoke pylint with :

from pylint.lint import Run
...

files = [ 'subdir/a.py', 'b.py' ]

for file in files:
 status = Run(['-E'] +
                       ['--rcfile', 'pylintrc'] +
                         [file], exit=False)
## To keep separate records of each file

Problem:
both files have this import
import  config.tc
It runs well because the module config is in the sys.path when run.

But pylint reports for subdir/a.py AND b.py
E: 16,0: No name 'tc' in module 'config'

If I revers the order of the files pylint reports OK for BOTH, probably
because config.tc matches the path from root directory of b.py, and then
when I check subdir/a.py, pylint somehow have a memory of the import.

Run totally separately, only subdir/a.py reports problem with the import of
config.tc.


I don't want the order of the files to matter due to that pylint keeps a
memory of the previous files imports and search paths. How do I clean out
the imports between the files?

-- 
Mats Hallingstr?m
System Tester and python coder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141003/cb9e0e82/attachment.html>

From sylvain.thenault at logilab.fr  Mon Oct  6 09:55:46 2014
From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=)
Date: Mon, 6 Oct 2014 09:55:46 +0200
Subject: [code-quality] Pylint keeps memory of imports between files.
In-Reply-To: <CA+CquU+aiy4LZrjfKQRqkDq1YCjKFuEqV8iwLsTP721Z3VvU1g@mail.gmail.com>
References: <CA+CquU+aiy4LZrjfKQRqkDq1YCjKFuEqV8iwLsTP721Z3VvU1g@mail.gmail.com>
Message-ID: <20141006075546.GA2943@logilab.fr>

On 03 octobre 07:51, Mats Hallingstr?m wrote:
> Hi Code Quality

Hi Mats,
 
> This may already be a reported bug, but then i need help in finding that
> issue.
> 
> 
> Situation:
> I invoke pylint with :
> 
> from pylint.lint import Run
> ...
> 
> files = [ 'subdir/a.py', 'b.py' ]
> 
> for file in files:
>  status = Run(['-E'] +
>                        ['--rcfile', 'pylintrc'] +
>                          [file], exit=False)
> ## To keep separate records of each file
> 
> Problem:
> both files have this import
> import  config.tc
> It runs well because the module config is in the sys.path when run.
> 
> But pylint reports for subdir/a.py AND b.py
> E: 16,0: No name 'tc' in module 'config'
> 
> If I revers the order of the files pylint reports OK for BOTH, probably
> because config.tc matches the path from root directory of b.py, and then
> when I check subdir/a.py, pylint somehow have a memory of the import.
> 
> Run totally separately, only subdir/a.py reports problem with the import of
> config.tc.
> 
> I don't want the order of the files to matter due to that pylint keeps a
> memory of the previous files imports and search paths. How do I clean out
> the imports between the files?

  from astroid import MANAGER
  MANAGER.clear_cache()

should do the trick.
-- 
Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42)
Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org

From mats.hallingstrom at gmail.com  Mon Oct  6 12:27:43 2014
From: mats.hallingstrom at gmail.com (=?UTF-8?Q?Mats_Hallingstr=C3=B6m?=)
Date: Mon, 6 Oct 2014 12:27:43 +0200
Subject: [code-quality] Pylint keeps memory of imports between files.
In-Reply-To: <20141006075546.GA2943@logilab.fr>
References: <CA+CquU+aiy4LZrjfKQRqkDq1YCjKFuEqV8iwLsTP721Z3VvU1g@mail.gmail.com>
 <20141006075546.GA2943@logilab.fr>
Message-ID: <CA+CquUKUnV-19zyJBWgkxvM4Die7T5g_tLbb9D_y4N6Dyb8YDA@mail.gmail.com>

Thanks!
I will try it

BR/
Mats

On 6 October 2014 09:55, Sylvain Th?nault <sylvain.thenault at logilab.fr>
wrote:

> On 03 octobre 07:51, Mats Hallingstr?m wrote:
> > Hi Code Quality
>
> Hi Mats,
>
> > This may already be a reported bug, but then i need help in finding that
> > issue.
> >
> >
> > Situation:
> > I invoke pylint with :
> >
> > from pylint.lint import Run
> > ...
> >
> > files = [ 'subdir/a.py', 'b.py' ]
> >
> > for file in files:
> >  status = Run(['-E'] +
> >                        ['--rcfile', 'pylintrc'] +
> >                          [file], exit=False)
> > ## To keep separate records of each file
> >
> > Problem:
> > both files have this import
> > import  config.tc
> > It runs well because the module config is in the sys.path when run.
> >
> > But pylint reports for subdir/a.py AND b.py
> > E: 16,0: No name 'tc' in module 'config'
> >
> > If I revers the order of the files pylint reports OK for BOTH, probably
> > because config.tc matches the path from root directory of b.py, and then
> > when I check subdir/a.py, pylint somehow have a memory of the import.
> >
> > Run totally separately, only subdir/a.py reports problem with the import
> of
> > config.tc.
> >
> > I don't want the order of the files to matter due to that pylint keeps a
> > memory of the previous files imports and search paths. How do I clean out
> > the imports between the files?
>
>   from astroid import MANAGER
>   MANAGER.clear_cache()
>
> should do the trick.
> --
> Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse
> (05.62.17.16.42)
> Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations
> D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
> CubicWeb, the semantic web framework:    http://www.cubicweb.org
>



-- 
Mats Hallingstr?m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141006/5b2b4ff7/attachment.html>

From clockworksaint at gmail.com  Fri Oct 17 18:23:08 2014
From: clockworksaint at gmail.com (Weeble)
Date: Fri, 17 Oct 2014 17:23:08 +0100
Subject: [code-quality] flake8 missing output issues
Message-ID: <CACBComE9pq_wK02jNNhsZNPE4_dFntmioZaELS9CqOYNcR1JQg@mail.gmail.com>

I'm having some trouble with flake8 when it's invoked from another
process, such as in vim. Sometimes its output goes missing - I just
get back an empty string. The problem vanishes if I
PYTHONUNBUFFERED=1, so I suspect it is related to how flake8 creates
multiple processes. I looked through the code and could see that it
creates multiprocessing sub-processes with daemon=True. Since these
sub-processes perform output on stdout, I believe they need to flush
it before telling the main thread that they are done, otherwise there
is a risk their output will be lost entirely if they are terminated
when the main process exits.

I find my problem goes away if I add "sys.stdout.flush()" to the
finally block in reporter.py's process_main(). I think this is a good
thing to do for the reason I described above.

However, I'm not *entirely* sure I understand what's going on. Every
single time I run flake8 from the command-line, all its output appears
as normal. So while there's a theoretical possibility that output
buffers might not be flushed, that doesn't actually seem to be what's
happening here. My suspicion was that the parent process was exiting
first, then the calling process (vim using the system() function) saw
it had exited and stopped reading its output, then the child process
flushed its output, but it was too late. That would explain why output
appears on the command-line, but is sometimes missed by vim. But I
can't actually figure out a way for this to happen. The parent process
should always terminate its daemon children as part of its exit
handler. So as far as I can tell they should all be dead by the time
it exits, and either they have flushed or their output is lost.

I posted about my problems on the Syntastic google group. You can find
a bit more detail about my original problem and what I tried there:

https://groups.google.com/forum/#!topic/vim-syntastic/Pdnp43Ok-3Y

Does anyone better understand what might be going on here?

Regards,

Weeble.

From sree at mahiti.org  Sat Oct 18 14:45:58 2014
From: sree at mahiti.org (Sreekanth S Rameshaiah)
Date: Sat, 18 Oct 2014 18:15:58 +0530
Subject: [code-quality] flake8 enabling clone digger integration
Message-ID: <CAD=RLAiMMx5F7wr5CfttAM+cwA-BM6WBA=pRofzFrRHUpAgq+A@mail.gmail.com>

Dear Developers,
Greetings !
I downloaded flake8 from gitlab and ifound it as a very useful tool to
ensure code quality.
Was wondering it was possible to integrate clone digger (
http://clonedigger.sourceforge.net/) into flake8.
Please advise.
Regards,
- sree
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141018/649aae24/attachment.html>

From graffatcolmingov at gmail.com  Sat Oct 18 20:09:53 2014
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Sat, 18 Oct 2014 13:09:53 -0500
Subject: [code-quality] flake8 enabling clone digger integration
In-Reply-To: <CAD=RLAiMMx5F7wr5CfttAM+cwA-BM6WBA=pRofzFrRHUpAgq+A@mail.gmail.com>
References: <CAD=RLAiMMx5F7wr5CfttAM+cwA-BM6WBA=pRofzFrRHUpAgq+A@mail.gmail.com>
Message-ID: <CAN-Kwu3=Q3-Fba_M0ZsB+80mVWt8A_TdfVQ-KArx+P-wjExGnw@mail.gmail.com>

On Sat, Oct 18, 2014 at 7:45 AM, Sreekanth S Rameshaiah <sree at mahiti.org> wrote:
> Dear Developers,
> Greetings !
> I downloaded flake8 from gitlab and ifound it as a very useful tool to
> ensure code quality.
> Was wondering it was possible to integrate clone digger
> (http://clonedigger.sourceforge.net/) into flake8.
> Please advise.
> Regards,
> - sree

Hi Sreekanth,

I'm not familiar with any existing integration with clone digger. That
said, Flake8 has a way of providing extensions:
https://flake8.readthedocs.org/en/2.2.3/extensions.html. Since I'm not
familiar with clone digger, I'm not sure how difficult that part of
things will be for you, but hopefully the flake8 side of things will
be easy.

Cheers,
Ian

From graffatcolmingov at gmail.com  Sat Oct 18 20:16:19 2014
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Sat, 18 Oct 2014 13:16:19 -0500
Subject: [code-quality] flake8 missing output issues
In-Reply-To: <CACBComE9pq_wK02jNNhsZNPE4_dFntmioZaELS9CqOYNcR1JQg@mail.gmail.com>
References: <CACBComE9pq_wK02jNNhsZNPE4_dFntmioZaELS9CqOYNcR1JQg@mail.gmail.com>
Message-ID: <CAN-Kwu1bRu0tLoA40Z+zgHcmFa+yOwVSqLaiwBBTrXB_SB2r-g@mail.gmail.com>

On Fri, Oct 17, 2014 at 11:23 AM, Weeble <clockworksaint at gmail.com> wrote:
> I'm having some trouble with flake8 when it's invoked from another
> process, such as in vim. Sometimes its output goes missing - I just
> get back an empty string. The problem vanishes if I
> PYTHONUNBUFFERED=1, so I suspect it is related to how flake8 creates
> multiple processes. I looked through the code and could see that it
> creates multiprocessing sub-processes with daemon=True. Since these
> sub-processes perform output on stdout, I believe they need to flush
> it before telling the main thread that they are done, otherwise there
> is a risk their output will be lost entirely if they are terminated
> when the main process exits.
>
> I find my problem goes away if I add "sys.stdout.flush()" to the
> finally block in reporter.py's process_main(). I think this is a good
> thing to do for the reason I described above.
>
> However, I'm not *entirely* sure I understand what's going on. Every
> single time I run flake8 from the command-line, all its output appears
> as normal. So while there's a theoretical possibility that output
> buffers might not be flushed, that doesn't actually seem to be what's
> happening here. My suspicion was that the parent process was exiting
> first, then the calling process (vim using the system() function) saw
> it had exited and stopped reading its output, then the child process
> flushed its output, but it was too late. That would explain why output
> appears on the command-line, but is sometimes missed by vim. But I
> can't actually figure out a way for this to happen. The parent process
> should always terminate its daemon children as part of its exit
> handler. So as far as I can tell they should all be dead by the time
> it exits, and either they have flushed or their output is lost.
>
> I posted about my problems on the Syntastic google group. You can find
> a bit more detail about my original problem and what I tried there:
>
> https://groups.google.com/forum/#!topic/vim-syntastic/Pdnp43Ok-3Y
>
> Does anyone better understand what might be going on here?
>
> Regards,
>
> Weeble.

Hey Weeble,

I'm rather certain you've hit the nail directly on the head. The
problem here is the default use of multiprocessing. If you're open to
sending a PR to fix this (since you seem to have found a solution, if
I understand you correctly), I'll happily and gratefully merge it and
cut a new minor release for this. I'm sincerely sorry you've run into
this problem.

In case you missed the announcement earlier this month, the repository
is https://gitlab.com/pycqa/flake8 now.

Cheers,
Ian

From sree at mahiti.org  Mon Oct 20 08:13:37 2014
From: sree at mahiti.org (Sreekanth S Rameshaiah)
Date: Mon, 20 Oct 2014 11:43:37 +0530
Subject: [code-quality] flake8 enabling clone digger integration
In-Reply-To: <CAN-Kwu3=Q3-Fba_M0ZsB+80mVWt8A_TdfVQ-KArx+P-wjExGnw@mail.gmail.com>
References: <CAD=RLAiMMx5F7wr5CfttAM+cwA-BM6WBA=pRofzFrRHUpAgq+A@mail.gmail.com>
 <CAN-Kwu3=Q3-Fba_M0ZsB+80mVWt8A_TdfVQ-KArx+P-wjExGnw@mail.gmail.com>
Message-ID: <CAD=RLAgEJNGLB7x7qfwtY4EOQ3X_Pn0vJEWhmZc3A=3Yn98COQ@mail.gmail.com>

Thanks Ian,
Will try the Flake8 extensions..
- sree

On 18 October 2014 23:39, Ian Cordasco <graffatcolmingov at gmail.com> wrote:

> On Sat, Oct 18, 2014 at 7:45 AM, Sreekanth S Rameshaiah <sree at mahiti.org>
> wrote:
> > Dear Developers,
> > Greetings !
> > I downloaded flake8 from gitlab and ifound it as a very useful tool to
> > ensure code quality.
> > Was wondering it was possible to integrate clone digger
> > (http://clonedigger.sourceforge.net/) into flake8.
> > Please advise.
> > Regards,
> > - sree
>
> Hi Sreekanth,
>
> I'm not familiar with any existing integration with clone digger. That
> said, Flake8 has a way of providing extensions:
> https://flake8.readthedocs.org/en/2.2.3/extensions.html. Since I'm not
> familiar with clone digger, I'm not sure how difficult that part of
> things will be for you, but hopefully the flake8 side of things will
> be easy.
>
> Cheers,
> Ian
>



-- 
Sreekanth S Rameshaiah
Executive Director
Mahiti Infotech Pvt. Ltd.
An ISO 9001:2008 & ISO 27001:2013 Enterprise

Phone:  +91 80 4905 8444
Mobile:  +91 98455 12611
www.mahiti.org
www.mahiti-infotech.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141020/826dac96/attachment.html>

From peter.bittner at gmx.net  Tue Oct 21 16:00:18 2014
From: peter.bittner at gmx.net (Peter Bittner)
Date: Tue, 21 Oct 2014 16:00:18 +0200
Subject: [code-quality] Pylint configuration in setup.cfg
Message-ID: <CAHywH5CpOzyBj3vxBhOfqn=yte1h9+J8ZaHZvt=gf6qbTjqpAA@mail.gmail.com>

Hi there the makers of Pylint,

has anyone ever thought about suggesting in the Pylint docs to place
configuration options in a Python project's `setup.cfg` file?

The interesting thing is that when you do a web search for "flake8 pylint
comparison" the opinions you find are somewhat, "flake8 and pylint are
complimentary tools [...] pylint is harder to configure".

So, yes, a bit more complex to configure. But after all the active
configuration can be generated, and it looks like all options can easily be
put into one of the standard places, such as `setup.cfg`. I've put all my
flake8 configuration (3 lines, really) and some additional pylint-specific
options into the setup.cfg file in my project [1], and ran pylint with it
as follows:

$ pylint --rcfile=setup.cfg organice

[1] https://github.com/bittner/django-organice/blob/develop/setup.cfg

If now, by magic, pylint would look for a setup.cfg file automatically
(maybe in the second place, after not finding the default resource file)
the command line would be reduced to "pylint <module>". Wouldn't that be
awesome?

What do you think? Would you accept an appropriate pull request on the docs
in the first place?

BTW, I found issue #121 on Bitbucket with the latest comment saying just
about the same in just one sentence:
https://bitbucket.org/logilab/pylint/issue/121/search-pylintrc-in-config-directory

Regards,
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141021/bba6dd0f/attachment.html>

From kay.hayen at gmail.com  Wed Oct 29 15:00:15 2014
From: kay.hayen at gmail.com (Kay Hayen)
Date: Wed, 29 Oct 2014 15:00:15 +0100
Subject: [code-quality] How to detect unused PyLint declarations
Message-ID: <CAMEZBD+CDoe_OkibSVf0Lv0YD9wUrhD_R5kX50N785Ai=i_Ajg@mail.gmail.com>

Hello,

I have coding rules that require me to annotate exceptions to rules
for PyLint, but occasionally it happens that I find PyLint rules disabled
that would no longer be necessary.

Is there a way or script, or anything to detect these automatically? I
was thinking of writing something that removes PyLint disablers one
by one, and checks if that doesn't generate PyLint warnings, and
warn about those. Didn't do it so far, but I feel tempted to do this
now.

However, to PyLint, this might be way more easier to implement, and
maybe it was done. I cannot find anything in the manpage though.

Yours,
Kay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141029/97f03cbd/attachment.html>