From vladimir at keleshev.com  Tue Apr  2 22:55:57 2013
From: vladimir at keleshev.com (Vladimir Keleshev)
Date: Tue, 02 Apr 2013 22:55:57 +0200
Subject: [code-quality] Common error format for static analysis tools
Message-ID: <1042251364936157@web4h.yandex.ru>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130402/c762af99/attachment.html>

From florent.xicluna at gmail.com  Tue Apr  2 23:18:30 2013
From: florent.xicluna at gmail.com (Florent)
Date: Tue, 2 Apr 2013 23:18:30 +0200
Subject: [code-quality] Pending proposals for Pyflakes
Message-ID: <CAPFzasOxSy3MChysWEMS7FFip6siLgBw6giYAaiEYexwTsw9ug@mail.gmail.com>

Hello,

there are two patches in Launchpad that I plan to merge within few days.

"import fu; del fu complains about RedefinedWhileUnused"
  https://bugs.launchpad.net/pyflakes/+bug/1162161
  https://github.com/pyflakes/pyflakes/pull/4

"ignore special locals of test frameworks"
  https://bugs.launchpad.net/pyflakes/+bug/1105525
  https://github.com/pyflakes/pyflakes/pull/5

If you are interested, you can review these issues and comment.

-- 
Florent
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130402/5398cf94/attachment.html>

From Ronny.Pfannschmidt at gmx.de  Tue Apr  2 23:22:00 2013
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Tue, 02 Apr 2013 23:22:00 +0200
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <1042251364936157@web4h.yandex.ru>
References: <1042251364936157@web4h.yandex.ru>
Message-ID: <515B4BF8.20509@gmx.de>

Hi,

I would like to rather see a format of lines of json documents

on top of that there could be a formatted output using newstyle 
formatstrings (and there could even be a default format string for your 
proposed format)

-- Ronny


On 04/02/2013 10:55 PM, Vladimir Keleshev wrote:
> Hi everyone,
> I'm very excited about the new mailing list. I wanted to share an old
> idea of mine, which is probably unoriginal, but can kick-start a discussion.
> What about all of our static analysis tools adopt a *common*
> machine-friendly error format, so that we could write editor plugins
> once, and then all of our tools would work interchangeably, and new
> tools could be adopted without need to update editor plugins.
> The proposed format could be triggered by a decided-upon option, e.g.
> --machine-friendly, with this usage-pattern:
>      usage: <tool> --machine-friendly <file>...
> The proposed format would consist of the following lines:
> </absolute/path/to/file>:<start-line>:<start-char>...<end-line>:<end-char>:
> <error-message>
> For example:
>      /home/username/git/project/project.py:65:4...65:7: Infinite `for` loop
>      /home/username/git/project/another.py:1:1...1:5: `re` imported, but
> not used
> This is very similar to what most static analysis tools (and compilers)
> do anyway. But it also adds <start-char> and <end-char> which allow for
> precise error-highlighting in the editor.
> What do you think?
> Best,
> Vladimir
>
>
> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> http://mail.python.org/mailman/listinfo/code-quality


From vladimir at keleshev.com  Tue Apr  2 23:33:26 2013
From: vladimir at keleshev.com (Vladimir Keleshev)
Date: Tue, 02 Apr 2013 23:33:26 +0200
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <515B4BF8.20509@gmx.de>
References: <1042251364936157@web4h.yandex.ru> <515B4BF8.20509@gmx.de>
Message-ID: <179451364938406@web22h.yandex.ru>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130402/e38654c8/attachment.html>

From florent.xicluna at gmail.com  Tue Apr  2 23:40:10 2013
From: florent.xicluna at gmail.com (Florent)
Date: Tue, 2 Apr 2013 23:40:10 +0200
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <1042251364936157@web4h.yandex.ru>
References: <1042251364936157@web4h.yandex.ru>
Message-ID: <CAPFzasMSWQtfjD5igVv+F_A+URp-v6L3XN0GKHA-=jahxCDbOA@mail.gmail.com>

Hello Vladimir,

I would like to give some information about what is already available
regarding some tools.

First, you might know that there's some effort to merge the output of
Pyflakes and Pep8 tools within Flake8.
Today Flake8 supports all the options of Pep8 (and can be extended for
plugins).
For example, a useful feature is to mimic the "pylint" format with `flake8
--format pylint`.

There's also a feature request which asks for JSON output for flake8.
https://bitbucket.org/tarek/flake8/issue/99

This is the current state for the Flake8 tool (and its libraries).
I don't know much about pylint and other tools.

-- 
Florent




2013/4/2 Vladimir Keleshev <vladimir at keleshev.com>

> Hi everyone,
>
> I'm very excited about the new mailing list. I wanted to share an old idea
> of mine, which is probably unoriginal, but can kick-start a discussion.
>
> What about all of our static analysis tools adopt a *common*
> machine-friendly error format, so that we could write editor plugins once,
> and then all of our tools would work interchangeably, and new tools could
> be adopted without need to update editor plugins.
>
> The proposed format could be triggered by a decided-upon option, e.g.
> --machine-friendly, with this usage-pattern:
>
>     usage: <tool> --machine-friendly <file>...
>
> The proposed format would consist of the following lines:
>
>
> </absolute/path/to/file>:<start-line>:<start-char>...<end-line>:<end-char>:
> <error-message>
>
> For example:
>
>     /home/username/git/project/project.py:65:4...65:7: Infinite `for` loop
>     /home/username/git/project/another.py:1:1...1:5: `re` imported, but
> not used
>
> This is very similar to what most static analysis tools (and compilers) do
> anyway. But it also adds <start-char> and <end-char> which allow for
> precise error-highlighting in the editor.
>
> What do you think?
>
> Best,
> Vladimir
>
> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> http://mail.python.org/mailman/listinfo/code-quality
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130402/65881c55/attachment-0001.html>

From florent.xicluna at gmail.com  Tue Apr  2 23:46:51 2013
From: florent.xicluna at gmail.com (Florent)
Date: Tue, 2 Apr 2013 23:46:51 +0200
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <CAPFzasMSWQtfjD5igVv+F_A+URp-v6L3XN0GKHA-=jahxCDbOA@mail.gmail.com>
References: <1042251364936157@web4h.yandex.ru>
	<CAPFzasMSWQtfjD5igVv+F_A+URp-v6L3XN0GKHA-=jahxCDbOA@mail.gmail.com>
Message-ID: <CAPFzasMTBqpgMa2sN9XKVVc=Qe3eBxT19kSYFu3Viz5CF_QyXw@mail.gmail.com>

Regarding the output, the Flake8 tool returns <row> and <col>.
And IIUC the pylint tool returns only <row>.

None of these tools is able to detect and return <end-line> and <end-char>.

-- 
Florent


2013/4/2 Florent <florent.xicluna at gmail.com>

> Hello Vladimir,
>
> I would like to give some information about what is already available
> regarding some tools.
>
> First, you might know that there's some effort to merge the output of
> Pyflakes and Pep8 tools within Flake8.
> Today Flake8 supports all the options of Pep8 (and can be extended for
> plugins).
> For example, a useful feature is to mimic the "pylint" format with `flake8
> --format pylint`.
>
> There's also a feature request which asks for JSON output for flake8.
> https://bitbucket.org/tarek/flake8/issue/99
>
> This is the current state for the Flake8 tool (and its libraries).
> I don't know much about pylint and other tools.
>
> --
> Florent
>
>
>
>
> 2013/4/2 Vladimir Keleshev <vladimir at keleshev.com>
>
>> Hi everyone,
>>
>> I'm very excited about the new mailing list. I wanted to share an old
>> idea of mine, which is probably unoriginal, but can kick-start a discussion.
>>
>> What about all of our static analysis tools adopt a *common*
>> machine-friendly error format, so that we could write editor plugins once,
>> and then all of our tools would work interchangeably, and new tools could
>> be adopted without need to update editor plugins.
>>
>> The proposed format could be triggered by a decided-upon option, e.g.
>> --machine-friendly, with this usage-pattern:
>>
>>     usage: <tool> --machine-friendly <file>...
>>
>> The proposed format would consist of the following lines:
>>
>>
>> </absolute/path/to/file>:<start-line>:<start-char>...<end-line>:<end-char>:
>> <error-message>
>>
>> For example:
>>
>>     /home/username/git/project/project.py:65:4...65:7: Infinite `for` loop
>>     /home/username/git/project/another.py:1:1...1:5: `re` imported, but
>> not used
>>
>> This is very similar to what most static analysis tools (and compilers)
>> do anyway. But it also adds <start-char> and <end-char> which allow for
>> precise error-highlighting in the editor.
>>
>> What do you think?
>>
>> Best,
>> Vladimir
>>
>> _______________________________________________
>> code-quality mailing list
>> code-quality at python.org
>> http://mail.python.org/mailman/listinfo/code-quality
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130402/9b8c98ec/attachment.html>

From sylvain.thenault at logilab.fr  Wed Apr  3 09:58:42 2013
From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=)
Date: Wed, 3 Apr 2013 09:58:42 +0200
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <CAPFzasMTBqpgMa2sN9XKVVc=Qe3eBxT19kSYFu3Viz5CF_QyXw@mail.gmail.com>
References: <1042251364936157@web4h.yandex.ru>
	<CAPFzasMSWQtfjD5igVv+F_A+URp-v6L3XN0GKHA-=jahxCDbOA@mail.gmail.com>
	<CAPFzasMTBqpgMa2sN9XKVVc=Qe3eBxT19kSYFu3Viz5CF_QyXw@mail.gmail.com>
Message-ID: <20130403075842.GA2576@logilab.fr>

Hi,

On 02 avril 23:46, Florent wrote:
> Regarding the output, the Flake8 tool returns <row> and <col>.
> And IIUC the pylint tool returns only <row>.

PyLint returns column information since 0.24, released almost 2 years ago.

> None of these tools is able to detect and return <end-line> and <end-char>.

None of these tool returns this information, though it would probably be possible
to return sensible values in most cases (depending on the pb detected).

-- 
Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42)
Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org

From dmalcolm at redhat.com  Wed Apr  3 20:16:33 2013
From: dmalcolm at redhat.com (David Malcolm)
Date: Wed, 03 Apr 2013 14:16:33 -0400
Subject: [code-quality] Common error format for static analysis tools
In-Reply-To: <1042251364936157@web4h.yandex.ru>
Message-ID: <1365012993.24820.47.camel@surprise>

Have you seen Firehose:
  https://pypi.python.org/pypi/firehose
?

It currently has parsers for 3 different analysis tools (for C/C++ as it
happens, though I'm sure it could support analyzers for other
languages), and my cpychecker tool has a branch that "natively" uses
Firehose, in that it generates errors by creating firehose.model objects
in memory, then writing them out to stderr (and optionally as XML).

I used this for my PyCon US 2013 talk on static analysis of Python
extension modules
http://lists.fedoraproject.org/pipermail/firehose-devel/2013-April/000030.html
specifically, I used the XML serialization format to reliably extract
warnings from a mass-rebuild of 370 packages, then used the JSON
serialization format to get them into mongodb, which is how I generated
the graphs in my talk.

I also have some report-generation code using this format:
http://lists.fedoraproject.org/pipermail/firehose-devel/2013-February/000005.html

The formats and APIs aren't yet set in stone, in case we need to make
changes for analyzers for python code.

Hope this is helpful
Dave



From graffatcolmingov at gmail.com  Thu Apr  4 15:14:56 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Thu, 4 Apr 2013 09:14:56 -0400
Subject: [code-quality] Yet another metric
Message-ID: <CAN-Kwu0tCf2XrEcLq0M3kr2Dc=tVKYLHYqqLeVA2Qd+C3=n4iA@mail.gmail.com>

So I was just introduced to pyroma
(https://pypi.python.org/pypi/pyroma,
https://bitbucket.org/regebro/pyroma) which would probably work nicely
as a plugin to flake8 or just as a good metric in general.

It doesn't exactly measure the quality of the code itself, BUT it does
measure the quality of a package which I'm sure would be useful to
some (if not many) of our users.

I had never thought of analyzing a package for quality beyond it's
source (in all candor), so I'm fairly excited about this tool.

From tarek at ziade.org  Thu Apr  4 15:22:04 2013
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 04 Apr 2013 15:22:04 +0200
Subject: [code-quality] Yet another metric
In-Reply-To: <CAN-Kwu0tCf2XrEcLq0M3kr2Dc=tVKYLHYqqLeVA2Qd+C3=n4iA@mail.gmail.com>
References: <CAN-Kwu0tCf2XrEcLq0M3kr2Dc=tVKYLHYqqLeVA2Qd+C3=n4iA@mail.gmail.com>
Message-ID: <515D7E7C.6050303@ziade.org>

On 4/4/13 3:14 PM, Ian Cordasco wrote:
> So I was just introduced to pyroma
> (https://pypi.python.org/pypi/pyroma,
> https://bitbucket.org/regebro/pyroma) which would probably work nicely
> as a plugin to flake8 or just as a good metric in general.
>
> It doesn't exactly measure the quality of the code itself, BUT it does
> measure the quality of a package which I'm sure would be useful to
> some (if not many) of our users.
>
> I had never thought of analyzing a package for quality beyond it's
> source (in all candor), so I'm fairly excited about this tool.
Interesting

Except:

...If you are checking on a PyPI package, and not a local directory or 
local package, pyroma will check the number of owners the package has on 
PyPI. It should be three or more, to minimize the "Bus factor", the risk 
of the index owners suddenly going off-line for whatever reason...

In reality, most companies/org use a single account on PyPI so you can't 
tell how many developers are behind - so this check seems unpractical.

Note that I added a similar thing in python's distutils: the check command.

That reminds me of the CheeseCake btw : http://pycheesecake.org/



> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> http://mail.python.org/mailman/listinfo/code-quality


-- 
Tarek Ziad? ? http://ziade.org ? @tarek_ziade


From graffatcolmingov at gmail.com  Thu Apr  4 15:29:25 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Thu, 4 Apr 2013 09:29:25 -0400
Subject: [code-quality] Yet another metric
In-Reply-To: <515D7E7C.6050303@ziade.org>
References: <CAN-Kwu0tCf2XrEcLq0M3kr2Dc=tVKYLHYqqLeVA2Qd+C3=n4iA@mail.gmail.com>
	<515D7E7C.6050303@ziade.org>
Message-ID: <CAN-Kwu0bs7J62abqyVyqp4F8o7mZ_6t2051xRBGd=arS+Ojw+Q@mail.gmail.com>

On Thu, Apr 4, 2013 at 9:22 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 4/4/13 3:14 PM, Ian Cordasco wrote:
>>
>> So I was just introduced to pyroma
>> (https://pypi.python.org/pypi/pyroma,
>> https://bitbucket.org/regebro/pyroma) which would probably work nicely
>> as a plugin to flake8 or just as a good metric in general.
>>
>> It doesn't exactly measure the quality of the code itself, BUT it does
>> measure the quality of a package which I'm sure would be useful to
>> some (if not many) of our users.
>>
>> I had never thought of analyzing a package for quality beyond it's
>> source (in all candor), so I'm fairly excited about this tool.
>
> Interesting
>
> Except:
>
> ...If you are checking on a PyPI package, and not a local directory or local
> package, pyroma will check the number of owners the package has on PyPI. It
> should be three or more, to minimize the "Bus factor", the risk of the index
> owners suddenly going off-line for whatever reason...
>
> In reality, most companies/org use a single account on PyPI so you can't
> tell how many developers are behind - so this check seems unpractical.
>
> Note that I added a similar thing in python's distutils: the check command.
>
> That reminds me of the CheeseCake btw : http://pycheesecake.org/

Well if it were a plugin to Flake8 it would only be checking the local
directory since Flake8 at no point (as you well know) downloads
packages for checks. The I hadn't noticed the check command in
distutils (I've never dug that far into it) and I'm looking at
CheeseCake now as well. Thanks!

From graffatcolmingov at gmail.com  Thu Apr  4 16:39:06 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Thu, 4 Apr 2013 10:39:06 -0400
Subject: [code-quality] Fwd: [Python-Dev] [Announcement] New mailing list
 for code quality tools including Flake8, Pyflakes and Pep8
In-Reply-To: <CAH_1eM07-HKPa8c_BZRx01v=Zqw9O7epc=CG9pwZ8LcHO+dp1Q@mail.gmail.com>
References: <CAN-Kwu0MoXRndfZ+nO3LU45j5MdAzD5DRz+dAUZanfmJ7+JW0w@mail.gmail.com>
	<CA+RO4nN=D8v4m0295mRfDz9ZaJkVVhU5N6iJ5Omygnt77pj04w@mail.gmail.com>
	<CAH_1eM07-HKPa8c_BZRx01v=Zqw9O7epc=CG9pwZ8LcHO+dp1Q@mail.gmail.com>
Message-ID: <CAN-Kwu1i-wQZ0LTaiOZ4jK79v0gCuvqwkSLyF_G3dw1-d2E6UA@mail.gmail.com>

Hey all,

Alfredo brought up an excellent question on python-dev following the
announcement of our new list. I don't think there are any
pyflakes/pylint/etc. checks run against CPython currently. Has anyone
thought about doing this and seeing how many errors or warnings there
are? I would hope there aren't that many, but with modules like
unittest I could see pep8 giving a lot of warnings.

---------- Forwarded message ----------
From: Charles-Fran?ois Natali <cf.natali at gmail.com>
Date: Thu, Apr 4, 2013 at 12:57 AM
Subject: Re: [Python-Dev] [Announcement] New mailing list for code
quality tools including Flake8, Pyflakes and Pep8
To: Alfredo Solano Mart?nez <asolano at icai.es>
Cc: Ian Cordasco <graffatcolmingov at gmail.com>, Python Dev
<Python-Dev at python.org>


> Are you planning to cover the code quality of the interpreter itself
> too? I've been recently reading through the cert.org secure coding
> practice recommendations and was wondering if there has is any ongoing
> effort to perform static analysis on the cpython codebase.

AFAICT CPython already benefits from Coverity scans (I guess the
Python-security guys receive those notifications). Note that this only
covers the C codebase.

cf

From tarek at ziade.org  Thu Apr  4 17:30:46 2013
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 04 Apr 2013 17:30:46 +0200
Subject: [code-quality] Fwd: [Python-Dev] [Announcement] New mailing
 list for code quality tools including Flake8, Pyflakes and Pep8
In-Reply-To: <CAN-Kwu1i-wQZ0LTaiOZ4jK79v0gCuvqwkSLyF_G3dw1-d2E6UA@mail.gmail.com>
References: <CAN-Kwu0MoXRndfZ+nO3LU45j5MdAzD5DRz+dAUZanfmJ7+JW0w@mail.gmail.com>
	<CA+RO4nN=D8v4m0295mRfDz9ZaJkVVhU5N6iJ5Omygnt77pj04w@mail.gmail.com>
	<CAH_1eM07-HKPa8c_BZRx01v=Zqw9O7epc=CG9pwZ8LcHO+dp1Q@mail.gmail.com>
	<CAN-Kwu1i-wQZ0LTaiOZ4jK79v0gCuvqwkSLyF_G3dw1-d2E6UA@mail.gmail.com>
Message-ID: <515D9CA6.80100@ziade.org>

On 4/4/13 4:39 PM, Ian Cordasco wrote:
> Hey all,
>
> Alfredo brought up an excellent question on python-dev following the
> announcement of our new list. I don't think there are any
> pyflakes/pylint/etc. checks run against CPython currently. Has anyone
> thought about doing this and seeing how many errors or warnings there
> are? I would hope there aren't that many, but with modules like
> unittest I could see pep8 giving a lot of warnings.

basically, most of the stdlib is non-pep8 ;)
>
> ---------- Forwarded message ----------
> From: Charles-Fran?ois Natali <cf.natali at gmail.com>
> Date: Thu, Apr 4, 2013 at 12:57 AM
> Subject: Re: [Python-Dev] [Announcement] New mailing list for code
> quality tools including Flake8, Pyflakes and Pep8
> To: Alfredo Solano Mart?nez <asolano at icai.es>
> Cc: Ian Cordasco <graffatcolmingov at gmail.com>, Python Dev
> <Python-Dev at python.org>
>
>
>> Are you planning to cover the code quality of the interpreter itself
>> too? I've been recently reading through the cert.org secure coding
>> practice recommendations and was wondering if there has is any ongoing
>> effort to perform static analysis on the cpython codebase.
> AFAICT CPython already benefits from Coverity scans (I guess the
> Python-security guys receive those notifications). Note that this only
> covers the C codebase.
>
> cf
> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> http://mail.python.org/mailman/listinfo/code-quality


-- 
Tarek Ziad? ? http://ziade.org ? @tarek_ziade


From florent.xicluna at gmail.com  Thu Apr  4 17:49:28 2013
From: florent.xicluna at gmail.com (Florent)
Date: Thu, 4 Apr 2013 17:49:28 +0200
Subject: [code-quality] Fwd: [Python-Dev] [Announcement] New mailing
 list for code quality tools including Flake8, Pyflakes and Pep8
In-Reply-To: <515D9CA6.80100@ziade.org>
References: <CAN-Kwu0MoXRndfZ+nO3LU45j5MdAzD5DRz+dAUZanfmJ7+JW0w@mail.gmail.com>
	<CA+RO4nN=D8v4m0295mRfDz9ZaJkVVhU5N6iJ5Omygnt77pj04w@mail.gmail.com>
	<CAH_1eM07-HKPa8c_BZRx01v=Zqw9O7epc=CG9pwZ8LcHO+dp1Q@mail.gmail.com>
	<CAN-Kwu1i-wQZ0LTaiOZ4jK79v0gCuvqwkSLyF_G3dw1-d2E6UA@mail.gmail.com>
	<515D9CA6.80100@ziade.org>
Message-ID: <CAPFzasPavYr_30mF4_gixus+WLqadCZCS97oNqc1pEdhm73DmA@mail.gmail.com>

About PEP8, see also: http://bugs.python.org/issue16960

IIRC Georg Brandl ran Pyflakes on the stdlib few years ago, and he removed
some unused imports and constructs.
Some other developers have also fixed errors reported by Pyflakes.

-- 
Florent


2013/4/4 Tarek Ziad? <tarek at ziade.org>

> On 4/4/13 4:39 PM, Ian Cordasco wrote:
>
>> Hey all,
>>
>> Alfredo brought up an excellent question on python-dev following the
>> announcement of our new list. I don't think there are any
>> pyflakes/pylint/etc. checks run against CPython currently. Has anyone
>> thought about doing this and seeing how many errors or warnings there
>> are? I would hope there aren't that many, but with modules like
>> unittest I could see pep8 giving a lot of warnings.
>>
>
> basically, most of the stdlib is non-pep8 ;)
>
>
>> ---------- Forwarded message ----------
>> From: Charles-Fran?ois Natali <cf.natali at gmail.com>
>> Date: Thu, Apr 4, 2013 at 12:57 AM
>> Subject: Re: [Python-Dev] [Announcement] New mailing list for code
>> quality tools including Flake8, Pyflakes and Pep8
>> To: Alfredo Solano Mart?nez <asolano at icai.es>
>> Cc: Ian Cordasco <graffatcolmingov at gmail.com>, Python Dev
>> <Python-Dev at python.org>
>>
>>
>>  Are you planning to cover the code quality of the interpreter itself
>>> too? I've been recently reading through the cert.org secure coding
>>> practice recommendations and was wondering if there has is any ongoing
>>> effort to perform static analysis on the cpython codebase.
>>>
>> AFAICT CPython already benefits from Coverity scans (I guess the
>> Python-security guys receive those notifications). Note that this only
>> covers the C codebase.
>>
>> cf
>> ______________________________**_________________
>> code-quality mailing list
>> code-quality at python.org
>> http://mail.python.org/**mailman/listinfo/code-quality<http://mail.python.org/mailman/listinfo/code-quality>
>>
>
>
> --
> Tarek Ziad? ? http://ziade.org ? @tarek_ziade
>
>
> ______________________________**_________________
> code-quality mailing list
> code-quality at python.org
> http://mail.python.org/**mailman/listinfo/code-quality<http://mail.python.org/mailman/listinfo/code-quality>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130404/fe716b96/attachment-0001.html>

From vladimir at keleshev.com  Mon Apr 15 20:59:46 2013
From: vladimir at keleshev.com (Vladimir Keleshev)
Date: Mon, 15 Apr 2013 20:59:46 +0200
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
Message-ID: <178071366052386@web12f.yandex.ru>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130415/499f7721/attachment.html>

From florent.xicluna at gmail.com  Wed Apr 17 11:26:16 2013
From: florent.xicluna at gmail.com (Florent)
Date: Wed, 17 Apr 2013 11:26:16 +0200
Subject: [code-quality] [ANN] Pyflakes 0.7
Message-ID: <CAPFzasNakRCZF_yUECNZphOcCM=EVQ_2cO9M_WaERpVun-7q_g@mail.gmail.com>

Hi,

I'm pleased to announce the release of Pyflakes 0.7.

It includes several fixes and enhancements:
  - Add --version and --help options.
  - Support `python -m pyflakes` (Python 2.7 and Python 3.x).
  - Add attribute `Message.col` to report column offset.
  - Do not report redefinition of variable for a variable used in a list
    comprehension in a conditional.
  - Do not report redefinition of variable for generator expressions and
    set or dict comprehensions.
  - Do not report undefined name when the code is protected with a
    `NameError` exception handler.
  - Do not report redefinition of variable when unassigning a module
imported
    for its side-effect.
  - Support special locals like `__tracebackhide__` for py.test.
  - Support checking doctests.
  - Fix issue with Turkish locale where `'i'.upper() == 'i'` in Python 2.

All the best,

-- 
Florent Xicluna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130417/79cdadb9/attachment.html>

From tarek at ziade.org  Wed Apr 17 11:33:44 2013
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 17 Apr 2013 11:33:44 +0200
Subject: [code-quality] [ANN] Pyflakes 0.7
In-Reply-To: <CAPFzasNakRCZF_yUECNZphOcCM=EVQ_2cO9M_WaERpVun-7q_g@mail.gmail.com>
References: <CAPFzasNakRCZF_yUECNZphOcCM=EVQ_2cO9M_WaERpVun-7q_g@mail.gmail.com>
Message-ID: <516E6C78.4040907@ziade.org>

On 4/17/13 11:26 AM, Florent wrote:
> Hi,
>
> I'm pleased to announce the release of Pyflakes 0.7.

looks good! thanks

btw, since they were tons of fixes since flake8 2.0 maybe we could 
release 2.1 ?

Cheers


-- 
Tarek Ziad? ? http://ziade.org ? @tarek_ziade


From graffatcolmingov at gmail.com  Wed Apr 17 14:49:34 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Wed, 17 Apr 2013 08:49:34 -0400
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <178071366052386@web12f.yandex.ru>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
	<178071366052386@web12f.yandex.ru>
Message-ID: <CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>

On Mon, Apr 15, 2013 at 2:59 PM, Vladimir Keleshev
<vladimir at keleshev.com> wrote:
> Hi Ian,
>
> I'm actually on the mailing list as well (since maybe 2 weeks ago). You
> might remember my email about proposing a common format for error messages.
> I think it's a great idea to coordinate development of our static-analysis
> tools. Adding pep257 to flake8 might be a good idea. Shall we call it
> fla257ke8? :o)
>
> But I'm not sure I understand the purpose of flake8. Wouldn't having a
> common error format between all the tools defeat the purpose of flake8? Then
> a dummy runner would suffice. Imagine a world where all static analysis
> tools have a common output format and same command-line interface and error
> conventions.
>
> Cheers,
> Vladimir

Hey Vladimir,

I'm awful with names, so I apologize for not putting your post there
together with the fact you work on PEP257.

flake8 (as soon as we update to pyflakes 0.7 which was just announced)
will have a consistent error format. (Prior to 2.0, our vendored copy
of pyflakes provided column reporting which was just introduced in
pyflakes 0.7 and was missing in every official version prior to it.)
As for the purpose of flake8, it was created to run pep8 and pyflakes
and optionally check the McCabe complexity score. It is already
extensible to check naming conventions via pep8, so extending it (most
likely through an add-on) to check doc-strings would be a logical
step.

I'm not readily familiar with pep257's error reporting, but in
skimming over the code I noticed the Error class. I'm sure if we had
to, we could monkey patch your tool (as we already monkey patch
pyflakes), to standardize the output for flake8 users. Due to the fact
we can do that (and already do), I'm not sure attempting to make a
standard way to report errors, warnings, etc. is an immediate
priority. It's certainly something to consider, but we will encounter
the issue where tools are old enough to have other tools written to
parse their output. They could add options for different styles of
reporting, but if they're anything like pep8, they already have those
options which makes this a non-issue.

Anyway, I look forward to working with you on integrating pep257 into
flake8 and I hope to hear more from you.

Cheers,
Ian

From graffatcolmingov at gmail.com  Wed Apr 17 14:50:49 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Wed, 17 Apr 2013 08:50:49 -0400
Subject: [code-quality] [ANN] Pyflakes 0.7
In-Reply-To: <516E6C78.4040907@ziade.org>
References: <CAPFzasNakRCZF_yUECNZphOcCM=EVQ_2cO9M_WaERpVun-7q_g@mail.gmail.com>
	<516E6C78.4040907@ziade.org>
Message-ID: <CAN-Kwu3GY-e76+JBWQ57kijou_Y0pqvA0NQnL2ZYKscdOqAbYg@mail.gmail.com>

On Wed, Apr 17, 2013 at 5:33 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 4/17/13 11:26 AM, Florent wrote:
>>
>> Hi,
>>
>> I'm pleased to announce the release of Pyflakes 0.7.

It looks awesome Florent!

>
> looks good! thanks
>
> btw, since they were tons of fixes since flake8 2.0 maybe we could release
> 2.1 ?

Let me look over the issues, once more and make sure nothing should be
fixed before 2.1, and then I'll reply here.

From graffatcolmingov at gmail.com  Wed Apr 17 15:04:47 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Wed, 17 Apr 2013 09:04:47 -0400
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
	<178071366052386@web12f.yandex.ru>
	<CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>
Message-ID: <CAN-Kwu1pXNibDLQCxajTYBfkm3HvGKjnMpD7ZnXb-eArm0V5Pw@mail.gmail.com>

On Wed, Apr 17, 2013 at 8:49 AM, Ian Cordasco
<graffatcolmingov at gmail.com> wrote:
> On Mon, Apr 15, 2013 at 2:59 PM, Vladimir Keleshev
> <vladimir at keleshev.com> wrote:
>> Hi Ian,
>>
>> I'm actually on the mailing list as well (since maybe 2 weeks ago). You
>> might remember my email about proposing a common format for error messages.
>> I think it's a great idea to coordinate development of our static-analysis
>> tools. Adding pep257 to flake8 might be a good idea. Shall we call it
>> fla257ke8? :o)
>>
>> But I'm not sure I understand the purpose of flake8. Wouldn't having a
>> common error format between all the tools defeat the purpose of flake8? Then
>> a dummy runner would suffice. Imagine a world where all static analysis
>> tools have a common output format and same command-line interface and error
>> conventions.
>>
>> Cheers,
>> Vladimir
>
> Hey Vladimir,
>
> I'm awful with names, so I apologize for not putting your post there
> together with the fact you work on PEP257.
>
> flake8 (as soon as we update to pyflakes 0.7 which was just announced)
> will have a consistent error format. (Prior to 2.0, our vendored copy
> of pyflakes provided column reporting which was just introduced in
> pyflakes 0.7 and was missing in every official version prior to it.)
> As for the purpose of flake8, it was created to run pep8 and pyflakes
> and optionally check the McCabe complexity score. It is already
> extensible to check naming conventions via pep8, so extending it (most
> likely through an add-on) to check doc-strings would be a logical
> step.
>
> I'm not readily familiar with pep257's error reporting, but in
> skimming over the code I noticed the Error class. I'm sure if we had
> to, we could monkey patch your tool (as we already monkey patch
> pyflakes), to standardize the output for flake8 users. Due to the fact
> we can do that (and already do), I'm not sure attempting to make a
> standard way to report errors, warnings, etc. is an immediate
> priority. It's certainly something to consider, but we will encounter
> the issue where tools are old enough to have other tools written to
> parse their output. They could add options for different styles of
> reporting, but if they're anything like pep8, they already have those
> options which makes this a non-issue.
>
> Anyway, I look forward to working with you on integrating pep257 into
> flake8 and I hope to hear more from you.
>
> Cheers,
> Ian

And after re-reading a couple Flake8 issues, I re-discovered this
project: https://github.com/fedora-static-analysis/firehose It was
suggested to us by the author itself and you'll probably find it very
interesting Vladimir. :)

I'm looking into it myself

From vladimir at keleshev.com  Wed Apr 17 15:31:36 2013
From: vladimir at keleshev.com (Vladimir Keleshev)
Date: Wed, 17 Apr 2013 15:31:36 +0200
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
	<178071366052386@web12f.yandex.ru>
	<CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>
Message-ID: <277541366205496@web22g.yandex.ru>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20130417/576e94f1/attachment.html>

From tarek at ziade.org  Wed Apr 17 15:51:45 2013
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 17 Apr 2013 15:51:45 +0200
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <178071366052386@web12f.yandex.ru>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
	<178071366052386@web12f.yandex.ru>
Message-ID: <516EA8F1.3030801@ziade.org>

On 4/15/13 8:59 PM, Vladimir Keleshev wrote:
> Hi Ian,
> I'm actually on the mailing list as well (since maybe 2 weeks ago). 
> You might remember my email about proposing a common format for error 
> messages. I think it's a great idea to coordinate development of our 
> static-analysis tools. Adding pep257 to flake8 might be a good idea. 
> Shall we call it fla257ke8? :o)
> But I'm not sure I understand the purpose of flake8. Wouldn't having a 
> common error format between all the tools defeat the purpose of flake8?

Having a common standard would be great indeed, but I am afraid every 
CLI will always have its own specifics way of displaying things - so the 
hard part
is to make sure every tool out there provide a way to get that specific 
output wether it's via a CLI option or via code (like the flake8 plugins)


 > Then a dummy runner would suffice.
> Imagine a world where all static analysis tools have a common output 
> format and same command-line interface and error conventions.

You've just described what flake8 is trying to achieve: being a dummy 
runner on the top of a bunch of tools  :)

but in order to perform this in a smart way, it needs to add a bit of 
glue to reorder the outputs etc.
It also provides a standard/stable interface for the user and other tools

So at the end, adding a new metrics can consist of simply creating a new 
flake8 plugin and
let flake8 take care of the CLI + configuration part.

Cheers
Tarek


-- 
Tarek Ziad? ? http://ziade.org ? @tarek_ziade


From dmalcolm at redhat.com  Wed Apr 17 17:41:51 2013
From: dmalcolm at redhat.com (David Malcolm)
Date: Wed, 17 Apr 2013 11:41:51 -0400
Subject: [code-quality] Code-quality & PEP257
In-Reply-To: <277541366205496@web22g.yandex.ru>
References: <CAN-Kwu1PaNTK4OUCK-pPcpo6H=jC9EeEcM6_CJTC--g6y4YN-w@mail.gmail.com>
	<178071366052386@web12f.yandex.ru>
	<CAN-Kwu3_KJ+FcF241wDFA_xXnfgo19xeTRzMgkji202BdBd-bg@mail.gmail.com>
	<277541366205496@web22g.yandex.ru>
Message-ID: <1366213311.30308.34.camel@surprise>

On Wed, 2013-04-17 at 15:31 +0200, Vladimir Keleshev wrote:
> Hi Ian,
>  
> I'm ready to assist you with integrating pep257 into flake8. Do you
> plan to vendorize it or make it a dependency? `Error` is the right
> class to look for. `__str__` formats the error. The default format
> looks like:
>  
>     path/file.py:14:4: PEP257 Class docstrings should have 1 blank
> line around them.
>  
> which looks pretty much the same as flake8's:
>  
>     coolproject/mod.py:97:1: F401 'shutil' imported but unused
>  
> except for dot in the end and no error/warning code. `check_source`
> will return a list of all errors in a string.
>  
> * * *
>  
> But still, as I was saying:
>  
> > Wouldn't having a
> > common error format between all the tools defeat the purpose of
> flake8?
>  
> About Firehose: I find it over-engineered. It is radically different
> from any current error formats. I think that only something simple can
> succeed when it comes to standards that cannot be enforced.

Hi!

I'm the creator of Firehose, so I clearly have a bias here :)

I guess the reason for the perceived complexity is that Firehose does a
lot of things:
* as well as capturing just an issue at a location in a source file, it
can also capture a trace of execution to reach the issue.  This data is
emitter by tools like clang-analyzer and my cpychecker tool, describing
a path through the source code that leads to a given problem.  (I
suspect that the current tools in flake8 don't do that level of data
output, so perhaps this complexity is redundant for your use-case)
* as well as capturing "issues" in the software under test (SUT),
Firehose can catch "failures" meaning a problem with the testing
software (e.g. a traceback, potentially associated with the line of code
in the SUT that broke the checker): if you have any of these, you don't
have full coverage of the software-under-test.   I've found this *very*
useful when doing mass-runs of my checker on large collections of code:
you can record the failures and come back to them later
* there's also the concept of "info" about the SUT: something that isn't
an "issue" per se, just data.  I use this for capturing McCabe
complexity of functions in C extension modules in cpychecker.
* potentially you may want to add CWE ID for errors, which in Firehose
is an optional field of a firehose.model.Issue.  cwe.mitre.org is a
sister project to CVE, attempting to classify the underlying kinds of
coding mistake that lead to security issues (e.g. "CWE-89" which means
"SQL Injection vulnerability", as opposed to CVE, which catalogs
specific instances of where a CWE occurred in a specific piece of code).

...and so on, this mail is probably too long already.

Also most of the firehose data model is optional.

But firehose isn't yet API-frozen (neither are the XML or JSON
serializations), so if you have ideas for improving things, let me know.

In any case, if you have a well-defined format, I'm sure we can write a
firehose.parsers module for slurping it into my data model, and a method
for getting it back out again.

Cheers; hope this is constructive
Dave

[...snip...]



From sylvain.thenault at logilab.fr  Thu Apr 18 14:50:24 2013
From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=)
Date: Thu, 18 Apr 2013 14:50:24 +0200
Subject: [code-quality] news from the Pylint front
Message-ID: <20130418125024.GE2439@logilab.fr>

Hi there,

first post to the code-quality mailing list. Among others, I've been developping and 
maintaining Pylint for the past 10 years with some fellows, and it seems this is now 
the place to be for code quality checkers, so here we go.

You may be aware, or not, that things are moving these days on the pylint front. 
Two noticeable things:

* pylint development is moving to bitbucket and away from its "parent" (Logilab) [1]

* a pylint sprint will be held in Toulouse (France) from June 17 to 19, with currently 
  2 or 3 people from Logilab and probably at least 3 from Google [2]

Following the bitbucket move, we're currently discussing about moving Pylint related
discussions from its current mailing-list (python-projects at lists.logilab.org) to this 
one. You can see from the archives [3] that this is not high traffic, but neither 
negligeable compared to this list current traffic. Of course I would like to know
opinion from people here before acting anything.

Last but not least, it would be great if some flake8 fellows would be available to 
sprint with us, so we can see how to integrate pylint there and share efforts.

Waiting for your feedbacks, cheers,

[1]?http://www.logilab.org/blogentry/129458
[2] http://www.logilab.org/133321
[3] http://lists.logilab.org/pipermail/python-projects/

-- 
Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42)
Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org

From tarek at ziade.org  Thu Apr 18 14:53:31 2013
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 18 Apr 2013 14:53:31 +0200
Subject: [code-quality] news from the Pylint front
In-Reply-To: <20130418125024.GE2439@logilab.fr>
References: <20130418125024.GE2439@logilab.fr>
Message-ID: <516FECCB.2070704@ziade.org>

On 4/18/13 2:50 PM, Sylvain Th?nault wrote:
> Hi there,
>
> first post to the code-quality mailing list. Among others, I've been developping and
> maintaining Pylint for the past 10 years with some fellows, and it seems this is now
> the place to be for code quality checkers, so here we go.
>
> You may be aware, or not, that things are moving these days on the pylint front.
> Two noticeable things:
>
> * pylint development is moving to bitbucket and away from its "parent" (Logilab) [1]
>
> * a pylint sprint will be held in Toulouse (France) from June 17 to 19, with currently
>    2 or 3 people from Logilab and probably at least 3 from Google [2]

Great news
> Following the bitbucket move, we're currently discussing about moving Pylint related
> discussions from its current mailing-list (python-projects at lists.logilab.org) to this
> one. You can see from the archives [3] that this is not high traffic, but neither
> negligeable compared to this list current traffic. Of course I would like to know
> opinion from people here before acting anything.
That would be very cool - it makes a lot of sense if we can all reunite 
in the same place

>
> Last but not least, it would be great if some flake8 fellows would be available to
> sprint with us, so we can see how to integrate pylint there and share efforts.
I cannot be physically present in Toulouse but I can probably commit 
some time online
on my side

> Waiting for your feedbacks, cheers,
>
> [1] http://www.logilab.org/blogentry/129458
> [2] http://www.logilab.org/133321
> [3] http://lists.logilab.org/pipermail/python-projects/
>


-- 
Tarek Ziad? ? http://ziade.org ? @tarek_ziade


From graffatcolmingov at gmail.com  Thu Apr 18 16:15:32 2013
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Thu, 18 Apr 2013 10:15:32 -0400
Subject: [code-quality] news from the Pylint front
In-Reply-To: <516FECCB.2070704@ziade.org>
References: <20130418125024.GE2439@logilab.fr>
	<516FECCB.2070704@ziade.org>
Message-ID: <CAN-Kwu3YnA7Thx0Mu6BcfdOFF3fqKuN7pt=4OdyFcbcfjePWAQ@mail.gmail.com>

On Thu, Apr 18, 2013 at 8:53 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 4/18/13 2:50 PM, Sylvain Th?nault wrote:
>>
>> Hi there,
>>
>> first post to the code-quality mailing list. Among others, I've been
>> developping and
>> maintaining Pylint for the past 10 years with some fellows, and it seems
>> this is now
>> the place to be for code quality checkers, so here we go.
>>
>> You may be aware, or not, that things are moving these days on the pylint
>> front.
>> Two noticeable things:
>>
>> * pylint development is moving to bitbucket and away from its "parent"
>> (Logilab) [1]
>>
>> * a pylint sprint will be held in Toulouse (France) from June 17 to 19,
>> with currently
>>    2 or 3 people from Logilab and probably at least 3 from Google [2]
>
>
> Great news
>
>> Following the bitbucket move, we're currently discussing about moving
>> Pylint related
>> discussions from its current mailing-list
>> (python-projects at lists.logilab.org) to this
>> one. You can see from the archives [3] that this is not high traffic, but
>> neither
>> negligeable compared to this list current traffic. Of course I would like
>> to know
>> opinion from people here before acting anything.
>
> That would be very cool - it makes a lot of sense if we can all reunite in
> the same place

What Tarek said x2. We made the mailing list with the hope of
consolidating conversation and having yet another place for running
ideas past each other.

>> Last but not least, it would be great if some flake8 fellows would be
>> available to
>> sprint with us, so we can see how to integrate pylint there and share
>> efforts.
>
> I cannot be physically present in Toulouse but I can probably commit some
> time online
> on my side

I don't think I'll have the cash or the time to fly in for it, but I
would definitely be up for sprinting with you guys from the US.

Welcome to the list!