[code-quality] Searching multiple setup.cfg files for flake8 section?

[Ian Cordasco]

On Tue, Jul 7, 2015 at 8:48 AM, Ian Lee <ianlee1521 at gmail.com> wrote:
>
>
> On Tuesday, July 7, 2015, Ned Batchelder <ned at nedbatchelder.com> wrote:
>>
>>
>>
>> On 7/7/15 5:05 AM, Willy Wu via code-quality wrote:
>>
>> Hey all,
>>
>> We use flake8 at Dropbox and are pretty happy with it.  But one small
>> issue is the way that flake8 configuration gets discovered.  We use the
>> per-project configuration via setup.cfg files which are checked into source
>> control, but flake8 only considers the first setup.cfg in the directory path
>> (regardless of whether it contains a flake8 section or not).  This forces us
>> to duplicate our [flake8] configuration across all of our various
>> setup.cfgs.
>>
>> Said in crude ascii form, this is a simplified version of our project
>> layout:
>> - repo root
>>   |- setup.cfg (default values)
>>   \- subservice #1
>>     \- setup.cfg (which has non-flake8 overrides for subservice #1)
>>   \- subservice #2
>>     \- setup.cfg (which similarly contains non-flake8 overrides for
>> subservice #2)
>>
>> But before I go off trying to patch flake8 code, is there relevant
>> background on why you guys don't try to discover the first setup.cfg with a
>> flake8 section, but instead just consider the first encountered setup.cfg?
>> We've seen teams accidentally break flake8 checking whenever they add
>> setup.cfg files for non-flake8 projects, and the "fix" is to copy/paste the
>> flake8 section into those new files.
>>
>> Or alternatively, should we be doing something different with our project
>> layout as a workaround?  It's entirely possible that we should be
>> configuring things differently overall...?
>>
>>
>> I'm interested in this question for coverage.py.
>
>
> This has been the bulk of the issue with solving this problem in pep8 as
> well. Namely answering the question of which setup.cfg files to read, when
> to stop (after reading the first with a particular section, or merge all
> found?) and also whether per user or per project settings should take
> precedence (additionally whether a config file specified with --config
> should be the only configuration read or whether it should be merged with
> per user and per project config, or be the only configuration used
> (similarly for individual command line options).
>
> I also agree with Ian's email asking for more information about your
> specific use.


So I'll answer Ned's question and this email at once. To Ned's
question of "Does setuptools or pip use all of the setup.cfg files it
can find?" (paraphrased) No. To the best of my knowledge, it uses the
one in the current directory only if it is there.

To expound on Ian's point, (not confusing at all that there are two
Ian's working on complimentary projects responding to the same thread)
Flake8 has a cap on pep8's version specifically because in 1.6.0 pep8
tried to solve some user confusion over this exact topic.

This broke config discovery and such with flake8 which caused a myriad
of bug reports which lead to the cap (as well as Flake8's strategy to
cap acceptable versions pre-emptively which lead to the thread with
the macports redistributor).

---

Now for something a little off-topic:

In my honest opinion, my mental model for per-user global configs
interacting with per-project configs is this:

- Read per-user global config
- Read per-project global config (notice the singular noun here)
- Merge the per-project into the per-user so that per-project takes
precedence. Reasoning: if I have, for my personal development
environment, a rule ignored that I don't like but I'm contributing to
a project that doesn't ignore it, pep8/flake8 should raise a warning
locally so that the CI jobs for that project do not have to tell me.

I think that's how most people expect that interaction to work. The
part where people's models differ is when having one or both of the
above AND specifying --config.

In this case, I think we should look at how we behave in similar situations:

If my user global config has one list of ignored errors and warnings,
and the project config has another, and I specify --ignore=... on the
command-line, then --ignore on the command-line wins and that's what
everyone expects. It doesn't follow very naturally, but as such, I'd
expect --config to be the only source of truth in that case. Why?
Because if you don't specify it explicitly, you want Flake8 to search
for other config files. When you specify it explicitly, you're
overriding that behaviour, just like when you specify --ignore
explicitly.

Here comes the fun and most convoluted part, let's say you have one or
both of (per-user global config, per-project global config) and (for
example) --ignore passed on the cli and --config passed on the CLI.
How does that win? Well, let's think of it like this:

--config wins over (per-user, per-project) configs (assuming we agree
with my perspective above)
--ignore wins over any configuration

So in that case, --ignore is respected over any configuration for
ignore in the config file passed in with --config, but the rest of it
is respected.

Everyone who is confused and has a headache now, raise your hand. *raises hand*

For what it's worth, I don't want this in a new pep8 release just yet.
And if it goes into one, it should (in my opinion) go into a pep8
2.0.0 release. This is a huge change that breaks the way pep8 works.
As such, other projects that either consume it or build upon it would
need time to properly prepare for this change. It should also be
documented extensively.