[code-quality] Searching multiple setup.cfg files for flake8 section?

[Ian Cordasco]

On Tue, Jul 7, 2015 at 12:56 PM, Willy Wu <wwu at dropbox.com> wrote:
> Hey Ian, answers inline:
>
> On Tue, Jul 7, 2015 at 4:36 PM, Ian Cordasco <graffatcolmingov at gmail.com>
> wrote:
>>
>> Hey Willy,
>>
>> Responses in-line:
>>
>> On Tue, Jul 7, 2015 at 4:05 AM, Willy Wu via code-quality
>> <code-quality at python.org> wrote:
>> > Hey all,
>> >
>> > We use flake8 at Dropbox and are pretty happy with it.  But one small
>> > issue
>> > is the way that flake8 configuration gets discovered.  We use the
>> > per-project configuration via setup.cfg files which are checked into
>> > source
>> > control, but flake8 only considers the first setup.cfg in the directory
>> > path
>> > (regardless of whether it contains a flake8 section or not).  This
>> > forces us
>> > to duplicate our [flake8] configuration across all of our various
>> > setup.cfgs.
>> >
>> > Said in crude ascii form, this is a simplified version of our project
>> > layout:
>> > - repo root
>> >   |- setup.cfg (default values)
>> >   \- subservice #1
>> >     \- setup.cfg (which has non-flake8 overrides for subservice #1)
>> >   \- subservice #2
>> >     \- setup.cfg (which similarly contains non-flake8 overrides for
>> > subservice #2)
>>
>> So... to be clear... you're putting all of your Flake8 configuration
>> in "repo root/setup.cfg" but you have other setup.cfg files in
>> sub-directories. How are you running Flake8 (or perhaps, where are you
>> running Flake8) such that it picks up the setup.cfg files in the
>> sub-directories? Am I correct in assuming that you are running it in
>> that sub-directory? If not, can you give me the command-invocation
>> that triggers this?
>
> Flake8 actually gets run automatically by our lint tools during code diffs,
> but ultimately the command that runs is:
> $ flake8 /Users/name/{repo_root}/{path_to_file}
>
> But the current working directory during the command that kicks off flake8
> is {repo_root}, so I was hoping that flake8 would search anywhere between
> the file's directory and the current working directory when trying to
> discover various setup.cfgs to consider.

Yeah I'm not sure that's desirable behaviour. We're then checking

$HOME (and variations of that based on operating system, and best
practice on that operating system)
$CWD
all paths between $CWD and $(dirname $file).

The problem is that only works for invocation of a single file. Given
your project layout, you could never just run

$ flake8 .

Flake8 would pick one configuration file over the other (probably
based on alphabetical order). As you can probably see from later
messages in this thread, our configuration discovery/selection is
already a mess and very very complicated (and looking as if it will
already get worse). We can't introduce that behaviour simply because
you're only running it against a single file. That would be worse than
a nightmare.

> I think the source of our confusion is that we use py.test as our unit test
> runner, and they only consider setup.cfg a match if it contains a [pytest]
> section, so there's kind of a tension between how you guys and pytest are
> resolving configs.

Except it's much more in line with how tox resolves configs. We can
find points and counter-points but saying X should behave like Y is
not a discussion worth having unless Y is a canonical implementation
of something (and there really is no canon in this kind of
configuration discovery/selection for Python tooling that looks in
several places).

> Let me know what else you need?

Which setup.cfg has the Flake8 config? Is it

- {repo_root}/setup.cfg
- {repo_root}/{subservice_x}/setup.cfg
- Something else?

Also, are each of the subservices actually their own packages or are
you simply using setup.cfg to hold config for tooling? If they are
separate packages there are different practices that could be followed
(separate repositories, etc.) but, again, that's not my place.

If your CI is so inflexible that you have to run flake8 from
repo_root, is it also so inflexible that you can't provide a path to
the setup.cfg?

Also, do the flake8 settings in each sub-service differ? If not, why
can't they all be kept in the single setup.cfg in repo_root? I would
imagine you'd want to enforce quality over an entire repository, not
selectively over subparts of it. Other large projects have
per-repository configs, not per-subproject configs. Wikimedia,
OpenStack, and a few others all do it per project. Some have
sub-packages that are in the repository and released from there, but
they all observe the same flake8 settings.

That said, I wonder if we stop searching for a config file when we've
found one of {setup.cfg, tox.ini, .flake8} even if it doesn't have a
flake8 section. That seems like a bug in pep8's config file discovery
and parsing. That still wouldn't help you though. =/