[code-quality] Searching multiple setup.cfg files for flake8 section?

[Willy Wu]

On Tue, Jul 7, 2015 at 9:16 PM, Ian Cordasco <graffatcolmingov at gmail.com>
wrote:

> On Tue, Jul 7, 2015 at 12:56 PM, Willy Wu <wwu at dropbox.com> wrote:
> > Hey Ian, answers inline:
> >
> > On Tue, Jul 7, 2015 at 4:36 PM, Ian Cordasco <graffatcolmingov at gmail.com
> >
> > wrote:
> >>
> >> Hey Willy,
> >>
> >> Responses in-line:
> >>
> >> On Tue, Jul 7, 2015 at 4:05 AM, Willy Wu via code-quality
> >> <code-quality at python.org> wrote:
> >> > Hey all,
> >> >
> >> > We use flake8 at Dropbox and are pretty happy with it.  But one small
> >> > issue
> >> > is the way that flake8 configuration gets discovered.  We use the
> >> > per-project configuration via setup.cfg files which are checked into
> >> > source
> >> > control, but flake8 only considers the first setup.cfg in the
> directory
> >> > path
> >> > (regardless of whether it contains a flake8 section or not).  This
> >> > forces us
> >> > to duplicate our [flake8] configuration across all of our various
> >> > setup.cfgs.
> >> >
> >> > Said in crude ascii form, this is a simplified version of our project
> >> > layout:
> >> > - repo root
> >> >   |- setup.cfg (default values)
> >> >   \- subservice #1
> >> >     \- setup.cfg (which has non-flake8 overrides for subservice #1)
> >> >   \- subservice #2
> >> >     \- setup.cfg (which similarly contains non-flake8 overrides for
> >> > subservice #2)
> >>
> >> So... to be clear... you're putting all of your Flake8 configuration
> >> in "repo root/setup.cfg" but you have other setup.cfg files in
> >> sub-directories. How are you running Flake8 (or perhaps, where are you
> >> running Flake8) such that it picks up the setup.cfg files in the
> >> sub-directories? Am I correct in assuming that you are running it in
> >> that sub-directory? If not, can you give me the command-invocation
> >> that triggers this?
> >
> > Flake8 actually gets run automatically by our lint tools during code
> diffs,
> > but ultimately the command that runs is:
> > $ flake8 /Users/name/{repo_root}/{path_to_file}
> >
> > But the current working directory during the command that kicks off
> flake8
> > is {repo_root}, so I was hoping that flake8 would search anywhere between
> > the file's directory and the current working directory when trying to
> > discover various setup.cfgs to consider.
>
> Yeah I'm not sure that's desirable behaviour. We're then checking
>
> $HOME (and variations of that based on operating system, and best
> practice on that operating system)
> $CWD
> all paths between $CWD and $(dirname $file).
>
> The problem is that only works for invocation of a single file. Given
> your project layout, you could never just run
>
> $ flake8 .
>
> Flake8 would pick one configuration file over the other (probably
> based on alphabetical order). As you can probably see from later
> messages in this thread, our configuration discovery/selection is
> already a mess and very very complicated (and looking as if it will
> already get worse). We can't introduce that behaviour simply because
> you're only running it against a single file. That would be worse than
> a nightmare.
>

So our flake8 might be a simpler case, where our lint tool invokes flake8
on every file in the diff individually rather than a bunch of files at
once.  Am I right to guess that if you do "flake8 .", then flake8 picks a
single per-project config file to apply across all files?


>
> > I think the source of our confusion is that we use py.test as our unit
> test
> > runner, and they only consider setup.cfg a match if it contains a
> [pytest]
> > section, so there's kind of a tension between how you guys and pytest are
> > resolving configs.
>
> Except it's much more in line with how tox resolves configs. We can
> find points and counter-points but saying X should behave like Y is
> not a discussion worth having unless Y is a canonical implementation
> of something (and there really is no canon in this kind of
> configuration discovery/selection for Python tooling that looks in
> several places).
>
> > Let me know what else you need?
>
> Which setup.cfg has the Flake8 config? Is it
>
> - {repo_root}/setup.cfg
> - {repo_root}/{subservice_x}/setup.cfg
> - Something else?
>

{repo_root}/setup.cfg is originally the file w/ flake8 configs, and it's
where we were hoping to consolidate toward.


> Also, are each of the subservices actually their own packages or are
> you simply using setup.cfg to hold config for tooling? If they are
> separate packages there are different practices that could be followed
> (separate repositories, etc.) but, again, that's not my place.
>

Each of the subservices is a separate service that gets deployed to its own
cluster in prod.  We actually used to do separate repos for separate
services, but lately we wanted to do away with the overhead of lots of
little repos so we've been merging repos.  This issue with multiple
setup.cfgs is the fallout of the repo merging.  :)


> If your CI is so inflexible that you have to run flake8 from
> repo_root, is it also so inflexible that you can't provide a path to
> the setup.cfg?
>
> Also, do the flake8 settings in each sub-service differ? If not, why
> can't they all be kept in the single setup.cfg in repo_root? I would
> imagine you'd want to enforce quality over an entire repository, not
> selectively over subparts of it. Other large projects have
> per-repository configs, not per-subproject configs. Wikimedia,
> OpenStack, and a few others all do it per project. Some have
> sub-packages that are in the repository and released from there, but
> they all observe the same flake8 settings.
>

The flake8 settings in the subservices generally don't differ, which is why
we were hoping to consolidate their declarations everywhere.  It's just
that the various subservices are overriding service-specific settings to
different values in their respective setup.cfgs.

Did you mention that flake8 can support a --config parameter that overrides
per-project config discovery?  Since one source of the issue is that
multiple different projects are all using the same setup.cfg file, a
reasonable workaround might be to invoke flake8 with
--config={repo_root}/flake8.cfg which would ideally have the [flake8]
section that we're looking for.


> That said, I wonder if we stop searching for a config file when we've
> found one of {setup.cfg, tox.ini, .flake8} even if it doesn't have a
> flake8 section. That seems like a bug in pep8's config file discovery
> and parsing. That still wouldn't help you though. =/
>

Yeah, that's exactly the problem - that we stop searching, even if the
discovered file doesn't have a flake8 section.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20150707/a0cab23b/attachment.html>