[code-quality] Libraries and Standardization

[Ian Stapleton Cordasco]

Hi all,

tl;dr: Two of us think it'd be good to collaborate even more and
standardize on more things between projects. We'd like your feedback.

So Claudiu and I had some time to chat while we were at PyCon. I think
much of the discussion was spurred by
https://github.com/PyCQA/meta/issues/25 and I think we came to some
solid agreement on things we'd like to see the PyCQA become and do.

We covered two main topics:

1. The joking name that I chose for our tiny organization is causing
confusion for others. For example, we have several tools that we
maintain that contribute to improving and maintaining people's code
quality. However, people often see two or more of those tools as
exclusive or as conflicting (e.g., pylint and flake8, pylint and
pycodestyle)

2. There are a lots of things that we all do similarly but just
slightly differently. It would be *fantastic* for our users if they
didn't have to learn 15 ways of doing things (I'm exaggerating of
course).


An Authority should be an Authority ... or at least act as something
approximating a single unit

The idea here is that we should do our best to explain to users that
there's no animosity between pylint and flake8. In fact, Flake8 runs
Pylint everytime we run our tests. Just like flake8 runs itself
against its own code base as well as Pylint, we should have a cohesive
story to users. There's lots of junk on the internet about why to use
one over the other.

I proposed that we might want to sell users on an approach like so: If
you've never used one of these tools before, start small. Flake8 seems
to be slightly easier for folks to get started with. Let's suggest
that as a first step kind of like sticking your toes in the water to
determine the temperature. Once you're happy with that and you want
something a little more advanced and powerful, start using Pylint.
Continue using both. They overlap but they don't entirely coincide.


You learned A, I learned B, they learned C, let's all learn from each other

The next step would be aggregating everyone's learnings. Pylint has
been around for nearly 2 decades (if I remember correctly) and Flake8
has been around for almost a decade (maybe it's been a decade? I don't
even know anymore). We have more and more tools. We all use
configuration files and we all have slightly different behaviour with
respect to how we handle them, find them, etc.

We could all standardize and document a standard for the organization.
In my opinion, that standard should be pyproject.toml (c.f.,
https://snarky.ca/clarifying-pep-518/). INI + ConfigParser is
terrible. TOML, while I don't like the fact that it hasn't been
properly specified or standardized, is at least a bit more flexible.
Further, the packaging community has chosen this and is unifying on
it. The PEP even specifically suggests people being able to use
sections named "tool.pylint" or "tool.flake8". It looks close enough
to INI that I don't think it'd be a pain for users to migrate.

I think as we make this migration, we should also consider migrating
to a simpler configuration structure - namely eliminating User
configuration (either in $HOME or in $XDG_CONFIG) as well as system
level configuration.

It makes sense that if we're standardizing on all of this for users,
we can also build libraries for standardizing things in the org: ideas
of some low hanging-fruit:

- Multiprocessing (-j/--jobs for concurrency)
- Configuration file management (e.g., format, parsing, locating, and merging)
- Naming error codes (e.g., standardizing on how pylint names things)
- Standardizing on configurability of running versus ignored checks
- And a few others

Naturally, this is just the things we discussed and we're by no means
the people making final decisions here, so we'd like you to weigh in.

Cheers,
Ian