[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Jason R. Coombs]

Jason R. Coombs <jaraco at jaraco.com> added the comment:

Although it feels the topic has shifted from the original concern (an unintentional incompatibility) to the broader topic of the API change to entry_points generally, I'm happy to address your comments:

> - I don't think they were discussed thoroughly, and when opposition was presented it was not listened to thoroughly: https://github.com/python/importlib_metadata/pull/278

Anyone reading that thread will see that I was responsive to your concerns, adapted the solution based on your concerns, and spent extra energy documenting the motivations for the change and exploring solutions until I came up with something that I believed would address the concerns and has since largely borne out that goal in the release.

> - the change significantly complicates importlib.metadata with lots of sneaky types (they all look like builtin types but do not act like them)

The solution that I ultimately settled on in python/importlib_metadata#278 does not use "sneaky" or "cutsey" types, but in fact reduces the number of magic types. EntryPoints is a tuple subclass, EntryPoint is a namedtuple subclass (with __iter__ magic deprecated), and SelectableGroups is a compatibility shim to be removed. Which types are sneaky and look like built-in types but do not act like them?

> - it simultaneously introduces new apis and old apis which will both be around for extended periods of time but immediately moves to DeprecationWarning

Given that backports are available, I saw no strong reason to delay the DeprecationWarning. I was open to the possibility that the transition could prove too taxing and the deprecation would have to be delayed. This early exposure means that most projects will already have addressed the deprecation concerns prior to the release of Python 3.10. In https://github.com/python/importlib_metadata/issues/298, the SQLAlchemy team helped me understand a nuanced use-case I hadn't considered, where libraries don't necessarily have the luxury of requiring newer backport releases, but in that case, I developed a solution that would provide future compatibility even with older stdlib and backport releases (through backports.entry_points_selectable). As far as I understand, there are no known use-cases that aren't satisfied by this design.

> - the new apis aren't remarkably better than the old apis -- the motivation was written as "typing issues" but without citing actual issues. in fact, the new items are significantly more complicated to type properly

The typing issues were discussed here (https://github.com/pypa/twine/pull/728#pullrequestreview-573342005) in the PR referenced by python/importlib_metadata#278. The magic casting of a two tuple to an item of a dict was found to be incompatible and unsupported by mypy (https://github.com/python/mypy/issues/9938). I further expanded on the motivations that led to this approach in https://github.com/python/importlib_metadata/issues/282 and https://github.com/python/importlib_metadata/issues/284.

So while Twine is able to declare types properly using this new design, it was infeasible to do so in the old design. After the compatibility layers are removed, `entry_points` will simply return an `EntryPoints` object, which presents an iterable of `EntryPoint` objects but with some facilities for selection. It's straightforward and clean. Please demonstrate the complication you see with the current approach.

The new APIs are not only easier to describe with types, but they are easier to describe in documentation.

> - the change breaks many significantly important projects, from perusing related issues it's at the very least flake8, pandas, virtualenv, astropy, pytest, hypothesis -- and hundreds more from a quick github code search

I'm not aware of a single breakage. The code is compatible for all known use-cases, but does present a DeprecationWarning in some usages. In the case of flake8, I've proposed a solution to avoid the DeprecationWarning and move to the preferred design, even without requiring an updated importlib_metadata backport.

I'm invested in providing as seamless a transition as possible to all projects, and I believe this change accomplishes that. Please let me know if there's a project or application where that's not the case.

> personally I think [all API changes] should be reverted 

Given the amount of adoption already, reverting these changes is likely to cause more disruption than moving forward with them.

If you would like to see the changes reverted or adapted further, please provide an example of a use-case that's broken by the current approach. So far, the only example presented above appears contrived and not an actual concern. It's common for Python minor releases to introduce undocumented changes that are technically incompatible but in ways that affect few or no users.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44246>
_______________________________________