[Distutils] Entry points: specifying and caching

[Donald Stufft]

> On Oct 20, 2017, at 9:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 20 October 2017 at 23:19, Donald Stufft <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> One that I was helping someone debug just the other day is that they’re super non-debuggable and the behavior when you have two things providing the same entry point is basically ???? (If I remember correctly, the behavior is that the first thing found is the one that “wins”, which means the ordering of sys.path and the names of the projects supply it is what determines it). This got exposed to the end user that they installed something that they thought was going to add support for something, but which silently did nothing because two different project happened to pick the same name for their entry point (not the group, it was two things providing plugins for the same system).
> 
> While I agree with this, I think that's a combination of pkg_resources itself being hard to debug in general, and the fact that pkg_resources doesn't clearly define the semantics of how it resolves name conflicts within an entry point group - as far as I know, it's largely an accident of implementation.
> 
> The interoperability spec is going to state that conflict resolution when the same name within a group is declared by multiple packages is the responsibility of the group consumer, so documenting the format should actually improve this situation, since it allows for the development of competing conflict resolution strategies in different runtime libraries.

I think it makes it *worse*, because now the behavior isn’t just a entrypoints weirdness, but now it changes based on which runtime library you use (which isn’t something that end users are likely to have much insight into) and it represents a footgun that package authors are unlikely to be aware of. If mycoolentrypointslib comes out that is faster, but changes some subtle behavior like this it’ll break people, but that is unlikely going to be an effect that people expect to happen just because they switched between two things both implementing the same standard.

So effectively this means that not only is the fact you’re using entrypoints part of your API, but now which entry point library you’re using at runtime is now also part of your API.

>  
> Of course there is the perennial entrypoints are super slow, which is partially the fault of pkg_resources does a bunch of import time logic, but also because scanning sys.path for all installed stuff is just slow.
> 
> Similar to the above, one of the goals of documenting the entry point file format is to permit libraries to compete in the development of effective entrypoint metadata caching strategies without needing to bless any particular one a priori, and without trying to manage experimental cache designs across the huge pkg_resources install base.

That goal can be achieved if it’s documented in setuptools.

>  
> They’re also somewhat fragile since they rely on the packaging metadata system at runtime, and a number of tools exclude that information (often times things that deploy stuff as a tarball/zipfile) which causes regular issues to be opened up for these projects when they get used in those environments.
> 
> This is true, and one of the main pragmatic benefits of adopting one of the purely import based plugin management systems. However, this problem will impact all packaging metadata based plugin management solutions, regardless of whether they use an existing file format or a new one.
>  
> Those are the ones I remember because they come up regularly (and people regularly come to me with issues with any project related to packaging in any way even for non packaging related features in those projects). I’m pretty sure there were more of them that I’ve encountered and seen projects encounter, but I can’t remember them to be sure.
> 
> I’m more familiar with why console_scripts entry point is not great and why we should stop using it since I regularly try to re-read all of pip’s issues and a lot of it’s issues are documented there.
> 
> I'm sympathetic to that, but I think even in that case, clearly documenting the format as an interoperability specification will help tease out which of those are due to the file format itself, and which are due to setuptools.setup specifically.

All of the ones I’m aware of are due to the file format itself, because they exist even without setuptools being involved at all.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20171020/93f37493/attachment-0001.html>