[Distutils] wheels on sys.path clarification (reboot)

[Donald Stufft]

On Jan 30, 2014, at 11:33 AM, Evgeny Sazhin <eugene at sazhin.us> wrote:

> On Thu, Jan 30, 2014 at 3:09 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 30 January 2014 05:33, Evgeny Sazhin <eugene at sazhin.us> wrote:
>>>> Eh, I think both 1 and 3 are things that are possibly reasonable to happen and
>>>> they are both things that I've contemplated as things to bring forward in
>>>> using xz as an alternative compression format. Even if #1 would need a major
>>>> revision of Wheel to happen adding official "support" for zip import means that
>>>> the change would have to be weighed against also breaking that backwards
>>>> compatibility.
>>> 
>>> 
>>> Eh, please don't break it!! Improve! Keep the ability to add wheels to python path.
>>> I don't care how, zip import, non-zip import, whatever - allow for jar-like behavior - people will thank you if it will work properly !;)
>>> I know, i'm probably alone in the desert...l;)
>> 
>> This is the biggest concern I see with "promoting" wheels being
>> directly importable via zipimport (I say "promoting" and not
>> "specifying" deliberately, but I don't want to get back into the
>> process debate).
>> 
>> People like Evgeny (no offense intended here, but your post is a good
>> example to use) coming from a Java background will see importable
>> wheels and *think* that they are similar in purpose to jars.
> 
> No offense taken - i'm happy to be a use case to add some perspective;))
> 
>> This is
>> not, and never will be, true. Python is not Java, deployment by
>> putting jar-like files on sys.path is not an advisable workflow for
>> Python - no matter what parallels with Java might make it look
>> attractive.
> 
> For pure python wheels - you're right - very attractive! And while i
> hear you guys saying it is not advisable to use wheels in sys.path i'm
> still far from seeing good reasoning. Please, correct me, but so far i
> saw 2:
> 
> 1. It is impossible to load C extensions
> 2. Error handling is bad
> 
> None of these reasons seems to be good enough to lead to a conclusion
> that wheel used as jar-like thing is not needed. Here is how i see it
> (again probably to my lack of knowledge):
> 
> 1. Yes there is a group of "elite" interesting projects that are using
> C extensions together with python code. What i don't understand is why
> are wheels supposed to be tailored to the needs of the hybrid projects
> instead of making pure python first priority? There is big crowd using
> C extensions, but how much bigger is the crowd who doesn't? Why law
> abiding pure python developer should jump through the hoops?

Because C extensions are not “elite”, they are pretty common. It’s somewhat
rare in my experience to see a non trivial project that doesn’t have a C
extension somewhere in it. 

> 
> 2. IF we are talking about error handling it seems that people
> generally do not consider jar-like behavior to be bad for pure python
> modules - It's just doesn't work very well (yet?). Is there really a
> problem with error handling for pure python? What if the error
> handling is improved? Is it still going to be bad? Why?

i consider the fact that they’ll only work in a subset of situations bad,
because it’s non obvious without digging through the code if any
particular project will support working from a place other than
a traditional install. This isn’t a new feature, Eggs have had this for
a long time and when confronted with bugs caused by zipped Eggs
the common solution is simply adding zip_safe=False to their project
which disables zipped eggs for that project. 

The name for the kind of feature this is, is an “attractive nuisance”. They
look like something that might work, especially to people coming to
Python from other languages such as Java, but if actually use the
feature in a non trivial fashion you’re going to end up with more
problems then you’ve solved when random projects simply don’t work
and the likelihood that their authors have no desire to support running
through a zipped file even if they *could* make it possible.

> 
>> 
>> Pointing out the implication that wheels can be put on sys.path in
>> specialised circumstances is entirely reasonable when talking to a
>> Python audience, but it triggers entirely the wrong associations when
>> a person with a Java background sees the statement.
>> 
>> Evgeny - for your purposes, you'd be better advised to see wheels as
>> similar in principle to Windows MSI installers (they aren't, but it
>> will avoid triggering incorrect analogies).
> 
> That is an awesome analogy, thank you!
> 
>> I don't have good
>> references or pointers to good Python development or deployment
>> practices, but you may want to ask around on python-list.
> 
> And that is my biggest concern (deployment). I believe these questions
> should be answered by wheels or python packaging specification.
> Not by third party tooling. The language should be able to work with
> artifacts/packages produced for the language by the language. Third
> party tools may allow for automation, continuous integration
> complicated deployments and whatnot. But the base must be there.
> 
> Thanks,
> Eugene
> 
>> (Be careful,
>> you may hear suggestions that you use eggs because they are similar to
>> jars - while that is true, the way Python works means that the
>> experience with eggs is far from being as natural as the jar
>> experience you're used to).
>> 
>> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140130/5f841814/attachment.sig>