[Distutils] wheels on sys.path clarification (reboot)

[Brett Cannon]

On Thu, Jan 30, 2014 at 12:46 PM, Evgeny Sazhin <eugene at sazhin.us> wrote:

> On Thu, Jan 30, 2014 at 11:59 AM, Donald Stufft <donald at stufft.io> wrote:
> >
> > On Jan 30, 2014, at 11:33 AM, Evgeny Sazhin <eugene at sazhin.us> wrote:
> >
> >> On Thu, Jan 30, 2014 at 3:09 AM, Paul Moore <p.f.moore at gmail.com>
> wrote:
> >>> On 30 January 2014 05:33, Evgeny Sazhin <eugene at sazhin.us> wrote:
> >>>>> Eh, I think both 1 and 3 are things that are possibly reasonable to
> happen and
> >>>>> they are both things that I've contemplated as things to bring
> forward in
> >>>>> using xz as an alternative compression format. Even if #1 would need
> a major
> >>>>> revision of Wheel to happen adding official "support" for zip import
> means that
> >>>>> the change would have to be weighed against also breaking that
> backwards
> >>>>> compatibility.
> >>>>
> >>>>
> >>>> Eh, please don't break it!! Improve! Keep the ability to add wheels
> to python path.
> >>>> I don't care how, zip import, non-zip import, whatever - allow for
> jar-like behavior - people will thank you if it will work properly !;)
> >>>> I know, i'm probably alone in the desert...l;)
> >>>
> >>> This is the biggest concern I see with "promoting" wheels being
> >>> directly importable via zipimport (I say "promoting" and not
> >>> "specifying" deliberately, but I don't want to get back into the
> >>> process debate).
> >>>
> >>> People like Evgeny (no offense intended here, but your post is a good
> >>> example to use) coming from a Java background will see importable
> >>> wheels and *think* that they are similar in purpose to jars.
> >>
> >> No offense taken - i'm happy to be a use case to add some perspective;))
> >>
> >>> This is
> >>> not, and never will be, true. Python is not Java, deployment by
> >>> putting jar-like files on sys.path is not an advisable workflow for
> >>> Python - no matter what parallels with Java might make it look
> >>> attractive.
> >>
> >> For pure python wheels - you're right - very attractive! And while i
> >> hear you guys saying it is not advisable to use wheels in sys.path i'm
> >> still far from seeing good reasoning. Please, correct me, but so far i
> >> saw 2:
> >>
> >> 1. It is impossible to load C extensions
> >> 2. Error handling is bad
> >>
> >> None of these reasons seems to be good enough to lead to a conclusion
> >> that wheel used as jar-like thing is not needed. Here is how i see it
> >> (again probably to my lack of knowledge):
> >>
> >> 1. Yes there is a group of "elite" interesting projects that are using
> >> C extensions together with python code. What i don't understand is why
> >> are wheels supposed to be tailored to the needs of the hybrid projects
> >> instead of making pure python first priority? There is big crowd using
> >> C extensions, but how much bigger is the crowd who doesn't? Why law
> >> abiding pure python developer should jump through the hoops?
> >
> > Because C extensions are not "elite", they are pretty common. It's
> somewhat
> > rare in my experience to see a non trivial project that doesn't have a C
> > extension somewhere in it.
>
> Isn't this a self contradictory statement? C extensions are quite
> common for non-trivial project. non-trivial is not common.
> Doesn't it mean that C extensions are rarely used for common trivial
> stuff? You also implying that there were non-trivial projects that
> didn't need C extensions. Or are you trying to make an argument that
> the overall majority of python projects have C extensions?


He's trying to argue that enough projects use C extensions that to have
special support for projects that don't use them is supporting a special
case. The Zen of Python says "Special cases aren't special enough to break
the rules" and explicitly having wheels support direct execution only when
they contain pure Python is supporting a special case. Obviously you can
argue over at what point something stops being a "special case", but
considering venvs isolate code well and are the officially supported way to
isolate code then that makes stuff that goes passed what venv can do a
special case.


> Is python
> such a bad language it cannot deal with simple stuff without C being
> involved?
>

The use of a C extension does not necessarily mean a shortcoming of Python.
People can write performance-critical code in C so as to get that last bit
of speed in heavily executed code in a tight loop, etc. It has nothing to
do with Python not being able to handle something and more that C code can
simply be faster when coded correctly; it's a feature and not a band-aid.

-Brett


>
>
> >
> >>
> >> 2. IF we are talking about error handling it seems that people
> >> generally do not consider jar-like behavior to be bad for pure python
> >> modules - It's just doesn't work very well (yet?). Is there really a
> >> problem with error handling for pure python? What if the error
> >> handling is improved? Is it still going to be bad? Why?
> >
> > i consider the fact that they'll only work in a subset of situations bad,
> > because it's non obvious without digging through the code if any
> > particular project will support working from a place other than
> > a traditional install.
>
> The necessity to dig through the code to understand if the project is
> runnable from zip is a responsibility of project provider - i.e. poor
> documentation.
> If the project is designed to be run that way the doc should say so.
>
>
> >This isn't a new feature, Eggs have had this for
> > a long time and when confronted with bugs caused by zipped Eggs
> > the common solution is simply adding zip_safe=False to their project
> > which disables zipped eggs for that project.
> >
> > The name for the kind of feature this is, is an "attractive nuisance".
> They
> > look like something that might work, especially to people coming to
> > Python from other languages such as Java, but if actually use the
> > feature in a non trivial fashion you're going to end up with more
> > problems then you've solved when random projects simply don't work
> > and the likelihood that their authors have no desire to support running
> > through a zipped file even if they *could* make it possible.
>
> OK, you're saying " if actually use the feature in a non trivial fashion"
> But was is non-trivial in putting pure python wheel in sys.path?
> Non-trivial would probably
> mean trying to load c extension from it, but that's obviously not
> supported,
> so i wouldn't even try.
>
>
> Thanks,
> Eugene
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140130/8ecc866e/attachment.html>