[Python-Dev] LZMA compression support in 3.3

[Dan Stromberg]

On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum <guido at python.org> wrote:

> On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg <drsalists at gmail.com>
> wrote:
> > IMO, we really, really need some common way of accessing C libraries that
> > works for all major Python variants.
>
> We have one. It's called writing an extension module.
>

And yet Cext's are full of CPython-isms.

I've said in the past that Python has been lucky in that it had only a
single implementation for a long time, but still managed to escape becoming
too defined by the idiosyncrasies of that implementation - that's quite
impressive, and is probably our best indication that Python has had
leadership with foresight.  In the language proper, I'd say I still believe
this, but Cext's are sadly not a good example.


> ctypes is a crutch because it doesn't realistically have access to the
> header files.


Well, actually, header files are pretty easy to come by.  I bet you've
installed them yourself many times.  In fact, you've probably even
automatically brought some of them in via a package management system of one
form or another without getting your hands dirty.

As a thought experiment, imagine having a ctypes configuration system that
looks around a computer for .h's and .so's (etc) with even 25% of the effort
expended by GNU autoconf.  Instead of building the results into a bunch of
.o's, the results are saved in a .ct file or something.  If you build-in
some reasonable default locations to look in, plus the equivalent of some
-I's and -L's (and maybe -rpath's) as needed, you probably end up with a
pretty comparable system.

(typedef's might be a harder problem - that's particularly worth discussing,
IMO - your chance to nip this in the bud with a reasoned explanation why
they can't be handled well!)

It's a fine crutch for PyPy, which doesn't have much of
> an alternative.


Wait - a second ago I thought I was to believe that C extension modules were
the one true way of interfacing with C code across all major
implementations?  Are we perhaps saying that CPython is "the" major
implementation, and that we want it to stay that way?

I personally feel that PyPy has arrived as a major implementation.  The
backup program I've been writing in my spare time runs great on PyPy (and
the CPython's from 2.5.x, and pretty well on Jython).  And PyPy has been
maturing very rapidly ('just wish they'd do 3.x!).

It's also a fine crutch for people who need something
> to run *now*. It's a horrible strategy for the standard library.
>

I guess I'm coming to see this as dogma.

If ctypes is augmented with type information and/or version information and
where to find things, wouldn't it Become safe and convenient?  Or do you
have other concerns?

Make a list of things that can go wrong with ctypes modules.  Now make a
list of things that can wrong with C extension modules.  Aren't they really
pretty similar - missing .so, .so in a weird place, and especially: .so with
a changed interface?  C really isn't a very safe language - not like
http://en.wikipedia.org/wiki/Turing_%28programming_language%29 or
something.  Perhaps it's a little easier to mess things up with ctypes today
(a recompile doesn't fix, or at least detect, as many problems), but isn't
it at least worth Thinking about how that situation could be improved?

If you have a better proposal please do write it up. But so far you
> are mostly exposing your ignorance and insisting dramatically that you
> be educated.
>

I'm not sure why you're trying to avoid having a discussion.  I think it's
premature to dive into a proposal before getting other people's thoughts.
Frankly, 100 people tend to think better than one - at least, if the 100
people feel like they can talk.

I'm -not- convinced ctypes are the way forward.  I just want to talk about
it - for now.  ctypes have some significant advantages - if we can find a
way to eliminate and/or ameliorate their disadvantages, they might be quite
a bit nicer than Cext's.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110827/859fea4e/attachment.html>