Build bugs in Python 2.2.1?

[Michael Hudson]

Hmm, I thought this deserved a proper reply, that I should take some
time writing, so of course I haven't got around to writing it yet.

Jonathan Hogg <jonathan at onegoodidea.com> writes:

> On 12/8/2002 14:14, in article lkvg6gmddn.fsf at pc150.maths.bris.ac.uk,
> "Michael Hudson" <mwh at python.net> wrote:
> 
> > Please don't get discouraged from working on distutils or Python's
> > build process!  But please also understand that there is a reason for
> > most of the extant stuff, and as Martin says, vague comments along the
> > lines of "it would be nice if ..." are not really helpful.
> 
> *sigh*
> 
> If I have appeared vague before, I'm sorry. I haven't come here with a
> solution because I don't have one. I tried to demonstrate where and why I
> think there is a problem and suggest a possible idea for a solution.

Well, time has passed sufficiently that I can't remember your possible
idea for a solution.  Was it to use autoconf for more or less
everything?

> In the interests of not seeming like a spoiled child (I was pretty tired
> last night when I gave up), I'll explain one more time. If someone doesn't
> understand or sees that I've missed something, please say so. I'm not trying
> to call-for-the-head-of distutils. It just doesn't work for me and here's
> why:
> 
> One of my day jobs is managing multi-platform UNIX sites. I use a system
> called Arusha for doing so. This allows me to manage packages by noting what
> kind of package it is and any important differences or exceptions that
> should be taken into account.
> 
> For most of the autoconf-style packages I install I can just specify
> something along the lines of:
> 
>     <prototype team="." name="GNU"/>
> 
> and it works. It gets built from source on each platform, installed,
> deployed onto all of the systems, and revealed into the users' namespace.
> For packages that have dependencies, for instance on OpenSSL, I can usually
> add a simple addition to this specifying the dependency and adding the
> necessary argument to allow configure to enable/find it:
> 
>     <configure>
>         <constraint>
>             <dependency type="normal" name="openssl" method="deploy"/>
>         </constraint>
>         <param name="ssldir">!field.dependencyDeployDir('openssl')</param>
>         <param name="configure_args">--with-ssl-dir=@param:ssldir@</param>
>     </configure>
> 
> This translates into an underlying shell script something along the lines
> of:
> 
>     CFLAGS=... LDFLAGS=... \
>     ./configure --prefix=... --with-ssl-dir=/._ark-deploy/openssl--0.96e
> 
> and that's all I need to do to make it work.

Hmm, it seems that the reason this works in Arusha is that Arusha has
support for these sort of things build in?  I mean there doesn't seem
to be much inherently simpler about this, it's just what everyone else
does.

> Unfortunately, life is not so simple with Python, because while having a
> configure front-end, the Python modules build process cannot be controlled
> with configure. I can't pass in the necessary CFLAGS and LDFLAGS to enable
> the SSL module to be built correctly either, because the flags are ignored
> by setup.py.

I do think setup.py should take more hints from the environment, but
I'm not sure how to do this reasonably.

Maybe it would be an idea to have *setup.py* take
--with-foo-dir=... style arguments?  Though getting them past
distutils spectactularly baroque options handling stuff might be a bit
exciting.

> To compile the SSL module I have to edit Modules/Setup. This sounds fine to
> most people, but the only way to do that in a repeatable way is to generate
> a patch. Since the patch file will vary depending on which version of
> OpenSSL I compile against (I have to be able to maintain multiple different
> versions) and possibly which platform I'm compiling for, the patch must be
> generated programmatically. The patch also varies depending on the version
> of Python so I have to maintain multiple patch templates.

Would it be any easier to patch setup.py instead?

> If someone can point out the blindingly obvious thing I've missed that will
> make all this horror go away, then please let me know.
> 
> It's quite funny in a way because Arusha is written in Python, and that's
> because I sold it to the main guy behind Arusha as being a fantastic
> language. And yet, Python is the hardest language to build using it ;-)

Well, this is partly because Python by its nature has a large and
mostly optional set of dependencies, some of which have a tendency to
hide in obscure corners of the file system (yes, I mean *you*, bsddb).

I know for a fact that *I* have no interest in converting setup.py to
the bizarre mix of m4 macros and sh that is configure.in, and *even
less* in maintaining said code.

Would being able to pass module specific arguments to setup.py make
your life easier?  You'd still have to fight the build process a bit,
but it might be less irritating than it currently is.

And to wedge a couple of comment in at the end because they didn't
seem to fit anywhere else: 

1) you realize that Python's setup.py hasn't really made your life
   harder than it was before?
2) I think that setup.py's main intent was to help the "average user",
   and whatever that means, I doubt it covers what you're up to.

Cheers,
M.

-- 
  This is the fixed point problem again; since all some implementors
  do is implement the compiler and libraries for compiler writing, the
  language becomes good at writing compilers and not much else!
                                 -- Brian Rogoff, comp.lang.functional