[Distutils] [RFC] eggbuild

[Manlio Perillo]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Il 04/11/2010 22:01, Alan Franzoni ha scritto:
> On 11/4/10 7:25 PM, Manlio Perillo wrote:
>>> What specific problem are you trying to address?
>>
>>
>> Distribute a Python package on Windows.
>>
>> Suppose package A depends on pure Python package B and on PyQT.
>> Right now I have two options:
>>
>> 1) create an installer using PyInstaller or py2exe
>> 2) have the user install Python (not a problem), PyQt
>>   (the "all included" installer) and then easy_install my application
>>
>>
>> I think this is a big problem.
>> Even if we assume that it is ok to create a PyInstaller installer, I
>> don't see why a Windows user must not be able to do:
>>     easy_install A
>>
>> and have B and PyQt (with external libraries) correctly installed on the
>> system.
> 
> 
> OK, I got it.
> 
> I must admit that my knowledge of Windows is not advanced enough to tell
> you whether this could work or if there're obvious issues.
> 

I will do some tests on Windows when I find some free time.
But it should work.

> This could be a problem on Linux as well, by the way. If I try
> installing, let's say, lxml, and libxml2 and libxml2-dev are not
> installed on my system, easy_install will fail.
> 

Yes.

But on Linux systems at least we have package managers, and a compiler
is almost always available,

> Those are my 2c for the "weak spots":
> 
> - you need to check recursively for all the shared libraries needed by
> python modules and extensions, and all the shared libraries needed by
> those shared libraries, and so on.

This is what I do.

> Eggbuilds might end up just like huge
> chroots.
> 
> If you want to, let's say, distribute something built with PyQt on
> Linux, you'll need PyQt, the QT Runtime, then? most probably X libs,
> libc and something else. Do you *really* want to distribute everything?
>

A simple solution is to specify, in the setup.cfg file, a list of "know"
libraries that you can assume are always available on the system.


> I've just checked for your EGG-INFO/native_libs: you don't seem to
> recurse libraries. Just an example I spotted: libgnutls requires zlib,
> but no libz.so.something is included in native libs.
> 

The reason libz.so is not included is because it is already required by
libpython.

I assume that libraries required by libpython are available on the
system, and I avoid to copy them in the egg.

> - you need to check for dlopen() calls as well (or LoadLibrary/Ex on
> Windows - but I don't know whether it's easy to look for them in PE
> executables or DLLs); this might not be trivial and will increase the
> number of libraries that need to be included.
> 

I assume only few projects use dlopen/LoadLibrary/Ex.
This problem, moreover, is so complex that can not even be solved by
advanced package managers; why should I try to solve it? :)

> 
> - there might be legal implications when redistributing shared
> libraries; it might not be easy to tell which license each library has,
> and a single non-redistributable library might prevent to release the
> whole package. 

Yes, this is indeed a problem.
It is resposibility of the egg builder to check this.

> GPL and LGPL mandates source distribution along binary
> distribution, are you ready to package and make all the sources
> available to the user? Remember that, as you will be the redistributor,
> this requirement must be met by you - you can't just tell people to
> fetch sources from this or that environment.
> 


> - reproducible builds will be almost impossible - if a dep changes on
> your system, you might end up without a clear understanding of what
> happened between one build and another - apparently identical - build.

As far as I can tell, the same can happen with PyInstaller.
And the same can happen, as an example, when riverbank builds the PyQt
installer.

> You may resort to chroot-building tools like pbuilder or mock on Linux,
> but this will be far more difficult on Windows.
> 
> My idea is that everything looks far too complex, and would work easily
> only for really simple cases. Everything else will just turn
> overcomplicated.
> 

It is indeed a complex problem, but PyInstaller is far more complex.

> You're really trying to port to Python what Java does through its
> classpath system and, to a greater extent, with Osgi; but Python
> philosophy is quite different, you'll end up distributing whole OSes!
> 
> 

I don't know Java :).



Regards   Manlio

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzTLycACgkQscQJ24LbaUSdoQCeJlSzjJ+iYR8gyqLu3RwKJFfk
e4IAn0VrfK97p4p0TCsNyBxje26Qu74T
=7xgA
-----END PGP SIGNATURE-----