[SciPy-dev] requesting feedback on/editing of scikits wiki-page

[David Cournapeau]

Alexander Schmolck wrote:
> David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:
>
>> Alexander Schmolck wrote:
>>> David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:
>> Well, if we have one example with many options, I guess it should not be 
>> too difficult to make it up-to-date. For example, for the package I 
>> would like to submit to scikits, I have a makefile which builds the pdf 
>> doc using some python scripts, and try to run those scripts to make sure 
>> everything is updated.
>
> Sorry, I'm not quite clear what you are getting at, so possibly this answer
> doesn't address your point at all but basically what I was trying to say is
> that if you use some real package, chances are that unless gets abandonned,
> someone will be pretty motivated to make sure its install works properly (even
> if something in setuptools changes) -- more so than if its just a toy for the
> benefit of *others*.
I was just saying that if the only drawback of having a "fake" package 
is that it is not updated, it should not be too difficult to have 
something in place to test it automatically.

But anyway, this is a very minor point (real package vs fake package).
>
> That's unfortunate -- I find numpy.distutils pretty impenetrable, and I also
> tend to believe that pretty much anything that's useful about it ought to
> eventually end up in setuptools, unless it's really specific to numpy or
> numerical python code -- IMO numpy having so much general-purpose building
> (and testing) facilities in numpy is a bad thing and increases the barrier of
> entry for scikits writers and other contributors of numerical python code
> unnecessarily, because the learning curve gets steeper and the number of wrong
> or largely equivalent choices and combinations that won't work grows.
I agree with you here, but that's a consequence of the history of numpy, 
I guess. Now, I don't find setuptools that much clearer myself. Maybe it 
is just because I have not used it extensively, but I always found 
setuptools (and distutils for that matter) quite complicated for 
anything not trivial.

I had to spend 2 hours to convert my project to setuptools, and for the 
only benefit to have support for namespace. And it still does not work 
because for some reasons, files automatically included by setuptools for 
install target are not included when using sdist target, whereas there 
is no difference between those targets with distutils... And I have not 
tested it yet on other platforms (I am always afraid of windows :) )
>  course if you manage to figure out what parts of numpy.distutils might be
> helpful for scikits-users, and how to use them, please do share your insights
> on the wiki -- using functionality from numpy.distutils is certainly better
> than roling one's own and will likely remain necessary for some time. But I do
> think that the longer term goal of the scipy-community should be to rely more
> on setuptools and less on numpy.distutils functionality (as far as
> non-special-purpose stuff is concerned; support for fortran stuff e.g. might
> well be special-purpose enough to keep it out of setuptools).
I don't know much about numpy.distutils, but if you take a quick look at 
numpy/distutils, it is around 1.9 Mb, of which:
    - 1 M of tests
    - 350 ko for fortran (fcompiler)
    - 300 ko for extending distutils (command)
    - half of the rest seems pretty specific to numpy too (system_info 
is, cpuinfo too, other I don't know).

You could say that the modules of numpy/scipy fall into 3 categories wrt 
distutils:
    - pure python packages: not difficult to handle with distutils
    - python packages which use their own C extension: not difficult to 
handle with distutils.
    - python packages which depend on C/Fortran code (other libraries): 
difficult to handle with distutils, if not impossible.

In my case, I use numpy.distutils to find a library and its header 
files. As far as I know, neither distutils or setuptools offer any 
facility for that.
   
>
> What happens? 
I got the following error when importing the package (from scikits 
import pyaudiolab)

<class 'pkg_resources.DistributionNotFound'>Traceback (most recent call 
last)

/usr/media/boulot/softwares/JapSoft/pykdic/src/<ipython console> in 
<module>()

/home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/__init__.py 
in <module>()
      2 # Last Change: Wed Jan 31 03:00 PM 2007 J
      3
----> 4 from info import __doc__
      5
      6 #from pyaudioio import play as linux_player

/home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/info.py 
in <module>()
     15 License: BSD-style (see LICENSE.txt in main source directory)
     16 """
---> 17 import version as vpy
     18 version = vpy.__version__
     19

/home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/version.py 
in <module>()
      3
      4 from pkg_resources import require
----> 5 __version__ = require('scikits.pyaudiolab')[0].version
      6 #__version__ = '0.7'
      7

/usr/lib/python2.5/site-packages/pkg_resources.py in require(self, 
*requirements)
    583         """
    584
--> 585         needed = self.resolve(parse_requirements(requirements))
    586
    587         for dist in needed:

/usr/lib/python2.5/site-packages/pkg_resources.py in resolve(self, 
requirements, env, installer)
    481                     dist = best[req.key] = env.best_match(req, 
self, installer)
    482                     if dist is None:
--> 483                         raise DistributionNotFound(req)  # XXX 
put more info here
    484                 to_activate.append(dist)
    485             if dist not in req:

<class 'pkg_resources.DistributionNotFound'>: scikits.pyaudiolab

If I uncommend the line __version__ = 
require('scikits.pyaudiolab')[0].version, everything is fine.
>
> If your code doesn't have any strange dependencies and you if email it to me
> (or point me to an url with a tgz), I could try and have a brief look at it,
> if you want.
You can find it here: 
http://www.ar.media.kyoto-u.ac.jp/members/david/archives/pyaudiolab-scikits.tbz2 
(this is a tarball of the current source directory, as setup.py sdist 
does not work yet...). The only dependency is libsndfile, which is 
easily available if you use linux.

David