[SciPy-dev] requesting feedback on/editing of scikits wiki-page

[Alexander Schmolck]

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> I was just saying that if the only drawback of having a "fake" package 
> is that it is not updated, it should not be too difficult to have 
> something in place to test it automatically.

Yes, but I'd personally attach far higher priority to fixing mlabwrap if a
test reveleaed a setup.py problem than fixing the scikits example package.
However, I do agree that such a package would be useful provided there is
someone or a group of people who'll take care of it, so if you feel like
writing it, and don't think maintenance will be a problem, by all means do --
and if you like we can also replace all the mlabwrap examples on the wiki with
``scikits.example`` (or whatever you want to call it).

>> That's unfortunate -- I find numpy.distutils pretty impenetrable, and I also
>> tend to believe that pretty much anything that's useful about it ought to
>> eventually end up in setuptools, unless it's really specific to numpy or
>> numerical python code -- IMO numpy having so much general-purpose building
>> (and testing) facilities in numpy is a bad thing and increases the barrier of
>> entry for scikits writers and other contributors of numerical python code
>> unnecessarily, because the learning curve gets steeper and the number of wrong
>> or largely equivalent choices and combinations that won't work grows.
> I agree with you here, but that's a consequence of the history of numpy, 
> I guess. 

Indeed and I'm sure that writing numpy.distutils was a good idea: distutils
has been languishing for ages without an active maintainer (IIRC) and
numpy/scipy clearly needed reasonable (and for some things even specialized)
build support. I just think that with the arrival of setuptools (which is
itself quite rich), it would be good to consolidate the two as much as
possible and move to a situation where numpy.distutils really only deals with
what can't and shouldn't be done by setuptools.

> Now, I don't find setuptools that much clearer myself. Maybe it is just
> because I have not used it extensively, but I always found setuptools (and
> distutils for that matter) quite complicated for anything not trivial.

Yes, setuptools alone is already is enough to chew on. On the other hand
setuptools really does offer extremely valuable functionality, first and
foremost CPAN like functionality (i.e. easy automatic package installation for
users) but also many really useful conveniences (such as the ``python setup.py
develop``; ``test`` also seems like a good idea) and it is officially blessed
for inclusion as distutils replacement -- so there's no way around dealing
with it anyway and it will be with us for some time.

So there seems more benefit in learning about setuptools than numpy.distutils,
especially for functionality that could also be used for non-numpy projects
(adding a dependency to numpy just so you can use numpy.distutils seems a bit
excessive, right?), which is why I tried to mostly stick to setuptools on the
wiki.

> I had to spend 2 hours to convert my project to setuptools, and for the 
> only benefit to have support for namespace. 

I can comisserate -- all the work I've done on mlabwrap off late falls into
the no-real-coding category, too.

However, I are you sure that namespace support is the only benefit? Surely its
a benefit if your users can say 'easy_install pyaudiolab' and it will just be
downloaded and installed, automatically also downloading numpy if necessary?

> And it still does not work because for some reasons, files automatically
> included by setuptools for install target are not included when using sdist
> target, whereas there is no difference between those targets with
> distutils... 

Hmm maybe a bug? I recommend asking on the distutils-list, I 've found it
quite helpful.

> And I have not tested it yet on other platforms (I am always afraid of
> windows :) )

Rightly so, as I can attest from windows-user reports from the mlabwrap-user
list.

> I don't know much about numpy.distutils, but if you take a quick look at 
> numpy/distutils, it is around 1.9 Mb, of which:
>     - 1 M of tests
>     - 350 ko for fortran (fcompiler)
>     - 300 ko for extending distutils (command)
>     - half of the rest seems pretty specific to numpy too (system_info 
> is, cpuinfo too, other I don't know).
>
> You could say that the modules of numpy/scipy fall into 3 categories wrt 
> distutils:
>     - pure python packages: not difficult to handle with distutils
>     - python packages which use their own C extension: not difficult to 
> handle with distutils.

>From what I hear setuptools makes good cross-platform support a whole lot
easier, but I can't comment on it from personal experience.

>     - python packages which depend on C/Fortran code (other libraries): 
> difficult to handle with distutils, if not impossible.

Right -- I wonder whether it might sense to include support for that in
setuptools, but it might be too specialized and better handled by
numpy.distutils.

> In my case, I use numpy.distutils to find a library and its header 
> files. As far as I know, neither distutils or setuptools offer any 
> facility for that.

Sounds like a good reason for using it then -- but wouldn't you agree that it
would be even nicer if setuptools would also offer this functionality and you
could use it for non-numpy-related projects? (not that I'd necessarily know if
setuptools did already offer something like this)

>> What happens? 
> I got the following error when importing the package (from scikits 
> import pyaudiolab)
[snipped]

I assume Robert Kern's tip solved the problem?

cheers,

'as