[SciPy-dev] mlabwrap scikit [Was: Re: scikits project]

[Robert Kern]

Alexander Schmolck wrote:
> Robert Kern <robert.kern at gmail.com> writes:

> Anyway, the choice between either layout (i.e. subproject internal/external
> {tags,branches,trunk}) seems not terribly important.

Fine. Please let's drop the issue, then.

>>> Maybe the structure I proposed would also make importing and exporting of
>>> projects slightly easier (because it mirrors the typical layout of an
>>> individual svn project). Speaking of which -- is there something I can do with
>>> the existing CVS so that it can be easily imported in the scikits svn (in
>>> which case we can get rid of what's already checked in), or would importing
>>> the CVS involve a hassle in any case, because then I'll just archive it on
>>> sourceforge.
>> I don't know. You'll have to read the cvs2svn documentation.
> 
> Sorry, I should have phrased this better. I have already read the cvs2svn docs
> and converted the CVS repository (just a single trunk, no branches or tags).
> If I put a tar.bz2 of that repository on the web, can someone with admin
> rights easily install it in lieu of the existing repository?

Is there no conversion tool that simply puts revisions into an existing
directory of a repository instead of making a new repository?

> Is Berkeley db
> fine as backend?

No. We use the fsfs.

>>>> If so it should also use numpy.distutils. Just make sure to import
>>>> setuptools first.
>>>>
>>>>   import setuptools
>>>>   from numpy.distutils.core import setup
>>>>   ...
>>> Is there a recipe/template for this somewhere? Googling "scipy setuptools"
>>> comes up with
>>>
>>>  <http://projects.scipy.org/ipython/ipython/wiki/SetupTools>
>>>
>>> as the first hit, which seems to indicate that setuptools is still a bit alpha
>>> and the docs can't be trusted if one wants something that actually works.
>> Fernando's a curmudgeon, and that page is old. Ignore him.  :-)
> 
> OK, I'm going the``ez_setup.py`` way.

No, ez_setup.py is deprecated. That part of the setuptools docs is out of date.
When the Cheeseshop comes back up read this page with up-to-date information
about how to get going with setuptools:

  http://cheeseshop.python.org/pypi/setuptools

>> Like I said, just import setuptools before you import numpy.distutils. Then use
>> numpy.distutils as normal to handle all of the building and stuff. setuptools
>> adds some keywords to setup() that you should also provide, namely
>>
>>   namespace_packages=['scikits'],
>>
>> That's all that's necessary. There's no particular magic to combining setuptools
>> and numpy.distutils. 
> 
> Well, it's not quite obvious how to fully take advantage of setuptools,
> though. One of the main reasons for using it is that it's meant to download
> and install depenedencies automatically, but that can't work if my setup.py
> import something from its sole dependency (numpy) to start with. Surely there
> must be some way to write packages that depend on numpy but can be installed
> automatically (and download numpy if required)?

Not really. The dependencies that you can specify are requirements for using the
package after it is installed, not requirements for building the package. The
structure of distutils setup.py files pretty much enforces this. setuptools
doesn't really get to do anything until setup() is called, i.e. after you've
used your dependencies.

You might be able to hack something together with
pkg_resources.WorkingSet.resolve(), though.

http://peak.telecommunity.com/DevCenter/PkgResources#workingset-methods-and-attributes

> And why do I need to use
> numpy.distutils in the first place?

<shrug> You don't strictly have to. numpy.get_include() is probably sufficient
for you, but it has the same problem.

> Another thing: should ``scikits/__init__.py`` be really empty? From
> 
> <http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages>
> 
> it looks a bit to me like it should contain:
> 
>   __import__('pkg_resources').declare_namespace(__name__)
> 
> ?

Yes, apologies.

> Finally, what is the preferred download url for scikits projects? Should I
> continue to host the file-release on SF, or should they go somewhere else?

I recommend putting them on the Cheeseshop.

> Same for the webpage.

scipy.org wiki if you can.

> So I think some kind of template for scikit authors would be useful and I'd
> suggest that once I've got setup.py etc. ironed out I put some info for other
> prospective scikit authors on a wikipage on scipy.org -- what would be a good
> place?

http://projects.scipy.org/scipy/scikits

> Finally, just to double check, does this directory structure look good to you: 
> 
> mlabwrap/
>    setup.py
>    README.txt
>    tests/                # N.B. renamed from 'test'
>      test_mlabwrap.py
>      ...
>    scikits/
>      __init__.py         # empty
>      mlabwrap/
>         __init__.py
>         _mlabwrap.py
>         mlabraw.py
>         -> awmstools.py
>         -> awmsmeta.py
> 
> ?

Yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco