[SciPy-Dev] Question about subpackage/submodule API

[josef.pktd at gmail.com]

On Sat, Feb 12, 2011 at 1:53 PM, Warren Weckesser
<warren.weckesser at enthought.com> wrote:
> The recent discussion about what to call the module in
> Pim Schellart's nice Lomb-Scargle contribution brought up
> a question for me that I have not seen discussed before.
> In many cases, a subpackage contains many modules,
> and the assorted functions and classes in these modules
> are made available in the (sub)package-level namespace
> by importing them from within __init__.py.  For example,
> scipy/stats contains distributions.py, kde.py, morestats.py,
> mstats.py, rv.py, stats.py and vonmises.py.  Except for
> mstats.py, the contents of all these are imported into
> the __init__.py namespace and included in __all__, so
> users can say, for example,
>
>>>> from scipy.stats import bayes_mvs
>
> instead of
>
>>>> from scipy.stats.more_stats import bayes_mvs
>
> Is this an *intentional* definition of an API?  For example,
> is it "wrong" for a user to refer to the submodule 'more_stats'
> explicitly when importing?  I think the answer is yes, because
> I don't think any promise is being made that the submodule
> won't be renamed or refactored as scipy development
> progresses.  But I'd like to be sure that that is everyone
> else's understanding.
>
> If that is not the case, then changes that I have made in the
> past have violated the deprecation policy.  For example, a
> while back I moved the functions in signal/filter_design.py
> that were related to FIR filters to their own module,
> fir_filter_design.py.  By making appropriate changes to
> signal/__init__.py, all the function were still importable
> from scipy.signal.  So, from my point of view, I didn't
> change the public API and no deprecation was necessary.
> But if scipy.signal.filter_design is part of the public
> API, then I should have made sure that something like
>
>>>> from scipy.signal.filter_design import firwin
>
> still worked after the refactor.
>
> Note: I'm not saying that *all* submodules should be private
> and have their objects exposed only through __init__.py.
> I'm just looking for some clarification of existing
> policy.

I haven't seen a discussion before either. I think the main reason to
rename any modules is if there are not all objects or functions
exposed in the namespace for the subpackage. I would never try to
rename stats.distributions, because even though the distribution
instances are exposed in stats, the classes itself are not. When we
added mstats to the import I added mstats as import module fore the
actual modules, as a sub namespace.

Some modules have unexposed helper function that are sometimes useful
for users to use.

On the other hand, I don't think there is many import by users that is
directly from stats.stats or stats.morestats or stats.mstats_extras.

I think until scipy 1.0 there is still more streamlining in the scipy
structure necessary, but I think we should move to a policy that also
internal module reorganization is deprecated, even if this wasn't so
in the past. If I remember correctly, there were other cases besides
your changes.

As a related aside: If we are ever allowed to make bigger changes,
then I'd love to break up scipy.stats. I find the import time for
scipy.stats (and the accompanying kitchen sink) pretty awful.

Josef


>
> Warren
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>