[SciPy-Dev] Question about subpackage/submodule API

[Travis Oliphant]

The policy in the past has been that the stable API is only one level down from the scipy namespace.  

So, developers should import the name from the top level namespace.

As a subpackage grows, I could see justification for one more level in the stable API --- e.g. scipy.signal.spectral. 


But, I would be opposed to an API that is deeper than that.

Travis 

--
(mobile phone of)
Travis Oliphant
Enthought, Inc.
1-512-536-1057
http://www.enthought.com

On Feb 12, 2011, at 1:39 PM, josef.pktd at gmail.com wrote:

> On Sat, Feb 12, 2011 at 2:35 PM,  <josef.pktd at gmail.com> wrote:
>> On Sat, Feb 12, 2011 at 1:53 PM, Warren Weckesser
>> <warren.weckesser at enthought.com> wrote:
>>> The recent discussion about what to call the module in
>>> Pim Schellart's nice Lomb-Scargle contribution brought up
>>> a question for me that I have not seen discussed before.
>>> In many cases, a subpackage contains many modules,
>>> and the assorted functions and classes in these modules
>>> are made available in the (sub)package-level namespace
>>> by importing them from within __init__.py.  For example,
>>> scipy/stats contains distributions.py, kde.py, morestats.py,
>>> mstats.py, rv.py, stats.py and vonmises.py.  Except for
>>> mstats.py, the contents of all these are imported into
>>> the __init__.py namespace and included in __all__, so
>>> users can say, for example,
>>> 
>>>>>> from scipy.stats import bayes_mvs
>>> 
>>> instead of
>>> 
>>>>>> from scipy.stats.more_stats import bayes_mvs
>>> 
>>> Is this an *intentional* definition of an API?  For example,
>>> is it "wrong" for a user to refer to the submodule 'more_stats'
>>> explicitly when importing?  I think the answer is yes, because
>>> I don't think any promise is being made that the submodule
>>> won't be renamed or refactored as scipy development
>>> progresses.  But I'd like to be sure that that is everyone
>>> else's understanding.
>>> 
>>> If that is not the case, then changes that I have made in the
>>> past have violated the deprecation policy.  For example, a
>>> while back I moved the functions in signal/filter_design.py
>>> that were related to FIR filters to their own module,
>>> fir_filter_design.py.  By making appropriate changes to
>>> signal/__init__.py, all the function were still importable
>>> from scipy.signal.  So, from my point of view, I didn't
>>> change the public API and no deprecation was necessary.
>>> But if scipy.signal.filter_design is part of the public
>>> API, then I should have made sure that something like
>>> 
>>>>>> from scipy.signal.filter_design import firwin
>>> 
>>> still worked after the refactor.
>>> 
>>> Note: I'm not saying that *all* submodules should be private
>>> and have their objects exposed only through __init__.py.
>>> I'm just looking for some clarification of existing
>>> policy.
>> 
>> I haven't seen a discussion before either. I think the main reason to
>> rename any modules is if there are not all objects or functions
>> exposed in the namespace for the subpackage.
> 
> (proofreading after the send)
> I think the main reason *not* to rename a module is if it contains
> objects, functions and classes, that are not exposed in the namespace
> of the subpackage.
> 
> Josef
> 
>> I would never try to
>> rename stats.distributions, because even though the distribution
>> instances are exposed in stats, the classes itself are not. When we
>> added mstats to the import I added mstats as import module fore the
>> actual modules, as a sub namespace.
>> 
>> Some modules have unexposed helper function that are sometimes useful
>> for users to use.
>> 
>> On the other hand, I don't think there is many import by users that is
>> directly from stats.stats or stats.morestats or stats.mstats_extras.
>> 
>> I think until scipy 1.0 there is still more streamlining in the scipy
>> structure necessary, but I think we should move to a policy that also
>> internal module reorganization is deprecated, even if this wasn't so
>> in the past. If I remember correctly, there were other cases besides
>> your changes.
>> 
>> As a related aside: If we are ever allowed to make bigger changes,
>> then I'd love to break up scipy.stats. I find the import time for
>> scipy.stats (and the accompanying kitchen sink) pretty awful.
>> 
>> Josef
>> 
>> 
>>> 
>>> Warren
>>> 
>>> 
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> 
>>> 
>> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev