[Matplotlib-devel] Reorganising the test files?
Antony Lee
antony.lee at berkeley.edu
Sun Jan 29 04:44:02 EST 2017
It looks like there are two issues at hand here.
- Should there be one test module per real module? My opinion is that the
answer is generally yes, but there are cases where this would be awkward,
so we shouldn't make it a hard rule. For example, `test_pickle.py` tests
the picklability of various objects, from simple ones to entire figures.
Of course, each of the tests could be moved to the test module
corresponding to the module where the object is defined, but I doubt this
would make things clearer.
- Should there be a `matplotlib.tests` package containing all the tests
(possibly replicating the matplotlib hierarchy) or one `.tests` subpackage
per package? Let's realize that matplotlib has a very flat hierarchy
compared to the other projects Nelle mentioned. Basically, the change
would entail moving
- test_axes to axes/tests
- test_backend_* to backends/tests
- test_cbook to cbook/tests
- test_triangulation to tri/tests
- test_style to style/tests
The other subpackages are projections and sphinxext, for which there does
not appear to be a dedicated test module. Frankly, which such a small
number of exceptions, I feel it would be more confusing to move the
corresponding tests out to their own subpackage. The situation is not the
same as in numpy, scipy, skimage, pandas, sklearn and sympy, which (I just
checked) all have more subpackages (i.e. folders) than submodules (i.e.
files) directly under the main package. (IPython has more direct
submodules, but many of them seem to be just shims left over from the big
split). For these projects, subpackages provide a clear partitioning
between parts of the code, and it is thus natural to split the tests in the
same manner.
Antony
2017-01-28 17:40 GMT-08:00 Matthias Bussonnier <bussonniermatthias at gmail.com
>:
> > numpy, scipy, skimage, pandas, sklearn, sympy, ipython all follow this
> standard.
>
> We do our best in IPython to try to "catch up" on standard, and right
> now we are not the best example.
> Like matplotlib our test harness is relatively complex (for different
> reasons than matplotlib). But we are doing
> our best to push things forward. Migrating to PyTest is one of the
> things we hope to one day achieve (or already start).
>
> "The bit split" helped, but in 2014 out tests layout was kind of horrible.
>
> I am not involved enough in matplotlib to tell you whether you should
> (or not) decide to change (that's your choice, and only core dev can
> likely correctly weight on cost/benefits balance), but IMHO, and the
> experience I have:
>
> - With IPython : non traditional package layout and test harness
> make it difficult for new/occasional contributors.
> - E.g in IPython a couple of time per year, we have people
> running `nosetest` from the root of the directory.
> - we have a specific guard against running iptest from source
> directory.
> - contributing to our test suite is harder
>
> That does not seem much, but integrating something small through time
> can lead to quite a lot.
>
> - You may not realise it but a lot of people when wondering how to
> do things will look up to "core" packages.
> - They either will not understand, potentially giving up.
> - Or will replicate a complex setup
>
> That add to the myth that "python packaging is hard", and I think that
> weighting on the overall community impact is important.
> I definitely feel a "layout context switching" when working on
> matplotlib. Test layout is not the only one, but it definitively
> interfere with my editor which smart search on nearby files.
>
> Catching up on "Standard" does not mean either changing all at once.
> AFAIU you do that for pep8 where the number of violation can only go down,
> so you can perfectly start a policy:
> - new test need to be next to thing they test.
>
> Sidenote: I don't see why a testing submodule should not be importable.
> Don't you have utility function that would be useful for others ?
> Wouldn't it bother you that only a test submodule can be imported but
> only by testing tools ?
>
> I'm going to guess "just post 2.0" might not be a good time to
> actually shuffle things around, but looking at "what would it break"
> (beyond people habits) can be a way to see a couple of assumptions
> that are not supposed to be here*, and lay out the foundation to
> moving things around if one day you decide to.
>
> Thanks !
> --
> Matthias
>
> [*] source: PyCon 2014, I deleted the test suite. IPython was refusing
> to start.
>
>
>
>
> On Sat, Jan 28, 2017 at 10:27 AM, Nelle Varoquaux
> <nelle.varoquaux at gmail.com> wrote:
> >>> Actually, it is really a pain that Matplotlib doesn't follow python
> >>> standards for any contributors that contributes to any other python
> >>> packages.
> >>
> >>
> >> very true, but...
> >>
> >> 1) If we were starting from scratcth that would be one thing, but a lot
> of
> >> thrash jsut to meet a "standard" is probably not worth it.
> >
> >
> > The benefits of following standards for maintainability has been
> > demonstrated over and over.
> >
> >>
> >> 2) I don't know how standardized this concept is, and personally, I
> don''t
> >> like it (if I understand it correctly, anyway)
> >
> >
> > It is extremely standard. Matplotlib is not the only project not
> following
> > standards, but it's probably in the 1% of projects not following
> standards.
> > More importantly, it is how we (the community) teach how to organize
> > packages and tests.
> >
> > numpy, scipy, skimage, pandas, sklearn, sympy, ipython all follow this
> > standard.
> > The only other mainstream scientific python package that may not entirely
> > follow this convention is cython.
> >
> >>
> >> I prefer that ALL tests be in one single directory tree, ideally with
> that
> >> tree mtaching the package / subpacakge/module tree of the project, ie:
> >
> >
> > The whole point of convention is to avoid this.
> >
> >>
> >>
> >> project
> >> __init__.py
> >> proj_mod1.py
> >> sub_package1
> >> __init__.py
> >> sub_mod1.py
> >> ...
> >>
> >> test_project
> >> test_proj_mod1.py
> >> test_sub_package1
> >> test_sub_mod1.py
> >>
> >> i.e. one test file for each module file.
> >>
> >> With this method, it would not be hard to find the tests that correspond
> >> to a particular python module.
> >>
> >> I'm ambivalent about whether the test files should be importable
> packages,
> >> and I think there is a lot to be said for keeping the test code outside
> the
> >> installable package tree altogether, but I definitely don't like them
> all
> >> intermingled.
> >>
> >> though I still don't advocate a lot of churn for this...
> >>
> >> -CHB
> >>
> >> --
> >>
> >> Christopher Barker, Ph.D.
> >> Oceanographer
> >>
> >> Emergency Response Division
> >> NOAA/NOS/OR&R (206) 526-6959 voice
> >> 7600 Sand Point Way NE (206) 526-6329 fax
> >> Seattle, WA 98115 (206) 526-6317 main reception
> >>
> >> Chris.Barker at noaa.gov
> >
> >
> >
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Matplotlib-devel at python.org
> > https://mail.python.org/mailman/listinfo/matplotlib-devel
> >
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel at python.org
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20170129/7a17f3fb/attachment-0001.html>
More information about the Matplotlib-devel
mailing list