[Pandas-dev] Unit test reorganization

Mon Jan 25 11:59:25 EST 2016

sounds good to me.

on other notes.  Planning on doing the 0.18.0 RC in say 2 weeks time.

I think that adding to_xarray to 0.18.0 is realistic, but
I think we need to push deprecating Panel to 0.19.0 simply to have some
time
for this to brew (and to_xarray) to mature. (*could* simply delay 0.18.0
for say
a month otherwise).

any objections?

On Mon, Jan 25, 2016 at 11:47 AM, Wes McKinney <wesmckinn at gmail.com> wrote:

> hi all,
>
> As part of code cleanup and reorganization, let's start creating a
> "quarantine" of test code for functionality (like Panel classes) that
> we are contemplating deprecating and later removing in 1.0, if that
> sounds like a good idea?
>
> - Wes
>
> On Wed, Jan 13, 2016 at 6:06 PM, Wes McKinney <wesmckinn at gmail.com> wrote:
> > On Wed, Jan 13, 2016 at 6:01 PM, Jeff Reback <jeffreback at gmail.com>
> wrote:
> >> so I agree +1 on moving all to pandas/tests
> >>
> >> - the indexing tests, which *mostly* are in test_indexing.py, though
> quite a
> >> few are in test_series/test_frame.py, should ideally be
> >> merged into a set of tests/indexing
> >>
> >> - io tests could be left alone I think
> >>
> >
> > Yeah, I think pandas/io/tests is the one definite exception where
> > there isn't much benefit
> >
> >> - stats tests are *mostly* deprecated
> >>
> >> - since going to deprecate panel + nd soon, I think makes sense to move
> >> these tests & code to pandas/deprecated, to keep separate
> >>
> >> - test_tslib.py should be integrated into tseries/test_timeseries.py
> >>
> >> - almost all of the Index tests are now in test_index (which sub-class
> being
> >> somewhat generically tested), but the time-series ones
> >> are in tseries/test_base, so these could be merged as well.
> >>
> >
> > Yep, it specifically would be good to collect 100% of the index data
> > structure machinery (including Datetime/Timedelta/PeriodIndex) in one
> > place (same for axis indexing as you said, since it got pretty
> > scattered)
> >
> >>
> >> Jeff
> >>
> >>
> >>
> >>
> >> On Wed, Jan 13, 2016 at 8:51 PM, Wes McKinney <wesmckinn at gmail.com>
> wrote:
> >>>
> >>> Another idea here I've been toying with to achieve better logical test
> >>> organization is to place all tests in the whole project under
> >>> pandas/tests. This way we can centralize all the tests relating to
> >>> some functional aspect of pandas in one place, rather than the status
> >>> quo where test code tends to be fairly close to its implementation
> >>> (but not always). A prime example of where I let this get disorganized
> >>> early on is time series functionality tests are somewhat scattered
> >>> across pandas/tests, pandas/tseries, etc. This way we can also collect
> >>> a single directory of "quarantined" pandas 0.x behavior that we are
> >>> contemplating changing in a 1.0 release.
> >>>
> >>> Thoughts on this + other ideas how to help organize the tests to help
> >>> mentally in approaching refactoring and internal changes?
> >>>
> >>> - Wes
> >>>
> >>> On Wed, Jan 13, 2016 at 1:16 PM, Wes McKinney <wesmckinn at gmail.com>
> wrote:
> >>> > OK, I got started with the biggest offender:
> >>> >
> >>> > https://github.com/pydata/pandas/pull/12032
> >>> >
> >>> > It would be great to take the same approach with the other large test
> >>> > modules, with a special eye for quarantining "leaky" internals code
> >>> > and segregating NumPy interoperability contracts. I didn't completely
> >>> > do this with test_frame.py but it's a good start.
> >>> >
> >>> > There's definitely plenty of code in the other top level test modules
> >>> > which may nest under tests/frame or tests/series
> >>> >
> >>> > - Wes
> >>> >
> >>> > On Mon, Jan 11, 2016 at 8:47 AM, Wes McKinney <wesmckinn at gmail.com>
> >>> > wrote:
> >>> >> On Sun, Jan 10, 2016 at 6:06 PM, Stephan Hoyer <shoyer at gmail.com>
> >>> >> wrote:
> >>> >>> On Fri, Jan 8, 2016 at 5:34 PM, Wes McKinney <wesmckinn at gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> Big #1 question is, how strongly do you feel about *shipping* the
> >>> >>>> test
> >>> >>>> suite in site-packages? Some other libraries with sprawling and
> >>> >>>> complex test suites have chosen not to ship them:
> >>> >>>> https://github.com/zzzeek/sqlalchemy
> >>> >>>
> >>> >>>
> >>> >>> I would prefer to include the test suite if possible, because the
> >>> >>> ability to
> >>> >>> type "nosetests pandas" makes it easy both for users to verify
> >>> >>> installations
> >>> >>> are working properly and for downstream distributors to identify
> and
> >>> >>> report
> >>> >>> bugs. The complete pandas test suite still runs in 20-30 minutes,
> so I
> >>> >>> think
> >>> >>> it's still fairly reasonable to use it for these purposes.
> >>> >>>
> >>> >>
> >>> >> Got it. I wasn't sure if this was something people still wanted to
> do
> >>> >> in practice with the burgeoning test suite.
> >>> >>
> >>> >>>>
> >>> >>>> Independently, I would support and help with starting a judicious
> >>> >>>> reorganization of the contents of pandas/tests. So I'm thinking
> like
> >>> >>>>
> >>> >>>> tests/
> >>> >>>>     dataframe/
> >>> >>>>     series/
> >>> >>>>     algorithms/
> >>> >>>>     internals/
> >>> >>>>     tseries/
> >>> >>>>
> >>> >>>> and so forth.
> >>> >>>
> >>> >>>
> >>> >>> This sounds like a great idea -- these files have really gotten
> out of
> >>> >>> control!
> >>> >>>
> >>> >>
> >>> >> Sounds good. I've been sorting through points of contact between
> >>> >> Series/DataFrame's implementation and internal matters (e.g. the
> >>> >> BlockManager) and figured it would be good to "quarantine" code that
> >>> >> makes assumptions about what's under the hood. I'll get the first
> >>> >> couple patches started and it can be a slow burn to break apart
> these
> >>> >> large files.
> >>> >>
> >>> >>> Cheers,
> >>> >>> Stephan
> >>> _______________________________________________
> >>> Pandas-dev mailing list
> >>> Pandas-dev at python.org
> >>> https://mail.python.org/mailman/listinfo/pandas-dev
> >>
> >>
> _______________________________________________
> Pandas-dev mailing list
> Pandas-dev at python.org
> https://mail.python.org/mailman/listinfo/pandas-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20160125/5322e81f/attachment-0001.html>