[SciPy-Dev] scipy.stats

[josef.pktd at gmail.com]

On Mon, May 31, 2010 at 10:43 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Mon, May 31, 2010 at 8:28 AM, <josef.pktd at gmail.com> wrote:
>>
>> On Mon, May 31, 2010 at 10:23 AM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Mon, May 31, 2010 at 8:16 AM, <josef.pktd at gmail.com> wrote:
>> >>
>> >> Since Travis seems to want to take back control of scipy.stats, I am
>> >> considering my role as inofficial maintainer as ended.
>> >>
>> >> I would have appreciated his help almost 3 years ago, when I started
>> >> to learn numpy, scipy, and started to submit patches for
>> >> scipy.stats.distributions.
>> >>
>> >> But by now, I have pretty strong opinions about statistics in python,
>> >> after almost  three years, I'm a bit tired of cleaning up the mess of
>> >> others (and want to clean up my own mess), and there are obviously big
>> >> philosophical differences for the development process between me and
>> >> Travis (no discussion, no review, no tests).
>> >> http://projects.scipy.org/scipy/log/trunk/scipy/stats/tests
>> >>
>> >> Watching the scipy changelog and checking any function that Travis
>> >> quietly commits is no fun (see mailing list for the introduction of
>> >> curve_fit or ask Stefan).
>> >>
>> >> I said early on that I would like to trust the results that
>> >> scipy.stats produces (although I don't find the mailing list thread
>> >> any more).
>> >>
>> >> I considered scipy to go into a stable direction like Python is,
>> >> kitchen sink for scientific programming, which might be slow-moving
>> >> but with high standards, and not a sandbox.
>> >>
>> >> Details are at
>> >> http://mail.scipy.org/pipermail/scipy-dev/2010-April/014058.html
>> >>
>> >> After my initial scipy.stats.distributions cleanup, test coverage was
>> >> at 91%, I have no idea where it is after this weekend.
>> >>
>> >> This is more about the process then the content, distributions was
>> >> Travis's baby (although unfinished), and most of his changes are very
>> >> good, but I don't want to look for the 5-10% (?) typos anymore.
>> >>
>> >
>> > Ah Josef, there are easier ways to lodge complaints than resignation ;)
>> > I
>> > agree that it was rude of Travis to make those changes without running
>> > them
>> > through the list, and he does tend to toss stuff in that others have to
>> > clean up, the same with c-code. But maybe we can manage to get him
>> > housebroken without all moving out.
>>
>> I think the discussion with him occurred already several times on the
>> lists.
>>
>
> But he forgets. In any case, I feel that you are currently the essential
> person in the stats area because of your hard work, expertise, and planning,
> and we need to fix things up so that you are happy. Make some suggestions.

My approach has been to block any untested, non-trivial changes, and
to review and test changes by others, so we don't increase the amount
of "technological debt".

The second part is testing and verification of existing functions
(finish the statistical review which has been lingering for several
years), which is slow going because I'm almost the only one doing it.

The third part is develop enhancements and prepare for a consistent
refactoring of "statistics in python" in a sandbox outside of scipy.
(Here, I admit I have a small conflict of interest getting things
quickly into scipy, because I'm more interested in "Statistics in
Python" than statistics in scipy.)

I have been a bit slow working on scipy trunk in the last year because
I worked more on statsmodels, and I didn't have always the time (or
didn't feel like) keeping up with recompiling numpy/scipy.

(My alternative is to develop pythonstats completely outside of
scipy.stats, maintaining a smaller set of verified functions and just
ignore scipy.stats for now.)

Josef

>
> Chuck
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>