[SciPy-User] peer review of scientific software

[Matthew Brett]

Hi,

On Tue, May 28, 2013 at 1:11 PM, Calvin Morrison <mutantturkey at gmail.com> wrote:
>
>
>
> On 28 May 2013 16:00, Matthew Brett <matthew.brett at gmail.com> wrote:
>>
>> Hi,
>>
>> On Tue, May 28, 2013 at 12:41 PM, Martin van Leeuwen
>> <vanleeuwen.martin at gmail.com> wrote:
>> > Hi,
>> >
>> > Nice article. The frustration for students without a formal programming
>> > background such as a bachelor in computer science is as big as that for
>> > students and Profs that do have such a background, I think.
>>
>> I found the article frustrating - it didn't seem to have much to add
>> to a general set of feelings (that most of us share) that writing code
>> properly is a good idea.
>>
>> The question that always comes up is - why?   Most scientists trained
>> in the ad-hoc get-it-to-work model have a rather deep belief that this
>> model is more or less OK, and that doing all that version-control,
>> testing stuff is for programming types with lots of time on their
>> hands.
>
>
> Yes and wearing lab vests, writing down procedures and documenting methods
> is all just for people with so much time...
>
> Version control, unit testing, proper practices in general, actually are
> time savers. I use version control on my own projects, because it helps me
> organize my code and allows me to stay organized. Coding well helps me read
> through my code easier and collaborate with others. The issue is not with
> the tools, it is with people refusing to learn how to use them.
>
> The ability to reproduce results is a very important aspect of science is it
> not? How can I know if your claims are true if you have hidden software that
> has never seen the light of day? How can you benefit the community if nobody
> can use your software?
>
>>  If we want to persuade those guys and gals, we have to come up
>> with something pretty compelling, and I don't think we have that yet.
>> I would love to see some really good data to show that we'd proceed
>> faster as scientists with more organized coding practice,
>
>
> Proceed faster as scientists individually? Maybe not. But as an aggregate,
> the community most certainly benefit from not having to reimplement tools,
> by developing tools for other people to use, by making it easier to
> collaborate and reasons that I can't even think of!
>
> What is the point of publishing works if you aren't publishing the tools?
> Who are you helping? The community, the populace as a whole, or your silly
> CV?

I have personally been doing these good things "since my youth", and
trying to teach other people to do the same.

I don't often have much success though, hence my email.

The response usually goes something like "I can't afford to waste
time, I've got a deadline / I've got to get tenure" etc.

If our response is a general 'oh but it's much better to do it that
way' - I can assure you, most of the time, that doesn't cut it.

As Nathaniel says - for those who are interested - just showing them
how to do this stuff can often be enough.   "Oh yes, I get it,
awesome".

For those who sense this as a threat to waste their time in tedious
detail - that isn't going to work.

We really have to persuade these people that - for a short investment
of time - they will reap major benefits - for themselves and / or for
their fellow scientists.   I don't know of much data to help with this
latter thing.  I can imagine data, but I don't know where it is, or if
it exists...

Cheers,

Matthew