[Scipy-organizers] Publication and review in SciPy

[Jacob Barhak]

Hi Andy,

You may wish to go over previous material. 

The CSD case is highly relevant here. And you were BCCd to the entire relevant communication for a few months. And we discussed this also in person.

Note that CSD admitted their editor did wrong. See the communications you already have. I took this as only a single example. There are more such phenomenon that should be stopped - not publication. Your approach of the editor standing valiantly at the gate and stopping the flow of bad publications may already be antiquated. New technology opens new paths we should explore and github has a good infrastructure to support this. 

Unblinding the review process is a viable solution that was not possible in the past. 

You strongly oppose this unblinding for some reason. You fail to give concrete examples while I gave you one close to SciPy - considering this my evidence seems stronger than your arguments at this point in time. 

And do recall that this is public, so your last remark regarding unread papers can be interpreted badly considering your strong affiliation with SciPy - you should rephrase your text here and issue a correction - I truly believe you wish SciPy proceedings to be read. Let us figure out how this can be accomplished. 

           Jacob 

Sent from my iPhone

On Nov 6, 2013, at 4:02 PM, Andy Ray Terrel <andy.terrel at gmail.com> wrote:

> Jacob,
> 
> First I appreciate your viewpoint and am glad you are able to give
> such vigor to this discussion.  I'm going to try to respond to your
> points here but I want to say that the SciPy proceedings is a small
> part of the overall scientific community.  The most effective use of
> our time is executing a well done, highly cited proceedings, not
> debating other journals' practices.
> 
> On Wed, Nov 6, 2013 at 2:37 PM, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>> Hi Sheila, Hi Katy, Hi James, Hi Andy,
>> 
>> Thanks you all for the active participation and rapid responses. Now this seems to move somewhere.
>> 
>> Sheila, you have interesting ideas in the link you sent. Yet this has to become more specific to be implemented. I still suggest a wiki page that will retain history of changes that multiple people can work on. Is it possible to open such a wiki in github and give public permissions for changes?
> 
> Please use google here.  Asking others to find out details about
> github wastes their time.
> 
>> 
>> More importantly, there is a discussion about blind review. I can speak of experience. Blind review seems like a good and noble idea. The idea of allowing someone to be protected by blindness while providing an opinion is very nice in theory - it is very similar to the idea of anonymous voting in government elections. I can certainly see the benefits. However, it seems the apademic system has grew to the point where this idea no longer works in practice well. This idea is actively abused al many levels these days and blind review may no longer be a valid solution. I can give many examples, yet I will concentrate on one example that was already witnessed by some in this community - the CSD journal that was associated with SciPy 2013.
>> 
>> The journal has rejected a paper without sending it to review and the editor who should have released it to review chose to remain blind, and even worse protected by the publisher who did not reveal the editors name. Note that the editorial board associated with the Journal is public and is listed on the journal web site. So in a sense, the incompetence of the editor who chose not to do their work now casts doubt on the entire editorial board that may very well be innocent and do their job well. Also, blindness can be used as a way to sneak attack good work without accountability. The assumption that all decision makers are honorable and good natured may not be valid at all cases - and supporting blindness opens the door to such cloak and dagger attacks.
> 
> This list is not an appropriate venue for you to air your dirty
> laundry with CSD.  As far as I can tell the editorial board did their
> job.  They protected their review board from a paper they thought
> would be rejected. Editors are the keepers of the journal and give no
> promise to review every paper submitted.  If someone submits a paper
> that is inappropriate to SciPy Proceedings our editors reserve the
> same exact right.
> 
> The proposal was to have a blind review but publish the review in the
> open.  This will allow other reviewers to respond to a review that is
> out of line.
> 
>> Furthermore there are other elements to take into account:
>> 
>> 1. The scientific community in a certain field may be very small at times and therefore identity may be deduced and competition/differences/conflicts of interest may exist anyway at several levels within the group yet not visible - an open non blind review is relevant in this case.
> 
> Our community is pretty large, the shimmer of hidden identity still
> helps folks give an honest review.
> 
>> 
>> 2. True experts typically try to gain recognition by publication - not hide it by having their name blinded. The argument of avoiding conflict is irrelevant - if a true expert sees a problem and does not report it, then they may be at fault - similar to a doctor not treating a patient. This is worse in my mind than stepping forward while risking a carrier. A carrier consists of past deeds not a future one wishes to obtain.
> 
> It is the duty of the editor to find experts, but since the reviews
> are published in the open, the submitter can refute any problems
> addressed by the review in the open.  Because other reviewers are
> open, others can refute them as well.
> 
>> 3. If someone writes a review we wish to know the level of expertise of that person - a persons name gives such indication. Again, the past career speaks rather than future prospects.
> 
> Reviews should be judged by the merits of their content not the
> prestige of the reviewer. Once again the open reviews and responses
> can address this.
> 
>> 4. A non blind review opens the opportunity for a conversation and several rounds of improvement - just like versions of a software. Reviews should no longer be one shot judgments. They should improve others work - I think this is more important and possible to obtain for an open review since the reviewer knows their review itself is under scrutiny.
> 
> This is still possible if a few reviewers are blind.
> 
>> 5. A blind review is funny in the sense that if I meet that person later should I conceal the fact that I was a reviewer? This makes communications awkward and not a good base for community communications - a white elephant will be in the room from the start.
> 
> That is your personal decision and emotions.  I regularly talk to
> folks about their papers without revealing that I'm a reviewer. The
> honesty of reviews is more important.
> 
>> I can think of other aspects. Yet these are enough to explain my position. I myself decided I no longer provide blind reviews for that reason.
> 
> Sorry, I have not been convinced at all.
> 
>> Note that some of level of compromise is possible by allowing blind review during the pre-publication period and then revealing the name of the reviewer post acceptance - yet in the rapid communications world we are having today, and especially in the open software community, do we really need to wait that long?
> 
> No.  The anonymous reviewers should always be anonymous, the
> protection of identity is to promote honest reviews not to speed to
> publication.  Public reviews can be always be public.
> 
>> In summary, I suggest we keep the entire process open rather than spend time on figuring out complicated ways to incorporate blindness. It is better to spend efforts on automating editorial support to help handle the papers flowing in and the invitation for reviews.
> 
> This has already be accomplished via the github model
> 
>> And to be a bit sarcastic, allow me to add the following question:
>> 
>> How can one be so open if one is so blind?
> 
> Who cares about openness if it is producing papers that are uncited and unread.
> 
>> This is a bit offensive for a reason to show the contradiction between openness and blindness - if you see synergy there then please explain.
>> 
>> I hope these arguments seem logical and I am open for counter arguments. I do hope to get some to figure out the best solution here.
>> 
>>            Jacob
>> 
>> 
>> Sent from my iPhone
> 
> -- Andy