[Scipy-organizers] Publication and review in SciPy

[Anthony Scopatz]

On Wed, Nov 6, 2013 at 2:02 PM, Andy Ray Terrel <andy.terrel at gmail.com>wrote:

>  The most effective use of our time is executing a well done, highly cited
> proceedings,


Here here!  I completely agree.

Also as communications chair, I think that we should now transition from
the current
discussion to more actually doing.  I applaud Sheila's issue that she
opened.  If you
want to actually work on this issue, please defer to Sheila and Stefan.  I
don't believe
that a lot of further good can really come from further discussion on this
thread.  The
main points have been made, let's reiterate them only in service to our
broader goal
of having great proceedings.

Be Well
Anthony


> not
> debating other journals' practices.
>
> On Wed, Nov 6, 2013 at 2:37 PM, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > Hi Sheila, Hi Katy, Hi James, Hi Andy,
> >
> > Thanks you all for the active participation and rapid responses. Now
> this seems to move somewhere.
> >
> > Sheila, you have interesting ideas in the link you sent. Yet this has to
> become more specific to be implemented. I still suggest a wiki page that
> will retain history of changes that multiple people can work on. Is it
> possible to open such a wiki in github and give public permissions for
> changes?
>
> Please use google here.  Asking others to find out details about
> github wastes their time.
>
> >
> > More importantly, there is a discussion about blind review. I can speak
> of experience. Blind review seems like a good and noble idea. The idea of
> allowing someone to be protected by blindness while providing an opinion is
> very nice in theory - it is very similar to the idea of anonymous voting in
> government elections. I can certainly see the benefits. However, it seems
> the apademic system has grew to the point where this idea no longer works
> in practice well. This idea is actively abused al many levels these days
> and blind review may no longer be a valid solution. I can give many
> examples, yet I will concentrate on one example that was already witnessed
> by some in this community - the CSD journal that was associated with SciPy
> 2013.
> >
> > The journal has rejected a paper without sending it to review and the
> editor who should have released it to review chose to remain blind, and
> even worse protected by the publisher who did not reveal the editors name.
> Note that the editorial board associated with the Journal is public and is
> listed on the journal web site. So in a sense, the incompetence of the
> editor who chose not to do their work now casts doubt on the entire
> editorial board that may very well be innocent and do their job well. Also,
> blindness can be used as a way to sneak attack good work without
> accountability. The assumption that all decision makers are honorable and
> good natured may not be valid at all cases - and supporting blindness opens
> the door to such cloak and dagger attacks.
> >
>
> This list is not an appropriate venue for you to air your dirty
> laundry with CSD.  As far as I can tell the editorial board did their
> job.  They protected their review board from a paper they thought
> would be rejected. Editors are the keepers of the journal and give no
> promise to review every paper submitted.  If someone submits a paper
> that is inappropriate to SciPy Proceedings our editors reserve the
> same exact right.
>
> The proposal was to have a blind review but publish the review in the
> open.  This will allow other reviewers to respond to a review that is
> out of line.
>
> > Furthermore there are other elements to take into account:
> >
> > 1. The scientific community in a certain field may be very small at
> times and therefore identity may be deduced and
> competition/differences/conflicts of interest may exist anyway at several
> levels within the group yet not visible - an open non blind review is
> relevant in this case.
>
> Our community is pretty large, the shimmer of hidden identity still
> helps folks give an honest review.
>
> >
> > 2. True experts typically try to gain recognition by publication - not
> hide it by having their name blinded. The argument of avoiding conflict is
> irrelevant - if a true expert sees a problem and does not report it, then
> they may be at fault - similar to a doctor not treating a patient. This is
> worse in my mind than stepping forward while risking a carrier. A carrier
> consists of past deeds not a future one wishes to obtain.
> >
>
> It is the duty of the editor to find experts, but since the reviews
> are published in the open, the submitter can refute any problems
> addressed by the review in the open.  Because other reviewers are
> open, others can refute them as well.
>
> > 3. If someone writes a review we wish to know the level of expertise of
> that person - a persons name gives such indication. Again, the past career
> speaks rather than future prospects.
> >
>
> Reviews should be judged by the merits of their content not the
> prestige of the reviewer. Once again the open reviews and responses
> can address this.
>
> > 4. A non blind review opens the opportunity for a conversation and
> several rounds of improvement - just like versions of a software. Reviews
> should no longer be one shot judgments. They should improve others work - I
> think this is more important and possible to obtain for an open review
> since the reviewer knows their review itself is under scrutiny.
> >
>
> This is still possible if a few reviewers are blind.
>
> > 5. A blind review is funny in the sense that if I meet that person later
> should I conceal the fact that I was a reviewer? This makes communications
> awkward and not a good base for community communications - a white elephant
> will be in the room from the start.
> >
>
> That is your personal decision and emotions.  I regularly talk to
> folks about their papers without revealing that I'm a reviewer. The
> honesty of reviews is more important.
>
> > I can think of other aspects. Yet these are enough to explain my
> position. I myself decided I no longer provide blind reviews for that
> reason.
> >
>
> Sorry, I have not been convinced at all.
>
> > Note that some of level of compromise is possible by allowing blind
> review during the pre-publication period and then revealing the name of the
> reviewer post acceptance - yet in the rapid communications world we are
> having today, and especially in the open software community, do we really
> need to wait that long?
> >
>
> No.  The anonymous reviewers should always be anonymous, the
> protection of identity is to promote honest reviews not to speed to
> publication.  Public reviews can be always be public.
>
> > In summary, I suggest we keep the entire process open rather than spend
> time on figuring out complicated ways to incorporate blindness. It is
> better to spend efforts on automating editorial support to help handle the
> papers flowing in and the invitation for reviews.
> >
>
> This has already be accomplished via the github model
>
> > And to be a bit sarcastic, allow me to add the following question:
> >
> > How can one be so open if one is so blind?
> >
>
> Who cares about openness if it is producing papers that are uncited and
> unread.
>
> > This is a bit offensive for a reason to show the contradiction between
> openness and blindness - if you see synergy there then please explain.
> >
> > I hope these arguments seem logical and I am open for counter arguments.
> I do hope to get some to figure out the best solution here.
> >
> >             Jacob
> >
> >
> > Sent from my iPhone
>
> -- Andy
> _______________________________________________
> Scipy-organizers mailing list
> Scipy-organizers at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-organizers
>