[Scipy-organizers] Publication and review in SciPy

Wed Oct 30 14:26:38 EDT 2013

I forgot to reply-all.

Hi all,

With respect to testing external parters (who have open and free
platforms in the sense of FLOSS*), it could be interesting and fun for
people to try building prototypes and/or extending systems -- but not
close to the conference. We could find interested volunteers who may
not have experience doing scientific programming but who are
interesting in open science and FLOSS projects and this would give
them a way to explore their interests and work with us before the
conference. Closer to the conference (months?) I would be less
interested in spending time on experimental prototypes.

But, there are so many exciting things going on,
<https://khinsen.wordpress.com/2013/09/27/activepapers-for-python/>
just for one example. I imagine people would find it fun to experiment
try out these ideas in a real context rather than a hypothetical one.
I know I would.

Disclosure, I work for Victoria Stodden on an open source project
related to this, so I am biased.

Ps. I do like the process of open review that I've seen people doing
on github using issues and pull requests.

* as opposed to something like the executable papers that Elsevier has.

On Tue, Oct 29, 2013 at 11:48 AM, Jacob Barhak <jacob.barhak at gmail.com> wrote:
> Hi Andy,
>
> Thanks for the explanation. However you should be aware that a review process is free in all publication methods I encounter. Even editorial board members are free.
>
> I was serving as an editorial board member for a while. No one pays anything and furthermore people avoid being reviewers.
>
> To collect 3 valid reviews for a paper you sometimes need to send about 50 to 70 invitations. And even people who promised to review many times do not complete it in reasonable time or just change their minds. There is no incentive for reviewing and despite importance people do not volunteer. The search for reviewers becomes a data entry job.
>
> The same problem is probably plaguing the entire scientific publication system. We are just seeing part of a bigger breakdown that had many negative effects.
>
> I believe this is time for new solutions and there are plenty of ideas. Yet the volunteer system of SciPy is nit an excuse not to push forward - everyone uses volunteers for review.
>
> And with regards to testing external partners - I disagree with you. You can tell a candidate that you will be testing them - it is quite regular in many occasions - why not a candidate publisher?
>
> I strongly support a github based review system.
>
>          Jacob
>
>
>
> Sent from my iPhone
>
> On Oct 29, 2013, at 10:46 AM, Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>
>> Hello All,
>>
>> Before we start discussing the future of SciPy Proceedings, we need to
>> have a little perspective on where things are and who is involved.
>> Let me try to summarize what has been going on.
>>
>> First, we have to ask why does SciPy have a proceedings to begin with.
>> I think one answer is that the current state of journals is hostile
>> towards publishing ideas our communities hold dear.  Things like
>> scientific software should be built to be usable and here is a code
>> showing something that is known but done in a way that is accessible.
>> As co-chair the intention was to get more recognition for the amazing
>> work coming out of our community, while a video of a talk is nice and
>> tweets draw interest, archived well done publications stand a longer
>> test of time.  By helping our community publish more we help the
>> scientific software field legitimize itself in the ever increasingly
>> competitive market (academic and otherwise).
>>
>> Second, we have to establish a means. Companies such as IOP and
>> Elsevier make money off such publications.  Even societies such as
>> SIAM and ACM draw the majority of their funds from journal and
>> conference proceedings.  SciPy Proceedings is entirely volunteer with
>> no revenue and this needs to be kept in mind when deciding these
>> wonderful goals, whose gonna do the work and why.  While I think we
>> had a much more positive review process last year than in the previous
>> few years, we still don't have the proceedings up in a readable,
>> archivable format. Without a surge of fresh hands helping out with
>> this portion of the conference, (actually reviewing, helping with the
>> publishing, and so on) I am hesitant to push forward on this pursuit.
>> (With that said, I've added two positions to the SciPy2014 board as
>> "Technical Committee Chairs" including Sheila who has a great deal of
>> experience with challenging the publishing norms.)
>>
>> Third, a bit of history.  Last year we pursued a route of having both
>> the proceedings that included our traditional 6 page documents with a
>> low review overhead and including a focus issue with a more
>> prestigious open journal, Computational Science and Discovery.  CSD
>> was a newer journal trying to push changes in the field, but
>> unfortunately have had many setbacks (including staffing problems)
>> that have basically hamstrung our interactions.  At SciPy2013 I had
>> invited several journal editors in scientific computing to be part of
>> a BOF that ended up being canceled due to their unavailability.  I was
>> pleased that the Center for Open Science was still able to have a
>> strong presence.  CSD is still interested in having the focus issue
>> for SciPy2013, but I am not very confident that it will happen (yes,
>> I'm emailing them quite a bit these days.)
>>
>> Finally, where are we.  Last year Jarrod and Stéfan built a review
>> system based on Github pull requests
>> (https://github.com/scipy/scipy_proceedings/pulls?direction=desc&page=1&sort=created&state=closed).
>> This replaced the previous system of putting reviews on the website,
>> but SciPy2013 Proceedings website has yet to be built
>> (https://github.com/scipy/scipy_proceedings/issues/70).  Jarrod has
>> stepped down as chair and we are still searching for the co-chair for
>> this role.  I think there is a lot of potential in this approach but
>> would agree it needs a few steps to make a well-done professional
>> procedure that will attract more attention.
>>
>> At this point of the juncture, I am convince if we want to change the
>> state of things we need to do it ourselves in a professional
>> well-publicized manner, or find a strong partner (investing in Jarrod
>> and Stefan's github review system, Center for Open Science, inSCIght).
>> I would prefer to take partners at face value and work with them to
>> produce a product, not a disguised test of their services. Finally in
>> a time of crisis in scientific research, we need to hold ourselves to
>> standards that are much higher than our many academic peers.
>>
>> -- Andy
>>
>>
>>
>> On Tue, Oct 29, 2013 at 9:38 AM, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>>> Hi Matt,
>>>
>>> The link you sent is relevant yet will take a long time to process - there are many ideas out there in that conference.
>>>
>>> As for your second remark regarding partnering. Well, you can have a very basic solution with little effort using github and just specifying how to use it properly. The 2013 github publication model combined with the 2012 open review policy may be a good base. From there on you can always build further. Yet first you should have a solid  simple base.
>>>
>>> If you wish to test an external partner for publication it is possible to test beforehand by submitting a paper and see how it is handled - I can help here if you have specifics in mind.
>>>
>>> Thanks for your fast response.
>>>
>>>        Jacob
>>>
>>> Sent from my iPhone
>>>
>>> On Oct 29, 2013, at 9:13 AM, Matthew Turk <matthewturk at gmail.com> wrote:
>>>
>>>> Hi Jacob,
>>>>
>>>> On Tue, Oct 29, 2013 at 10:01 AM, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>>>>> Hello to all SciPy organizers.
>>>>>
>>>>> This is submitted here after an email conversation with some of the organizers pointing towards an ineffective journal publication venue in 2013. Andy invited me to send the conversation here to address a larger pool of opinions in SciPy.
>>>>>
>>>>> The traditional journal publication system is quite broken and cannot keep up with technological changes. Here are some examples:
>>>>> 1. The review processes are cumbersome blind and long
>>>>> 2. Journal publications are not geared towards code publication
>>>>> 3. Version control and sharing are not embedded in most of those systems
>>>>>
>>>>> The changing landscape of technology may call for other publication alternatives for the SciPy proceedings that do not need to rely on old journal type publication.
>>>>>
>>>>> Journal publications are still used for promotion and other recognition within the scientific community, yet if the traditional system is so broken, then it is time for a better alternative. SciPy is a good base for forming such an alternative.
>>>>
>>>> I find this to be an extremely interesting avenue, and SciPy is indeed
>>>> a good venue for opening up these discussions.  Last year, Will
>>>> Schroeder's keynote touched upon the work being done through the
>>>> Insight Journal, which also attempts to address many of these
>>>> shortcomings.  The WSSSPE workshop at SC13 this year has several
>>>> contributions that discuss publishing models, too:
>>>> http://wssspe.researchcomputing.org.uk/contributions/ .
>>>>
>>>>>
>>>>> I really liked the path taken in 2012 where reviews were being asked and openly stored with the paper - a non blind review. I would like to see more of this approach. This is more similar to testing software where someone has to sign on a product.
>>>>>
>>>>> I would suggest some elements that make sense to me to keep publication effective:
>>>>>
>>>>> 1. Use github or a similar repository or a wiki to publish SciPy proceedings - this will allow linking to code, video, slides, etc.
>>>>>
>>>>> 2. Emphasize electronic publication over traditional paper formatting. Which can be accomplished using simple RST or MD or similar non demanding non time consuming formatting.
>>>>>
>>>>> 3. Ensure high quality that is accountable by using open non blind review process.
>>>>>
>>>>> Note that the latter review process can continue even after publication and paper submitters may be asked to participate in open review as part of participating in SciPy.
>>>>>
>>>>> There are just a few ideas. I would appreciate a discussion on those issues to help improve SciPy in the future and use its innovative spirit to influence the scientific community in better directions.
>>>>
>>>> Attempting to move the proceedings to a non-traditional journal, or
>>>> even start one, could be a very beneficial both for SciPy the
>>>> conference and the community.  My main reaction to this is that there
>>>> are so many possible partners out there, both within the python/scipy
>>>> community as well as in the broader "open science" or even
>>>> computational science communities, that we would really need to ensure
>>>> we have as many partners in this as possible, which might make it
>>>> broader than we can pull off for 2014 proceedings.
>>>>
>>>> -Matt
>>>>
>>>>>
>>>>>       Jacob
>>>>>
>>>>> Sent from my iPhone
>>>>> _______________________________________________
>>>>> Scipy-organizers mailing list
>>>>> Scipy-organizers at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/scipy-organizers
>>> _______________________________________________
>>> Scipy-organizers mailing list
>>> Scipy-organizers at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-organizers
> _______________________________________________
> Scipy-organizers mailing list
> Scipy-organizers at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-organizers

-- 
sheila at codersquid.com