deep learning

[Emmanuelle Gouillart]

[changing the title of the thread since one of your comments deviated
quite a lot from the topic of "task board"]

> One more point (yet it is better to start a discussion in a RFC :) ) I'd like
> to bring to the discussion. I remember a short discussion with Stefan, that
> there was a decision upon that `skimage` is a community driven-project. I
> understand, that it is probably mostly used for educational purposes, but we
> shouldn't as a project fall behind the progress in Image Processing and
> Computer Vision. I see that many modern Deep Learning frameworks start to
> implement their own image processing routines just to have a common ground for
> their functionality. I believe we could still catch the train. So, I propose to
> consider moving `skimage` infrastructure onto `theano`. That is a hell lot of
> work, of course, but I'm asking just for a discussion yet.
> From this section, actually, one more pragmatic topic arises. Even if we keep
> image processing part up to the community, I think that we have to put more our
> (core team) efforts on the backend part of the project (not just testing and
> documentation part, but also e.g. make easier function chaining - https://
> github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone
> outside the core team to have enough knowledge, experience and confidence to
> implement this and some other features. I.e. I believe that we have to make
> some necessary contributions to stay competitive.

I have to say that I disagree a lot with this analysis.

1. First of all, I don't think that scikit-image is used primarily for
educational purposes. If you browse through the research articles that
cite the scikit-image paper, you'll find that people have used
scikit-image as a building block for scientific software and applications
or for research in various fields such as medical science, materials
science, plant research, etc. I know several industrial groups which are
using scikit-image for their daily R&D workflow. Tens of thousands of
people are browsing every month through the examples of the gallery to
discover how to perform specific image processing operations.

That said, you're totally right to raise the question of "staying
competitive". But it's a tricky issue, since in my opinion, being
competitive is a subtle mixture of usability, features, performance
(probably in this order, but it's a matter of debate). 

2. About Deep Learning: deep learning is very cool, trendy and powerful,
it can blow your mind for some operations, yet it cannot do everything in
image processing. Labeling images with kittens is not the sole purpose
that our users have :-). Furthermore, deep learning require intensive
computations, that are best done on the GPU. Some packages like keras
provide elegant solutions for deep learning and are compatible with
scikit-image, thanks to the common use of numpy arrays. 

Can some scikit-image contributors work on a deep learning package for
image processing using theano, tensorflow, keras and some of the
knowledge/features of scikit-image? Probably. Does it have to be in
scikit-image? I don't think so (although it could be a project of the
scikit-image organization). Let's try not to build a monster package with
everything and the kitchen sink, that quickly becomes uninstallable and
unusable.

Using theano instead of NumPy arrays would be to favor one image
processing workflow at the expense of many others, so I don't think it's
wise. To quote David Waley-Farley: "Yep. Theano, Numba, numexpr, Cython
and PyPy shall one day all merge to form Numtron, defender of sanity,
runner of fast numerics.". But before we're there, NumPy is the common
denominator for the scientific packages.

It would be interesting to know which are the deep learning frameworks
that you're mentioning, to see how they tackle image processing. I'd be
interested in knowing more about this.  

3. For function chaining, it might be interesting to take a look at
scikit-learn's pipeline 
http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html

My 2 cents,
Emma