[scikit-learn] Roc curve from multilabel classification has slope

[José Ismael Fernández Martínez]

That's indeed the case, there are ties in my predictions. In response to
"plotting one ROC curve for every class in your result", it's also part of
my analysis. Thank you very much.

Ismael

2017-01-08 3:27 GMT-06:00 Roman Yurchak <rth.yurchak at gmail.com>:

> José, I might be misunderstanding something, but wouldn't it make more
> sens to plot one ROC curve for every class in your result (using all
> samples at once), as opposed to plotting it for every training sample as
> you are doing now? Cf the example below,
>
> http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
>
> Roman
>
> On 08/01/17 01:42, Jacob Schreiber wrote:
> > Slope usually means there are ties in your predictions. Check your
> > dataset to see if you have repeated predicted values (possibly 1 or 0).
> >
> > On Sat, Jan 7, 2017 at 4:32 PM, José Ismael Fernández Martínez
> > <ismaelfm_ at ciencias.unam.mx <mailto:ismaelfm_ at ciencias.unam.mx>> wrote:
> >
> >     But is not a scikit-learn classifier, is a keras classifier which,
> >     in the functional API, predict returns probabilities.
> >     What I don't understand is why my plot of the roc curve has a slope,
> >     since I call roc_curve passing the actual label as y_true and the
> >     output of the classifier (score probabilities) as y_score for every
> >     element tested.
> >
> >
> >
> >     Sent from my iPhone
> >     On Jan 7, 2017, at 4:04 PM, Joel Nothman <joel.nothman at gmail.com
> >     <mailto:joel.nothman at gmail.com>> wrote:
> >
> >>     predict method should not return probabilities in scikit-learn
> >>     classifiers. predict_proba should.
> >>
> >>     On 8 January 2017 at 07:52, José Ismael Fernández Martínez
> >>     <ismaelfm_ at ciencias.unam.mx <mailto:ismaelfm_ at ciencias.unam.mx>>
> >>     wrote:
> >>
> >>         Hi, I have a multilabel classifier written in Keras from which
> >>         I want to compute AUC and plot a ROC curve for every element
> >>         classified from my test set.
> >>
> >>         <image1.PNG>
> >>
> >>         Everything seems fine, except that some elements have a roc
> >>         curve that have a slope as follows:
> >>
> >>         enter image description here
> >>         <https://i.stack.imgur.com/XCNCA.png>I don't know how to
> >>         interpret the slope in such cases.
> >>
> >>         Basically my workflow goes as follows, I have a
> >>         pre-trained |model|, instance of Keras, and I have the
> >>         features |X| and the binarized labels |y|, every element
> >>         in |y| is an array of length 1000, as it is a multilabel
> >>         classification problem each element in |y| might contain many
> >>         1s, indicating that the element belongs to multiples classes,
> >>         so I used the built-in loss of |binary_crossentropy| and my
> >>         outputs of the model prediction are score probailities. Then I
> >>         plot the roc curve as follows.
> >>
> >>
> >>         The predict method returns probabilities, as I'm using the
> >>         functional api of keras.
> >>
> >>         Does anyone knows why my roc curves looks like this?
> >>
> >>
> >>         Ismael
> >>
> >>
> >>
> >>         Sent from my iPhone
> >>
> >>         _______________________________________________
> >>         scikit-learn mailing list
> >>         scikit-learn at python.org <mailto:scikit-learn at python.org>
> >>         https://mail.python.org/mailman/listinfo/scikit-learn
> >>         <https://mail.python.org/mailman/listinfo/scikit-learn>
> >>
> >>
> >>     _______________________________________________
> >>     scikit-learn mailing list
> >>     scikit-learn at python.org <mailto:scikit-learn at python.org>
> >>     https://mail.python.org/mailman/listinfo/scikit-learn
> >>     <https://mail.python.org/mailman/listinfo/scikit-learn>
> >
> >     _______________________________________________
> >     scikit-learn mailing list
> >     scikit-learn at python.org <mailto:scikit-learn at python.org>
> >     https://mail.python.org/mailman/listinfo/scikit-learn
> >     <https://mail.python.org/mailman/listinfo/scikit-learn>
> >
> >
> >
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
> >
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170112/f1c49db5/attachment-0001.html>