[scikit-learn] Model trained in 0.17 gives entirely different results in 0.15

[Dale T Smith]

Use conda or a virtualenv to handle compatibility issues. Then you can control when upgrades occur. I’ve used conda with good effect to handle version issues such as yours.

Otherwise, use PMML. The Data Mining Group maintains a list of PMML producers and consumers. I think there is a Python wrapper for JPMML which is what you can use for a consumer.

http://dmg.org/pmml/products.html

__________________________________________________________________________________________
Dale Smith | Macy's Systems and Technology | IFS eCommerce | Data Science and Capacity Planning
| 5985 State Bridge Road, Johns Creek, GA 30097 | dale.t.smith at macys.com

From: scikit-learn [mailto:scikit-learn-bounces+dale.t.smith=macys.com at python.org] On Behalf Of Shi Yu
Sent: Wednesday, August 3, 2016 3:03 PM
To: Scikit-learn user and developer mailing list
Subject: Re: [scikit-learn] Model trained in 0.17 gives entirely different results in 0.15

⚠ EXT MSG:
Hi Andy,

Thanks for the feedback.  Indeed we think it would be a good idea to enforce version persistence something like in serialVersionUID Java here.

We deployed models trained on our laptop onto our clusters, and ran into this issue and paid a serious lesson for that.

Best,

Shi


---------- Forwarded message ----------
From: Andreas Mueller <t3kcit at gmail.com<mailto:t3kcit at gmail.com>>
Date: Wed, Aug 3, 2016 at 1:29 PM
Subject: Re: [scikit-learn] Model trained in 0.17 gives entirely different results in 0.15
To: Scikit-learn user and developer mailing list <scikit-learn at python.org<mailto:scikit-learn at python.org>>

Hi Shi.
In general, there is no guarantee that models built with one version will work in a different version.
In particular, loading in an older version when built in a newer version seems something that's tricky to achieve.

We might want to warn the user when doing this. The docs are not very explicit about this.

Opened an issue:
https://github.com/scikit-learn/scikit-learn/issues/7135

Andy

On 08/02/2016 05:02 PM, Shi Yu wrote:
Hello,

We trained SVM models in scikit-learn 0.17 and saved it as pickle files. When loading the models back in a lower version of scikit-learn 0.15, the outputs are entirely different.  Basically for binary classification problem, for the same test data,  it swapped the probabilities and gave an opposite prediction.  In 0.17 the probability is [0.02668825,  0.97331175] and the prediction is 1.  In 0.15 the probability is [0.97331175, 0.02668825] and the prediction is 0.

I wonder is anyone seeing the same issue, or it has been notified.  I could provide more details for error replication if required.

Best,

Shi


_______________________________________________

scikit-learn mailing list

scikit-learn at python.org<mailto:scikit-learn at python.org>

https://mail.python.org/mailman/listinfo/scikit-learn


_______________________________________________
scikit-learn mailing list
scikit-learn at python.org<mailto:scikit-learn at python.org>
https://mail.python.org/mailman/listinfo/scikit-learn

* This is an EXTERNAL EMAIL. Stop and think before clicking a link or opening attachments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20160803/cab7b1c8/attachment.html>