[scikit-learn] RFE with logistic regression

[Benoît Presles]

I did the same tests as before adding fit_intercept=False and:

1. I have got the same problem as before, i.e. when I execute the RFE 
multiple times I don't get the same ranking each time.

2. When I change the solver to 'sag' 
(classifier_RFE=LogisticRegression(C=1e9, verbose=1, max_iter=10000, 
fit_intercept=False, solver='sag')), it seems that I get the same 
ranking at each run. This is not the case with the 'saga' solver.
The ranking is not the same between the solvers.

3. With C=1, it seems that I have the same results at each run for all 
solvers (liblinear, sag and saga), however the ranking is not the same 
between the solvers.


How can I get reproducible and consistent results?


Thanks for your help,
Best regards,
Ben



Le 24/07/2018 à 18:16, Stuart Reynolds a écrit :
> liblinear regularizes the intercept (which is a questionable thing to
> do and a poor choice of default in sklearn).
> The other solvers do not.
>
> On Tue, Jul 24, 2018 at 4:07 AM, Benoît Presles
> <benoit.presles at u-bourgogne.fr> wrote:
>> Dear scikit-learn users,
>>
>> I am using the recursive feature elimination (RFE) tool from sklearn to rank
>> my features:
>>
>> from sklearn.linear_model import LogisticRegression
>> classifier_RFE = LogisticRegression(C=1e9, verbose=1, max_iter=10000)
>> from sklearn.feature_selection import RFE
>> rfe = RFE(estimator=classifier_RFE, n_features_to_select=1, step=1)
>> rfe.fit(X, y)
>> ranking = rfe.ranking_
>> print(ranking)
>>
>> 1. The first problem I have is when I execute the above code multiple times,
>> I don't get the same results.
>>
>> 2. When I change the solver to 'sag' or 'saga' (classifier_RFE =
>> LogisticRegression(C=1e9, verbose=1, max_iter=10000), solver='sag'), it
>> seems that I get the same results at each run but the ranking is not the
>> same between these two solvers.
>>
>> 3. With C=1, it seems I have the same results at each run for the
>> solver='liblinear', but not for the solvers 'sag' and 'saga'. I still don't
>> get the same results between the different solvers.
>>
>>
>> Thanks for your help,
>> Best regards,
>> Ben
>>
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn