[scikit-learn] RFE with logistic regression

[Benoît Presles]

Dear scikit-learn users,

I am using the recursive feature elimination (RFE) tool from sklearn to 
rank my features:

from sklearn.linear_model import LogisticRegression
classifier_RFE = LogisticRegression(C=1e9, verbose=1, max_iter=10000)
from sklearn.feature_selection import RFE
rfe = RFE(estimator=classifier_RFE, n_features_to_select=1, step=1)
rfe.fit(X, y)
ranking = rfe.ranking_
print(ranking)

1. The first problem I have is when I execute the above code multiple 
times, I don't get the same results.

2. When I change the solver to 'sag' or 'saga' (classifier_RFE = 
LogisticRegression(C=1e9, verbose=1, max_iter=10000), solver='sag'), it 
seems that I get the same results at each run but the ranking is not the 
same between these two solvers.

3. With C=1, it seems I have the same results at each run for the 
solver='liblinear', but not for the solvers 'sag' and 'saga'. I still 
don't get the same results between the different solvers.


Thanks for your help,
Best regards,
Ben