[scikit-learn] Specifying exceptions to ParameterGrid
Jaidev Deshpande
deshpande.jaidev at gmail.com
Wed Nov 23 05:15:07 EST 2016
Hi,
Sometimes when using GridSearchCV, I realize that in the grid there are
certain combinations of hyperparameters that are either incompatible or
redundant. For example, when using an MLP, if I specify the following grid:
grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',
'invscaling', 'adaptive']}
then it yields the following ParameterGrid:
[{'learning_rate': 'constant', 'solver': 'sgd'},
{'learning_rate': 'constant', 'solver': 'adam'},
{'learning_rate': 'invscaling', 'solver': 'sgd'},
{'learning_rate': 'invscaling', 'solver': 'adam'},
{'learning_rate': 'adaptive', 'solver': 'sgd'},
{'learning_rate': 'adaptive', 'solver': 'adam'}]
Now, three of these are redundant, since learning_rate is used only for the
sgd solver. Ideally I'd like to specify these cases upfront, and for that I
have a simple hack (
https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38).
Using that yields a ParameterGrid as follows:
[{'learning_rate': 'constant', 'solver': 'adam'},
{'learning_rate': 'invscaling', 'solver': 'adam'},
{'learning_rate': 'adaptive', 'solver': 'adam'}]
which is then simply removed from the original ParameterGrid.
I wonder if there's a simpler way of doing this. Would it help if we had an
additional parameter (something like "grid_exceptions") in GridSearchCV,
which would remove these dicts from the list of parameters?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161123/976884d7/attachment.html>
More information about the scikit-learn
mailing list