cancel
Showing results for 
Search instead for 
Did you mean: 

Hyperparameter selection when training models for first time

Francisco99
Blue LED

Hi, I'm interested about how DR chooses Hyperparameters for each model. Can be that DR chooses them by a Nested Cross Validation technique? 

Best regards.

Labels (1)
4 Replies
lhaviland
Community Team
Community Team

Hi, and welcome to the community!

I can confirm that yes, DataRobot uses nested validation

Thank you for your question

Francisco99
Blue LED

Thanks for your answer!

 

So If I train models with 5-Fold Cross Validation, the procedure of DataRobot to search hyperparameters would be doing inner 5-Fold CV in each fold of the initial 5 partitions and then choosing the best set hyperparameters along all folds? 

0 Kudos
Lukas
Data Scientist
Data Scientist

Pretty much, yes.

 

On the inner CV folds that is.

The outer CV isn't used for tuning, only for model selection. And then we have the holdout of course. 

Francisco99
Blue LED

Ok, thanks! now is clearer. And if I use a train - validation procedure with DataRobot, the hyperparameters are chosen by the performance in validation set after training in train set? I mean that there is not a cross validation procedure when selecting hyperparameters with this approach, I'm right?

0 Kudos