Hyperparameter selection when training models for first time

[Francisco99]

Hi, I'm interested about how DR chooses Hyperparameters for each model. Can be that DR chooses them by a Nested Cross Validation technique? 

Best regards.

[Francisco99]

Ok, thanks! now is clearer. And if I use a train - validation procedure with DataRobot, the hyperparameters are chosen by the performance in validation set after training in train set? I mean that there is not a cross validation procedure when selecting hyperparameters with this approach, I'm right?

[Lukas]

Pretty much, yes.

On the inner CV folds that is.

The outer CV isn't used for tuning, only for model selection. And then we have the holdout of course. 

[Francisco99]

Thanks for your answer!

So If I train models with 5-Fold Cross Validation, the procedure of DataRobot to search hyperparameters would be doing inner 5-Fold CV in each fold of the initial 5 partitions and then choosing the best set hyperparameters along all folds? 

[Linda]

Hi, and welcome to the community!

I can confirm that yes, DataRobot uses nested validation

Thank you for your question 👍