By using the autopilot option I was able to build a regression model (light boosted tree) with a R2 of about 60% on test set. Then I deployed the model and tested it on a blind dataset. The R2 this time decreased from 60% to about 35%.
I tried to improve the model accuracy but w/o success.
Hi firstname.lastname@example.org - Here's a quick response that may help. Have you tried Get More Accuracy? Below the button you can see which modeling mode and feature list DataRobot suggests for the next modeling run, to get more accuracy.