cancel
Showing results for 
Search instead for 
Did you mean: 

Challenger mode and retraining

seanwu95
Blue LED

Hi, for challenger mode, does it retrain with new data, but keeping my current hparams and model architecture? Or it actually runs new feature engineering and architecture search?

If I want to use the current "champion model" HParam and architecture, but retraining the model as new data comes in, how do I do it? Thanks

2 Replies
lhaviland
Community Team
Community Team

Hi @seanwu95  - Until other community members answer your question, here's a quick post about challenger models you can have a look at (if you haven't already!) 

-linda

0 Kudos
doyouevendata
DataRobot Employee
DataRobot Employee

Note the challenger models exist separate from any retraining.  A deployment in DataRobot can have 0 to many challenger models behind it.  Scoring requests come in and are satisfied by the currently set active model behind the deployment.  Scoring request data is accrued over time so that it can be run through challenger models at a later period, to preserve scoring resources during requests.  Eg. Sunday mornings at 3am, score the prior week's data through each challenger model.  They can then be evaluated, where a new champion may be chosen.

Although we are in the process of developing some automated retraining as well, at present one would leverage the Python SDK to observe triggers and orchestrate actions, such as making a decision to retrain, and orchestrating the process of getting the latest data into DataRobot to train and potentially deploy a model (on its own, as a challenger to an existing deployment, or replacing the champion model in an existing deployment.)