A-B testing models?

[krista-kelly]

Hi Is there a way to deploy 2 models to do A/B testing? hoping yes and that you can give an example?

[zmarkan]

Hi @krista-kelly 

A/B testing of models is possible with our MLOps product. MLOps has a feature called Challenger Models. The idea is that you deploy multiple models at the same time - one Champion model, and the others as Challenger models. While only the Champion model is used to make predictions, all models' performance is constantly evaluated and if a Challenger model performs better than the Champion you can easily replace it.

The feature is described in this blog post:
https://www.datarobot.com/blog/introducing-mlops-champion-challenger-models/ and we will share a tutorial once one is available.