On Thursday (April 23rd) DataRobot hosted a webinar discussing how effective Machine Learning Operations (MLOps) is key to getting the most from machine learning. We presented use cases and discussed potential problems with model management (including ML issues resulting from the global COVID-19 pandemic). In addition, we discussed proven and scalable methodologies for production model deployment, monitoring, and lifecycle management.
Click here to view the on-demand webinar "AI in Turbulent Times: Managing Models in Uncertain Times" and learn some MLOps best practices as presented by DataRobot experts.
We ran two polls during the webinar--wonder how you would answer each?
During the live presentation, we answered numerous questions but couldn't get to all of them. We've compiled those questions and answers and attached them to this post for your reference.
After watching the webinar
To follow up with Seph and Rajiv, click Reply and post your thoughts and questions. Let us know what you'd like to hear about!
For drift, we can look at both the variables/features for a model as well as the target. By measuring this over time, we can see how these change over time.
The other component is measuring the accuracy of the model over time. For this, you do need to evaluate the predictions and the actual value. We recommend starting to measure this as soon as you deploy the model and get back the actual values. This will involve keeping a history of predictions. But even if you start evaluating accuracy after your model has been deployed for some time, you can still start to evaluate how your model is decaying over time.