Introduction to Model Monitoring

Showing results for 
Search instead for 
Did you mean: 

Introduction to Model Monitoring

You can find the latest information for deploying and monitoring in the DataRobot public documentation. Also, click ? in-app to access the full platform documentation for your version of DataRobot.

(Article updated July 2020)

This article showcases the Model Monitoring capabilities that come with DataRobot, which include service health, data drift, and accuracy monitoring.


Deploying a model as a REST API through DataRobot’s deploy method, means you have access to the wide range of monitoring capabilities available in the Deployments page.

Figure 1. Landing page of DataRobotFigure 1. Landing page of DataRobot


Figure 2. Deployments pageFigure 2. Deployments page

You’ll find a summary of all deployments that you have created (using DataRobot) or that have been shared with you. At the top of the page, you’ll see a summary for these deployments: how many there are, how many predictions have happened recently, and (most importantly) a synopsis of service health, data drift, and accuracy.

Below this information. You’ll find a list of all of the available deployments. To do a deep dive into one of them, you can just click on it.


When you select a model from the Deployments inventory page, DataRobot opens to that model’s Overview page. The overview provides a model-specific summary that describes the deployment, including the information you provided when creating the deployment and any model replacement activity.

You will see information similar to Figure 3.

Figure 3. Overview tabFigure 3. Overview tab

Service Health

The Service Health tab tracks metrics about a deployment’s ability to respond to prediction requests quickly and reliably. You can use this information to understand bottlenecks and assess capacity, which is critical to proper provisioning.

Figure 4. Service Health tabFigure 4. Service Health tab


Data Drift

When you deploy a model through DataRobot, training data is automatically uploaded with the model. The Data Drift dashboard leverages that data to help you to analyze a model’s performance after it has been deployed. The dashboard provides three interactive visualizations to identify the health of a deployed model over a specified time interval.

Figure 5. Data Drift tabFigure 5. Data Drift tab


The Accuracy tab enables you to analyze the performance of model deployments over time, using standard statistical measures and visualizations. Use this tool to determine whether a model’s quality is decaying and if you should consider replacing it. The Accuracy tab renders insights based on the problem type and its associated metrics—metrics that vary depending on regression or binary classification projects.


Figure 6. Accuracy tabFigure 6. Accuracy tab

Monitoring and Notification Settings

DataRobot provides a wide range of options to customize your deployments (as shown in Figures 7 and 8). For example, you can set a specific threshold for data drift based on your tolerance. When data drift is larger than that defined limit, DataRobot will flag the deployment’s data drift as either yellow or red, depending on the defined threshold.

Figure 7. Monitoring settingsFigure 7. Monitoring settings

Lastly, you can set the notifications you receive as well as the frequency for the notifications. Do you want to be warned only when something important happens, or do you want to receive an email notification every 2 hours even when your deployment is working as intended?

Figure 8. Notification settingsFigure 8. Notification settings

Sharing a Deployment

Every deployment in DataRobot can be shared with your colleagues. Figure 9 shows the procedure for sharing a deployment: simply insert the email of the person you want to share your project with.

Figure 9. Sharing a deploymentFigure 9. Sharing a deployment

There are also different roles you can assign based on the governance protocols of your company. An individual can be a consumer, a user, or an owner of a deployment.

More Information

See the DataRobot public documentation: Manage deploymentsOptimization metricsDeployment roles and permissions

Version history
Last update:
‎09-13-2021 12:37 PM
Updated by: