For this article, let's assume you have a deployment and are actively making predictions, but now you want to see how well your model is performing. To do this, you need to upload the actual outcome data and associate it with the predictions that were made.
To track the model accuracy we need to first import the actuals data into the AI Catalog. The AI Catalog is your own dedicated storage resource and provides a centralized way to manage data sets from different data sources. We won't go into the many features it has except to say that you will upload and store your actuals data here.
To do so, click the AI Catalog tab at the top of the screen and then click Add to catalog. Select the source of the data you want to upload and then select the data; in this example we’re uploading a local file.
Figure 1. AI Catalog
Navigate back to the Deployment dashboard (click the Deployments tab at the top of the screen) and select your deployment.
Now, to return to the previous page where we created the deployment, click the Settings tab, and we see the Actuals section is now enabled. Click AddData to locate the actuals data from the AI Catalog. From here, indicate the Actuals Response column (which holds your actual outcome results), the Association ID column to link back to the predictions made, an optional column name to keep a record of what action was taken given the result, along with a timestamp if you want to keep track of when the actual values were obtained.
Figure 2. Actuals data entry
Click the Accuracy tab and you’ll see how the predictions perform in comparison to the actual outcomes.
If you’re a licensed DataRobot customer, search the in-app Platform Documentation for Settings tab.