cancel
Showing results for 
Search instead for 
Did you mean: 

Model Selection via Custom Metrics

Model Selection via Custom Metrics

When it comes to evaluating machine learning model performance, DataRobot provides many of the standard metrics out-of-the box, either on the model leaderboard or as part of a model insight. However, depending on the industry, you may need to sort your DataRobot leaderboard by a specific metric not natively supported by DataRobot. This AI Accelerator demonstrates how one can leverage DataRobot's python client to extract predictions, compute custom metrics, and sort their DataRobot models accordingly. The topics covered are as follows:

 

1. Setup: import libraries and connect to DataRobot
2. Build models with Autopilot
3. Retrieve predictions and actuals
4. Sort models by Brier Skill Score (BSS)
5. Sort models by Rate@Top1%
6. Sort models by return-on-investment (ROI)

Labels (4)
0 Replies