Do I need to re-run the training process if I want to use a different metric?

cancel
Showing results for 
Search instead for 
Did you mean: 

Do I need to re-run the training process if I want to use a different metric?

Sometimes you might want to evaluate a model’s performance by using a different metric. DataRobot allows you to access a different metric without re-training the models.

How to change the project metric through the GUI

  1. Select the Models tab.
    The Metric menu in the Leaderboard shows the current metric.
    lhaviland_0-1609269189527.png
  2. Select a different metric: the values for that metric are shown

How to access different metrics through the API

Although it is not yet possible to change the metrics for the whole project through API similar to GUI, you can still retrieve the scores for each model.

Here is the example on how to access AUC score for a particular model using DataRobot Python Client.

  1. Retrieve the model object.
  2. Call .metrics function on the model object. This returns a list of metrics and associated scores.
  3. Extract the particular metric from the list.

    For example, model.metrics['AUC'] will provide output similar to this:

    {'backtesting': None,
    'holdout': 0.69721,
    'backtestingScores': None,
    'crossValidation': 0.6956260000000001,
    'validation': 0.70403}​

You can also request model.metrics['AUC']['crossValidation'] to retrieve the score for Cross Validation only.

For more information, see the in-app API Documentation. Also, you can read the DataRobot Python Client documentation from here.

- Updated December 2020

Labels (1)
Version history
Revision #:
6 of 6
Last update:
3 weeks ago
Updated by:
 
Contributors