Custom Metrics for Monitoring

[Salvadiswar]

I'm currently using Trial version of DataRobot. I'm curious to know does the MLOps platform support the addition of custom metrics for monitoring into the platform through MLOps agent.

Ex: I need to add a new metrics which defines my model accuracy instead of existing metrics available. Is it possible to add those metrics into the UI through MLOps agents.

[IraWatt]

Hi Salvadiswar,

No I don't believe it is possible to add metrics into the UI. The metrics available in MLOps from the dropdown menu are the same as those supported by the Accuracy tab (Accuracy Available in MLOps ), and the Accuracy tab does not support custom metrics (Accuracy tab). 

Though I'm sure you could use the API to use your own custom metrics to trigger some action.

import datarobot as dr

project = dr.Project.get('5cc899abc191a20104ff446a')
model = project.get_models()[0]

deployment = Deployment.get(deployment_id='5c939e08962d741e34f609f0')
deployment.model['id'], deployment.model['type']
>>> ('5c0a979859b00004ba52e431', 'Decision Tree Classifier (Gini)')

#Calculate custom metric

[Salvadiswar]

Thank you @IraWatt