cancel
Showing results for 
Search instead for 
Did you mean: 

Unable to generate data drift in my deployment

Unable to generate data drift in my deployment

Hi,

 

I had a new deployment in DataRobot using the mode of [external deployment](https://docs.datarobot.com/en/docs/mlops/deployment/deploy-workflows/ext-cus-model-ext-env.html). The model predictions are made externally but monitored by DataRobot MLOps agent.

 

My deployment was set up with "target monitoring" and "feature drift tracking" enabled. There is also a baseline training dataset.

 

However, in my dashboard there is no Data Drift information.

Screen Shot 2022-11-16 at 12.13.14.png

 

I had the prediction stat and data reported using `

report_deployment_stats` and `
report_predictions_data` API. There had been more than 1000 prediction data submitted.
 
Why is there no data drift in my dashboard?
 
Any help and tips are appreciated.
 
Thank you.
Labels (5)
0 Kudos
2 Replies

Hi DS_Di!

We've discussed this issue internally and came up with some thoughts, that may help to understand what happened better:

Is the agent running and connected to DataRobot MLOps? Check the local agent logs and then check the Service Health tab for service stats:
 
If you can see service stats, the agent is connected and the spooler is configured correctly. Check the local agent logs:
  • If the logs indicate successfully reported predictions, then it is most likely a monitoring issue: for example, the drift baseline wasn't provided or can't be accessed due to a permissions issue.

If you can't see service stats, this could be the result of a number of issues:

Thanks Bogdan for outlining the diagnosis steps. I have actually manged to resolve this issue. It's because one of the columns I passed in as prediction data does not comply with Datarobot requirement. The data drift is working now after I corrected it.

One suggestion for future deveopment is whethere there can be more meaningful error message returned from API calls. My API calls are sent to DataRobot without error, so I was asuuming everything I passed in was correct.

Speaking of the agent log, it's in a special fs_spool format which is not very friendly to be read.