Using Custom Metrics to effectively monitor Generative AI

Showing results for 
Search instead for 
Did you mean: 

About this Accelerator

While more and more users have started building Generative AI solutions, their organizations have the tough job of monitoring these applications. Monitoring Generative AI Solutions or LLMOps is a multi-faceted endeavor and requires more than simple out of the box metrics. 

Each business is unique and has different metrics or different angles to monitor their solutions and models on. DataRobot provides tools like custom metrics that allow organizations to monitor their generative solutions easily and at the same time holistically. Custom Metrics allow organizations to define even complex metrics and record the measurements on the deployed Generative AI model. 

With this notebook, we aim to not only show how to monitor generative AI solutions effectively but also the different segments the metrics fall into. This segmentation framework can help organizations define and decide the metrics to effectively monitor their Generative models. 



Labels (1)
Version history
Last update:
‎10-20-2023 07:40 AM
Updated by: