Unsupervised Learning with no Target

[opamuk]

Hi all,

I am aware of the anomaly detection of DataRobot. However, thinking of a dataset with no target variable, how can I proceed in performing an anomaly detection on my data? Can you share some experiences or know-how on this?

What I want to achieve:

Let's say, I have many records about a hospital's performed services. It can be that the services have faulty entries (let's say 5 times the same service for the same case) and I want to show this. The historical faulty services have not been flagged, therefore no target field to learn from.

Thanks in advance.

Regards,

Oktay

[opamuk]

Hi Emily, Thank you very much. I am looking forward for the new feature in the next release. I will try to arbitrary pick a target and continue. Regards, Oktay

[emily · Accepted solution]

Hi Oktay, 

I did a little digging on the specific release for the UX anomaly detection for autoML, and it is available as an early release feature.  Your admin will have to turn on the feature flag for you under the "settings" tab.    Keep in mind that this is not GA, and there could still be bugs in the feature.  

Once activated, you simply have to import your dataset and when you click on the empty "target" box an option will appear in orange below that says "Enable unsupervised mode (no target)". 

Alternatively, you can start a project and arbitrarily pick a column as a target and get the anomaly scores under the "Insights tab".  Autopilot automatically does some anomaly detection with each run.   This won't generally appear near the top of the leaderboard.  However, you can search for it and then retrain it on a larger percentage of the data (see below).  You can also check the Repository at the top and search for more anomaly detection blueprints.

   Here is some more information on anomaly detection in the platform.

I hope this helps!

Best wishes, 

Emily

[emily]

Hi Oktay, 

What version of DataRobot do you have? Are you on premise or cloud? 

Emily