What is the prediction and its strength

[Bruce]

I asked this recently on the other forum but maybe this is more the right place.

A classification prediction comes with what looks like a number between 0 and 1, call it the certainty, and also a collection of feature-strength-value triples. 

But, what exactly is strength? 

If one is computing a continuous variable from continuous variables then this could be the partial derivative (marginal statistics). And since the certainty is a continuous variable between 0 and 1, this is a valid measure. But, is that what it is? At the very least it should be scaled by some measure of spread of the input variables. And what if the input variable is an integer or even catagorical?

Can someone clarify this for me?

[TravisB]

Hi Bruce, in a classification problem the output values are propensity scores, which can then be converted to discrete predicted values by applying threshold(s).   A propensity score is not really the certainty, and I often caution folks not to (necessarily) think of it as true real-world probability either - unless the model is very accurate and very well-calibrated.  It can be more useful to think of propensity scores as relative - as many classification use cases tend to end up being a ranking exercise.  Say for example ordering by descending propensity score to understand highest likelihood or risk amongst the individuals that were scored.

Re: your question on strength-feature-value triples, I understand this as referring to the prediction explanations which can be provided alongside the predictions.  For models which support it, these are derived from SHAP values (Shapley Additive Explanations) - and for models which don't, from XEMP (Exemplar Based Explanations).  Prediction Explanations are documented here:

https://docs.datarobot.com/en/docs/modeling/analyze-models/understand/pred-explain/index.html

To answer your question, integer and categorical features are catered for - the high-level interpretation is:

These are the (say top 3 or whatever was specified) feature-values, their direction positive or negative, and a simple granular representation of their magnitude, which contributed to the propensity score for this individual.  So the 'strength' is the relative marginal influence on the predicted outcome, according to the feature value's numeric SHAP score for this individual.  The SHAP values are ordered by descending magnitude and the top X are shown - and these will be different per row/individual. 

This link to the docs goes into some detail on SHAP:

https://docs.datarobot.com/en/docs/modeling/reference/model-detail/shap.html

This general reference may also also useful:

https://shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainab...

Hope this helps.