cancel
Showing results for 
Search instead for 
Did you mean: 

Inconsistency between PE in training and deployment

callmejack
Blue LED

Inconsistency between PE in training and deployment

When we checkout the prediction explanation in training it shows the values and explanation for top (let's say) 5 features.

They all have had positive impacts and it totally makes sense.

When I deploy the model for 5 PE, I get results with negative values. I expect to get top 5 features for each record the same as what I see in the UI of leaderboard for my selected model.

 

Labels (2)
0 Kudos
1 Reply
zach.ingbretsen
Data Scientist
Data Scientist

Hi Jack,

 

When you see prediction explanations on the leaderboard, DataRobot intentionally picks extreme predictions (some high, some low) to use as examples. In that situation, it's natural to expect that the most influential features for those predictions will be pointing in the same direction. The highest predictions likely have more features that are +, and the lowest predictions likely have more features that are -.

 

However, if you are looking at prediction explanations for more typical rows, there are likely mixed signals being sent; some features might be + while others might be - for a given row. This is something that naturally arises from the data and the models that result. The final prediction takes all the features into account.