cancel
Showing results for 
Search instead for 
Did you mean: 

Retrieving SHAP feature impact job

Retrieving SHAP feature impact job

What is the suggested method to initiate SHAP impact job and then later retrieve the results?
I’ve tried different permutations of this:

 

feature_impact_job = dr.ShapImpact.create(project_id, model_id)
feature_impact_job_id = feature_impact_job.id

# later using the same job id
m = dr.Model.get(project_id, model_id)

feature_impact_job = m.get(project, str(feature_impact_job_id))
#feature_impact_job = dr.ModelJob.get(project, str(feature_impact_job_id))
#feature_impact_job = dr.model.get(project, str(feature_impact_job_id))
feature_impacts = feature_impact_job.get_result_when_complete(max_wait = 19200*5)

 

 

But apparently get() seems incapable of getting a SHAP job. The API documentation illustrates how to initiate a SHAP job but doesn’t show how to retrieve that job, so I’m at a loss. Spent many hours hitting my head against this

 

 

One of the errors I get (depending on which variant I try) is 404 'message' 'Not found'.   But another error message I got (when I tried another variant) is that there's no job with that id number.

 

And btw, in case that's a question, I _did_ set the project to SHAP mode in advanced options. 

Labels (1)
0 Kudos
7 Replies

Hi there,

If you're creating a ShapImpact like so:

feature_impact_job = dr.ShapImpact.create(project_id, model_id)

 

I'd suggest trying:

my_results = dr.ShapImpact.get(project_id, model_id)


As seen in docs here:

https://datarobot-public-api-client.readthedocs-hosted.com/en/v3.0.1/autodoc/api_reference.html?high...

 

Let me know if that's not helpful and I'll try and make sure myself or another employee at DataRobot can help you.

Thanks!

Thanks Crussell,

I see, so keep hitting it repeatedly in a while-loop until it doesn't give a 404 error if/when the job isn't complete?

0 Kudos

So I believe since the `.create()` methods returns a Job you can use the method `get_result_when_complete()` like so:

 

feature_impact_job = dr.ShapImpact.create(project_id, model_id)
feature_impact = feature_impact_job.get_result_when_complete()

 

Docs for that method here:

https://datarobot-public-api-client.readthedocs-hosted.com/en/v3.0.1/autodoc/api_reference.html#data...

0 Kudos

Thanks Crussell,

I see, so keep hitting it repeatedly in a while-loop until it doesn't give a 404 error if/when the job isn't complete?

 

BTW apparently the code I posted didn't format correctly at first.  Fixed now-- maybe that helps clarify what I've been doing (and why)

0 Kudos

Ok, so that worked, I put the shap job directly into the pandas df and retrieved it in the second for-loop:

 

# First for-loop
project_id = project.id
model_id = most_accurate_model.id
shap_impact_job = dr.ShapImpact.create(project_id = project_id, model_id = model_id)
df.at[row_num, 'shap_impact_job'] = shap_impact_job

# Second for-loop
shap_impact_job = df.at[row_num, 'shap_impact_job'] 
shap_impact = shap_impact_job.get_result_when_complete()
jenD
DataRobot Employee
DataRobot Employee

joshuamailman@northwesternmutual.com  Can we mark this solved?

0 Kudos

Just adding an example of some functionality here using a loop. If you want to go grab a number of different SHAP request results efficiently, you can use two loops in this way. The first loop makes the requests, letting DataRobot start that work in parallel across modeling workers, and then the second loop retrieves and stores the results.

 

shap_requests = []
project_id = project.id
for model in models:
    model_id = model.id
    shap_impact_job_temp = dr.ShapImpact.create(
        project_id = project_id, 
        model_id = model_id)
    shap_requests.append(shap_impact_job_temp)
shap_outputs = []
for shap_request in shap_requests:
    shap_impact_temp = shap_request.get_result_when_complete()
    shap_outputs.append(shap_impact_temp)

 

0 Kudos