cancel
Showing results for 
Search instead for 
Did you mean: 

Example monitoring notebook is not recording predictions

Example monitoring notebook is not recording predictions

Hello there,

I'm running this notebook in colab (custom_model_notebook) following the instructions provided in the notebook. The mlops agent starts and then exits with "Folder provided './tmp/ta' for FSRecordSpooler does not exist" in the log even though I created the spool folder running the code provided in the cell. 

I went ahead with running the cells. After running the prediction cells, I see a file (fs_spool.1) in the spool directory. A new model package was created in the DataRobot UI but no prediction was recorded. 

All cell outputs match with the original notebook except for the mlops agent status cell. 

What am I missing? 

Labels (4)
0 Kudos
1 Solution

Accepted Solutions

Hi @abhi_mle, it seems that when the agent runs, if you set a relative path in the config file (e.g., `./tmp/ta`) it is actually relative to the agent's bin directory and not relative to the directory where you started the agent.

 

So, if you set the directory to "./tmp/ta" in the config file, you would need to run the following (assuming you are using the 8.2.7 version of the agent):

!mkdir -p ./datarobot_mlops_package-8.2.7/bin/tmp/ta

and set the SPOOL_DIR the same way :

SPOOL_DIR = "./datarobot_mlops_package-8.2.7/bin/tmp/ta"

I don't know if this is new behavior for the agent, but I'll see if I can get the notebook updated.

 

I would probably recommend setting an absolute path, rather than a relative path.

View solution in original post

4 Replies

Hi abhi_mle!

Could you show what directory Colab sees, by running "!ls" in a notebook cell? This error might be because the colab file system might works differently than the local FS.

Hi @Bogdan Tsal-Tsalko , thanks for your reply.

On the colab notebook,
!ls
custom-models datarobot_mlops_package-8.2.7 sample_data tmp
!ls -a ./tmp
. .. ta
 
 
0 Kudos

Hi @abhi_mle, it seems that when the agent runs, if you set a relative path in the config file (e.g., `./tmp/ta`) it is actually relative to the agent's bin directory and not relative to the directory where you started the agent.

 

So, if you set the directory to "./tmp/ta" in the config file, you would need to run the following (assuming you are using the 8.2.7 version of the agent):

!mkdir -p ./datarobot_mlops_package-8.2.7/bin/tmp/ta

and set the SPOOL_DIR the same way :

SPOOL_DIR = "./datarobot_mlops_package-8.2.7/bin/tmp/ta"

I don't know if this is new behavior for the agent, but I'll see if I can get the notebook updated.

 

I would probably recommend setting an absolute path, rather than a relative path.

Hi @zach.ingbretsen @Bogdan Tsal-Tsalko ,
mlops agent was running after I created spool directory relative to the agent's bin directory following your answer.

Really appreciate your quick responses and support.

0 Kudos