Eureqa fine tuning

redenton
Linear Actuator

Eureqa fine tuning

Every time I run a Eureqa model, I reset very many of the fine tuning parameters to the same values. Is there a way to save the set of parameters, so I don't have to start from scratch every time?

9 Replies

REdenton there's a documentation topic that seems to have the answer you need - https://docs.datarobot.com/en/docs/modeling/analyze-models/describe/eureqa.html#export-model-paramet... Hope it helps you

You could use composable ML (not available for trial users), which lets you save your favorite advanced tuning parameters in a blueprint.   

Thank you for getting back to me! I will try that and report as to whether that works.

0 Kudos

@tobysimpson The export option is interesting, I guess I can use it as a reference. But is there a way to import those settings for a new Eureqa run?  Then I wouldn't have to redo all the settings manually.

Hi @redenton looks like @Lukas gave you some ideas for using Python to do advanced tuning. If you've tried this out, can you share your findings with the Community? Interested to hear what you may have done for your use case.

- Linda

0 Kudos

@Linda

If I understand correctly, the suggestion is to use Python with a DataRobot API tokin to run DataRobot locally. As I understand it, this option is not available for a free account. I will pay to use DataRobot once I've used up my free time, but that hasn't happened yet. Perhaps if I paid for some extra time, I could do it, but I'm not familiar with Python and suspect that I don't want to try to use this approach. Thanks, Richard

0 Kudos

@redenton The Getting Started in Trial post (https://community.datarobot.com/t5/ai-cloud-platform-trial/getting-started-in-the-ai-cloud-platform-...) indicates you can use Python and R with the Trial:
A: Yes, the trial fully supports API access. DataRobot is accessible via API (see the developers site). There is also an R client and a Python client, with lots of examples on our community GitHub.

So I think the suggestion from @Lukas may be doable for you. 

- Linda

0 Kudos

Hi @redenton,

 

I can maybe get you started on how to do things in python, it's not very complex (but a little complex). We also have the same functionality in R, if you are more familiar with that.

 

I will stick to python for now, since I know it well.

 

The first thing you need is python and the datarobot library installed (pip install datarobot)

 

Then you will need an API token. You can create one via the menu top right corner -> Dev tools

Lukas_0-1641974538681.png

And here create a new key

Lukas_1-1641974568775.png

 

 

Next, let's assume you have a project created already, with an existing eureqa model.

If we navigate to that model, we can find a project_id and a model_id in the URL (the long strings)

 

Lukas_2-1641974647821.png

 

Now we're ready to start with our python script.

 

import datarobot as dr

# Connect to datarobot with your token
client = dr.Client(endpoint='https://app2.datarobot.com/api/v2', token='XYZ')

# Find the right project
project = dr.Project.get('6156fdd3000984791f39660c')

# and then the Eureqa model
eureqa_model = dr.Model.get(project.id, '6157042bbdcc650eb28a82d8')

# Now we can start the tuning session
tuning_session = eureqa_model.start_advanced_tuning_session()

A tuning session allows us to modify any parameter from any task (where 'task' is something like a preprocessing step, or a model, and parameter is then one of the hyperparameters we are interested in)

For the model I selected I see these tasks:

Lukas_3-1641975016621.png

 

And when I ask for the parameters specific to the Eureqa model, I get a long list of parameters, exactly what we're after.

Lukas_4-1641975065211.png

 

So, let's say we want to enable a few of the building blocks, we can do something like this

# get all the parameters for all the taskes
params = tuning_session.get_parameters()

# Predefine a few tuning settings we are interested in changing
settings = {
'building_block__less-than':4,
'building_block__round':3,
'building_block__sine':5,
'building_block__square_root':2
}

# now select those tasks we are interested in
params_to_tune = [p for p in params['tuning_parameters'] if
p['task_name'] == 'Eureqa Classifier (Quick Search: 250 Generations)'
and p['parameter_name'] in settings]

These params to tune now have this form, but we can ignore most of this noise, all we need is the names and the ID

Lukas_5-1641975237228.png

So, to tune the model, all that's left is this:

 

for p in params_to_tune:
task_name = p['task_name']
parameter_name = p['parameter_name']
parameter_id = p['parameter_id']
value = settings[parameter_name]
tuning_session.set_parameter(value, task_name, parameter_name, parameter_id)

tuning_session.run()

You should now see a new eureqa model in the web interface with your tuning settings.

 

For convenience, here's the full code again to copy/adjust/paste:

 

import datarobot as dr

client = dr.Client(endpoint='https://app2.datarobot.com/api/v2', token='XYZ')
project = dr.Project.get('6156fdd3000984791f39660c')
eureqa_model = dr.Model.get(project.id, '6157042bbdcc650eb28a82d8')
tuning_session = eureqa_model.start_advanced_tuning_session()

# Run, and note down the actual task name for your model
tuning_session.get_task_names()

tuning_session.get_parameter_names('Eureqa Classifier (Quick Search: 250 Generations)')
params = tuning_session.get_parameters()
settings = {
'building_block__less-than':4,
'building_block__round':3,
'building_block__sine':5,
'building_block__square_root':2
}

params_to_tune = [p for p in params['tuning_parameters'] if
p['task_name'] == 'Eureqa Classifier (Quick Search: 250 Generations)'
and p['parameter_name'] in settings]

for p in params_to_tune:
task_name = p['task_name']
parameter_name = p['parameter_name']
parameter_id = p['parameter_id']
value = settings[parameter_name]
tuning_session.set_parameter(value, task_name, parameter_name, parameter_id)

tuning_session.run()

 

I hope that helps!

 

Kind regards,

Lukas