Setting Up OTV Project in Python

[dd]

I'm having issues with customizing the backtests

spec = dr.DatetimePartitioningSpecification(datetime_partition_column = "date",
                                            holdout_start_date=datetime(2023,5,11),
                                            holdout_end_date=datetime(year=2023, month=7, day=10),
                                            #number_of_backtests = 3,
                                            use_time_series = False)
#spec.number_of_backtests = 2
spec.backtests=[ dr.BacktestSpecification(0,
                                         validation_start_date = datetime(2022,11,10), 
                                         validation_end_date = datetime(2023,5,10)),

                dr.BacktestSpecification(1,
                                         validation_start_date = datetime(2022,5,9), 
                                         validation_end_date = datetime(2022,11,9))]


partitioning_preview = dr.DatetimePartitioning.generate_optimized(project.id, spec,"target")
# the preview generated is based on the project's data
project.set_datetime_partitioning = spec
print(partitioning_preview.to_dataframe())
cleanFL = set(feature_list.Feature_list) - {"Date","target"}
featurelist = project.create_featurelist('CleanFL2', list(cleanFL))

#%% md

.  Whenever I set spec.backtests I get the following error when I run this code. I tried so many variations, but still cannot get it to work.  Any help will be appreciated

InvalidUsageError: Only one of (gap_duration, validation_duration, validation_start_date) or (validation_start_date, validation_end_date, primary_training_start_date, primary_training_end_date) can be used to configure backtests.

[crussellwalker]

Hi there, I hope I can help.

I'm looking at the docs here:

https://datarobot-public-api-client.readthedocs-hosted.com/en/latest-release/autodoc/api_reference.h...

It looks as though for your two `BacktestSpecification` in your supplied code you need to provide the parameters: `primary_training_start_date` and `primary_training_end_date`

I hope that's helpful - please let us know if that doesn't get you what you need - thanks!

[dd · Accepted solution]

I solved it, one has to first set the number of backtests and then set backtests

spec = dr.DatetimePartitioningSpecification(datetime_partition_column = "Date",
                                            holdout_start_date=datetime(2023,5,11),
                                            holdout_end_date=datetime(year=2023, month=7, day=10),
                                            number_of_backtests= 2,
                                            backtests=[ dr.BacktestSpecification(index=0,
                                                        primary_training_start_date=datetime(year=2016, month=11, day=19),
                                                     primary_training_end_date=datetime(year=2022, month=11, day=10),
validation_start_date = datetime(2022,11,10),
validation_end_date = datetime(2023,5,10) ),
                                                        dr.BacktestSpecification(index=1,
                                         primary_training_start_date=datetime(year=2016, month=11, day=19),
                                         primary_training_end_date=datetime(year=2022, month=5, day=9),
validation_start_date = datetime(2022,5,9), 
validation_end_date = datetime(2022,11,9))],
use_time_series = False)

[dd]

Thank you for the quick response.  I did on previous version, but the same problem.  I just figured out:  I need to set the number of backtests before modifying them.

[crussellwalker]

That's great - I appreciate you letting us know.