Based on the default modelling options, there is holdout data (20%) that is held.
I understand that once we unlock the holdout data, there is no going back.
My questions are:
1. When do we unlock the holdout data? At the very end of the process? If so, does it mean we can no longer run any other model, as the results would be affected?
2. Or can new models be run after unlocking the holdout data? Asking because I see that the validation, cross-validation and holdout scores are all different for the new model that I run. Is there a reference you may refer me to, in terms of how unlocking the holdout data affects new models being run after that?