One suggestion for a statistical solution is to run the models with and without those variables, export the error metric for each row - how yo do this will depend on which metric you are using. You will then end up with 2 or 3 distributions of data. An error distribution for the model with spatial/temporal data and an error distribution for the model(s) without spatial/temporal data. You can then do a non-parametric comparison (such as Wilcoxon) to see if those distributions are significantly different. If they are not, then you have statistical evidence that the temporal/spatial features are not significant.
You can support this finding with a lack of feature impact and minimal effects on partial dependence in the model that includes those features as well.
I hope this helps,