@tore.christensen Thanks for the detailed response and yes you're right JDBC URL starts with `jdbc:databricks:...` and that's why i'm not able to establish a connection.
@gustavo.lira @andrius-senulis So, what are your thoughts on how we should proceed?
Regarding how to proceed; am I understanding correctly that you are now getting a green connection?
If you have successfully created a "Databricks (Spark)" data connection in the Data Connection UI here: https://app.datarobot.com/classic-account/data-connections
Then you should be able to successfully set up a Job Definition using that connection.
Just in case I misunderstood something I want to emphasize the correct connection to choose is as seen below marked with a green circle:
Here is how i got my "correct" jdbc url by the way:
I log in to Databricks, go to my compute cluster configuration, scroll to the buttom for advanced options, click odbc/jdbc and pick the version before 2.6.22
Getting jdbc url details
Thanks, @tore.christensen! The DJBC URL that starts with "spark" was the key I was missing. I was able to establish a connection using the Databricks (Spark) adapter and write back predictions.