Skip to main content

Use Existing Cluster

Specifies whether an existing Databricks cluster is used instead of automatically creating a new job cluster. If this option is enabled, specify the existing cluster name or ID in the Job Cluster setting.

When enabled, the generated Databricks Asset Bundle job tasks reference the cluster with existing_cluster_id, taking the ID from the Job Cluster setting. When disabled, the tasks use job_cluster_key and the job defines its own cluster from the databricks.yml variable named by the same setting. The option has no effect when Job Cluster is Serverless.

When to Change This Setting​

Keep the default N so each job provisions its own compute (or runs serverless). Enable it to run the jobs on an always-on cluster you manage yourself, for example to avoid job-cluster startup time in tightly scheduled environments.

SettingInteraction
Job ClusterHolds the existing cluster ID when this setting is enabled

Notes​

  • This setting is part of the Databricks settings category.
  • The default value for this setting is N.