Ast Adf Azure Databricks Linked Service Element
X-Ref:
- Ast Adf Azure Databricks Linked Service Node
- Ast Adf Linked Service Base Node
- Ast Adf Root Object Node
- Ast Scope Boundary Node
Attributes
| Attribute | API Type | Default | Description |
|---|---|---|---|
| Name | String | Specifies the name of the object. This name can be used to reference this object from anywhere else in the program. This is a required property | |
| WorkspaceUri | String | Databricks URL with the Azure Region specified appropriately. For example: https://westeurope.azuredatabricks.net This is a required property | |
| AccessToken | String | This is required in order to authenticate Azure Databricks access. They need to be generated from the databricks workspace. | |
| Authentication | AdfAzureDatabricksAuthenticationType | AccessToken | Specifies how the Databricks cluster access should be authenticated. |
| ClusterVersion | String | This is the cluster's version of Spark. | |
| EncryptedCredential | String | Specifies the encrypted credential that will be used to authenticate the linked service. | |
| ExistingClusterId | String | This is the Cluster ID of an an already created Interactive Cluster. In the event that the cluster stops responding, you may need to restart it. Databricks recommends using new clusters for increased reliability. | |
| InstancePoolId | String | The id of an existing instance pool that will be used for all runs of this activity. | |
| LogDestination | String | Specify a location to deliver Spark driver, worker, and event logs. | |
| LogicalDisplayFolder | String | Specifies a path that should be used for organizing the tree display of this object in the Logical View in BimlStudio. | |
| NodeType | String | This property encodes the resources available to each of the Spark nodes present in the cluster. This is required for new clusters. | |
| WorkerCount | String | The intended number of worker nodes for a cluster. Either specify a string formatted Int32 for a precise number, or specify two Int32s, sperated by a colon, in order to specify auto-scale range. For example "1:10", where 1 is the min and 10 is the max. | |
| WorkerScaleOption | AdfAzureDatabricksWorkerScaleOption | Fixed | Specifies whether a Databricks cluster should autoscale workers or use a fixed worker count. |
| WorkspaceResourceId | String | Workspace resource id for databricks REST API. |
Singleton Children
| Child | API Type | Description |
|---|---|---|
| <AccessTokenKVS /> | AstAdfKeyVaultSecretNode | Defines a field in a Linked Service that references a key vault secret. This allows users to reference a password without putting it explicitly in the Azure Data Factory json. |
| <ConnectVia /> | AstAdfLinkedServiceConnectViaNode | Specifies the integration runtime that should be used to connect to the selected linked service. |
Collection Children
| Child | API Type | Description |
|---|---|---|
| <AdfAnnotations> <AdfAnnotation /> </AdfAnnotations> | String | This is the collection of annotations that will be emitted in the json. |
| <Annotations> <Annotation /> </Annotations> | AstAnnotationNode | This is a collection of annotation items that can be used to specify documentation, tags, or other information. Annotations are particularly useful for storing information about nodes that can be used by BimlScript code. |
| <CustomTags> <CustomTag /> </CustomTags> | AstAdfDatabricksBaseParameterNode | Custom tags for your Databricks linked service. |
| <InitScripts> <InitScript /> </InitScripts> | String | XXX |
| <SparkConfigurations> <Configuration /> </SparkConfigurations> | AstAdfDatabricksBaseParameterNode | Spark configuration property values for the Spark program environment. Key/Value format. |
| <SparkEnvVars> <EnvVar /> </SparkEnvVars> | AstAdfDatabricksBaseParameterNode | Spark configuation property values for the Spark program environment. Key/Value format. |