Configuring an ADF Linked Service Connection for Azure Cosmos DB for NoSQL
For information on how to enable a connection for use with linked services, see Configuring a Linked Service Connection.
After selecting Azure Cosmos DB for NoSQL from the Linked Service Type dropdown, the form required for creating an Azure Cosmos DB for NoSQL Linked Service will appear.
The Azure Cosmos DB for NoSQL option is offered on connections that use the Azure Cosmos DB for NoSQL System Type.
Datasets, Sources and Sinks
An Azure Cosmos DB for NoSQL connection serves the source system, the landing area and the persistent landing area stages.
The dataset addresses each object as a collection, taking the collection name from the TargetObjectName parameter.
Each Azure Cosmos DB for NoSQL object is read from the collection that the dataset names, and the copy source passes the TargetSchemaName and TargetObjectName parameters to the dataset.
An Azure Cosmos DB query for the collection (for example, SELECT c.id, c.name FROM c) is the object's Override SQL, or the text of the object's Override SQL extension point when the Override SQL is empty. The query is written to the ADF query property of the copy source, and BimlFlex appends nothing to the query it is given. An object with no query builds a copy source without one, so the whole collection is copied.
The copy source reads the setting keys below from the object's settings. Each key is a setting of the Setting Editor's Azure group, in the section Data Factory Copy Source Members, with the label the table names. These settings are blank by default. A value reaches an object as a Setting Override on that object. Add on the Azure group's Overrides tab opens the Add Setting Override form, where the Attribute Type is Object, the Connection and the Object select the object, the Setting is picked by its label (shown as Page Size (Data Factory Copy Source Members)), and the Override Value holds the member's text. A key that is empty or holds the default in the table is not written to the copy source.
| Setting Key | Label | ADF Property | Default |
|---|---|---|---|
pageSize | Page Size | pageSize | 0 |
detectDatetime | Detect Datetime | detectDatetime | false |
preferredRegions | Preferred Regions | preferredRegions | No default |
The preferredRegions value is a comma separated list of region names (for example, West US, East US), and each region is written as one entry of the preferredRegions list.
As a landing area or persistent landing area in a Data Factory or Fabric data pipeline project, the connection is the target of the copy. The copy sink writes to the collection the dataset names, and the copy passes the TargetSchemaName and TargetObjectName parameters to the dataset.
The sink carries no pre copy script. Documents are inserted into the collection by default, and replaced by key when the object's write behavior setting is upsert. The write behavior setting is the writeBehavior setting key of the object. The write batch size and the other sink members keep their defaults.
Choosing an Authentication Method
| Method | Credentials | Additional Settings |
|---|---|---|
| Account Key | The account key inside the Connection String, or an Azure Key Vault secret on the Account Key (Azure Key Vault) tab | None |
| Service Principal | The Tenant, the Service Principal Id and the Service Principal Credential, a key or a certificate entered as a sensitive value or an Azure Key Vault reference | Account Endpoint, Database Name, Service Principal Credential Type, Azure Cloud Type |
| System Assigned Managed Identity | The identity of the data factory | Account Endpoint, Database Name |
| User Assigned Managed Identity | The Credential that names the factory credential | Account Endpoint, Database Name |
Recommendation: Store the account key in Azure Key Vault on the Account Key (Azure Key Vault) tab, or the whole Connection String on the Connection String (Azure Key Vault) tab, rather than entering the key as plaintext.
Complete Example: Account Key in Azure Key Vault
| Field | Example Value |
|---|---|
| Connect via Integration Runtime | AutoResolveIntegrationRuntime |
| Connection String | AccountEndpoint=https://contoso-cosmos.example.com:443/;Database=sales |
| Azure Key Vault Linked Service | bfx-kv |
| Secret Name (Account Key (Azure Key Vault) tab) | BFX-SRC-CosmosNoSqlAccountKey |
The Connection String then carries the account endpoint and the database, and the account key comes from the secret.
Required Fields
Common Required Fields
An Azure Cosmos DB for NoSQL Linked Service connection selects one of four Authentication Types. The common required fields are required on every connection, and each Authentication Type adds the fields its section below names. A connection with no Authentication Type stored uses Account Key.
These common required fields are:
Authentication Type Specific Required Fields
- Account Key: Connection String (or Azure Key Vault)
- Service Principal: Account Endpoint, Tenant, Service Principal Id, Service Principal Credential (or Azure Key Vault)
- System Assigned Managed Identity: Account Endpoint
- User Assigned Managed Identity: Account Endpoint, Credential
Connect via Integration Runtime
Connect via Integration Runtime names the integration runtime for an Azure Cosmos DB for NoSQL Linked Service connection.
The default value is AutoResolveIntegrationRuntime.
To use a custom runtime, select one of the Azure Integration Runtimes saved in BimlFlex settings, or select + New and enter the name of the runtime.
A runtime added through + New is saved to the custom integration runtimes.
The custom values that appear in this dropdown can be maintained in Settings under Azure - AzureIntegrationRuntime.
The runtime is stored in the connection's LS_ConnectVia_IntegrationRuntime column. Any value other than AutoResolveIntegrationRuntime is written to the linked service as the runtime it connects through.
Azure Data Factory connects to Azure Cosmos DB for NoSQL through the Azure integration runtime or a self-hosted integration runtime. The Microsoft documentation names the self-hosted integration runtime for a data store located in a private network.
Connection String
The Connection String is used under the Account Key Authentication Type. The Connection String identifies the Azure Cosmos DB for NoSQL account and the database, in the account endpoint form the Microsoft documentation gives, AccountEndpoint=<EndpointUrl>;AccountKey=<AccessKey>;Database=<Database> (for example, AccountEndpoint=https://contoso-cosmos.example.com:443/;AccountKey=<AccessKey>;Database=sales).
It carries the account endpoint, the database and, unless the account key is stored in Azure Key Vault on the Account Key (Azure Key Vault) tab, the account key.
The Connection String field accepts the value as plaintext in the LS_ConnectionString column, or an Azure Key Vault reference on the Connection String (Azure Key Vault) tab, stored in the LS_ConnectionStringKVS_ fields, that resolves to the connection string at runtime. It maps to the ADF connectionString property.
It is suggested that Azure Key Vault be used in place of manually entering the Connection String.
Account Key (Azure Key Vault)
The Account Key (Azure Key Vault) tab takes an Azure Key Vault secret whose value is the account key, named by the Azure Key Vault Linked Service, the Secret Name and an optional Secret Version.
The reference is stored in the connection's LS_AccountKeyKVS_ fields and maps to the ADF accountKey property as an Azure Key Vault secret reference that resolves to the account key at runtime.
With this reference set, the Connection String carries the account endpoint and the database and no AccountKey.
The tab is optional.
Authentication Type
The Authentication Type select chooses how the linked service signs in to the Azure Cosmos DB for NoSQL account. It offers Account Key, Service Principal, System Assigned Managed Identity and User Assigned Managed Identity, stored as AccountKey, ServicePrincipal, SystemAssignedManagedIdentity and UserAssignedManagedIdentity.
The choice is stored in the connection's LS_AuthenticationType column and decides which members the linked service carries. It renders on every connection.
A connection with no Authentication Type stored opens on Account Key, keeps the column blank until a type is chosen, and builds the linked service with the account key.
Under Account Key the linked service carries the connection string and the account key. Under the three other types it carries the account endpoint and the database and no connection string.
A save keeps the members of the chosen Authentication Type and clears the members of the other types.
Account Endpoint
The Account Endpoint is the endpoint URL of the Azure Cosmos DB for NoSQL account (for example, https://contoso-cosmos.example.com:443/).
It renders under Service Principal, System Assigned Managed Identity and User Assigned Managed Identity, where it is required.
It is stored in the connection's LS_Endpoint column and maps to the ADF accountEndpoint property.
Database Name
The Database Name names the Azure Cosmos DB for NoSQL database (for example, sales).
It renders under Service Principal, System Assigned Managed Identity and User Assigned Managed Identity, where it is optional.
It is stored in the connection's LS_Db column and maps to the ADF database property.
Tenant
The Tenant identifies the tenant of the service principal.
It renders under Service Principal, where it is required.
It is stored in the connection's LS_Tenant column and maps to the ADF tenant property.
Service Principal Id
The Service Principal Id identifies the service principal that signs in.
It renders under Service Principal, where it is required.
It is stored in the connection's LS_ServicePrincipalId column and maps to the ADF servicePrincipalId property.
Service Principal Credential Type
The Service Principal Credential Type select names the kind of Service Principal Credential, ServicePrincipalKey for a key or ServicePrincipalCert for a certificate.
It renders under Service Principal, where it is optional. With no type selected and a Service Principal Credential set, the linked service carries ServicePrincipalKey.
It is stored in the connection's LS_ServicePrincipalCredentialType column and maps to the ADF servicePrincipalCredentialType property.
Service Principal Credential
The Service Principal Credential is the key or the certificate of the service principal.
It renders under Service Principal, where it is required.
The field accepts the credential as a sensitive value on the Service Principal Credential tab, stored in the LS_ServicePrincipalCredential column, or an Azure Key Vault reference on the field's Azure Key Vault tab, stored in the LS_ServicePrincipalCredentialKVS_ fields, that resolves to the credential at runtime. It maps to the ADF servicePrincipalCredential property.
A save keeps the sensitive value or the Azure Key Vault reference, whichever tab is selected, and clears the other.
Azure Cloud Type
The Azure Cloud Type select offers None, AzurePublic, AzureChina and AzureUsGovernment.
It renders under Service Principal, where it is optional. A blank Azure Cloud Type writes no member.
It is stored in the connection's LS_AzureCloudType column and maps to the ADF azureCloudType property.
Credential
The Credential names the factory credential of the user assigned managed identity.
It renders under User Assigned Managed Identity, where it is required.
It is stored in the connection's LS_Credential column and maps to the ADF credential property as a reference to that credential.
Azure Data Factory Linked Service Additional Information
For additional information on ADF Azure Cosmos DB for NoSQL Linked Service and its connection requirements see the Azure Data Factory Azure Cosmos DB for NoSQL Connector documentation.