Configuring an ADF Linked Service Connection for Azure Data Lake Storage Gen1
For information on how to enable a connection for use with linked services, see Configuring a Linked Service Connection.
Field labels in this walkthrough follow the underlying Azure Data Factory property names, with the connection column that carries each value named alongside. The exact labels shown in the BimlFlex Connection Editor may differ.
After selecting Data Lake Gen 1 from the Linked Service Type dropdown, the form required for creating an Azure Data Lake Storage Gen1 Linked Service will appear.
Microsoft has retired the Azure Data Lake Storage Gen1 service and asks its users to migrate to Azure Data Lake Storage Gen2. For a Gen2 connection, see Configuring an ADF Linked Service Connection for Data Lake Gen 2.
File Format Connections
Azure Data Lake Storage Gen1 is selected as the Linked Service Type of a connection whose System Type is one of the file formats Delimited Format, Avro Format, Binary Format, JSON Format, ORC Format, Parquet Format, XML Format or Delta Lake.
The Data Lake Gen 1 option appears in the Linked Service Type dropdown when the connection's Connection Type is Azure Data Lake Store.
Choosing an Authentication Method
| Method | Credentials | Additional Settings |
|---|---|---|
| Service Principal | Tenant, Service Principal Id and Service Principal Key | dataLakeStoreUri |
| System-assigned Managed Identity | None | dataLakeStoreUri |
| User-assigned Managed Identity | Credential | dataLakeStoreUri |
For a sink, the Additional Settings also carry subscriptionId and resourceGroupName.
Recommendation: Store the Service Principal Key in Azure Key Vault rather than entering it as plaintext.
Complete Example: Service Principal with Azure Key Vault
| Field | Example Value |
|---|---|
| Connect via Integration Runtime | AutoResolveIntegrationRuntime |
Data Lake Store URI (Additional Settings key dataLakeStoreUri) | https://contosolake.azuredatalakestore.net/webhdfs/v1 |
| Tenant | contoso.onmicrosoft.com |
| Service Principal Id | <application-client-id> |
| Azure Key Vault Linked Service | bfx-kv |
| Secret Name (for Service Principal Key) | BFX-LND-DataLakeGen1ServicePrincipalKey |
Required Fields
Common Required Fields
Most required fields for an Azure Data Lake Storage Gen1 Linked Service connection depend on the authentication method used. There are some fields that are required regardless of the authentication method.
These common required fields are:
Required when the store is written to as a sink:
Optional fields are:
Authentication Method Dependent Required Fields
The remaining field requirements are dependent on the authentication method.
Required fields for Service Principal:
- Tenant
- Service Principal Id
- Service Principal Key (or Azure Key Vault)
Required fields for System-assigned Managed Identity:
- None beyond the common required fields
Required fields for User-assigned Managed Identity:
- Credential, set in the
LS_Credentialcolumn outside the connection form
Connect via Integration Runtime
Connect via Integration Runtime names the integration runtime for an Azure Data Lake Storage Gen1 Linked Service connection.
The default value is AutoResolveIntegrationRuntime.
To use another runtime, select it from the dropdown or choose + New to add one.
When a custom value is saved in the linked service form, it will be added to the custom integration runtimes.
The custom values that appear in this dropdown can be maintained in Settings under Azure - AzureIntegrationRuntime.
The runtime is stored in the connection's LS_ConnectVia_IntegrationRuntime column. Any value other than AutoResolveIntegrationRuntime is written to the linked service as the runtime it connects through.
Data Lake Store URI
The Data Lake Store URI identifies the Azure Data Lake Store account.
It takes the form https://<accountname>.azuredatalakestore.net/webhdfs/v1 or adl://<accountname>.azuredatalakestore.net/.
The Data Lake Store URI is stored in the connection's Additional Settings (LS_AdditionalSettings) under the key dataLakeStoreUri, and maps to the ADF dataLakeStoreUri property.
Subscription ID and Resource Group Name
The Subscription ID is the Azure subscription that holds the Data Lake Store account, and the Resource Group Name is the resource group that holds it. Azure Data Factory requires both when the store is written to as a sink.
The Subscription ID is stored in the connection's Additional Settings (LS_AdditionalSettings) under the key subscriptionId, and maps to the ADF subscriptionId property.
The Resource Group Name is stored in the connection's Additional Settings under the key resourceGroupName, and maps to the ADF resourceGroupName property.
Account Name
The Account Name is stored in the connection's Additional Settings (LS_AdditionalSettings) under the key accountName, and maps to the ADF accountName property.
It is optional. When the key is absent, the linked service carries no accountName property.
Authentication Method
The Azure Data Lake Storage Gen1 Linked Service connection can use Service Principal, System-assigned Managed Identity or User-assigned Managed Identity authentication.
BimlFlex does not read the connection's Authentication Type column for this connector. The fields that are filled in decide the form.
When the connection's LS_EncryptedCredential column holds an Encrypted Credential value, it is also written to the linked service.
The connection form offers no Encrypted Credential field, so the LS_EncryptedCredential column is set outside the form.
Service Principal
Service Principal authentication signs in with a Microsoft Entra application that has been granted access to the Data Lake Store.
The Tenant is stored in the connection's LS_Tenant column and maps to the ADF tenant property.
The Service Principal Id supplies the application's client id and is stored in the connection's LS_ServicePrincipalId column. It maps to the ADF servicePrincipalId property.
The Service Principal Key supplies the application's key as plaintext in the LS_ServicePrincipalKey column, or as an Azure Key Vault reference held in the LS_ServicePrincipalKeyKVS_ fields that resolves to the key at runtime. It maps to the ADF servicePrincipalKey property.
The Service Principal Key and the Tenant are written to the linked service when the Service Principal Id is set.
It is suggested that Azure Key Vault be used in place of manually entering the Service Principal Key.
System-assigned Managed Identity
System-assigned Managed Identity authentication uses the managed identity of the data factory, which is granted access to the Data Lake Store. Leave the Tenant, Service Principal Id and Service Principal Key empty. The linked service then carries only the Data Lake Store information, which is the form Azure Data Factory uses for this method.
User-assigned Managed Identity
User-assigned Managed Identity authentication uses a credential created in the data factory for a user-assigned managed identity.
When the connection's LS_Credential column names a credential, it is written to the linked service as the ADF credential reference.
The connection form offers no Credential field, so the LS_Credential column is set outside the form.
Azure Data Factory Linked Service Additional Information
For additional information on ADF Azure Data Lake Storage Gen1 Linked Service and its connection requirements see the Azure Data Factory Azure Data Lake Storage Gen1 Connector documentation.