Configuring an ADF Linked Service Connection for Amazon S3 Compatible Storage
For information on how to enable a connection for use with linked services, see Configuring a Linked Service Connection.
Field labels in this walkthrough follow the underlying Azure Data Factory property names, with the connection column that carries each value named alongside. The exact labels shown in the BimlFlex Connection Editor may differ.
After selecting Amazon S3 Compatible from the Linked Service Type dropdown, the form required for creating an Amazon S3 Compatible Storage Linked Service will appear.
File Format Connections
Amazon S3 Compatible Storage is selected as the Linked Service Type of a connection whose System Type is one of the file formats Delimited Format, Avro Format, Binary Format, JSON Format, ORC Format, Parquet Format, XML Format or Delta Lake.
The Amazon S3 Compatible option appears in the Linked Service Type dropdown when the connection's Connection Type is Amazon S3.
Choosing an Authentication Method
| Method | Credentials | Additional Settings |
|---|---|---|
| Access Key | Access Key ID and Secret Access Key | forcePathStyle (optional) |
Recommendation: Store the Secret Access Key in Azure Key Vault rather than entering it as plaintext.
Complete Example: Access Key with Azure Key Vault
| Field | Example Value |
|---|---|
| Connect via Integration Runtime | AutoResolveIntegrationRuntime |
| Access Key ID | <access-key-id> |
| Service URL | https://objects.contoso.com |
| Azure Key Vault Linked Service | bfx-kv |
| Secret Name (for Secret Access Key) | BFX-LND-S3CompatibleSecretAccessKey |
Required Fields
Common Required Fields
An Amazon S3 Compatible Storage Linked Service connection uses a single Authentication Type, so the same fields are required on every connection.
These common required fields are:
Optional fields are:
Connect via Integration Runtime
Connect via Integration Runtime names the integration runtime for an Amazon S3 Compatible Storage Linked Service connection.
The default value is AutoResolveIntegrationRuntime.
To use another runtime, select it from the dropdown or choose + New to add one.
When a custom value is saved in the linked service form, it will be added to the custom integration runtimes.
The custom values that appear in this dropdown can be maintained in Settings under Azure - AzureIntegrationRuntime.
The runtime is stored in the connection's LS_ConnectVia_IntegrationRuntime column. Any value other than AutoResolveIntegrationRuntime is written to the linked service as the runtime it connects through.
Authentication Type
The Amazon S3 Compatible Storage Linked Service connection signs in with an access key: the Access Key ID and the Secret Access Key.
Access Key ID
The Access Key ID identifies the access key the connector signs in with.
It is stored in the connection's LS_AccessKeyId column and maps to the ADF accessKeyId property.
Secret Access Key
The Secret Access Key field accepts the secret access key as plaintext in the LS_SecretAccessKey column, or an Azure Key Vault reference held in the LS_SecretAccessKeyKVS_ fields that resolves to the key at runtime.
It maps to the ADF secretAccessKey property.
It is suggested that Azure Key Vault be used in place of manually entering the Secret Access Key.
Service URL
The Service URL is the custom S3 endpoint of the storage service, in the form https://<service url> (for example, https://objects.contoso.com).
It is stored in the connection's LS_ServiceUrl column and maps to the ADF serviceUrl property.
When the column is empty, the linked service carries no serviceUrl property.
Force Path Style
Force Path Style selects S3 path-style access instead of virtual hosted-style access.
Supported values are true and false. Whether a store needs path-style access is described in that store's own documentation.
Force Path Style is stored in the connection's Additional Settings (LS_AdditionalSettings) under the key forcePathStyle, and maps to the ADF forcePathStyle property.
When the key is absent, the linked service carries no forcePathStyle property, and Azure Data Factory uses false, its default.
Azure Data Factory Linked Service Additional Information
For additional information on ADF Amazon S3 Compatible Storage Linked Service and its connection requirements see the Azure Data Factory Amazon S3 Compatible Storage Connector documentation.