Configuring an ADF Linked Service Connection for SFTP
For information on how to enable a connection for use with linked services, see Configuring a Linked Service Connection.
Field labels in this walkthrough follow the underlying Azure Data Factory property names, with the connection column that carries each value named alongside. The exact labels shown in the BimlFlex Connection Editor may differ.
After selecting SFTP from the Linked Service Type dropdown, the form required for creating an SFTP Linked Service will appear.
The SFTP option is offered when the connection's Connection Type is SFTP.
File Format System Types
An SFTP Linked Service is selected on a connection whose System Type is a file format. The file format System Types that offer it are Delimited Format, Avro Format, Binary Format, JSON Format, ORC Format, Parquet Format and XML Format.
Choosing an Authentication Method
| Method | Credentials | Additional Settings |
|---|---|---|
| Basic | Username and Password | None |
| SSH Public Key | Username and a private key, with a pass phrase when the key is protected by one | None |
Recommendation: Store the Password in Azure Key Vault rather than entering it as plaintext.
Complete Example: Basic Authentication with Azure Key Vault
| Field | Example Value |
|---|---|
| Connect via Integration Runtime | AutoResolveIntegrationRuntime |
| Host | sftp.contoso.com |
| Port | 22 |
| Authentication Type | Basic |
| Username | bfx-reader |
| Azure Key Vault Linked Service | bfx-kv |
| Secret Name (for Password) | BFX-SRC-SftpPassword |
Required Fields
Common Required Fields
Most required fields for an SFTP Linked Service connection depend on the Authentication Type used. There are some fields that are required regardless of Authentication Type.
These common required fields are:
Optional fields are:
Authentication Type Dependent Required Fields
The remaining field requirements are dependent on Authentication Type.
Required fields for Basic:
- Password (or Azure Key Vault)
Required fields for SSH Public Key:
- One of the private key path, the private key content, or an Azure Key Vault reference to the private key content
- Pass phrase (or an Azure Key Vault reference to it), when the private key is protected by one
Connect via Integration Runtime
Connect via Integration Runtime names the integration runtime for an SFTP Linked Service connection.
The default value is AutoResolveIntegrationRuntime.
To use another runtime, select it from the dropdown or choose + New to add one.
When a custom value is saved in the linked service form, it will be added to the custom integration runtimes.
The custom values that appear in this dropdown can be maintained in Settings under Azure - AzureIntegrationRuntime.
The runtime is stored in the connection's LS_ConnectVia_IntegrationRuntime column. Any value other than AutoResolveIntegrationRuntime is written to the linked service as the runtime it connects through.
Host
The Host is the name or IP address of the SFTP server (for example, sftp.contoso.com).
It is stored in the connection's LS_Host column and maps to the ADF host property.
Port
The Port is the port on which the SFTP server is listening.
It is stored in the connection's LS_Port column and maps to the ADF port property. BimlFlex writes it to the linked service when the column is set.
When the Port is not set, Azure Data Factory uses port 22.
Authentication Type
The SFTP Linked Service connection can use Basic or SSH public key authentication.
The value is stored in the connection's LS_AuthenticationType column and maps to the ADF authenticationType property: Basic for Basic authentication and SshPublicKey for SSH public key authentication.
The Authentication Type dropdown in the BimlFlex Connection Editor offers Basic. It offers no SSH public key entry.
A connection whose LS_AuthenticationType column holds SshPublicKey is written with the SshPublicKey authentication type.
Any other value makes the build report an error, because the linked service accepts Basic or SshPublicKey.
For Basic, a Username and Password are required.
For SSH Public Key, a Username and a private key are required, with a pass phrase when the key is protected by one.
Basic
Basic authentication signs in to the SFTP server with a Username and Password.
The Username is stored in the connection's LS_Username column and maps to the ADF userName property.
The Password field accepts the password as plaintext in the LS_Password column, or an Azure Key Vault reference held in the LS_PasswordKVS_ fields that resolves to the password at runtime. It maps to the ADF password property.
It is suggested that Azure Key Vault be used in place of manually entering the Password.
SSH Public Key
SSH public key authentication signs in to the SFTP server with a Username and a private key.
The Username is stored in the connection's LS_Username column and maps to the ADF userName property.
The form shows the User Name field only while the Authentication Type is Basic. A value held in the column is kept when the connection is saved, and the build writes it whatever the Authentication Type.
A Password held on the connection is also written whatever the Authentication Type.
The SFTP form has no field for the private key or the pass phrase. The build reads these connection columns by name and writes each one whenever it is filled, whatever the Authentication Type:
| Column | ADF Property | Value |
|---|---|---|
LS_PrivateKeyPath | privateKeyPath | The absolute path to the private key file on the integration runtime machine |
LS_PrivateKeyContent | privateKeyContent | The private key content, written as a secure string |
LS_PrivateKeyContentKVS_StoreName, LS_PrivateKeyContentKVS_SecretName, LS_PrivateKeyContentKVS_SecretVersion | privateKeyContent | An Azure Key Vault reference to the secret that holds the private key content |
LS_PassPhrase | passPhrase | The pass phrase that decrypts the private key, written as a secure string |
LS_PassPhraseKVS_StoreName, LS_PassPhraseKVS_SecretName, LS_PassPhraseKVS_SecretVersion | passPhrase | An Azure Key Vault reference to the secret that holds the pass phrase |
An Azure Key Vault reference is written when both its _StoreName column, the Azure Key Vault linked service, and its _SecretName column are filled. The _SecretVersion column is optional.
When LS_PrivateKeyContentKVS_StoreName is filled, LS_PrivateKeyContent is not written. When LS_PassPhraseKVS_StoreName is filled, LS_PassPhrase is not written.
Fill one of LS_PrivateKeyPath, LS_PrivateKeyContent or the LS_PrivateKeyContentKVS_ reference.
When LS_PrivateKeyPath and the private key content are both filled, the build writes both, and on a connection that uses SSH public key authentication it reports an error that asks for one and only one of them.
The build reports the same error when none of the three is filled.
The private key path applies only when Connect via Integration Runtime names a self-hosted integration runtime. The private key content is the Base64 encoded SSH private key in OpenSSH format. The pass phrase is required when the private key file or the key content is protected by one.
The values held in these columns are kept when the connection is saved. When the connection holds an Azure Key Vault reference for the private key content or the pass phrase, saving the connection clears the matching inline column. Keep these columns blank on a connection that uses Basic authentication, because the build writes them on any connection where they are filled.
Skip Host Key Validation
Skip Host Key Validation is carried in the connection's LS_SkipHostKeyValidation column and maps to the ADF skipHostKeyValidation property.
BimlFlex writes the property as true when the column is empty or true, so the linked service skips host key validation.
The BimlFlex Connection Editor shows no field for this setting.
Azure Data Factory Linked Service Additional Information
For additional information on ADF SFTP Linked Service and its connection requirements see the Azure Data Factory SFTP Connector documentation.