Configuring an ADF Linked Service Connection for Amazon Redshift
For information on how to enable a connection for use with linked services, see Configuring a Linked Service Connection.
Field labels in this walkthrough follow the BimlFlex Connection Editor, with the connection column that carries each value and the Azure Data Factory property it maps to named alongside.
After selecting Amazon Redshift from the Linked Service Type dropdown, the form required for creating an Amazon Redshift Linked Service will appear.
Source Only Connector
An Amazon Redshift connection is used as a source. It is not available as a target.
The Amazon Redshift Linked Service Type is offered on connections that use the Amazon Redshift System Type.
Each object is read through a dataset parameterized on schema and table, and the query on the copy source reads the object.
The dataset takes the object's schema from the TargetSchemaName parameter and its table from the TargetObjectName parameter, and the copy source passes both parameters.
BimlFlex composes the query from the object's columns and source filters: the object's Where SQL and its parameter filters, then its Group By SQL and Order By SQL, then the text of the object's Append Source SQL extension point. A filled Override SQL on the object, or the text of the object's Override SQL extension point when the Override SQL is empty, takes the place of the composed column query, and the source filters and the appended text follow it the same way.
BimlFlex does not author the Amazon Redshift UNLOAD to Amazon S3 that Azure Data Factory offers for large copies.
Choosing an Authentication Method
| Method | Credentials |
|---|---|
| Basic | User Name and Password |
The Amazon Redshift linked service signs in with the User Name and Password. BimlFlex does not read the connection's Authentication Type column for this connector, and no authentication type is written to the linked service.
Recommendation: Store the Password in Azure Key Vault rather than entering it as plaintext.
Complete Example: Basic Authentication with Azure Key Vault
The values below are made up for illustration.
| Field | Example Value |
|---|---|
| Connect via Integration Runtime | AutoResolveIntegrationRuntime |
| Server | examplecluster.abc123xyz789.us-west-2.redshift.amazonaws.com |
| Port | 5439 |
| Database Name | analytics |
| User Name | bfx_reader |
| AKV Linked Service (Password, Azure Key Vault tab) | bfx-kv |
| Secret Name (Password, Azure Key Vault tab) | BFX-RedshiftPassword |
| SSL Mode | Verify Full |
From these values the build writes a linked service with version set to 2.0, the server, port, database and username, a password that references the secret in Azure Key Vault, and sslMode set to verify-full.
Required Fields
These fields are required for every Amazon Redshift Linked Service connection:
Connect via Integration Runtime
Connect via Integration Runtime names the integration runtime for an Amazon Redshift Linked Service connection.
The default value is AutoResolveIntegrationRuntime.
To use another runtime, select it from the dropdown or choose + New to add one.
The custom values that appear in this dropdown can be maintained in Settings under Azure, in the AzureIntegrationRuntime setting.
The value is stored in the connection's LS_ConnectVia_IntegrationRuntime column and maps to the ADF connectVia property.
When the value is AutoResolveIntegrationRuntime, no connectVia property is written and the default Azure integration runtime is used.
For version 2.0, Azure Data Factory supports the Azure integration runtime and the self-hosted integration runtime version 5.60 or above, and no driver install is needed.
Server
The Server is the IP address or host name of the Amazon Redshift server (for example, examplecluster.abc123xyz789.us-west-2.redshift.amazonaws.com).
It is stored in the connection's LS_Server column and maps to the ADF server property.
Database Name
The Database Name is the name of the Amazon Redshift database.
It is stored in the connection's LS_Db column and maps to the ADF database property.
Port
The Port is the TCP port that the Amazon Redshift server listens on.
It is stored in the connection's LS_Port column and maps to the ADF port property.
When the Port is empty, no port property is written and the service default 5439 applies.
User Name
The User Name is the name of a user who has access to the database.
It is stored in the connection's LS_Username column and maps to the ADF username property.
The build reports an error when the User Name is empty.
Password
The Password field accepts the password as plaintext in the LS_Password column on the Password tab, or an Azure Key Vault reference on the Azure Key Vault tab that resolves to the password at runtime.
The Azure Key Vault reference is stored in the LS_PasswordKVS_ fields: AKV Linked Service, Secret Name, and an optional Secret Version.
It is written when both the AKV Linked Service and the Secret Name are filled.
Either form maps to the ADF password property. Fill in one of the two forms.
When the connection holds both a plaintext password and an Azure Key Vault reference, or neither, the build reports an error that asks for exactly one of the two.
It is suggested that Azure Key Vault be used in place of manually entering the Password.
Encrypted Credential
When the connection's LS_EncryptedCredential column is filled, its value is written to the ADF encryptedCredential property.
The Amazon Redshift form has no field for it, and a value stored in the column is kept when the connection is saved.
Connection Settings
An Amazon Redshift linked service at version 2.0 takes two optional connection settings.
Each setting is stored in the connection's Additional Settings (LS_AdditionalSettings) under the key in the table, and maps to the ADF property of the same name.
| Setting | Key and ADF Property | Values | Service Default |
|---|---|---|---|
| Ssl Mode | sslMode | The SSL certificate verification mode: Verify_full, Verify_ca, Required, Preferred, Allowed or Disabled, written as verify-full, verify-ca, require, prefer, allow or disable | verify-full |
| Ssl Insecure | sslInsecure | Whether the driver checks the authenticity of the IdP server certificate: true or false, also read from 1 or Y and from 0 or N | false |
On the ADF Linked Service tab of the BimlFlex Connection Editor, the SSL Mode dropdown and the SSL Insecure checkbox set the two keys.
The SSL Mode dropdown offers Not set, Verify Full, Verify CA, Required, Preferred, Allowed and Disabled, and Not set removes the key.
A checked SSL Insecure stores true, and a cleared checkbox removes the key.
A setting whose key is absent or empty is not written to the linked service, and the service default applies.
An Ssl Insecure of false is not written either, and the service default false applies. An Ssl Mode of Verify_full is written as verify-full.
The Additional Settings value is a JSON object, for example {"$v":1,"settings":{"sslMode":"Verify_ca","sslInsecure":"true"}}.
When the stored value is not valid JSON, the build reports a warning that names the connection, and neither setting is written.
Azure Data Factory Linked Service Additional Information
For additional information on ADF Amazon Redshift Linked Service and its connection requirements see the Azure Data Factory Amazon Redshift Connector documentation.