Setting Up Connections, Projects, and Batches
This walkthrough creates a complete BimlFlex solution from scratch, showing every configuration value and explaining why each choice is made. By the end you will have a working source-to-staging pipeline ready to build.
Prerequisites
- BimlFlex App installed and connected to a BimlFlex metadata database (Installation Guide)
- A source database accessible from your development machine (this example uses AdventureWorksLT on SQL Server — see AdventureWorks Setup)
- A target SQL Server database for staging (any empty database will work)
- BimlStudio installed and licensed (BimlStudio Setup)
Step 1: Create the Source Connection
Navigate to Connections in the BimlFlex App and click + to create a new connection.
| Field | Value | Why This Value |
|---|---|---|
| Connection | AWLT_SRC | Short, descriptive name following the [SOURCE]_SRC convention |
| Connection Type | OLEDB | OLEDB provides the broadest SQL Server compatibility for SSIS; use ADONET if targeting ADF-only |
| System Type | SQL Server | Matches the AdventureWorksLT database engine |
| Integration Stage | Source System | This connection represents where data originates |
| Connection String | Data Source=localhost;Initial Catalog=AdventureWorksLT;Provider=SQLNCLI11;Integrated Security=SSPI; | Standard SQL Server connection string; adjust Data Source for your server name |
What these choices do together: The combination of Connection Type (OLEDB), System Type (SQL Server), and Integration Stage (Source System) tells BimlFlex this is an on-premises SQL Server source. BimlFlex will generate appropriate OLEDB source components for SSIS or Copy Activity sources for ADF based on the project's integration template.
Save the connection.
Step 2: Create the Staging Connection
Click + again to create the staging target.
| Field | Value | Why This Value |
|---|---|---|
| Connection | AWLT_STG | Matches the source naming with _STG suffix for staging |
| Connection Type | OLEDB | Must match source connection type for SSIS; use ADONET for ADF |
| System Type | SQL Server | Staging is on the same SQL Server platform in this example |
| Integration Stage | Staging Area | Tells BimlFlex this is the intermediate processing zone |
| Connection String | Data Source=localhost;Initial Catalog=BFX_STG;Provider=SQLNCLI11;Integrated Security=SSPI; | Points to your empty staging database |
Save the connection.
Step 3: Create the Batch
Navigate to Batches and click + to create a new batch.
| Field | Value | Why This Value |
|---|---|---|
| Batch | LOAD_AWLT | Descriptive name for the orchestration container |
| Precedence Constraint | Success | The next step only runs if the previous succeeds — use Completion if you want execution to continue regardless |
| Threads | 4 | Number of parallel threads within the batch; 4 balances throughput vs. server load for a development environment |
| Containers | 0 | Number of sub-batches; 0 means all objects run in a single container. Increase for large projects that need grouped execution |
Save the batch.
Step 4: Create the Project
Navigate to Projects and click + to create a new project.
| Field | Value | Why This Value |
|---|---|---|
| Project | EXT_AWLT | EXT prefix indicates an extract project — a naming convention that helps organize multiple projects |
| Integration Template | SSIS | Generates SSIS packages. Change to ADF for Azure Data Factory pipelines. See Choosing an Integration Template for guidance |
| Source Connection | AWLT_SRC | The connection we created in Step 1 |
| Target Connection | AWLT_STG | The staging connection from Step 2 |
| Batch | LOAD_AWLT | The batch from Step 3 — this project runs inside this batch |
How these relate: The Project ties together a source, a target, a batch (execution container), and an integration template (code generation engine). When you build, BimlFlex generates one pipeline per source object, all grouped under this project within the batch.
Save the project.
Step 5: Import Source Metadata
With the project created, import the source schema:
- Navigate to Connections → select
AWLT_SRC→ click Import Metadata - Select the tables you want to stage (e.g.,
SalesLT.Customer,SalesLT.Product,SalesLT.SalesOrderHeader) - Click Import
BimlFlex creates Objects and Columns for each imported table. These appear under the AWLT_SRC connection in the Objects list.
For details, see Importing Metadata.
Step 6: Build and Verify
Open BimlStudio, connect to the same metadata database, select the customer and version, and build the project:
- Click Build in the BimlStudio ribbon
- Check the output pane for errors
- On success, find the generated SSIS project (.dtproj) or ADF ARM templates in the output directory
For build details, see Building a BimlStudio Project.
What to Do Next
- Add a Data Vault layer: Create a Persistent Staging (PSA) connection and a Data Vault connection, then use the Data Vault Accelerator to auto-generate Hubs, Links, and Satellites
- Switch to ADF: Change the project's Integration Template to
ADF, configure Linked Services on your connections (see Configuring a Linked Service), and rebuild - Add a Data Mart: Create a Data Mart project sourcing from Data Vault — see Data Mart Configuration