Batch Extension Points
The Batch category has the following available Extension Points defined.
Batch Connection
Configure additional Connection references to the Batch package. This only applies to SSIS deployments.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the SSIS Batch Package to which the connection will be added. |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchConnection" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<!-- Note that the connection must be a valid connection defined using BimlFlex connection metadata. Connections can also be added manually in BimlStudio if required. -->
<Connection ConnectionName="MY_SOURCE" />
What It Does
Adds extra connection manager references to an SSIS batch package so that tasks inside the package can communicate with databases or files not already wired up by BimlFlex metadata. Without this extension point, the batch package only contains connections that BimlFlex derives automatically from the configured source and target connections. Use it whenever a pre-process or post-process task in the same batch needs to reach a system that is outside the normal load path.
Where It Fires
Consumed in 90.flx-i-get-pkg-batch.biml, 90.flx-i-get-pkg-sql-batch.biml, 90.flx-i-get-pkg-sql-dm-batch.biml, and 90.flx-i-get-pkg-snowflake-sql-batch.biml. The emitted <Connection> element is injected into the Connections section of the SSIS batch package. The framework retrieves it via ObjectViewModel.GetBatchExtensionPoint().
When to Use It
- A BatchPreProcess or BatchPostProcess task must query a control database (e.g., an audit or logging server) that is not part of the regular BimlFlex metadata connections.
- You are orchestrating a cross-system load where the batch
LOAD_BFX_DVneeds to verify row counts against theAWLT_SRCsource connection before proceeding. - A file-system connection is needed so that a Script Task in the batch can write a manifest file at the end of each run. Use this instead of manually editing the generated package in BimlStudio when you want the connection to regenerate consistently.
- You need to reference a secondary staging connection (e.g.,
BFX_STG) inside a data mart batch (LOAD_BFX_DM) to perform a cross-database validation step. - Share a single connection reference across multiple batches. Your TLOG processing has five batch variants (
LOAD_PSA_TLOGS_COT,LOAD_PSA_TLOGS_CT,LOAD_PSA_TLOGS_RT,LOAD_PSA_TLOGS_T,LOAD_PSA_TLOGS_TCT) that all need access to theIF_ASDW_PSAconnection for pre/post-process SQL tasks. Use a comma-separated target list to apply the same<Connection ConnectionName="IF_ASDW_PSA" />to all five batches in a single extension point file.
Prerequisites
- The connection referenced in
ConnectionNamemust already be defined in BimlFlex metadata (App or Excel) or manually in BimlStudio. - The target batch (e.g.,
LOAD_BFX_DV) must exist in BimlFlex metadata. - SSIS deployment model; this extension point does not apply to ADF or SQL-only builds.
Implementation Steps
- Open the BimlFlex App or BimlStudio and navigate to the Extension Points editor.
- Create a new Extension Point and set its type to BatchConnection.
- Set the
targetattribute to the batch that needs the connection. - Define the
<Connection>element with a validConnectionName.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchConnection" target="LOAD_BFX_DV" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<Connection ConnectionName="AWLT_SRC" />
- Build the project in BimlStudio and verify the batch package now lists the new connection manager.
Example
Before -- the generated batch package contains only its default connections:
<Connections>
<Connection ConnectionName="BimlCatalog" />
</Connections>
After -- with the extension point applied, the additional connection appears:
<Connections>
<Connection ConnectionName="BimlCatalog" />
<Connection ConnectionName="AWLT_SRC" />
</Connections>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
Using a ConnectionName that does not exist in BimlFlex metadata. | Build error: "Could not resolve connection reference." | Add the connection in BimlFlex metadata first, or define it manually in BimlStudio. |
Omitting the target attribute so the connection is injected into every batch. | All batch packages receive the extra connection, increasing deployment size and confusion. | Set target="LOAD_BFX_DV" (or the specific batch) on the extension point directive. |
| Adding a connection that BimlFlex already generates automatically. | Duplicate connection manager warnings or build errors. | Check the generated package before adding; only add connections the framework does not already provide. |
Related Extension Points
- BatchPreProcess / BatchPostProcess -- the tasks in these containers are the most common consumers of connections added through BatchConnection.
- BatchVariable -- if the extra connection needs a parameterized connection string, combine with a batch variable.
- PackageVariable (object-level) -- for adding connections inside individual child packages rather than the batch.
Batch Package Configurations
Add PackageConfigurations for when you have unselected Use Project Deployment. This only applies to SSIS deployments.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the SSIS Batch Package to which the configurations will be added. |
Outputs
Name | Type | Description |
|---|---|---|
| ObjectInherit | Boolean | XXXX |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPackageConfigurations" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<!-- Use BatchPackageConfigurations to add a Configurations if you have unselected Use Project Deployment in the BimlStudio project. When using Package Deployment instead of Project Deployment you might need to specify additional configurations.
Configurations can also be added to the Batch using the PackageConfigurations combined with CustomOutput.ObjectInherit = true; -->
<# CustomOutput.ObjectInherit = true; #>
<PackageConfiguration Name="LOAD_MY_Configurations">
<ExternalFileInput ExternalFilePath="C:\Varigence\Configurations\MY_BATCH_Configurations.dtsConfig" FileUsageType="ExistingFile" RelativePath="false" />
</PackageConfiguration>
What It Does
Injects SSIS Package Configuration entries into a batch package so that connection strings, variables, and other runtime properties can be read from external .dtsConfig files or SQL Server tables at execution time. This is only relevant when the BimlStudio project is set to Package Deployment Model rather than the more modern Project Deployment Model. It allows environment-specific values (server names, file paths, credentials) to be swapped without rebuilding the package.
Where It Fires
Consumed in 90.flx-i-get-batch-configurations.biml. The emitted <PackageConfiguration> element is injected into the PackageConfigurations section of the batch package. This extension point is only used when the project is not using Project Deployment Model. Setting CustomOutput.ObjectInherit = true causes child packages to inherit the same configurations.
When to Use It
- Your organization uses Package Deployment Model and needs to externalize connection strings for
AWLT_SRCorBFX_STGso that the same package runs in DEV, TEST, and PROD without rebuilding. - A DBA requires that all SSIS packages on the server read credentials from a shared
.dtsConfigfile maintained outside of source control. - You want to override a batch-level variable (e.g., a file path or snapshot date) at runtime via an XML configuration file. Use this instead of ParameterBindings when you are on Package Deployment Model, since parameter bindings only work with Project Deployment.
- You need to propagate configurations down to child packages executed by the batch by enabling
ObjectInherit.
Prerequisites
- The BimlStudio project must be set to Package Deployment Model (not Project Deployment).
- The
.dtsConfigfile or SQL Server configuration table must exist in the target environment before the package runs. - The batch (e.g.,
LOAD_BFX_DM) must exist in BimlFlex metadata.
Implementation Steps
- Create or obtain the
.dtsConfigXML configuration file for the target environment. - Open the Extension Points editor and create a new BatchPackageConfigurations extension point.
- Set the
targetattribute to the specific batch. - Define the configuration entry pointing to the external file or SQL table.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPackageConfigurations" target="LOAD_BFX_DM" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<# CustomOutput.ObjectInherit = true; #>
<PackageConfiguration Name="LOAD_BFX_DM_EnvConfig">
<ExternalFileInput ExternalFilePath="C:\Config\LOAD_BFX_DM.dtsConfig" FileUsageType="ExistingFile" RelativePath="false" />
</PackageConfiguration>
- Build in BimlStudio. If
ObjectInheritis true, verify that child packages also contain the configuration.
Example
Before -- the generated batch package has no package configurations section:
<Package>
<Tasks>...</Tasks>
</Package>
After -- the configuration entry is present, allowing runtime value injection:
<Package>
<PackageConfigurations>
<PackageConfiguration Name="LOAD_BFX_DM_EnvConfig">
<ExternalFileInput ExternalFilePath="C:\Config\LOAD_BFX_DM.dtsConfig"
FileUsageType="ExistingFile" RelativePath="false" />
</PackageConfiguration>
</PackageConfigurations>
<Tasks>...</Tasks>
</Package>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
| Using this extension point with Project Deployment Model enabled. | The configuration entries are silently ignored; the package uses project parameters instead. | Switch to ParameterBindings for Project Deployment, or change the project to Package Deployment. |
Forgetting CustomOutput.ObjectInherit = true when child packages also need the configuration. | Child packages fail at runtime because they cannot resolve externalized connection strings. | Add <# CustomOutput.ObjectInherit = true; #> to the extension point code. |
Pointing to a .dtsConfig file path that does not exist on the execution server. | Runtime error: "The configuration file cannot be found." | Ensure the file is deployed to the same path on all SSIS servers, or use a UNC path. |
| Defining a configuration name that conflicts with a BimlFlex-generated configuration. | Unexpected property overrides or build warnings about duplicate names. | Use a distinct, descriptive name that will not collide with generated entries. |
Related Extension Points
- ParameterBindings -- the Project Deployment Model alternative to package configurations for passing values into child packages.
- BatchVariable -- often combined with configurations to define the variable that receives the externalized value.
- BatchPreProcess -- may consume variables populated by the configuration at runtime.
Batch Post Process
Configure logic that will be injected after the main Batch process. This only applies to SSIS deployments.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the Batch to which the SSIS Batch Package process will be added. |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPostProcess" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<!-- This can be any anything defined within the SSIS Control Flow. -->
<!-- The Batch postprocess needs to be connected to the relevant output from the preceding step, -->
<!-- the example below connects to the Main sequence container's standard output. -->
<Container Name="SEQC - Post Process" ConstraintMode="Parallel">
<PrecedenceConstraints>
<Inputs>
<Input OutputPathName="SEQC - Main.Output" EvaluationOperation="Constraint" EvaluationValue="Success"/>
</Inputs>
</PrecedenceConstraints>
<Tasks>
<ExecuteSQL Name="SQL - Set SnapshotDate" ConnectionName="BimlCatalog" ResultSet="None">
<DirectInput>EXEC [ssis].[SetConfigVariable] 'LOAD_DATAMART', 'LOAD_DATAMART.SnapshotDate', 'SnapshotDate', @VariableValue</DirectInput>
<Parameters>
<Parameter Name="@VariableValue" VariableName="User.SnapshotDate" Length="-1" DataType="String" />
</Parameters>
</ExecuteSQL>
</Tasks>
</Container>
What It Does
Injects custom SSIS Control Flow logic that executes after the main batch processing completes. This is the place to add cleanup tasks, notification calls, audit logging, snapshot persistence, or any finalization step that should only run once all child packages in the batch have finished. The post-process container is a sibling to the main sequence container and is connected via a precedence constraint.
Where It Fires
Consumed in 90.flx-i-get-pkg-batch.biml, 90.flx-i-get-pkg-sql-batch.biml, 90.flx-i-get-pkg-sql-dm-batch.biml, and 90.flx-i-get-pkg-snowflake-sql-batch.biml. The emitted Biml is injected as a sibling container to the main SEQC - Main container. It fires after the main batch execution completes, and the precedence constraint typically connects from SEQC - Main.Output.
When to Use It
- After
LOAD_BFX_DMfinishes loading all data mart tables, persist a snapshot date to the BimlCatalog so downstream reporting knows the as-of date. - Send an email or call a stored procedure to notify operations staff that the
LOAD_BFX_DVbatch completed successfully. - Execute a cleanup script that truncates staging tables in
BFX_STGafter all data has been promoted to the data vault. - Write a completion row to a custom audit table in
BFX_ODSthat external monitoring tools poll for SLA tracking. Use this instead of BatchPreProcess when the logic must only run after all child packages have succeeded. - Trigger a downstream process (e.g., start a cube refresh) by inserting a control row or calling an external API via a Script Task.
Prerequisites
- The batch (e.g.,
LOAD_BFX_DV,LOAD_BFX_DM) must exist in BimlFlex metadata. - Any connection referenced inside the post-process container (e.g.,
BimlCatalog) must be available to the batch package. Use BatchConnection to add it if it is not already present. - If the post-process uses a custom variable, define it first with BatchVariable.
Implementation Steps
- Open the Extension Points editor and create a new BatchPostProcess extension point.
- Set the
targetattribute to the batch that needs post-processing. - Define a
<Container>with a<PrecedenceConstraints>input connected toSEQC - Main.Output. - Add the desired tasks inside the container.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPostProcess" target="LOAD_BFX_DV" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<Container Name="SEQC - Post Process" ConstraintMode="Parallel">
<PrecedenceConstraints>
<Inputs>
<Input OutputPathName="SEQC - Main.Output" EvaluationOperation="Constraint" EvaluationValue="Success" />
</Inputs>
</PrecedenceConstraints>
<Tasks>
<ExecuteSQL Name="SQL - Log Batch Complete" ConnectionName="BimlCatalog" ResultSet="None">
<DirectInput>EXEC [ssis].[LogBatchComplete] '<#= batch.Name #>'</DirectInput>
</ExecuteSQL>
</Tasks>
</Container>
- Build and verify that the post-process container appears after the main sequence container in the generated package.
Example
Before -- the batch package control flow ends with the main container:
<Container Name="SEQC - Main">
<!-- child ExecutePackageTasks -->
</Container>
After -- the post-process container follows, connected via a precedence constraint:
<Container Name="SEQC - Main">
<!-- child ExecutePackageTasks -->
</Container>
<Container Name="SEQC - Post Process" ConstraintMode="Parallel">
<PrecedenceConstraints>
<Inputs>
<Input OutputPathName="SEQC - Main.Output" EvaluationOperation="Constraint" EvaluationValue="Success" />
</Inputs>
</PrecedenceConstraints>
<Tasks>
<ExecuteSQL Name="SQL - Log Batch Complete" ConnectionName="BimlCatalog" ResultSet="None">
<DirectInput>EXEC [ssis].[LogBatchComplete] 'LOAD_BFX_DV'</DirectInput>
</ExecuteSQL>
</Tasks>
</Container>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
Missing or incorrect PrecedenceConstraints input path. | The post-process container either never executes or runs in parallel with the main container. | Set OutputPathName="SEQC - Main.Output" and EvaluationValue="Success". |
| Referencing a connection not available to the batch package. | Runtime error: "The connection manager is not found." | Add the connection via BimlFlex metadata or use the BatchConnection extension point. |
Setting EvaluationValue="Completion" when you only want to run on success. | The post-process runs even when child packages fail, leading to misleading audit entries. | Use EvaluationValue="Success" to gate execution on successful completion. |
Naming the container the same as another extension point container (e.g., SEQC - PreProcess). | Build errors due to duplicate container names within the package. | Choose a unique container name such as SEQC - Post Process or SEQC - Cleanup. |
Related Extension Points
- BatchPreProcess -- the counterpart that fires before the main batch execution; use both together for setup/teardown patterns.
- BatchVariable -- define any custom variables the post-process tasks need.
- BatchConnection -- add connections that the post-process tasks reference.
- ParameterBindings -- if post-process values need to flow from batch variables into child packages.
Batch Pre Process
Configure logic that will be injected before the main Batch process. This only applies to SSIS deployments.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the SSIS Batch Package to which the process will be added. |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPreProcess" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<!-- This can be any anything defined within the SSIS Control Flow. -->
<Container Name="SEQC - Get SnapShotDate Parameters" ConstraintMode="Parallel">
<Tasks>
<ExecuteSQL Name="SQL - Get SnapshotDate" ConnectionName="BimlCatalog" ResultSet="SingleRow">
<Results>
<Result Name="0" VariableName="User.SnapshotDate" />
</Results>
<DirectInput>EXEC [ssis].[GetConfigVariable] 'LOAD_DATAMART', 'LOAD_DATAMART.SnapshotDate', 'SnapshotDate', @VariableValue, @ExecutionID</DirectInput>
<Parameters>
<Parameter Name="@VariableValue" VariableName="User.SnapshotDate" Length="-1" DataType="String" />
<Parameter Name="@ExecutionID" Direction="Input" DataType="Int64" VariableName="User.ExecutionID" />
</Parameters>
</ExecuteSQL>
</Tasks>
</Container>
What It Does
Injects custom SSIS Control Flow logic that executes before the main batch processing begins. This is the place to add initialization steps such as retrieving configuration values, setting snapshot dates, validating prerequisites, or performing pre-load truncations. The pre-process container fires after BimlFlex orchestration start but before any child packages execute.
Where It Fires
Consumed in 90.flx-i-get-pkg-batch.biml, 90.flx-i-get-pkg-sql-batch.biml, 90.flx-i-get-pkg-sql-dm-batch.biml, and 90.flx-i-get-pkg-snowflake-sql-batch.biml. The emitted Biml is wrapped in <Container Name="SEQC - PreProcess"> and inserted before the main sequence container. It fires after the orchestration start step, connected from SEQC - BimlFlex Package Start.Output.
When to Use It
- Before
LOAD_BFX_DMruns, retrieve a snapshot date from the BimlCatalog and store it in a user variable so all child packages use the same point-in-time reference. - Validate that the
AWLT_SRCsource system is reachable by running a lightweight connectivity check before kicking off the fullEXT_AWLT_SRCextraction batch. - Truncate landing tables in
BFX_STGbefore starting a full-refresh batch load, ensuring no stale data remains from the previous run. - Query a control table to determine whether the batch should execute at all (e.g., check a "run enabled" flag) and set a variable that a precedence constraint can evaluate. Use this instead of BatchPostProcess when the logic must complete before any child packages begin.
- Call a stored procedure to acquire a distributed lock or semaphore so that parallel batch executions do not collide.
- Execute a Snowflake stored procedure before the batch starts. Use a Snowflake SQL custom task to call a stored procedure (e.g.,
CALL "rdv"."flex_WW_VW_WW_STAGE"()) that prepares staging views or materializes temporary tables before the batch's child packages begin extracting data. The custom task references batch-level Snowflake variables for connection details.
Prerequisites
- The batch (e.g.,
EXT_AWLT_SRC,LOAD_BFX_DV) must exist in BimlFlex metadata. - Any user variables referenced in the pre-process (e.g.,
User.SnapshotDate) must be defined via the BatchVariable extension point. - Connections used inside the container (e.g.,
BimlCatalog) must be available to the batch package.
Implementation Steps
- Open the Extension Points editor and create a new BatchPreProcess extension point.
- Set the
targetattribute to the specific batch. - Define a
<Container>with the initialization tasks. - Reference any user variables that will hold the retrieved values.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchPreProcess" target="LOAD_BFX_DM" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<Container Name="SEQC - Get SnapshotDate" ConstraintMode="Parallel">
<Tasks>
<ExecuteSQL Name="SQL - Get SnapshotDate" ConnectionName="BimlCatalog" ResultSet="SingleRow">
<Results>
<Result Name="0" VariableName="User.SnapshotDate" />
</Results>
<DirectInput>EXEC [ssis].[GetConfigVariable] 'LOAD_BFX_DM', 'LOAD_BFX_DM.SnapshotDate', 'SnapshotDate', @VariableValue, @ExecutionID</DirectInput>
<Parameters>
<Parameter Name="@VariableValue" VariableName="User.SnapshotDate" Length="-1" DataType="String" />
<Parameter Name="@ExecutionID" Direction="Input" DataType="Int64" VariableName="User.ExecutionID" />
</Parameters>
</ExecuteSQL>
</Tasks>
</Container>
- Build and open the generated package to confirm the pre-process container appears before
SEQC - Mainin the control flow.
Example
Before -- the batch package control flow starts directly with the main container:
<Container Name="SEQC - BimlFlex Package Start">...</Container>
<Container Name="SEQC - Main">
<!-- child ExecutePackageTasks -->
</Container>
After -- the pre-process container is inserted between start and main:
<Container Name="SEQC - BimlFlex Package Start">...</Container>
<Container Name="SEQC - Get SnapshotDate" ConstraintMode="Parallel">
<Tasks>
<ExecuteSQL Name="SQL - Get SnapshotDate" ConnectionName="BimlCatalog" ResultSet="SingleRow">
<Results>
<Result Name="0" VariableName="User.SnapshotDate" />
</Results>
<DirectInput>EXEC [ssis].[GetConfigVariable] 'LOAD_BFX_DM', 'LOAD_BFX_DM.SnapshotDate', 'SnapshotDate', @VariableValue, @ExecutionID</DirectInput>
<Parameters>
<Parameter Name="@VariableValue" VariableName="User.SnapshotDate" Length="-1" DataType="String" />
<Parameter Name="@ExecutionID" Direction="Input" DataType="Int64" VariableName="User.ExecutionID" />
</Parameters>
</ExecuteSQL>
</Tasks>
</Container>
<Container Name="SEQC - Main">
<!-- child ExecutePackageTasks -->
</Container>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
| Referencing a user variable that has not been declared via BatchVariable. | Runtime error: "Variable User.SnapshotDate is not found in the Variables collection." | Add a corresponding BatchVariable extension point to declare the variable with a default value. |
Using a container name that collides with the BimlFlex-generated SEQC - PreProcess wrapper. | Build error or duplicate container names in the package. | Use a descriptive, unique name such as SEQC - Get SnapshotDate rather than the reserved wrapper name. |
Forgetting that the pre-process container is automatically connected from SEQC - BimlFlex Package Start.Output. | Manually adding a precedence constraint to Package Start, resulting in duplicate connections. | Omit explicit precedence constraints to Package Start; the framework handles this wiring. |
| Placing long-running validation logic that delays the entire batch. | The main batch does not start until the pre-process finishes, causing SLA breaches. | Keep pre-process logic lightweight; move heavy operations to a dedicated preceding batch. |
Related Extension Points
- BatchPostProcess -- the counterpart that fires after the main batch execution; pair with PreProcess for setup/teardown patterns.
- BatchVariable -- required to declare any user variables the pre-process reads or writes.
- BatchConnection -- add connections that pre-process tasks reference if they are not already in the batch.
- ParameterBindings -- pass variables populated during pre-process down to child packages.
Batch Variable
Configure additional Variables for the Batch Package, or override default BimlFlex variables. This only applies to SSIS deployments.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the SSIS Batch Package to which the variables will be added. |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchVariable" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<Variable Name="CurrentModifiedDate" DataType="String" Namespace="User">1900-01-01</Variable>
What It Does
Adds custom SSIS package-level variables to the batch package, or overrides default values of variables that BimlFlex generates automatically. These variables can then be used by pre-process and post-process tasks, Script Tasks, expressions, and parameter bindings within the batch. The framework includes duplicate-detection logic so that adding a variable with the same name as an existing one replaces it rather than causing a build error.
Where It Fires
Consumed in 90.flx-i-get-pkg-batch.biml, 90.flx-i-get-pkg-sql-batch.biml, 90.flx-i-get-pkg-sql-dm-batch.biml, 90.flx-i-get-pkg-snowflake-sql-batch.biml, and 90.flx-i-get-pkg-parameter-variable.biml. The emitted <Variable> elements are injected into the <Variables> section of the batch package. The framework uses GetVariableNodes() parsing to prevent duplicate variables when a variable with the same name already exists.
When to Use It
- A BatchPreProcess task needs a
User.SnapshotDatevariable to store a date retrieved from the BimlCatalog before theLOAD_BFX_DMbatch runs. - You want to override the default value of a BimlFlex-generated variable (e.g., change the default timeout or retry count) for a specific batch like
EXT_AWLT_SRC. - A Script Task inside the batch needs a
User.NotificationEmailstring variable to hold the recipient address for completion alerts. - You are using ParameterBindings to pass a variable into child packages and need to first declare that variable at the batch level. Use this instead of hardcoding values inside Execute SQL Tasks when multiple tasks in the same batch need the same value.
- You need a flag variable (e.g.,
User.RunFullRefreshasBoolean) that a precedence constraint evaluates to decide whether to execute a full or incremental load path. - Define Snowflake connection variables at the batch level. Add variables for
SnowflakeAccount,SnowflakeDatabase,SnowflakeRegion,SnowflakeUser,SnowflakePassword,SnowflakeWarehouse, andSnowflakeSchemato the batch package so that all child packages inheriting from this batch can reference these values for Snowflake custom tasks.
Prerequisites
- The batch (e.g.,
LOAD_BFX_DV,LOAD_BFX_DM) must exist in BimlFlex metadata. - The variable data type must be a valid SSIS data type (
String,Int32,Int64,Boolean,DateTime, etc.). - SSIS deployment model; this extension point does not apply to ADF or SQL-only builds.
Implementation Steps
- Open the Extension Points editor and create a new BatchVariable extension point.
- Set the
targetattribute to the specific batch. - Define one or more
<Variable>elements withName,DataType,Namespace, and a default value.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="BatchVariable" target="LOAD_BFX_DM" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<Variable Name="SnapshotDate" DataType="String" Namespace="User">1900-01-01</Variable>
<Variable Name="RunFullRefresh" DataType="Boolean" Namespace="User">false</Variable>
- Build the project and inspect the generated batch package to confirm the variables appear in the Variables pane.
Example
Before -- the batch package contains only BimlFlex-generated variables:
<Variables>
<Variable Name="ExecutionID" DataType="Int64" Namespace="User">0</Variable>
</Variables>
After -- custom variables are added alongside the generated ones:
<Variables>
<Variable Name="ExecutionID" DataType="Int64" Namespace="User">0</Variable>
<Variable Name="SnapshotDate" DataType="String" Namespace="User">1900-01-01</Variable>
<Variable Name="RunFullRefresh" DataType="Boolean" Namespace="User">false</Variable>
</Variables>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
Using an incorrect DataType (e.g., String for a value that should be DateTime). | Runtime type-conversion errors when the variable is used in an expression or parameter mapping. | Match the DataType to how the variable will be consumed (e.g., DateTime if used in date comparisons). |
Omitting the Namespace attribute. | The variable is created in the default namespace, and tasks referencing User.VariableName cannot find it. | Always include Namespace="User" for custom variables. |
| Defining a variable that conflicts with a BimlFlex system variable without intending to override it. | Unexpected behavior because the framework's duplicate-detection replaces the system variable's default value. | Review the generated package's default variables before adding; choose unique names unless you intentionally override. |
| Declaring variables but never consuming them in any task. | No error, but the package accumulates unused variables that confuse future maintainers. | Only add variables that are referenced by PreProcess, PostProcess, ParameterBindings, or expression-based logic. |
Related Extension Points
- BatchPreProcess / BatchPostProcess -- the most common consumers of batch variables for initialization and finalization logic.
- ParameterBindings -- passes batch variables into child packages via Execute Package Task parameter bindings.
- BatchPackageConfigurations -- an alternative way to externalize variable values at runtime when using Package Deployment Model.
- PackageVariable (object-level) -- for adding variables to individual child packages rather than the batch.
Parameter Bindings
Configure Parameter Bindings for an Execute Package Task. This only applies to SSIS deployments. The Extension Point configures parameters from the Batch package that can be sent to the individual process packages through the EPT task in the Batch package.
Parameters
Name | Type | Description |
|---|---|---|
| batch | BimlFlexModelWrapper.BatchesWrapper | Contains all information related to the Batch Package to which the Parameter binding will be added. |
Template
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="ParameterBindings" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<!-- The below example is adding a ParameterBinding to a ExecutePackageTask. ParameterBindings is normally combined with a PackageParameter Extension Point -->
<ParameterBindings>
<ParameterBinding Name="SnapshotDate" VariableName="User.SnapshotDate" />
</ParameterBindings>
What It Does
Adds custom parameter bindings to the Execute Package Tasks within a batch package, enabling the batch to pass variable values down to individual child packages at runtime. This is the Project Deployment Model mechanism for sharing context -- such as snapshot dates, execution flags, or environment identifiers -- from the orchestrating batch into each child package that it executes. It replaces the need for package configurations in modern SSIS deployments.
Where It Fires
Consumed in 90.flx-i-get-pkg-batch.biml. The emitted <ParameterBindings> element is injected into the ExecutePackageTask's ParameterBindings section for each child package executed by the batch. This extension point only applies when using Project Deployment Model.
When to Use It
- The
LOAD_BFX_DMbatch retrieves a snapshot date in a BatchPreProcess step and needs to passUser.SnapshotDateto each child package (e.g., loadingSalesLT.Customerfact tables) so they all use the same point-in-time reference. - A global
User.RunFullRefreshflag set at the batch level must flow into every child package inLOAD_BFX_DVto switch between delta and full-load logic. - You need to pass an environment identifier (e.g.,
User.EnvironmentCode) from theEXT_AWLT_SRCbatch into child packages so that connection expressions resolve to the correct server. Use this instead of BatchPackageConfigurations when you are on Project Deployment Model, since package configurations are ignored in that mode. - An execution correlation ID must be propagated from the batch to child packages for end-to-end audit tracing.
Prerequisites
- The BimlStudio project must use Project Deployment Model (the default for modern SSIS).
- The batch variable being bound must be declared via BatchVariable (or be a BimlFlex-generated variable).
- Each child package must have a matching PackageParameter extension point that declares the receiving parameter with the same name and data type.
- The batch (e.g.,
LOAD_BFX_DM) must exist in BimlFlex metadata.
Implementation Steps
- Ensure the batch-level variable exists by creating a BatchVariable extension point (e.g.,
User.SnapshotDate). - Ensure each child package has a matching parameter by creating a PackageParameter extension point.
- Open the Extension Points editor and create a new ParameterBindings extension point.
- Set the
targetattribute to the batch. - Define the
<ParameterBindings>element mapping batch variables to child package parameters.
<#@ extension bundle="BimlFlex.bimlb" extensionpoint="ParameterBindings" target="LOAD_BFX_DM" #>
<#@ property name="batch" type="BimlFlexModelWrapper.BatchesWrapper" #>
<ParameterBindings>
<ParameterBinding Name="SnapshotDate" VariableName="User.SnapshotDate" />
<ParameterBinding Name="RunFullRefresh" VariableName="User.RunFullRefresh" />
</ParameterBindings>
- Build and inspect the generated batch package. Each Execute Package Task should show the parameter bindings in its properties.
Example
Before -- the Execute Package Task in the batch has no custom parameter bindings:
<ExecutePackage Name="EPT - Load_Customer" PackageName="Load_Customer">
<ExternalProjectPackage Package="Load_Customer.dtsx" />
</ExecutePackage>
After -- parameter bindings pass batch variables into the child package:
<ExecutePackage Name="EPT - Load_Customer" PackageName="Load_Customer">
<ExternalProjectPackage Package="Load_Customer.dtsx" />
<ParameterBindings>
<ParameterBinding Name="SnapshotDate" VariableName="User.SnapshotDate" />
<ParameterBinding Name="RunFullRefresh" VariableName="User.RunFullRefresh" />
</ParameterBindings>
</ExecutePackage>
Common Mistakes
| Mistake | Symptom | Fix |
|---|---|---|
| The child package does not have a matching PackageParameter with the same name. | Runtime error: "The parameter SnapshotDate does not exist in the package." | Create a PackageParameter extension point on each child package that declares the parameter with the same Name and DataType. |
| Using this with Package Deployment Model instead of Project Deployment. | Parameter bindings are ignored; the child package never receives the values. | Switch to Project Deployment Model, or use BatchPackageConfigurations for Package Deployment. |
| Mismatched data types between the batch variable and the child package parameter. | Runtime type-cast error or silent data truncation. | Ensure the DataType in BatchVariable matches the DataType in PackageParameter (e.g., both String or both DateTime). |
| Forgetting to declare the batch variable via BatchVariable. | Build error: "Variable User.SnapshotDate is not defined." | Add a BatchVariable extension point that declares the variable before referencing it in a binding. |
Binding to a system variable (e.g., System::PackageName) instead of a user variable. | Unexpected behavior or read-only variable errors. | Only bind user-scoped variables (User.VariableName) that you control. |
Related Extension Points
- BatchVariable -- required to declare the batch-level variables that parameter bindings reference.
- PackageParameter (object-level) -- required on each child package to declare the receiving parameter.
- BatchPreProcess -- commonly used to populate the batch variable before it is passed to child packages.
- BatchPackageConfigurations -- the Package Deployment Model alternative when Project Deployment is not used.