Varigence is excited to release the 2022 R3 version of the BimlFlex platform for data solution automation!
- Please make sure databases and projects are backed up before upgrading.
- Please email email@example.com with any installation or upgrade issues.
BimlFlex 2022 R3 is installed and upgraded through a single consolidated installer.
Build 188.8.131.52, release date: 30 Mar 2023
- BimlFlex Developer Setup. This installer includes all BimlFlex components
- BimlFlex Runtime Setup. This installer includes the required runtime components for servers that will execute SSIS packages
Manual database installation and update scripts should be used to update BimlFlex and BimlCatalog databases that require
Multi-Factor Authentication due to this reported issue: Microsoft DacServices does not work with SqlClient 5.0.
- BimlFlex Database Manual Deployment. This script includes all BimlFlex components for a manual database deployment.
- BimlCatalog Database Manual Deployment. This script includes all BimlCatalog components for a manual database deployment.
2022 R3 - New Features
- Added various Script Activity related Extension Points, including Snowflake Scale Up and Down, Snowflake Create Stage, Staging Area, and Data Vault processing. Additional information is found in the updated Extension Point documentation
- BimlFlex now supports a new REST API and sFTP Linked Service that can be generated as part of a project's connection. Configuring these new types will support the generation of a corresponding Linked Service in ADF
- For file types,
parquetis now available as a supported file connection
- A new Display Folder feature has been added to the Connection Editor. Using this feature will create a new top-level group (folder) in the Treeview, or add the connection to an existing top-level group
2022 R3 - Improvements
Various Application Improvements
- Added improved support for metadata import from a DB2 source using OLEDB. This resolves a few known issues, most importantly a syntax problem related to a missing DATA_COMPRESSION statement
- Various metadata editor components of the BimlFlex App now have updated icons representing the technology that applies. For example, connections display an icon representing the system type they are configured for. Projects and Batches show an icon representing the integration template they are associated with. This provides an easy visual reminder of the technology used in the connection
Azure Data Factory (ADF)
- The 'Truncate Landing', or 'SQL_TRUNCATE_LND' Stored Procedure call in source-to-staging Execute Pipelines has been replaced by a 'Pre Copy Script', which has been added to the Copy Activity. This addresses a reported issue where Copy Activity timeout and retry values did not correctly work with the truncate step. The Copy Activity would populate the Landing Area, but upon retry this data was not truncated which resulted in duplicates. By adding the Pre Copy Script to perform the truncation, these operations are performed in a more functionally cohesive way.
For users that use the 'ADF Pre Copy Script' Extension Point, please make sure that this is updated to include the necessary truncate statements. The statement used is @concat('TRUNCATE TABLE [',pipeline().parameters.TargetSchemaName,'].[',pipeline().parameters.TargetTableName,']').
The timeout value of the Script Activities as generated by BimlFlex has been updated to 30 minutes (formerly 10 minutes) by default. Various extension points have been added to allow users to override the Script Activities for more fine-grained configurations (see below)
Two new Linked Service options have been added for Azure Data Factory. These can be defined using the Connection Editor: Rest API and sFTP. By selecting these system types and enabling
Cloud, users can configure the details to generate these connections as Linked Services.
2022 R3 - Bug Fixes
Various Application Fixes
- Fixed a bug that incorrectly triggered a validation warning when an object has more than one parameter defined
- Fixed a bug where changing the 'FromDate' or 'ToDate' in the Version Editor unintentionally disabled the ability to save
Data Vault Accelerator
- Fixed a bug where sometimes columns that have been split into a different Satellite would still appear in the original Satellite
- Addressed an issue where the appending or removal of the Record Source column would not always perform the change. This happened very infrequently, but has now been consistently resolved
- Fixed a bug which prevented the Temporal Table Pattern Name to be correctly applied to the generated PSA tables when they are enabled for the SQL Server Temporal Table feature
- Fixed a bug where adding a new object from the Connection Editor incorrectly displayed mandatory fields
- Added some quality of life fixes in the Connection Editor related to guarding changes when switching forms. For example, unsaved changes are now properly detected when switching between a connection string and Azure Key Vault definition. Also, earlier saved changes that have been made redundant because a different configuration was selected are now consistently cleared. For example, switching between authentication modes or using key vaults. In exception cases, this metadata that was incorrectly retained and would sometimes lead to build issues
- Address an user interface issue where the 'navigate to object' button would not always navigate to the correct object
- Fixed a bug where the SQL Source Expression was not correctly applied when generating Stored Procedures to load Data Mart objects
- Fixed a bug where Snowflake for SSIS projects incorrectly displayed validation errors intended for Azure Data Factory projects
- Fixed a bug where the generated Stored Procedures would not correctly use the database name as part of the fully qualified name for Managed Instances where all solution connections are defined in separate databases. Managed Instances support cross-query databases, and this bug prevented this for Staged Queries
- Fixed a bug where quotes were not added to column names in rare cases, leading to issues when using umlauts and similar special characters. Special characters in column names should now work again
- In rare cases, when using the Pushdown Extraction feature in combination with Snowflake, some column names would not have double quotes. This only occurs in this combination, and has now been resolved
- Fixed a bug where configurations that have been set to 'default' would not correctly generate constraints for Staging Area and Persistent Staging Area objects
- Fixed a bug that would crash BimlStudio if any bundle other than the default BimlFlex.bimlb bundle would be used