Skip to main content

Release Notes

Varigence is pleased to release BimlFlex 2027.

note
  • We recommend testing in a non-production environment before upgrading your production environment.
  • Please make sure databases and projects are backed up before upgrading.
  • Please email [email protected] with any installation or upgrade issues.
Important for Upgrade

Ten retired Azure Data Factory connectors are removed in this release. Microsoft has removed these connectors from the Azure Data Factory service: nine were retired on 31 December 2024, and the Amazon Marketplace Web Service connector was removed after Amazon shut down the underlying MWS API on 31 March 2024. Existing pipelines that use them no longer execute, and new workloads cannot be created. The affected connector types are:

  • Amazon Marketplace Web Service (MWS)
  • Concur
  • Couchbase
  • Drill
  • HBase
  • Magento
  • Marketo
  • PayPal
  • Phoenix
  • Zoho

BimlFlex no longer offers these connector types for new connections and no longer emits them to Azure Data Factory. Existing metadata configured with one of these connector types produces a validation error identifying the retired connector.

For any affected connection, migrate it to a supported connector or remove it from your metadata. Until this is done, the build reports a validation error identifying the retired connector. Because Microsoft has removed these connectors from the Azure Data Factory service, they are non-functional in deployed pipelines regardless of this change.

Important for Upgrade

Four more retired Azure Data Factory connectors are removed in this release. The Azure Data Factory service retired these connectors on 31 December 2024. The affected connector types are:

  • Azure Database for MariaDB
  • Oracle Eloqua
  • Oracle Service Cloud
  • Salesforce Marketing Cloud

BimlFlex no longer offers these connector types for new connections and no longer emits them to Azure Data Factory. Existing metadata configured with one of these connector types produces validation error CON_21005013 identifying the retired connector; migrate the affected connections to a supported connector or remove them from your metadata.

Important for Upgrade

Scoped overrides that previously never applied now apply. A defect in five metadata lookups caused certain scoped overrides (batch-scoped setting and configuration overrides, connection-scoped and project-scoped extension points, and parent-project setting inheritance) to be silently ignored at build time. BimlFlex 2027 fixes these lookups, and every previously-ignored override of these shapes takes effect on your next build, which can change generated output.

Before building with BimlFlex 2027, review the overrides in your metadata: the BimlFlex app (2026 release line) flags every affected row with warning ATT_21005007 ("dormant scoped override"), and the audit query under Upgrade Considerations below lists the same rows directly from your BimlFlex database. Remove or adjust any override you do not want to take effect.

Installation​

BimlFlex 2027 is installed and upgraded through a single consolidated installer.

Build number and download links will be published when BimlFlex 2027 is released.

What's New in BimlFlex 2027​

BimlFlex 2027 carries one idea end to end. Define the warehouse once, in metadata, and let the product do the rest: model it, generate it, see it, and trust it. A new guided Dimensional Accelerator takes you from source objects to a published star-schema model. If you're starting from an existing warehouse instead, you can classify imported tables from the import editor to attach them into your model. The generation engine underneath is not new (BimlFlex has generated production dimensional load logic for years) and in this release it extends to Microsoft Fabric: Data Factory pipelines emitted in Fabric's native git JSON format, from the same metadata that already targets SQL Server, SSIS, SSAS, Azure Data Factory, Snowflake, and Databricks. The same Fabric build can also emit Power BI Direct Lake semantic-model definitions (TMDL) as an opt-in preview. Before anything deploys, you can run the build from inside the app, browse every generated asset with its Biml, and trace objects and columns across the whole generated warehouse, column-level wherever your mappings, keys, and expressions establish the link. And generation stays deterministic: the same metadata produces the same output every build, with no LLM in the generation path. When validation flags something, guidance on what the finding means and how to fix it arrives in banners and tooltips where you work. The release rounds that out with a rebuilt dashboard, applicability-aware settings, bulk editing, and continued hardening of the server API.

Dimensional Accelerator​

BimlFlex 2027 introduces the Dimensional Accelerator: a guided path from source objects to a published star-schema model. Pick a target project, select your sources, preview the dimensional model, and publish, with a built-in walkthrough for first-time users. Point it at your staging or persistent-staging tables and it derives the star schema for you (dimensions, facts, surrogate keys, and change-type defaults) in one pass. If you've built your Data Vault in BimlFlex with point-in-time tables, the accelerator derives your dimensional marts directly from it: hubs with PITs become dimensions, links become facts. The source-view SQL for every derived dimension and fact is written into your metadata automatically, and the accelerator detects hand edits so your customizations are never overwritten.

The accelerator accelerates modeling. The load code itself still comes from the generation engine BimlFlex has shipped for years across SQL Server, Snowflake, and Databricks, and now extended to Microsoft Fabric: slowly changing dimensions (Type 1 and Type 2), surrogate keys, and inferred-member handling.

Microsoft Fabric Generation​

Define your warehouse once in BimlFlex metadata and generate implementations across SQL Server, SSIS, SSAS, Azure Data Factory, Snowflake, Databricks, and now Microsoft Fabric. BimlFlex compiles your data-warehouse metadata into Microsoft Fabric Data Factory pipelines (120+ pipelines for a full Data Vault solution in a single build) emitted in Fabric's native git JSON format with pipeline schedules included. Build a full Data Vault or Data Mart solution targeting Microsoft Fabric Warehouse: BimlFlex generates the orchestration pipelines with typed Warehouse connections and a workspace connections manifest for provisioning.

Fabric output is generated through typed, validated translators across the pipeline constructs BimlFlex generates for Fabric (the same first-class treatment as our Azure Data Factory output, not string-templated JSON), and a built-in guard rejects any raw-JSON override that collides with a typed property. Where your model uses scheduled triggers or connections Fabric cannot represent, the build produces explicit per-item warnings and a machine-readable connections manifest with degradation states, not silent drops.

Semantic Models (Preview)​

BimlFlex 2027 adds Power BI Direct Lake semantic-model generation as an opt-in preview: enable the SemanticModelEnable setting value and the build emits semantic-model definitions (TMDL) alongside your Fabric output. The generator emits the folder-and-file layout Fabric uses for semantic models in git: a Name.SemanticModel definition folder with directLake partitions bound to OneLake. Every emitted semantic model is verified two ways: byte-exact golden files, and a round-trip through Microsoft's own Tabular Object Model serializer, so the TMDL BimlFlex produces is TMDL Microsoft's tooling parses. Varigence has generated Analysis Services Tabular semantic models from metadata for years in BimlStudio; the new TMDL work brings that same discipline to Power BI Direct Lake on Fabric.

Build, Preview, and Extension Points​

Run your BimlFlex build from inside the app: a dedicated Build page starts the compiler against your metadata and surfaces the results, no separate IDE round-trip. Before you run it, preview what the build will produce: an in-app tree of every generated asset with the underlying Biml for each node. Extension points are now first-class in the app too: author them in a Biml-aware Monaco editor, bind them to targets, and run Validate with diagnostics that jump you to the exact line and column.

The Build page also checks your metadata for readiness problems before you run a build, surfacing them as actionable rows right on the page. And when a build produces diagnostics, they arrive grouped by severity and code, with each finding deep-linking straight to the entity that caused it.

Global Lineage​

Trace any object or column across the whole generated warehouse. BimlFlex builds a global lineage graph from your metadata and build snapshot, down to column level wherever authored mappings, key compositions, expressions, or conservative column-name matching establish the link, with object-level flow everywhere else. Both a graph view and a text trace view are available from a single lineage entry point in the navigation. You can also see impact as a flat, scannable panel, click through to the object's editor, and pick lineage targets with a column-aware filter.

Settings Intelligence and the Dashboard​

BimlFlex now knows which of its hundreds of settings actually apply to your platform and stage: inapplicable settings are hidden by default (one click shows all), applicability is badged, and overriding a dormant setting warns you before it silently does nothing. The per-setting reference documentation has also been expanded: each setting is now documented individually in the BimlFlex Settings Reference. The settings editor itself is more transparent: every setting shows its description, its default, and where its current value comes from, so you can see at a glance what has been overridden and by what. Search matches the setting key, its description, and its grouping, and you can restore any changed setting to its default in one click, or filter to @modified to review exactly what you've changed. Settings also link straight to their documentation pages; links appear only where the doc page exists.

The dashboard was rebuilt from the ground up: a responsive, accessible layout with a live model-health score and more than 20 automated checks across completeness, naming, modeling, and data quality; each finding links directly to where you fix it.

Bulk Editing​

Edit metadata in bulk without leaving the product: a grid-based editor lets you change many rows across entity types in a detached, validated working set (you see every pending change and its validation state before an explicit save) with named column profiles for repeatable views.

Import and Brownfield Attach​

Bringing an existing warehouse into BimlFlex? Classify imported tables directly from the import editor to attach an existing database's tables into your model. Import also shows a summary strip of the settings in play, and BimlFlex checkpoints your workspace before the operation runs.

Validation Guidance​

Validation findings explain themselves: a finding can carry an explanation, a remediation, and a documentation link, delivered in banners and tooltips where you work. Written guidance ships for Connection, Project, Object, Column, Setting, and Version validation rules: what the finding means and how to fix it.

Navigation now follows how you actually deliver a warehouse: the left nav is organized by lifecycle stage, lineage lives at a single merged entry point, and first-run users are guided to import metadata and resumed where they left off.

Engineering Quality​

Ongoing engineering-quality work continued through this release: recent work closed input-validation gaps in the server API, including strict entity-type validation and constraining file reads to the application content root.

What This Release Does Not Do​

  • Generation, not deployment. BimlFlex generates Fabric Data Factory pipelines as files in Fabric's native git JSON format; getting them into a workspace is your deployment process. This release does not deploy to Fabric, and Fabric notebook support is not part of the typed output set.
  • Fabric target scope. The supported Fabric solution target in this release is Fabric Warehouse. Lakehouse solution templates are not part of this release.
  • Semantic models are a preview. Semantic-model definitions (TMDL) are emitted as build output files; getting them into a Power BI or Fabric workspace is your deployment process. Direct Lake on OneLake is the supported flavor in this preview.
  • Data Vault to star requires PITs. The vault-to-marts derivation works from hubs with point-in-time tables; hubs without a PIT are excluded with a diagnostic rather than guessed at.
  • Lineage granularity is evidence-based. Column-level edges appear where authored mappings, key compositions, expressions, or conservative name matching establish them; elsewhere lineage is object-level flow. Very large traces degrade gracefully with a notice.
  • How Fabric output is verified. Fabric emission is regression-guarded by a byte-diff golden-output harness: deterministic output, verified build over build.
  • Escape hatches stay explicit. Typed translators cover the constructs BimlFlex generates for Fabric; a deliberate raw-JSON escape hatch remains for constructs outside the typed set, with a guard that rejects collisions with typed properties.
  • Windows is required. BimlFlex runs on Windows.
  • Your metadata stays in your estate. BimlFlex works on metadata in your own database, reached with credentials you hold, inside your own environment.

Upgrade Considerations​

Scoped Setting and Extension Point Overrides Now Apply​

Earlier BimlFlex versions contained a defect in five metadata lookups: each filtered on a foreign key unrelated to the scope being queried, so overrides of the shapes below were silently ignored at build time. BimlFlex 2027 corrects the lookups. Overrides that never applied before now apply, which can change generated output on your first build after upgrading:

  • Batch-scoped setting overrides: SettingValue and {ConfigurationKey}_ConfigurationValue / {ConfigurationKey}ConfigurationValue rows attached to a Batch. This includes AppendBatchName, so emitted package names can change, and the Snowflake connection family (account, warehouse, database, and scale settings), so Snowflake connections can be repointed.
  • Connection-scoped and project-scoped extension points: ExtensionPointTarget rows attached only to a Connection or only to a Project. Extension points registered at these scopes that never fired will start firing, injecting their custom Biml into generated artifacts.
  • Parent-project setting inheritance: child projects now inherit SettingValue overrides entered on their parent project.
  • Object- and column-scoped extension points now also participate in connection-scope lookups: object/column-scoped ExtensionPointTarget rows always carry a Connection reference, so the corrected connection lookup considers them as well. An injection only occurs where a connection-scope extension point with a matching name and target is actually authored, but if you use object- or column-scoped extension points, review them alongside the audit results below.

The BimlFlex app (2026 release line) already flags every affected row with warning ATT_21005007 ("dormant scoped override"). Before building with BimlFlex 2027, review those warnings, or run the audit query below against your BimlFlex database, and remove or adjust any override you do not want to take effect.

Edge case: if a CustomAttribute row was created directly in the database with two scope foreign keys (the BimlFlex app never creates these), the first-match ordering between it and a single-scope row can change.

The audit query lists every row that will start taking effect. It applies the same conditions as warning ATT_21005007; key comparisons are case-sensitive to match the build behavior. It lists single-scope rows only; object/column-scoped extension-point rows (last bullet above) are not listed even though connection-scope lookups now also consider them:

DECLARE @zero UNIQUEIDENTIFIER = '00000000-0000-0000-0000-000000000000';

SELECT ca.[UID],
       ca.[AttributeKey],
       ca.[AttributeValue],
       ca.[AttributeProperty],
       ca.[BatchUID],
       ca.[ProjectUID],
       ca.[ConnectionUID],
       [Shape] = CASE
         WHEN ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'SettingValue'
              AND ca.[BatchUID] IS NOT NULL AND ca.[BatchUID] <> @zero
           THEN 'Batch-scoped setting override'
         WHEN ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'SettingValue'
           THEN 'Parent-project setting override (children will inherit)'
         WHEN ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'ExtensionPointTarget'
              AND ca.[ConnectionUID] IS NOT NULL AND ca.[ConnectionUID] <> @zero
           THEN 'Connection-scoped extension point'
         WHEN ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'ExtensionPointTarget'
           THEN 'Project-scoped extension point'
         ELSE 'Batch-scoped configuration override'
       END
FROM [app].[CustomAttributes] ca
WHERE ISNULL(ca.[IsDeleted], 0) = 0
  AND (
    -- Batch-scoped SettingValue override
    (ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'SettingValue'
      AND ca.[BatchUID] IS NOT NULL AND ca.[BatchUID] <> @zero
      AND (ca.[ConnectionUID] IS NULL OR ca.[ConnectionUID] = @zero))
    OR
    -- SettingValue override on a project that is some project's parent
    (ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'SettingValue'
      AND ca.[ProjectUID] IS NOT NULL AND ca.[ProjectUID] <> @zero
      AND (ca.[BatchUID] IS NULL OR ca.[BatchUID] = @zero)
      AND (ca.[ConnectionUID] IS NULL OR ca.[ConnectionUID] = @zero)
      AND (ca.[ObjectUID] IS NULL OR ca.[ObjectUID] = @zero)
      AND (ca.[ColumnUID] IS NULL OR ca.[ColumnUID] = @zero)
      AND EXISTS (SELECT 1
                  FROM [app].[Projects] p
                  WHERE ISNULL(p.[IsDeleted], 0) = 0
                    AND p.[ParentProjectUID] = ca.[ProjectUID]
                    AND p.[UID] <> ca.[ProjectUID]))
    OR
    -- Batch-scoped configuration override, both key conventions:
    -- '{Key}_ConfigurationValue' and '{Key}ConfigurationValue'
    (LEN(ca.[AttributeKey]) > LEN('ConfigurationValue')
      AND RIGHT(ca.[AttributeKey], LEN('ConfigurationValue')) COLLATE Latin1_General_CS_AS = 'ConfigurationValue'
      AND ca.[BatchUID] IS NOT NULL AND ca.[BatchUID] <> @zero
      AND (ca.[ProjectUID] IS NULL OR ca.[ProjectUID] = @zero))
    OR
    -- Connection-only ExtensionPointTarget
    (ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'ExtensionPointTarget'
      AND ca.[ConnectionUID] IS NOT NULL AND ca.[ConnectionUID] <> @zero
      AND (ca.[BatchUID] IS NULL OR ca.[BatchUID] = @zero)
      AND (ca.[ProjectUID] IS NULL OR ca.[ProjectUID] = @zero)
      AND (ca.[ObjectUID] IS NULL OR ca.[ObjectUID] = @zero)
      AND (ca.[ColumnUID] IS NULL OR ca.[ColumnUID] = @zero))
    OR
    -- Project-only ExtensionPointTarget
    (ca.[AttributeKey] COLLATE Latin1_General_CS_AS = 'ExtensionPointTarget'
      AND ca.[ProjectUID] IS NOT NULL AND ca.[ProjectUID] <> @zero
      AND (ca.[BatchUID] IS NULL OR ca.[BatchUID] = @zero)
      AND (ca.[ConnectionUID] IS NULL OR ca.[ConnectionUID] = @zero)
      AND (ca.[ObjectUID] IS NULL OR ca.[ObjectUID] = @zero)
      AND (ca.[ColumnUID] IS NULL OR ca.[ColumnUID] = @zero))
  );

Retired Azure Data Factory Connectors​

Azure Data Factory connector types that Microsoft has removed from the service are no longer supported by BimlFlex: Concur, Couchbase, Drill, HBase, Magento, Marketo, PayPal, Phoenix, and Zoho (retired 31 December 2024), and Amazon Marketplace Web Service (MWS; removed after Amazon shut down the underlying MWS API on 31 March 2024). These connector types are no longer offered when creating new connections, and BimlFlex no longer emits them to Azure Data Factory. Existing metadata configured with one of these connector types produces a validation error identifying the retired connector; migrate the affected connections to a supported connector or remove them. See Important for Upgrade above for details. Deprecation warnings for these connector types are being introduced in the BimlFlex 2026 release line ahead of this removal.

Four more connector types retired by the Azure Data Factory service on 31 December 2024 are also removed: Azure Database for MariaDB, Oracle Eloqua, Oracle Service Cloud, and Salesforce Marketing Cloud. These connector types are no longer offered when creating new connections, and BimlFlex no longer emits them to Azure Data Factory. Existing metadata configured with one of these connector types produces validation error CON_21005013 identifying the retired connector; migrate the affected connections to a supported connector or remove them.

Support​

For questions or issues related to BimlFlex 2027:

For installation or upgrade assistance, please email [email protected] with detailed information about your environment and any issues encountered.