Account
Skip to main content

File-Mode Metadata

BimlFlex stores its metadata in one of two ways. In file mode, the warehouse model (the sources, objects, relationships, and settings that define a BimlFlex project) is a set of human-readable JSON files in a git working tree, called a file store. In database mode, the same model is rows in a metadata database. Both storage models are supported. A new metadata capability is built for both the database store and the file store unless its own page says otherwise, and the BimlFlex Metadata MCP Server is such a page. File mode is the direction for how BimlFlex stores metadata, never a limit on what database mode can do.

This page explains what a file store is, why the model lives in files, how BimlFlex reads and writes the files, and what file mode means for database-backed projects.

Why files​

A data warehouse model is a design artifact, and the way engineering teams manage a design artifact is the way they manage source code. Storing the model as files makes it:

  • Versionable. The model lives in source control alongside the rest of the solution. Every change to the warehouse design is recorded with its full history, and any prior state of the model can be recovered exactly.
  • Diffable and reviewable. A change to the model reads as a change to files, so teams can see precisely what changed and review modeling decisions through ordinary git workflows (branches, diffs, and merge review) with the same rigor they apply to code.
  • A foundation for tooling. A readable file representation gives external tooling and automation a stable surface. Tools can read the model and propose changes that a person reviews before anything lands.

The file store​

A file store is a directory under git. The model is a set of JSON documents. An entity is either its own file or nested inside a parent's document, and it is identified by its name property.

  • Identity is the name. An entity's identity is its top-level name property in the JSON document. Filenames are a convention. A rename in a designer updates the references to the entity in the same save, and two entities with the same name in one store is a load error that names both entities.
  • One writer at a time. The store gives the read and write guarantee of VSCode or Visual Studio over files on disk. When the files change on disk and nobody is mid-save, BimlFlex reloads and serves the new state. A save whose files changed on disk since they were read is refused once, with each affected file named and what happened to it (modified or deleted), and nothing is written. There is no automatic merging or arbitration between versions.
  • Git is the history layer. The store writes working-tree files only. A save in the desktop app never commits as a side effect, and the store keeps no history machinery of its own. History, undo, and recovery are git operations, through the app's git surfaces or any git client.
  • Secrets stay out of the files. Connection strings, passwords, and keys are not written into the metadata files. A migrated store carries them as named placeholders. Their values sit in a filled, git-ignored local secrets file beside the files, and a blank shareable template beside it lists the names for other machines.
  • Build output holds connection secrets. The output of a build carries working connection strings, in file mode and in database mode alike, and is handled like a secrets file. The database deployment scripts under the output's Deploy folder (ssdt-deploy.<database>.ps1) carry them on every build, and the SSIS connection manager (.conmgr) and project (.dtproj) files carry them wherever an SSIS project is emitted.

The on-disk shape of a file store, its layout, the names and references in its files, its secret references and its diagnostic codes, is described in the File Store Reference.

Reading and editing the files in the app​

The Files item of the left navigation is the in-app surface over the file store, and the Compare view lives inside the Files workspace. Files renders in both storage modes. Compare renders only in file mode, and shows a file tree on the left and a side-by-side diff between any two points on the right.

  • Files shows the store's working tree with a git status badge on each file. An open file renders in a Monaco viewer with schema-fed validation, completion, and hover documentation, and shows who last changed it and when. Entity references in a file are click-through to that entity's designer, and entity editors carry an open-file action that lands on the entity's file.
  • History and comparison are per file: the commit log for the file, its content at any commit, and a side-by-side diff between any two points or against the working tree.
  • Finding is filter-as-you-type in the tree and a fuzzy finder (Ctrl+P style) over file paths and entity names.
  • Editing a file's text in the app runs through the same save path as a designer save, with the same refusal when the file changed on disk underneath you. Text the file format does not accept is refused with the file and position named, before anything is written. Accepted text lands exactly as you typed it.

Editing the files directly​

The files are plain text and can be edited by hand or by tooling, and edits you make are saved exactly as you typed them. When you later save an entity from a BimlFlex designer, BimlFlex rewrites that entity's file in its canonical form, with a fixed layout and property order. A file that looks reformatted in a diff after a designer save is expected: it is the same model written in the standard layout, not a corrupted file.

Migrating a database to files​

BimlFlex includes a one-time migration tool that reads a BimlFlex metadata database and writes the same metadata as a file store, from the Migrate to Files page in the app or with the bfxc export command line. See Migrate a BimlFlex database to files.

Where this leaves database-backed metadata​

Database mode is supported. Existing BimlFlex projects that use a metadata database keep working as they do today, with no change to how they are edited, built, or maintained. A new capability is built for both stores unless its own page says otherwise. Migrating to files is the way to take up the git-based workflow of a file store, not a requirement for new capabilities.

Keep two distinct concepts separate. How BimlFlex stores its metadata, in a database or in files, is independent of how BimlFlex deploys your data warehouse. Warehouse deployment to your target platform works the same way regardless of the metadata storage model, and it is unaffected by file mode either way.