Migrate a BimlFlex database to files

BimlFlex includes a one-time migration tool that reads a BimlFlex metadata database and writes the same metadata as a set of files ready for git. The result is a file-mode metadata store that BimlFlex opens directly.

The tool is available in two forms that run the same conversion and produce the same files and the same report:

• In the app. The Migrate to Files page in the Administration group of the left navigation. Use this when the database is the one the app is connected to and you want to open the result in BimlFlex right away.
• On the command line. The bfxc export command. Use this for scripted or repeated migrations, or when the database is reachable from a machine that does not run the app.

Each run migrates one customer and one version, and a file store holds one version. To keep more than one version as files, run the tool once per version, each into its own directory.

Before you start

• The source. A BimlFlex database at a supported version. In the app, the source is the connection the app is currently using. On the command line, the source is a connection string.
• The target directory. The directory must be new, empty, a fresh git repository (only .git and .gitignore present), or an existing BimlFlex file store, which is re-exported in place. Any other non-empty directory is refused and nothing is written.
• Secrets. Connection strings, passwords, keys and other secret values are never written into the metadata files. With the default option, the run writes your values into a filled, git-ignored local secrets file beside a blank shareable template, so the first build from the migrated store works on that machine without re-entering anything. Decide whether you want that or the secret values dropped. See The secrets files.
• A validation pass first. Both forms offer a dry run that validates the whole conversion and writes nothing. Run it before the real migration so the report shows any findings while there is still nothing on disk.

The two forms side by side

Choice	In the app	On the command line
Source database	The current connection, shown read-only	--source <connection string> or the BFXC_SOURCE environment variable
Customer	The current customer, switched in the toolbar as usual	--customer <name or uid>
Version	The Version select, one version per run	--version <name> (the current version when omitted)
Target directory	The Target directory field	--repo <directory>
Secrets	Write a secrets template (default) writes the filled .bimlflex/secrets.env, which is git-ignored, and the blank .bimlflex/secrets.template.env, which stays committable to share. Strip secrets writes neither, and every value is re-entered in the app	--secrets externalize (default) or --secrets strip, with the same two outcomes
Dangling optional references	Drop optional references that no longer resolve	--force
Validate without writing	Validate, or the Validate only, write nothing option with Migrate	--dry-run
The report	Shown on the results panel and offered by Download report	--report <path>, or the default location described below

In the app

Open the page

Connect the app to the database you want to migrate, then open Administration in the left navigation and select Migrate to Files. The item appears while the app is connected to a database. In file mode there is no database to migrate, and the page explains that a database connection is needed and links to Setup.

Above the form, a readiness strip confirms that the migration tool is available, that the current connection is a database, and that no other migration is running. When a row is blocked, its message says why, and the Validate and Migrate buttons stay disabled until the condition clears. One migration runs at a time.

Source

The source is shown read-only from the current connection: the server, the database, the authentication kind, and the current customer. Switch customers in the toolbar as you do everywhere else in the app.

Choose the Version to migrate. The select lists the current customer's versions and defaults to the version in context. One version is exported per run.

Target

Enter the Target directory as an absolute path on the machine the app runs on, for example C:\repos\metadata. The rule from Before you start applies: new, empty, a fresh git repository, or an existing BimlFlex file store to re-export in place. Anything else is refused before the run starts, and nothing is written.

Options

• Write a secrets template (default) or Strip secrets. Secret values are not written into the metadata files either way. With the template option, the run writes your values into the filled local secrets file at .bimlflex/secrets.env inside the target, and the blank shareable template at .bimlflex/secrets.template.env beside it. The filled file is git-ignored and the template stays committable. With the strip option, neither file is written and every secret value is re-entered in the app. See The secrets files.
• Drop optional references that no longer resolve (default off). Without this, a dangling optional reference refuses the run and the report names it.
• Validate only, write nothing (default off). Runs the whole conversion through serialization and self-verification and stops before the write.
• Initialize a git repository in the target (default on). Lets the finished migration open directly in BimlFlex, because opening a local file store needs a git work tree. The repository is initialized only when the target does not exist yet or is empty, so a target that already is a git repository is left as it is. The option is skipped on a validation run, which creates nothing.

Validate and Migrate

Validate runs the conversion as a dry run regardless of the option above. It reads the database, runs every stage, and reports the outcome without touching the target directory. A validation of a new path leaves that path absent.

Migrate runs the conversion and writes the files. Both buttons are disabled while a run is active, while the readiness strip shows a blocked row, and until a target directory is entered.

The run

The run card shows the state (Queued, Running, Succeeded, Completed with warnings, Refused, Failed, Cancelled, or Timed out), the start time and elapsed time, and the live output of the tool as it arrives. The output prints one line per stage: the preflight with the database version, customer, version and reference-integrity findings, the extract with row counts, the migration steps applied, the model build with entity counts, the serialization and self-verification with the file count, the write with the target path and the number of secret template lines and filled secret values, and a closing report line with the exit code.

Cancel stops the run. A run cancelled before the write stage wrote nothing. A run cancelled during the write leaves the tool's staged directories (.bfxc-tmp-* and .bfxc-old-*) in the target for manual recovery, and the results panel says so with the path. The write never destroys the previous generation of files.

The results

When the run ends, the results panel shows:

• The verdict. Succeeded, Completed with warnings, Refused (nothing was written, and the findings carry the reason), Failed, Cancelled, or Timed out. A validation run reports "Validated, nothing written".
• Source and target. The server, database, customer, version, database version, export time, and the target directory, as recorded in the report.
• Entity counts. How many entities of each kind were exported.
• Findings. Error, warning and forced-warning totals, then the findings grouped by code. Expand a group to see each finding's severity, entity, key, detail, and the action the tool took.
• Steps. Each conversion step with its status, so a step that was applied, verified as a no-op, skipped, or stubbed is visible.
• Secrets guidance. When the run wrote the secrets files, the panel says that secret values were not written into the metadata files, that your values are in place in the local secrets file, which is git-ignored, so keep it out of source control, and that the shareable template at <target>/.bimlflex/secrets.template.env lists the named slots for other machines. When some secret values were already masked in the database, each one carries a required action in the findings. The panel renders a Required actions list ahead of the guidance, and the verdict states "This run is not complete until the required actions below are done." See Values the database only held masked.

The actions are Open in BimlFlex, Download report, Copy target path, and Run again, which returns to the form with the same values. A Recent runs list lets you reopen a result after navigating away. The report is kept with the run and is not written into the target directory.

Open in BimlFlex

After a successful migration, Open in BimlFlex connects the target directory as a local file store. The app switches into file mode over the migrated store, with the migrated files showing as uncommitted changes ready to commit. Opening a local store needs a git work tree, which is why the initialize option is on by default. When the run did not initialize a git repository, the button is replaced by the instruction to run git init in the target and connect from Setup.

From the command line

bfxc is installed with BimlFlex. The installer places it in a bfxc folder inside the BimlFlex application folder. Run it from that folder, or add the folder to your PATH.

bfxc export

bfxc export --source <connection string> --customer <name or uid> --version <name> --repo <directory>
            [--secrets externalize|strip] [--force] [--dry-run] [--report <path>]

Option	Meaning
--source	The connection string of the BimlFlex database. May be omitted when the BFXC_SOURCE environment variable is set.
--customer	The customer to export, by name or uid. Required.
--version	The version to export, by name. The customer's current version when omitted. One version per run.
--repo	The target directory. The same rule applies as in the app: new, empty, a fresh git repository, or an existing BimlFlex file store to re-export in place. Anything else is refused and nothing is written.
--secrets	externalize (default) writes the filled .bimlflex/secrets.env inside the target, which is git-ignored, and the blank .bimlflex/secrets.template.env, which stays committable to share. strip writes neither and drops the secret values. Neither writes a secret value into the metadata files.
--force	Drops optional references that no longer resolve instead of refusing the run.
--dry-run	Validates the whole conversion, including serialization and self-verification, and writes nothing.
--report	Where to write the JSON report.
--log <file>	Also writes the output as JSON lines to a file. Available on every bfxc command.
--quiet	Suppresses the console output. Available on every bfxc command.

Keeping the connection string off the command line

A connection string that carries a password should not appear on the process command line, where any process listing on the machine can read it. Set the BFXC_SOURCE environment variable in the shell instead and omit --source. When both are present, --source wins.

set BFXC_SOURCE=<connection string>
bfxc export --customer Contoso --version "Version 1" --repo C:\repos\metadata

In PowerShell, set the variable with $env:BFXC_SOURCE = "<connection string>" before running the command.

Output, report, and exit codes

The tool prints one line per stage, the same lines the app shows in its live output, and ends with a report line naming the report path and the exit code.

The report is a JSON file with the source and target, the entity counts, the findings, the steps, and the exit code. When the run writes the files, the report defaults to .bimlflex/conversion-report.json inside the target. When the run is a dry run or is refused, nothing is written to the target, so the report defaults to conversion-report.json in the working directory. Pass --report to choose the location.

Exit code	Meaning
0	Succeeded.
2	Completed with warnings. The files were written and the report lists the warnings.
3	Refused by validation or policy. Nothing was written. The findings carry the reason.
1	Failed.

bfxc version prints the tool version, the file format version it writes, and the database version it targets, as one JSON object.

The secrets files

Secret values are never written into the migrated metadata files, in either form and with either secrets option. Where the database held a connection string, a password, or a key, the files carry a named placeholder instead. The value behind each placeholder lives in one of two places on the machine: the filled local secrets file, or the credential store the app keeps on that machine.

Where each value lives

With the template option, a run that writes the files also writes the secrets files in the .bimlflex folder inside the target:

• .bimlflex/secrets.env is the filled local secrets file. It carries the values read from the database, one line per placeholder with the placeholder name and its value, and is written when at least one value was read. This is the file the build resolves named secrets from.
• .bimlflex/secrets.template.env is the blank shareable template. It lists every placeholder name with an empty value, so it shows which secrets the store needs without carrying any of them.

The target's .gitignore gets rules that ignore the filled file and every per-environment secrets file beside it and keep the template committable. When the database holds no secrets, neither file is written.

The first build

A build from the migrated store on that machine works right away, without typing any value by hand, and the app shows the same values it showed in database mode. The build takes each named secret from the filled local secrets file when that file carries it, and from the credential store the app keeps on that machine for any name the file does not carry. The file is read at build time and is never copied into the credential store, so every secret lives in exactly one place.

A malformed filled file refuses the build with a message that names the file and the line, and never the value.

Values the database only held masked

A value the source database stores masked cannot be recovered from it. The slot is listed in the template and left out of the filled file, and the run raises a finding with a required action for it, carried in the report and listed under Required actions on the results panel. The run is not complete until each such value is entered, either on its line in .bimlflex/secrets.env or in the app's credential store.

Source control

The filled file stays on the machine and out of the repository. The template carries no values, so it is safe to share, and it is not ignored, so it is committed with the store. On another machine, fill a copy of the template as that machine's .bimlflex/secrets.env, or enter the values in that machine's app credential store.

Running the migration again

Exporting into the same directory again regenerates the .bimlflex folder, including both secrets files. A value you typed by hand into .bimlflex/secrets.env, for a slot the database only holds masked, is lost on the next export and its required action is raised again. A value entered in the app's credential store survives a re-export.

The strip option

With the strip option, neither file is written and every secret value is re-entered in the app.

Open, review, and commit

The migration leaves the target directory uncommitted. Nothing is committed on your behalf. Open the store in BimlFlex, with Open in BimlFlex or by connecting to the directory from Setup, review the files as uncommitted changes, and commit them yourself through the app's git surfaces or any git client. That first commit is the starting point of the store's history.

note

Files in a BimlFlex file store can be edited by hand. When you later save an entity from a BimlFlex designer, its file is rewritten in the standard canonical form, so a reformatted file after a designer save is expected and is not corruption. See Editing the files directly.

The migration can be run again into the same directory with the same customer and version. An existing BimlFlex file store is re-exported in place, and files whose content did not change show no change in git. The export replaces the store's contents, so a different version exported into the same directory replaces the version that was there.

Migrating in the other direction

The rare case of moving a file store back into a database is available on the command line as bfxc import:

bfxc import --repo <directory> --target <connection string> [--customer <name or uid>] [--version <name>]
            [--force] [--verify] [--dry-run] [--report <path>]

The import is additive into an empty customer version of the target database. A version that already holds metadata is refused. --target may be omitted when the BFXC_TARGET environment variable is set, the same way --source works for export. During import, store: placeholder names resolve from both .bimlflex/secrets.template.env and .bimlflex/secrets.env under the store, and the filled .bimlflex/secrets.env wins when both carry the same name. env: placeholder names resolve from environment variables. Any placeholder that does not resolve is reported as a finding and its value must be re-entered in the target database.