Initial Configuration
Once the BimlFlex installation is completed, it is time to set up the environment for a BimlFlex project. The initial setup of BimlFlex include the steps to set up the full development environment:
- Connecting to the BimlFlex database
- Loading sample metadata (optional)
- Creating the BimlStudio project
BimlFlex App
Welcome to BimlFlex!The BimlFlex Dashboard is the first screen you will see when opening BimlFlex. The dashboard provides a quick overview of the metadata in the active Customer and Version, and supports the most common tasks and navigation options.
The primary source BimlFlex uses to generate data solutions assets is design metadata, simply referred to as 'metadata'. Metadata is most commonly in the form of Objects with corresponding properties, such as Columns and Settings.
Although it is relatively straightforward to define individual metadata objects, it can take considerable time and effort to create, update and maintain the number of objects that would typically be involved in an enterprise data solution.
The BimlFlex Application allows users to add and update all the metadata required to automate complex data solutions, without ever writing a single line of code.
Developing data solutions is not always easy, and may need many specific designs and modifications to fully meet requirements. To be able to manage this complexity in a streamlined way, BimlFlex provides a variety of editors, configurations, overrides and design features.
The BimlFlex Applicationis the main design area to configure your intended data solution. It consists of a number of metadata editors that are visible in the left-hand navigation panel.
Metadata Connection
To begin designing your data solution, start the BimlFlex Applicationand configure it to connect to the BimlFlex database.
Set up the Connection through the Database Setup Pane, which can be accessed from the Database Settings Cog in the top menu bar. The database setup pane contains options to specify connectivity to the BimlFlex database. The Update Button can be pressed to refresh the database contents when connection details have been modified.
Once connection has been established, a new Customer can be created. Or, an existing Customer and Version can be selected.
New Customer Creation
The Customer is the top-level grouping of metadata in the BimlFlex database. A customer represents an independent logical area for which the metadata is separated from other customers.
Customers can be used to identify different companies, teams or initiatives. Any metadata defined for a specific customer can not be used by other customers in BimlFlex.
A customer can contain multiple Versions to manage changes in metadata over time, but can also be used for a more fine-grained approach to separate metadata. For example, if you want to organize BimlFlex to use a single customer to represent the company, versions can be defined to represent separate teams or initiatives.
A customer can have multiple versions of metadata. An initial version is created for a new customer.
Use the New Customer Button in the BimlFlex Applicationto add the customer to the BimlFlex database.
The customer can also be created through:
- BimlStudio > BimlFlex Ribbon > Open Bundle > New Customer
- BimlFlex Excel Add-in > BimlFlex Ribbon > Metadata Connection pane > New Customer
Load Sample Metadata
BimlFlex provides several sets of sample metadata and configurations.
These samples support various common use-cases and scenarios and are intended to assist adopting specific features for certain architectures and technical environments. The full list of available samples is available in the samples documentation.
For example, samples cover how to load files using SQL Server Integration Services (SSIS), how to implement delete detection to load data into Azure, or how to deliver a Data Vault model on Snowflake.
In the BimlFlex Application, use the Load Sample Metadata button in the Getting Started pane on the Dashboard to load sample metadata to the current active Customer.
Tip
The Varigence YouTube channel contains various introduction videos about using BimlFlex and BimlStudio. This video provides an overview of how to load sample metadata into BimlFlex.
BimlStudio Configuration
With the BimlFlex database available, and a new customer created, the metadata can be connected to via BimlStudio so that the metadata can be compiled into the various data solution artifacts - table scripts, procedures and data logistics processes.
Open the BimlStudio application and create a new, empty BimlFlex project.
Note
BimlStudio and the BimlFlex Excel Add-in will prompt for a License Key the first time a new installation starts. If a trial key is required, please contact Varigence BimlFlex Support at support@bimlflex.com.
Creating the BimlFlex project
From the start page, create a new BimlFlex Project.
Place the project in a suitable folder, the folder will contain source code for the project and should be managed through a source control system.
Once the new BimlStudio project is created, it is possible to open it from the folder by opening the project file directly, or by opening the project from BimlStudio.
At this stage, a new BimlFlex project has been created in BimlStudio, but this has not yet been connected to the metadata repository.
Tip
The Varigence YouTube channel contains various introduction videos about using BimlFlex and BimlStudio. This video provides an overview of creating a new BimlFlex project in BimlStudio.
Connect to the BimlFlex Database
The BimlStudio project needs to access the metadata in the BimlFlex database in order to generate output.
Note
The databases are created and updated through the BimlFlex installer. It is also possible to set up the databases from the BimlFlex ribbon tab in BimlStudio, or directly by using the Setup BimlFlex Databases button. Additional information is available in theBimlFlex database installation section.
To connect the BimlStudio project to the BimlFlex database, connectivity details to the metadata database need to be provided.
When the connectivity to the BimlFlex database is established, a customer and version can be selected from the drop-down lists. Selecting a customer and version here and saving the project will ensure the right output is generated.
Tip
Connectivity from BimlStudio to the BimlFlex database can always be opened by clicking the Open Bundle option in the BimlFlex ribbon.
Important
After updating a Customer or Version, close and re-open the application.
When the BimlStudio project is associated with a customer and version, the metadata can be refreshed on an ongoing basis by clicking the Refresh Metadata option in the BimlFlex ribbon. This will reload the metadata, parse and validate it, and update the Logical View in BimlStudio.
The following properties are available to configure the metadata connection:
| Field | Description |
|---|---|
| Server | The address of the server where the BimlFlex database is hosted. |
| Authentication | The credentials used to access the server and database. |
| Database | The name of your BimlFlex Database. This dropdown list is populated with all databases available through the server connection. Clicking Update will refresh the list of available databases from the connection defined. |
| Customer | The BimlFlex Customer used to configure the project metadata. Click Update to refresh the dropdown with a list of Customers in the BimlFlex database. A new customer can be created with the New Customer button. |
| Version | The BimlFlex Metadata Version to be built and used for code generation in BimlStudio. Click Update to refresh the list of Versions in the BimlFlex database associated with the selected Customer. |
| Use My Connection Strings | Use the Connection Strings for the current user that were set while the Use My Connection String option was enabled. |
| Use My Exclusions | Use the Exclusions defined in the BimlFlex Metadata. |
| Use Quick Parse | Only load placeholder objects into the logical model. This allows for faster processing by not populating the full Biml representation in memory |
BimlStudio Project Settings
For further, more detailed configuration, the BimlStudio project options include settings such as the target SQL Server version, SSIS and ADF versions, and whether to use the project or package deployment model for SSIS.
Most of these are optional to change, but when using SSIS the target version and deployment method needs to be correct for the packages to open correctly in Visual Studio, and run correctly on the destination SSIS server.
If this is the first time a solution is being built, the user should check their project settings to ensure that the correct target versions are set.
The project settings can be accessed via the 'File' menu in BimlStudio, and then selecting 'Project Settings' from the menu. Within the dialog, users may select settings for Biml Engine, Build, Target Versions, Options, and Externals.
Important
Once all settings are completed, restart BimlStudio and reopen the BimlFlex project to load the metadata.
The most important settings to configure initially are the Build and Target Versions settings.
Note
Users should also check that their output path is correctly configured. If one is not provided then the generated assets will be created in the output folder of the directory that contains the .mst file.
For additional information, please refer to the BimlStudio reference documentation.
Access Project Settings by clicking File then Project Settings.
Build Output
In the build section, the output locations and options for built artifacts. Use these settings to direct where the output artifacts will be created.
To access or configure the project's output path, click File then Output Settings. Select Build from the left side of the menu.
The desired output path may be entered manually in the Output Path field. Alternatively, click the Browse... button to locate and select the output folder from the path selection dialog that appears.
Select or deselect the Clean Output Folder option, as desired.
Cleaning the output folder will scrub the folder of any old files, so that the folder only contains assets related to the most recent build.
Target Versions
In the target version section, the correct versions of the technical environment can be specified. These settings will direct the Biml compiler to output the generated artifacts in a way that matches the version, so these settings should match the target architecture.
SQL and DDL
- SQL Server, this corresponds to the SQL Version to use for Scripts and SSDT Projects
- DDL Build Mode, always leave as None for BimlFlex projects. BimlFlex provides direct script options and SSDT Projects for DDL instead
Data Integration
- ADF Build Mode. ADF can be managed through multiple, separate JSON files or Azure ArmTemplate files
- SSIS, the SSIS Server version to build for
- Use Project Deployment, should the build spawn Project Deployment ready projects and packages. This is the preferred deployment methodology for SSIS since SQL Server 2012
SQL Server Analysis Services
- SSAS Multidimensional, not included in BimlFlex projects
- SSAS Tabular, not included in BimlFlex projects