Object Editor

BimlFlex Objects are data sets defined in metadata.

Objects closely mirror a Relational Database table, though not every object will translate directly to a table because BimlFlex also supports flat files, Excel documents and, via use of Extensions Points, APIs calls. Instead of using multiple entity types to handle these different sources of data, BimlFlex defines each as an object and uses the Object Type field to annotate what type of object is being referenced.

Objects are created and edited using the Object Editor in the BimlFlex Application.

Overview

Objects are displayed in a hierarchical Treeview on the left Side Navigation menu. The treeview is organized by Project > Source Connection > Objects, with the number of tables within a given Object displayed in parenthesis.

Note

Detailed descriptions of all Object Editor fields and options are available in the Reference Documentation.

Selecting an Object with multiple tables will display a secondary list of all editable options within. The user may navigate to the Object editor screen by selecting any Object from the displayed list.

Selecting an Object with a single table will immediately navigate the user to the Object editor screen.

The treeview for Objects is expandable/collapsible by clicking the icons.

Selecting the checkbox(es) icon will enable users to bulk archive multiple Objects at one time.

Any instance of the icon will serve as a Global Navigation Transition. Clicking this icon will navigate the user directly to the referenced Connection, Object, or Column.

Toggling Descendants will enable or disable all parameters for both the currently selected node and all of its descendant nodes. This option is enabled by default.

The Search field is contained within the treeview menu, and will be hidden when collapsed. The Search field will return results in real-time, only presenting Objects containing the term being searched.

Details Tab

The Details Tab focuses on general Object information and configuration. This tab is used to define and create the Object itself.

Details Tab - Action Buttons

Icon	Action	Description
	Save	This will save the current set of staged changes. The Save button is will only enable if the Object has changes staged and there are no major validation issues with the current Object properties.
	Discard	This will Discard any unsaved changes and revert to last saved form.
	Archive	This will hard delete the selected Object. This will result in the physical removal of the selected record from the metadata database. The data will no longer be accessible by the BimlFlex Applicationand will require a Database Administrator to restore, if possible. Clicking Archive creates an Archive Object Dialog.
	Clone	This will bring up Clone Table Dialog that is used primarily in the creation and loading of a Data Mart.
	Duplicate	This will create a duplicate of the selected Object. A Duplicate Table Dialog will appear asking for a Target Table Name and a new Object will be created using all of the selected Object's current properties.
	Refresh	This will trigger a refresh of the metadata for the selected Object.
	Accelerate	This will transition to the Data Vault Accelerator and select the current active Object. See the annotated link for details on use and function.
	Diagram	This will transition to the Schema Diagram and select the current active Object. See the annotated link for details on use and function.
	Lineage	This will transition to the Data Lineage and select the current active Object. Data Lineage provides and insight to the data linage of the Object. See the annotated link for details on use and function.
	Excluded	This will determine if the Object and its metadata will be exluded along with the rest of the BimlFlex solution. This is designed to be paired with the Use My Exclusions (Locally) global setting to allow for multiple developers to work on different functional areas without deleting or globally excluding entities.
	Deleted	This will soft delete the currently selected Object. This will remove the Object and all associated entities from processing and validation. For information on how to add excluded or deleted items back to the scope, please read our tips and tricks section.

Additional Dialogs

Archive Object Dialog

Confirmation warning against the dangers of archiving. You are required to confirm by both the check box and the Ok button.

Warning

Archiving is a permanent removal of the selected entity from its associated table in the BimlFlex Database. The best practice is to first use the Deleted flag (soft delete) as an indication that the connection may need to be removed.

Clone Table Dialog

The Clone Table feature is primarily provided to allow for quick generation of Dimensions and Facts, but it can also be used to quickly define a new target Object for a given selected (source) Object. The location where the new target object is created is derived from the Project configuration of which the selected object is part of.

For example, if a given object is part of a project that targets a Data Vault Connection, the cloned object will be created in this connection.

The most common use case for using the clone feature is to create a Data Mart target for a manually created view, typically sourcing from the Data Vault. This view has already been created for the intended target state and structure.

Note

Please refer to the Data Mart section for more information on delivering data using Data Marts and Dimensional Models.

When cloning an object, please provide the intended Target Schema, choose your Target Object Type, set your Clone Table Options and click Save.

Clone Table Options

Option	Description
Overwrite Existing	When enabled, if the Object or associated Columns already exist, the Object and/or Columns will be overridden with current settings. When disabled only not existing entities will be created.
Add Target Column Mappings	This will automatically map the appropriate Target Columns to the columns of the source object.
Add Identity Column	Adds an Identity Column that can be used as the Surrogate Key (SK) for the cloned table that can be defined as Primary Key. The identity column automatically provides an incremental number value so that each record is guaranteed to be unique. This is common for a Dimension Object to support Type 2 (history tracking) values. It is generally not used when designing a Fact Object, unless there is a specific use case. Please note that when providing this column, the Primary Key is automatically enabled and the ordinal position will be set to '1'.

Duplicate Table Dialog

This will create a copy of the selected Object and properties along with a copy of all the Columns.

Important

Attributes and Parameters are not copied over.

Duplicate Table Options

Option	Description
Overwrite Existing	When enabled if the Object or associated Columns already exists, the Object and/or Columns will be overridden with current settings. When disabled only not existing entities will be created.

Details Tab - Standard Fields

The standard fields hold the bulk of the metadata that will be used to integrate the Object. Additional fields to manage the modeling and the ability to toggle Object level persistence is provided.

Model Overrides - Standard Fields

Allowed Values

Object Types

Object Types are configured using the Object Editor and drive the behavior and purpose of the Object for the design and code generation process. The available object types are dependent on the way the Connection is used in a BimlFlex Project. Depending on the connection's Integration Stage and System Type some options may, or may not be, available.

Change Type	Code	Description	Allowed for Integration Stage	Allowed for System Type
Bridge	BRG	Defines the object as a Data Vault Bridge entity.	Business Data Vault, Data Vault	All
Dimension	DIM	Defines the object as a Dimension Table in a Dimensional Model / Data Mart.	Data Mart	All
Exclude in Accelerator	EXC	Flags an object to not be included in the Data Vault Accelerator.	Source System	All
Fact	FCT	Defines the object as a Fact Table in a Dimensional Model / Data Mart.	Data Mart	All
Flat File	FIL	Determines that the object is a flat file.	File Export, Source System	Avro Format, Binary Format, Delimited Format, JSON Format, ORC Format, Parquet Format, Ragged Right Format , XML Format
Hierarchy Link	HAL
Hub	HUB	Defines the object as a Data Vault Hub entity.	Business Data Vault, Data Vault	All
Ignore	IGN	Setting an object to Ignore will remove it from any processing and/or visualisation in BimlFlex. No Staging Area or Persistent Staging Area objects will be derived.	All	All
Json	JSON	Determines that the object is a flat file in JSON format. Only applies when used for a REST API connection.	Source System	Rest Service
Link	LNK	Defines the object as a Data Vault Link entity.	Business Data Vault, Data Vault	All
Link Satellite	LSAT	Defines the object as a Data Vault Link-Satellite entity.	Business Data Vault, Data Vault	All
Point In Time	PIT	Defines the object as a Data Vault Point-In-Time (PIT) entity.	Business Data Vault, Data Vault	All
Reference	REF	Determines that this object will be generated as a stand-alone historized table in a Data Vault context.	Business Data Vault, Data Vault	All
Reference Satellite	RSAT	Determines that this object will be generated as a stand-alone (unconnected) Satellite in a Data Vault context.	Business Data Vault, aw Data Vault	All
Satellite	SAT	Defines the object as a Data Vault Satellite entity.	Business Data Vault, Data Vault	All
Stored Procedure	SP	For source objects only, determines the the object is a stored procedure whose output must be ingested into the data solution by BimlFlex.	Source System	Azure SQL Database, Azure Synapse, Microsoft SQL Server
Staged Query	STQ	Defines the object as an object that is derived from other existing objects. This will be treated as a data source, but no Staging Area and/or Persistent Staging Area components will be generated. For more details, please refer to the Staged Query Concept.	Source System	Azure SQL Database, Azure Synapse, Microsoft SQL Server, Snowflake Data Warehouse
Table	TBL	This is the default object type in BimlFlex. A Table property for the object means that no specific behaviour is applied by the BimlFlex code generation processes.	Business Data Vault, Landing Area, Master Data Services, Persistent Staging Area, Data Vault, Source System, Staging Area	Azure Synapse, COZYROC Excel File, COZYROC Salesforce, DB2, Dynamics CRM, Microsoft SQL Server, MySql, Oracle, Oracle RDB, PostgreSQL, Salesforce, Salesforce Marketing Cloud, Salesforce Service Cloud, Snowflake Data Warehouse, Sybase, Teradata
CDC All	TCA	Determines that the object is configured for Change Data Capture, querying the full history of available changes. BimlFlex will generate specific code to query the object.	Source System	Microsoft SQL Server
CDC Last	TCL	Determines that the object is configured for Change Data Capture, but only retrieves the last value every time data is loaded. BimlFlex will generate specific code to query the object.	Source System	Microsoft SQL Server
Change Tracking	TCT	Determines that the object is configured for Change Tracking. BimlFlex will generate specific code to query the object.	Source System	Microsoft SQL Server

Details Tab - SQL Overrides Fields

The majority of the SQL Overrides are provided to handle the various scenarios where overrides to the default SQL generated out of the box by BimlFlex. These fields are to be consider more advanced than the standard use cases and should only be used by a DBA or Architect that is familiar with the SQL required should use the SQL properties. The ability to inherit Columns or other properties are also provide along with the ability to annotate a dependant on another Object.

Field	Description
From SQL	Optional alternative FROM clause to be used during generation of the SQL statement.
Object Alias	Defines an alias (short name) for the Object. This is relevant in conjunction with the JOIN and/or Where SQL properties.
Join SQL	Optional additional JOIN clause(s) to be used during generation of the SQL statement. Commonly used when data needed to generate a complete Integration Key requires the use of a Foreign Key or additional relationship.
Where SQL	Optional additional WHERE clause to be used during generation of the SQL statement. Used to generate an filter that otherwise could not be generated by the use of a standard parameter.
Override SQL	Optional alternative SQL statement to be used. This needs to be an entire SQL statement and it will ignore any dynamically generated logic by BimlFlex. This should only be used as a last resort and only if there is no way to dynamically generate the statement required.
Select By SQL	Optional DISTINCT or TOP N SELECT clause. This field is limited to 1000 characters.
Group By SQL	Optional GROUP BY clause to be used during generation of the SQL statement. Used when the source Object requires a condensing of grain. Prior to using this clause, ensure that the generated SQL statement will not generate any unwanted results and that the grain can actually be condensed with the designed statement with issue.
Order By SQL	Optional ORDER BY clause to be used during generation of the SQL statement.
Create SQL	Optional alternative SQL statement to be used for the creation of the database artifact. As with [Override SQL] this is should be the complete DDL statement and should only be used if the result can not be generated dynamically.
Source Create SQL	Optional alternative SQL statement to be used for the creation of the database artifact on it's source system. As with Override SQL this is should be the complete DDL statement and should only be used if the result can not be generated dynamically.
Depends On Object	When an Object is selected, ensures that the linked Object must be processed prior to the processing of the current Object. This is only used for Objects that can not have their dependencies generated dynamically. BimlFlex maintains and enforces dependency trees for Objects generated through the tool.
Inherit From Object	When an Object is selected, uses the columns from the entered Object, similar to a Roleplaying Dimension. Used when the same table is used to process multiple business concepts. Only additional Columns unique to the current Object will show on the Columns list. Inherited Columns will only show in the Inherit From Object.
Same As Inherited	Defines if the Inherit From Object is the same Object. When set to true this will instruct BimlFLex to reuse the inherited tables for loading.
Use Inherited Name	Defines if the Inherit From Object has the same Object. When set to true this will instruct BimlFlex to reuse the inherited tables Object Name for loading.

[//]: # (TODO: Add an example column for the required/expected syntax for each field.)

Columns Tab

The Columns Tab provides quick access to all Columns included in the Object.

Action Buttons

Icon	Action	Description
	Add	Add will open an Add Column Dialog for creating a new Column and assign it to the current entity. When adding an Column via this button Connection and Object will be pre-filled on the created dialog box.
	Save	This will save the currently set of staged changes. The Save button is only enabled if the Column has changes staged and there are no major validation issues with the current Column properties.
	Archive	This will hard delete the selected Attribute. This will result in the physical removal of the selected record from the metadata database. The data will no longer be accessible by the BimlFlex Applicationand will require a Database Administrator to restore.	Archive Column
	Refresh	This will trigger a refresh of the metadata for the selected Columns.
	Integration Key	This will bring up the dialog to create an Integration Key.	Generate Derived Integration Key

Additional Dialogs

Add Column Dialog

Creates a Column and associates it with the current entity. Once entry is finished ensure that the Save button is clicked.

Archive Column Dialog

Confirmation box warning against the dangers of archiving. You are required to confirm by both the check box and the Ok button.

Warning

Archiving is a permanent removal of the selected entity from its associated table in the BimlFlex Database. The best practice is to first use the Deleted flag (soft delete) as an indication that the connection may need to be removed.

Generate Derived Integration Key Dialog

The Generate Derived Integration Key dialog will allow you to quickly generate an Integration Key. The selected Columns from the prior screen should already be selected and a Column Name is automatically populated based on the associated Objects Model Override Name or, if that is empty, the Object Name. The "_BK" suffix can be controlled with the Setting AppendIntegrationKey.

Generate Derived Integration Key Options

Option	Description
Add Record Source	This will prefix the @@rs to the Column Expression. @@rs will generate the value currently assigned to Record Source on the associated Connection.
Is Integration Key	This will flag the generated Column to become the Integration Key for the associated Object. Once selected it will automatically update Column Ordinal to "0" and also Is Primary Key to true.
Is Primary Key	This will flag the generated Column as a Primary Key for the associated Object.

Columns Tab Views

All Columns assigned to the current entity show here. New items can be entered entered via the Add action button. Existing items can be edited by double clicking the field needing adjustment. All the views have the same links to the Column.

Navigational Transitions

Item	Action	Entity Documentation
Column Value	Navigate to Column Editor, selecting clicked Column	Column Documentation

Overview

The Overview is available to give quick access to the more physical properties of the Columns relating to the selected entity. These are representations of the physical Source System and represent the raw definitions of the object in it's original form. Quick access is also provided for the various keys used across the separate Integration Layers or Integration Stages.

Settings

The Settings view allows for quick access to all the Boolean and flag properties associated with the Columns. In addition to the keys the ability to flag database concepts (such as IDENTITY or NULL), persistence, whether or not the Column is Derived, and the ability to flag as Excluded or Deleted.

Expressions

The Expressions view allows you to quickly view and manage all the expressions that may be used in the CAST, CONVERT, or general transformations that may be performed on any given Column. All values in this view are used to determine the ETL/ELT logic and transformation dataflow.

Mappings

The Mappings view shows the references and targets associated with the Columns. References are the equivalent of a Foreign Key in a Relational Database, though the enforcement of an actual constraint is configurable. References control how and where other tables relate to each other. Targets are the the Object and Column that the entity will populate in the associated Project's Target Connection. It is recommended that you use the Data Vault Accelerator or the Schema Diagram to populate these fields. This view is intended as easy way to reference and maintain previously entered metadata.

Overrides

The Overrides view is available to allow for quick management of modeled metadata from the Data Vault Accelerator or the Schema Diagram. This view is intended as easy way to reference and maintain previously entered metadata.

Attributes Tab

The Attributes Tab provides a view of any Configurations or Settings overrides that have been applied to the directly to the selected Object.

Note

This is exclusive to the Object level. Additional overrides may be present on any grains higher or lower than the Object.

Action Buttons

All Attributes assigned to the current entity show here. New items can be entered via the Add action button. Existing items can be edited by clicking the Value field to select the Attribute and using the Edit action button. Various navigational transitions are available on this form and outlined below.

The following options are available:

Icon	Action	Description
	Add	Add will create a new Attribute and assign it to the current entity. When adding an attribute via this button, Attribute Type and the respective linked fields, will be pre-filled on the created Add Attribute Dialog.
	Save	Save will save the currently set of staged changes. The Save button is only enabled if any Attribute has changes staged and there are no major validation issues with the current list of Attribute properties.
	Discard	This will Discard any unsaved changes and revert to last saved form.

When checking a single Attribute in the overview two additional options will become visible. When selecting two or more attributes only the Archive option will be visible.

Icon	Action	Description
	Archive	Archive will hard delete the selected Attribute. This will result in the physical removal of the selected record from the BimlFlex database. The data will no longer be accessible by the BimlFlex Applicationand will require a Database Administrator to restore. Clicking Archive will create an Archive Attribute Dialog.
	Edit	Edit will open an Edit Attribute Dialog to edit the currently selected Attribute. After edits are complete, be sure to click the Save button to confirm the changes.

Additional Dialogs

Add Attribute Dialog

Creates an Attribute and associates it with the current entity. Once entry is finished ensure that the Save button is clicked.

Edit Attribute Dialog

Edit the selected Attribute. Once entry is finished ensure that the Save button is clicked.

Archive Attribute Dialog

Confirmation warning against the dangers of archiving. You are required to confirm by both the check box and the Ok button.

Warning

Archiving is a permanent removal of the selected entity from its associated table in the BimlFlex Database. The best practice is to first use the Deleted flag (soft delete) as an indication that the connection may need to be removed.

Attribute Tab View

The Attribute Tab provides a different way to review the properties of Attributes associated with the selected entity.

Overview

The following details are shown in this view:

Component	Type	Description
Object	Value	The name of the Objects as associated with the current Connection. See the Object Editor documentation for more details.
Column	Navigational Value	Define the scope of the Attribute. See the Attribute Editor for more details.
Attribute	Navigational Value	Define the scope of the BimlFlex process that the Attribute is applied to. Attribute Type is required and must be a valid Attribute Type.
Value	Value	The value to be returned when the custom attribute is accessed by the framework. See the Attribute Editor for more details.
Property	Field	Define the value for a custom attribute if it is not specified in the Attribute Value field. See the Attribute Editor for more details.

Navigational Transitions

From the Attribute Tab, it is possible to navigate directly to the following areas of BimlFlex:

Item	Action
Object Value	Navigate to the Object Editor, by clicking <img class="icon-inline" src="../../static/svg/navigate.svg style="width: 18px"/>
Column Value	Navigate to Column Editor, selecting clicked Column
Attribute Value	Navigate to Attribute Editor, selecting clicked Attribute

Parameters Tab

The Parameters Tab provides a view of any Parameters that have been applied to the directly to the selected Object.

Note

This is exclusive to the Object level. Additional overrides may be present on any grains higher or lower than the Object.

Action Buttons

All Parameters assigned to the current entity show here. New items can be entered entered via the Edit action button. Various navigational transitions are available on this form and outlined below.

Icon	Action	Description
	Add	Add will create a new Parameter and assign it to the current entity. When adding an attribute via this button Attribute Type, along with the respective linked fields, will be pre-filled on the Add Parameter Dialog.
	Save	Save will save the currently set of staged changes. The Save button is only enabled if any Object has changes staged and there are no major validation issues with the current list of Object properties.
	Discard	This will Discard any unsaved changes and revert to last saved form.

When checking a single Parameter in the overview two additional options will become visible. When selecting two or more parameters only the Archive option will be visible.

Icon	Action	Description
	Archive	Archive will hard delete the selected Parameter. This will result in the physical removal of the selected record from the BimlFlex database. The data will no longer be accessible by the BimlFlex Applicationand will require a Database Administrator to restore. Clicking Archive will create an Archive Parameter Dialog.
	Edit	Edit will open an Edit Parameter Dialog to edit the currently selected Parameter. After edits are complete, be sure to click the Save button to confirm the changes.

Additional Dialogs

Add Parameter Dialog

Creates a Parameter and associates it with the current entity. Once entry is finished ensure that the Save button is clicked.

Edit Parameter Dialog

Edit the selected Parameter. Once entry is finished ensure that the Save button is clicked.

Archive Parameter Dialog

Confirmation warning against the dangers of archiving. You are required to confirm by both the check box and the Ok button.

Warning

Archiving is a permanent removal of the selected entity from its associated table in the BimlFlex Database. The best practice is to first use the Deleted flag (soft delete) as an indication that the connection may need to be removed.

Parameters Tab View

The Parameters Tab View provides a different way to review the properties of the Parameters associated with the selected entity.

Overview

Parameters Tab Table Components

Component	Type	Description
Object	Value	The name of the Object within the current Source Connection. See Objects Documentation.
Column	Value	The name of the Column within the current entity. See Columns Documentation.
Parameter	Navigational Value	The name of the Parameter that is being applied to the selected entity. See Parameters Documentation.
Operator	Value	Operator to use in the Parameter when applied to the source Column.
Default	Date	The parameter load value to use for the first load.
Data Type	Value	The data type to use for the Parameter. Must be a valid data type. See Valid Data Types.

Navigational Transitions

From the Parameters Tab, it is possible to navigate directly to the following areas of BimlFlex:

Item	Action
Object	Navigate to Object Editor, by clicking <img class="icon-inline" src="../../static/svg/navigate.svg style="width: 18px"/>
Column	Navigate to Column Editor, by clicking <img class="icon-inline" src="../../static/svg/navigate.svg style="width: 18px"/>
Name	Navigate to Parameter Editor, selecting the clicked Parameter