Silver Layer - Data Integration
The Silver layer contains cleansed, validated, and integrated data. BimlFlex supports two modeling approaches for implementing the Silver layer, with Data Vault being the recommended approach for enterprise scenarios.
Silver Layer Characteristics
- Cleansed: Data quality rules applied
- Conformed: Standardized formats and values
- Integrated: Data from multiple sources unified
- Historized: Changes tracked over time
BimlFlex Implementation Options
Option 1: Data Vault (Recommended)
Data Vault provides maximum flexibility and is the recommended approach for complex enterprise scenarios:
| Component | Purpose | Description |
|---|---|---|
| Hubs | Core Business Concepts | Unique business keys (Customer, Product, etc.) |
| Links | Relationships | Associations between Hubs |
| Satellites | Context/Attributes | Descriptive data with full history |
Benefits of Data Vault for Silver Layer:
- Flexibility: Easily accommodate new sources and requirements
- Auditability: Complete history of all changes
- Scalability: Parallel loading without dependencies
- Agility: Rapid integration of new data sources
tip
For comprehensive Data Vault guidance, see the Delivering Data Vault documentation.
Option 2: Normal Form
For simpler use cases, traditional Normal Form (3NF) modeling is supported:
- Familiar relational patterns
- Reduced complexity for straightforward integrations
- Suitable when Data Vault benefits aren't required
Business Data Vault
For additional transformation beyond the Raw Data Vault, BimlFlex supports Business Data Vault patterns:
- Business rules applied to create derived entities
- Calculated attributes and soft business rules
- Bridge and Point-in-Time (PIT) tables for query optimization
See: Business Data Vault
Platform-Specific Considerations
Microsoft Fabric
- Silver tables as managed Delta tables
- Unity with Fabric Data Warehouse for T-SQL access
- Direct Query from Power BI where appropriate
Databricks
- Unity Catalog managed tables
- Delta Lake for ACID transactions
- SQL or notebook-based transformations
Snowflake
- Standard Snowflake tables
- Time Travel for historical queries
- Dynamic Data Masking for governance