dbt

dbt for transforming data in the warehouse with SQL.

Core concepts

• Models — SQL that produces a table or view
• Tests — Uniqueness, not null, referential integrity, custom
• Seeds — CSV files loaded as tables
• Sources — Raw tables; freshness and lineage
• Incremental models — Only process new/changed rows

Project structure

models/
  staging/     # Light cleaning, naming
  intermediate/# Joins, prep for marts
  marts/       # Business-ready tables

Best practices

• Staging — One model per source table; standardize names and types
• Incremental — Use unique_key and incremental_strategy
• Refs — Use ref() for dependencies; never hardcode schema/table
• Docs — Add descriptions; dbt docs generate for lineage