dbt: Error handling in custom materializations for dbt


🚨 Proper Error Handling in dbt Custom Materializations with raise_database_error

In dbt, when writing custom materializations, it’s important to handle errors gracefully and consistently. Instead of allowing Python or Jinja errors to crash silently or give cryptic messages, dbt provides a native mechanism:

βœ… raise_database_error(msg)

This built-in dbt function:

  • Halts execution with a clear, user-defined error,
  • Logs it through dbt’s native system,
  • Maintains a full stack trace,
  • Makes your custom materialization easier to debug and maintain.

🧱 Example: Custom Materialization with Error Handling

Let’s create a custom materialization called safe_table that:

  • Builds a table normally,
  • But raises a helpful error if the unique_key config is missing.

πŸ“ File: macros/materializations/safe_table.sql

{% materialization safe_table, default %}

  {% set target_relation = api.Relation.create(
      identifier=model['alias'],
      schema=model['schema'],
      database=model['database']
  ) %}

  {% set unique_key = config.get('unique_key') %}

  {% if not unique_key %}
    {{ exceptions.raise_database_error(
      "The 'unique_key' config is required for the 'safe_table' materialization."
    ) }}
  {% endif %}

  -- If no error, proceed with standard table creation
  {% do run_query("drop table if exists " ~ target_relation) %}

  {% do run_query("create table " ~ target_relation ~ " as (select * from " ~ this) %}

  {{ return({'relations': [target_relation]}) }}

{% endmaterialization %}

βœ… How It Works

  • If the user forgets to set unique_key in their model: {{ config(materialized='safe_table') }} dbt will raise: Database Error in model my_model (models/my_model.sql) The 'unique_key' config is required for the 'safe_table' materialization.
  • This error:
    • Is clean and specific,
    • Shows up in dbt run output and logs,
    • Makes it easy to fix the issue.

🎯 Why Use raise_database_error()?

FeatureBenefit
βœ… Integrated stack traceBetter debugging in CI and local runs
βœ… Clean log outputShows up in dbt Cloud and CLI logs
βœ… Stops execution safelyAvoids partial state or silent failures
βœ… Easy to testTrigger and assert error conditions in CI

πŸš€ Bonus Tip: Use raise_compiler_error() for compile-time issues

  • Use raise_database_error() for runtime errors (e.g., SQL execution problems).
  • Use raise_compiler_error() for compile-time issues (e.g., bad configs, missing variables).

Leave a Reply

Your email address will not be published. Required fields are marked *