Glossary#
- Cosmos#
An open-source Python package that allows you to write data transformations using dbt and then use Apache Airflow®’s orchestration to integrate dbt projects into end-to-end workflows.
- DAG#
An Airflow term derived from the mathematical structure called Directed Acyclic Graph. A DAG provides a model that includes everything needed to execute an Airflow workflow. See DAG.
- DbtDag#
A Cosmos class that wraps a full Airflow DAG around a dbt project, rendering each dbt node as an Airflow task. Use
DbtDagwhen the entire Airflow DAG is dedicated to running a dbt project. See Cosmos core concepts.- DbtTaskGroup#
A Cosmos class that embeds a dbt project as an Airflow
TaskGroupinside a larger DAG. UseDbtTaskGroupwhen dbt is one stage of a broader pipeline (e.g. ingestion → dbt → reporting). See Cosmos core concepts.- dbt deps#
A dbt command that pulls the most recent version of dependencies listed in your
packages.ymlfile.- Callbacks#
User-defined functions that Cosmos invokes after the dbt command completes, during post-execution handling and before the temporary project directory is cleaned up. Useful for custom logging, alerting, or side effects without modifying the operator. Configure them on Cosmos operators via
callbackandcallback_args; when usingDbtDagorDbtTaskGroup, these can be passed throughoperator_args. See Callbacks.- ExecutionConfig#
The Cosmos class used to configure the execution mode and related runtime options such as the dbt executable path and invocation mode. See Execution Config.
- ExecutionMode#
Describes where and how Cosmos runs dbt commands (e.g.
LOCAL,WATCHER,DOCKER). Configured viaExecutionConfig. See Choose an execution mode.- Interceptors#
An optional list of callables (new in Cosmos 1.14) that run before Cosmos builds the dbt command for each task. Each callable receives
(context, operator)and may modifyoperator.varsandoperator.env; the modified values are then used when building and running the dbt command. Useful for injecting runtime variables or environment values per task run. Configured viaoperator_args={"interceptors": [...]}onDbtDagorDbtTaskGroup. See Operator arguments.- InvocationMode#
Controls how Cosmos calls dbt:
DBT_RUNNERimports dbt as a Python library in the same process (no subprocess overhead, lower CPU and memory usage), whileSUBPROCESSspawns a separate process (better isolation when dbt and Airflow share a Python environment). Running as a subprocess roughly doubles memory usage compared toDBT_RUNNER. See Invocation modes.- LoadMode#
The method Cosmos uses to parse your dbt project (e.g.
DBT_MANIFEST,DBT_LS). Configured viaRenderConfig. See Parsing Methods.- Manifest#
A dbt artifact that contains a full representation of the dbt project’s resources, including all node configurations and resource properties. See Manifest JSON file in the dbt docs.
- Partial parsing#
A dbt feature, enabled in Cosmos since v1.4, that skips re-parsing files that have not changed. Each Airflow node performs a full dbt project parse only once; subsequent task runs reuse the cached
partial_parse.msgpackartifact, reducing parse time and CPU usage. See Partial parsing.- Profile#
The authentication information used by dbt to connect to your data warehouse. See profile.
- ProfileConfig#
The Cosmos class that determines which data warehouse Cosmos connects to when executing dbt SQL. See Connect to your database.
- Profile mapping#
A Cosmos-provided resource that translates Airflow connections into dbt profiles. See Use a profile mapping.
- profiles.yml#
The file where dbt stores connection information for each data warehouse connection. See Connect to your database, Use your own profiles.yml file, or The profiles.yml file in the dbt docs.
- ProjectConfig#
The Cosmos class used to specify where your dbt project is located and any project variables that should be used for rendering and execution.
- node_converters#
A
RenderConfigoption that accepts a dictionary mapping aDbtResourceTypeto a callable, allowing users to replace how specific node types are rendered as Airflow tasks. See Render Config.- RenderConfig#
The Cosmos class that controls how a dbt project is turned into an Airflow DAG or TaskGroup, including node selection, test behaviour, and source rendering. See Render Config.
- SourceRenderingBehavior#
Controls whether dbt source nodes are rendered as Airflow tasks.
NONE(default) skips source nodes entirely;ALLrenders every source;WITH_TESTS_OR_FRESHNESSrenders only sources that have tests or a freshness check defined. Configured viaRenderConfig.- TestBehavior#
Controls when and how Cosmos runs dbt tests.
AFTER_EACH(default) adds a test task after each model;AFTER_ALLruns all tests in a single task at the end;BUILDruns tests as part ofdbt buildinside the model task;NONEskips tests entirely. Configured viaRenderConfig. See Testing behavior.