Invocation Modes#

Added in version 1.4.

For ExecutionMode.LOCAL execution mode, Cosmos supports two invocation modes for running dbt:

  1. InvocationMode.SUBPROCESS: In this mode, Cosmos runs dbt cli commands using the Python subprocess module and parses the output to capture logs and to raise exceptions.

  2. InvocationMode.DBT_RUNNER: In this mode, Cosmos uses the dbtRunner available for dbt programmatic invocations to run dbt commands. In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than InvocationMode.SUBPROCESS. This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve Airflow and dbt dependencies conflicts when using this mode.

The invocation mode can be set in the ExecutionConfig as shown below:

from cosmos.constants import InvocationMode

dag = DbtDag(
    # ...
    execution_config=ExecutionConfig(
        execution_mode=ExecutionMode.LOCAL,
        invocation_mode=InvocationMode.DBT_RUNNER,
    ),
)

If the invocation mode is not set, Cosmos will attempt to use InvocationMode.DBT_RUNNER if dbt is installed in the same environment as the worker, otherwise it will fall back to InvocationMode.SUBPROCESS.

This Page