Project config#

The cosmos.config.ProjectConfig allows you to specify information about where your dbt project is located and project variables that should be used for rendering and execution. It takes the following arguments:

  • dbt_project_path: The full path to your dbt project. This directory should have a dbt_project.yml file. On Airflow 3 (for example, Astronomer deployments) DAGs run from a versioned DAG bundle whose filesystem location can differ between DAG parsing and task execution, so a hardcoded absolute path may not exist on the worker. Derive the path from your DAG file instead, e.g. (Path(__file__).parent / "dbt/my_dbt_project").absolute().as_posix(). See Getting Started with Cosmos on Astro.

  • models_relative_paths: (new in v1.16) The path(s) to your models directories, relative to the dbt_project_path. Accepts a single path or a list of paths, mirroring dbt’s own support for multiple model-paths. This defaults to ["models"]. Replaces models_relative_path, deprecated since v1.16 and will be removed in Cosmos 2.0.

  • seeds_relative_paths: (new in v1.16) The path(s) to your seeds directories, relative to the dbt_project_path. Accepts a single path or a list of paths. This defaults to ["seeds"]. Replaces seeds_relative_path, deprecated since v1.16 and will be removed in Cosmos 2.0.

  • snapshots_relative_paths: (new in v1.16) The path(s) to your snapshots directories, relative to the dbt_project_path. Accepts a single path or a list of paths. This defaults to ["snapshots"]. Replaces snapshots_relative_path, deprecated since v1.16 and will be removed in Cosmos 2.0.

  • manifest_path: The absolute path to your manifests directory. This is only required if you’re using Cosmos’ manifest parsing mode. Cosmos supports both local and remote paths for manifest parsing (e.g. S3 URL). See Parsing Methods for more details.

  • project_name : The name of the project. If dbt_project_path is provided, the project_name defaults to the folder name containing dbt_project.yml. If dbt_project_path is not provided, and manifest_path is provided, project_name is required as the name can not be inferred from dbt_project_path

  • dbt_vars: (new in v1.3) A dictionary of dbt variables for the project rendering and execution. This argument overrides variables defined in the dbt_project.yml file. The dictionary of variables is dumped to a yaml string and passed to dbt commands as the –vars argument. Variables are only supported for rendering when using RenderConfig.LoadMode.DBT_LS and RenderConfig.LoadMode.CUSTOM load mode. Variables using Airflow templating will only be rendered at execution time, not at render time.

  • env_vars: (new in v1.3) A dictionary of environment variables used for rendering and execution. Rendering with env vars is only supported when using RenderConfig.LoadMode.DBT_LS load mode.

  • install_dbt_deps: (new in v1.9) Run dbt deps during DAG parsing and task execution if True (default). To control dbt deps per DAG run via an Airflow template, set ExecutionConfig.install_dbt_deps instead, which overrides this value at task execution time only.

  • copy_dbt_packages: (new in v1.10) Copy the dbt project dbt_packages instead of creating symbolic links, so Cosmos can run dbt deps incrementally.

  • partial_parse: (new in v1.4) If True, then attempt to use the partial_parse.msgpack if it exists. This is only used for the LoadMode.DBT_LS load mode, and for the ExecutionMode.LOCAL and ExecutionMode.VIRTUALENV execution modes. Due to the way that dbt partial parsing works, it does not work with Cosmos profile mapping classes. To benefit from this feature, users have to set the profiles_yml_filepath argument in ProfileConfig.

Project config example#

from cosmos.config import ProjectConfig

config = ProjectConfig(
    dbt_project_path="/path/to/dbt/project",
    models_relative_paths=["custom_models_folder", "other_models_folder"],
    seeds_relative_paths=["custom_seeds_folder"],
    snapshots_relative_paths=["custom_snapshots_folder"],
    manifest_path="/path/to/manifests",
    env_vars={"MY_ENV_VAR": "my_env_value"},
    dbt_vars={
        "my_dbt_var": "my_value",
        "start_time": "{{ data_interval_start.strftime('%Y%m%d%H%M%S') }}",
        "end_time": "{{ data_interval_end.strftime('%Y%m%d%H%M%S') }}",
    },
)