.. This file is autogenerated by `docs/scripts/generate_mappings.py`. Do not edit by hand. .. toctree:: :caption: Profiles self AthenaAccessKey GoogleCloudServiceAccountFile GoogleCloudServiceAccountDict GoogleCloudOauth DatabricksToken PostgresUserPassword RedshiftUserPassword SnowflakeUserPassword SnowflakeEncryptedPrivateKeyFilePem SnowflakeEncryptedPrivateKeyPem SnowflakePrivateKeyPem SparkThrift ExasolUserPassword TrinoLDAP TrinoCertificate TrinoJWT VerticaUserPassword Profiles Overview ========================== Cosmos supports two methods of authenticating with your database: - using your own dbt profiles.yml file - using Airflow connections via Cosmos' profile mappings If you're already interacting with your database from Airflow and have a connection set up, it's recommended to use a profile mapping to translate that Airflow connection to a dbt profile. This is because it's easier to maintain a single connection object in Airflow than it is to maintain a connection object in Airflow and a dbt profile in your dbt project. If you don't already have an Airflow connection, or if there's no readily-available profile mapping for your database, you can use your own dbt profiles.yml file. Regardless of which method you use, you'll need to tell Cosmos which profile and target name it should use. Profile config is set in the ``cosmos.config.ProfileConfig`` object, like so: .. code-block:: python from cosmos.config import ProfileConfig profile_config = ProfileConfig( profile_name="my_profile_name", target_name="my_target_name", # choose one of the following profile_mapping=..., profiles_yml_filepath=..., ) dag = DbtDag(profile_config=profile_config, ...) Using a profile mapping ++++++++++++++++++++++++++++++++++++ Profile mappings are utilities provided by Cosmos that translate Airflow connections to dbt profiles. This means that you can use the same connection objects you use in Airflow to authenticate with your database in dbt. To do so, there's a class in Cosmos for each Airflow connection to dbt profile mapping. You can find the available profile mappings on the left-hand side of this page. Each profile mapping is imported from ``cosmos.profiles`` and takes two arguments: * ``conn_id``: the Airflow connection ID to use. * ``profile_args``: a dictionary of additional arguments to pass to the dbt profile. This is useful for specifying values that are not in the Airflow connection. This also acts as an override for any values that are in the Airflow connection but should be overridden. Below is an example of using the Snowflake profile mapping, where we take most arguments from the Airflow connection but override the ``database`` and ``schema`` values: .. code-block:: python from cosmos.profiles import SnowflakeUserPasswordProfileMapping profile_config = ProfileConfig( profile_name="my_profile_name", target_name="my_target_name", profile_mapping=SnowflakeUserPasswordProfileMapping( conn_id="my_snowflake_conn_id", profile_args={ "database": "my_snowflake_database", "schema": "my_snowflake_schema", }, ), ) dag = DbtDag(profile_config=profile_config, ...) Note that when using a profile mapping, the profiles.yml file gets generated with the profile name and target name you specify in ``ProfileConfig``. Disabling dbt event tracking -------------------------------- .. note: Deprecated in v.1.4 and will be removed in v2.0.0. Use dbt_config_vars=DbtProfileConfigVars(send_anonymous_usage_stats=False) instead. .. versionadded:: 1.3 By default `dbt will track events `_ by sending anonymous usage data when dbt commands are invoked. Users have an option to opt out of event tracking by updating their ``profiles.yml`` file. If you'd like to disable this behavior in the Cosmos generated profile, you can pass ``disable_event_tracking=True`` to the profile mapping like in the example below: .. code-block:: python from cosmos.profiles import SnowflakeUserPasswordProfileMapping profile_config = ProfileConfig( profile_name="my_profile_name", target_name="my_target_name", profile_mapping=SnowflakeUserPasswordProfileMapping( conn_id="my_snowflake_conn_id", profile_args={ "database": "my_snowflake_database", "schema": "my_snowflake_schema", }, disable_event_tracking=True, ), ) dag = DbtDag(profile_config=profile_config, ...) Dbt profile config variables -------------------------------- .. versionadded:: 1.4.0 The parts of ``profiles.yml``, which aren't specific to a particular data platform `dbt docs `_ .. code-block:: python from cosmos.profiles import SnowflakeUserPasswordProfileMapping, DbtProfileConfigVars profile_config = ProfileConfig( profile_name="my_profile_name", target_name="my_target_name", profile_mapping=SnowflakeUserPasswordProfileMapping( conn_id="my_snowflake_conn_id", profile_args={ "database": "my_snowflake_database", "schema": "my_snowflake_schema", }, dbt_config_vars=DbtProfileConfigVars( send_anonymous_usage_stats=False, partial_parse=True, use_experimental_parse=True, static_parser=True, printer_width=120, write_json=True, warn_error=True, warn_error_options={"include": "all"}, log_format='text', debug=True, version_check=True, ), ), ) dag = DbtDag(profile_config=profile_config, ...) Using your own profiles.yml file ++++++++++++++++++++++++++++++++++++ If you don't want to use Airflow connections, or if there's no readily-available profile mapping for your database, you can use your own dbt profiles.yml file. To do so, you'll need to pass the path to your profiles.yml file to the ``profiles_yml_filepath`` argument in ``ProfileConfig``. For example, the code snippet below points Cosmos at a ``profiles.yml`` file and instructs Cosmos to use the ``my_snowflake_profile`` profile and ``dev`` target: .. code-block:: python from cosmos.config import ProfileConfig profile_config = ProfileConfig( profile_name="my_snowflake_profile", target_name="dev", profiles_yml_filepath="/path/to/profiles.yml", ) dag = DbtDag(profile_config=profile_config, ...)