GoogleCloudOauth#

Maps Airflow GCP connections to dbt BigQuery profiles that uses oauth via gcloud,

if they don’t use key file or JSON.

https://docs.getdbt.com/docs/core/connect-data-platform/bigquery-setup#oauth-via-gcloud

https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp.html

This profile mapping translates Airflow connections with the type google_cloud_platform into dbt profiles. To use this profile, import it from cosmos.profiles:

from cosmos.profiles import GoogleCloudOauthProfileMapping

profile = GoogleCloudOauthProfileMapping(
    conn_id = 'my_google_cloud_platform_connection',
    profile_args = { ... },
)

While the profile mapping pulls fields from Airflow connections, you may need to supplement it with additional profile_args. The below table shows which fields are required, along with those not required but pulled from the Airflow connection if present. You can also add additional fields to the profile_args dict.

dbt Field Name

Required

Airflow Field Name

project

True

extra.project

dataset

True

extra.dataset

Some notes about the table above:

  • This table doesn’t necessarily show the full list of fields you can pass to the dbt profile. To see the full list of fields, see the link to the dbt docs at the top of this page.

  • If the Airflow field name starts with an extra., this means that the field is nested under the extra field in the Airflow connection. For example, if the Airflow field name is extra.token, this means that the field is nested under extra in the Airflow connection, and the field name is token.

  • If there are multiple Airflow field names, the profile mapping looks at those fields in order. For example, if the Airflow field name is ['password', 'extra.token'], the profile mapping will first look for a field named password. If that field is not present, it will look for extra.token.

This Page