airflow.providers.dbt.cloud.hooks.dbt

Module Contents

Classes

TokenAuth

Helper class for Auth when executing requests.

JobRunInfo

Type class for the job_run_info dictionary.

DbtCloudJobRunStatus

dbt Cloud Job statuses.

DbtCloudHook

Interact with dbt Cloud using the V2 API.

Functions

fallback_to_default_account(func)

Decorator which provides a fallback value for account_id. If the account_id is None or not passed

airflow.providers.dbt.cloud.hooks.dbt.fallback_to_default_account(func)[source]

Decorator which provides a fallback value for account_id. If the account_id is None or not passed to the decorated function, the value will be taken from the configured dbt Cloud Airflow Connection.

class airflow.providers.dbt.cloud.hooks.dbt.TokenAuth(token)[source]

Bases: requests.auth.AuthBase

Helper class for Auth when executing requests.

__call__(self, request)[source]
class airflow.providers.dbt.cloud.hooks.dbt.JobRunInfo[source]

Bases: airflow.typing_compat.TypedDict

Type class for the job_run_info dictionary.

account_id :int[source]
run_id :int[source]
class airflow.providers.dbt.cloud.hooks.dbt.DbtCloudJobRunStatus[source]

Bases: enum.Enum

dbt Cloud Job statuses.

QUEUED = 1[source]
STARTING = 2[source]
RUNNING = 3[source]
SUCCESS = 10[source]
ERROR = 20[source]
CANCELLED = 30[source]
TERMINAL_STATUSES[source]
classmethod check_is_valid(cls, statuses)[source]

Validates input statuses are a known value.

classmethod is_terminal(cls, status)[source]

Checks if the input status is that of a terminal type.

exception airflow.providers.dbt.cloud.hooks.dbt.DbtCloudJobRunException[source]

Bases: airflow.exceptions.AirflowException

An exception that indicates a job run failed to complete.

class airflow.providers.dbt.cloud.hooks.dbt.DbtCloudHook(dbt_cloud_conn_id=default_conn_name, *args, **kwargs)[source]

Bases: airflow.providers.http.hooks.http.HttpHook

Interact with dbt Cloud using the V2 API.

Parameters

dbt_cloud_conn_id (str) -- The ID of the dbt Cloud connection.

conn_name_attr = dbt_cloud_conn_id[source]
default_conn_name = dbt_cloud_default[source]
conn_type = dbt_cloud[source]
hook_name = dbt Cloud[source]
static get_ui_field_behaviour()[source]

Builds custom field behavior for the dbt Cloud connection form in the Airflow UI.

connection(self)[source]
get_conn(self, *args, **kwargs)[source]

Returns http session for use with requests

Parameters

headers -- additional headers to be passed through as a dictionary

list_accounts(self)[source]

Retrieves all of the dbt Cloud accounts the configured API token is authorized to access.

Returns

List of request responses.

Return type

List[requests.models.Response]

get_account(self, account_id=None)[source]

Retrieves metadata for a specific dbt Cloud account.

Parameters

account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

Returns

The request response.

Return type

requests.models.Response

list_projects(self, account_id=None)[source]

Retrieves metadata for all projects tied to a specified dbt Cloud account.

Parameters

account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

Returns

List of request responses.

Return type

List[requests.models.Response]

get_project(self, project_id, account_id=None)[source]

Retrieves metadata for a specific project.

Parameters
  • project_id (int) -- The ID of a dbt Cloud project.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

Returns

The request response.

Return type

requests.models.Response

list_jobs(self, account_id=None, order_by=None, project_id=None)[source]

Retrieves metadata for all jobs tied to a specified dbt Cloud account. If a project_id is supplied, only jobs pertaining to this job will be retrieved.

Parameters
  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • order_by (Optional[str]) -- Optional. Field to order the result by. Use '-' to indicate reverse order. For example, to use reverse order by the run ID use order_by=-id.

  • project_id (Optional[int]) -- The ID of a dbt Cloud project.

Returns

List of request responses.

Return type

List[requests.models.Response]

get_job(self, job_id, account_id=None)[source]

Retrieves metadata for a specific job.

Parameters
  • job_id (int) -- The ID of a dbt Cloud job.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

Returns

The request response.

Return type

requests.models.Response

trigger_job_run(self, job_id, cause, account_id=None, steps_override=None, schema_override=None, additional_run_config=None)[source]

Triggers a run of a dbt Cloud job.

Parameters
  • job_id (int) -- The ID of a dbt Cloud job.

  • cause (str) -- Description of the reason to trigger the job.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • steps_override (Optional[List[str]]) -- Optional. List of dbt commands to execute when triggering the job instead of those configured in dbt Cloud.

  • schema_override (Optional[str]) -- Optional. Override the destination schema in the configured target for this job.

  • additional_run_config (Optional[Dict[str, Any]]) -- Optional. Any additional parameters that should be included in the API request when triggering the job.

Returns

The request response.

Return type

requests.models.Response

list_job_runs(self, account_id=None, include_related=None, job_definition_id=None, order_by=None)[source]

Retrieves metadata for all of the dbt Cloud job runs for an account. If a job_definition_id is supplied, only metadata for runs of that specific job are pulled.

Parameters
  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • include_related (Optional[List[str]]) -- Optional. List of related fields to pull with the run. Valid values are "trigger", "job", "repository", and "environment".

  • job_definition_id (Optional[int]) -- Optional. The dbt Cloud job ID to retrieve run metadata.

  • order_by (Optional[str]) -- Optional. Field to order the result by. Use '-' to indicate reverse order. For example, to use reverse order by the run ID use order_by=-id.

Returns

List of request responses.

Return type

List[requests.models.Response]

get_job_run(self, run_id, account_id=None, include_related=None)[source]

Retrieves metadata for a specific run of a dbt Cloud job.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • include_related (Optional[List[str]]) -- Optional. List of related fields to pull with the run. Valid values are "trigger", "job", "repository", and "environment".

Returns

The request response.

Return type

requests.models.Response

get_job_run_status(self, run_id, account_id=None)[source]

Retrieves the status for a specific run of a dbt Cloud job.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

Returns

The status of a dbt Cloud job run.

Return type

int

wait_for_job_run_status(self, run_id, account_id=None, expected_statuses=DbtCloudJobRunStatus.SUCCESS.value, check_interval=60, timeout=60 * 60 * 24 * 7)[source]

Waits for a dbt Cloud job run to match an expected status.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • expected_statuses (Union[int, Sequence[int], Set[int]]) -- Optional. The desired status(es) to check against a job run's current status. Defaults to the success status value.

  • check_interval (int) -- Time in seconds to check on a pipeline run's status.

  • timeout (int) -- Time in seconds to wait for a pipeline to reach a terminal status or the expected status.

Returns

Boolean indicating if the job run has reached the expected_status.

Return type

bool

cancel_job_run(self, run_id, account_id=None)[source]

Cancel a specific dbt Cloud job run.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

list_job_run_artifacts(self, run_id, account_id=None, step=None)[source]

Retrieves a list of the available artifact files generated for a completed run of a dbt Cloud job. By default, this returns artifacts from the last step in the run. To list artifacts from other steps in the run, use the step parameter.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • step (Optional[int]) -- Optional. The index of the Step in the Run to query for artifacts. The first step in the run has the index 1. If the step parameter is omitted, artifacts for the last step in the run will be returned.

Returns

List of request responses.

Return type

List[requests.models.Response]

get_job_run_artifact(self, run_id, path, account_id=None, step=None)[source]

Retrieves a list of the available artifact files generated for a completed run of a dbt Cloud job. By default, this returns artifacts from the last step in the run. To list artifacts from other steps in the run, use the step parameter.

Parameters
  • run_id (int) -- The ID of a dbt Cloud job run.

  • path (str) -- The file path related to the artifact file. Paths are rooted at the target/ directory. Use "manifest.json", "catalog.json", or "run_results.json" to download dbt-generated artifacts for the run.

  • account_id (Optional[int]) -- Optional. The ID of a dbt Cloud account.

  • step (Optional[int]) -- Optional. The index of the Step in the Run to query for artifacts. The first step in the run has the index 1. If the step parameter is omitted, artifacts for the last step in the run will be returned.

Returns

The request response.

Return type

requests.models.Response

test_connection(self)[source]

Test dbt Cloud connection.

Was this entry helpful?