airflow.providers.dbt.cloud.operators.dbt

Module Contents

Classes

DbtCloudRunJobOperatorLink

Operator link for DbtCloudRunJobOperator. This link allows users to monitor the triggered job run

DbtCloudRunJobOperator

Executes a dbt Cloud job.

DbtCloudGetJobRunArtifactOperator

Download artifacts from a dbt Cloud job run.

Bases: airflow.models.BaseOperatorLink

Operator link for DbtCloudRunJobOperator. This link allows users to monitor the triggered job run directly in dbt Cloud.

name = Monitor Job Run[source]

Link to external system.

Note: The old signature of this function was (self, operator, dttm: datetime). That is still supported at runtime but is deprecated.

Parameters

  • operator – The Airflow operator object this link is associated to.

  • ti_key – TaskInstance ID to return link for.

Returns

link to external system

class airflow.providers.dbt.cloud.operators.dbt.DbtCloudRunJobOperator(*, dbt_cloud_conn_id=DbtCloudHook.default_conn_name, job_id, account_id=None, trigger_reason=None, steps_override=None, schema_override=None, wait_for_termination=True, timeout=60 * 60 * 24 * 7, check_interval=60, additional_run_config=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Executes a dbt Cloud job.

See also

For more information on how to use this operator, take a look at the guide: Trigger a dbt Cloud Job

Parameters

  • dbt_cloud_conn_id (str) – The connection ID for connecting to dbt Cloud.

  • job_id (int) – The ID of a dbt Cloud job.

  • account_id (Optional[int]) – Optional. The ID of a dbt Cloud account.

  • trigger_reason (Optional[str]) – Optional. Description of the reason to trigger the job.

  • steps_override (Optional[List[str]]) – Optional. List of dbt commands to execute when triggering the job instead of those configured in dbt Cloud.

  • schema_override (Optional[str]) – Optional. Override the destination schema in the configured target for this job.

  • wait_for_termination (bool) – Flag to wait on a job run’s termination. By default, this feature is enabled but could be disabled to perform an asynchronous wait for a long-running job run execution using the DbtCloudJobRunSensor.

  • timeout (int) – Time in seconds to wait for a job run to reach a terminal status for non-asynchronous waits. Used only if wait_for_termination is True. Defaults to 7 days.

  • check_interval (int) – Time in seconds to check on a job run’s status for non-asynchronous waits. Used only if wait_for_termination is True. Defaults to 60 seconds.

  • additional_run_config (Optional[Dict[str, Any]]) – Optional. Any additional parameters that should be included in the API request when triggering the job.

Returns

The ID of the triggered dbt Cloud job run.

template_fields = ['dbt_cloud_conn_id', 'job_id', 'account_id', 'trigger_reason', 'steps_override',...[source]
execute(context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

on_kill()[source]

Override this method to cleanup subprocesses when a task instance gets killed. Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up or it will leave ghost processes behind.

class airflow.providers.dbt.cloud.operators.dbt.DbtCloudGetJobRunArtifactOperator(*, dbt_cloud_conn_id=DbtCloudHook.default_conn_name, run_id, path, account_id=None, step=None, output_file_name=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Download artifacts from a dbt Cloud job run.

See also

For more information on how to use this operator, take a look at the guide: Download run artifacts

Parameters

  • dbt_cloud_conn_id (str) – The connection ID for connecting to dbt Cloud.

  • run_id (int) – The ID of a dbt Cloud job run.

  • path (str) – The file path related to the artifact file. Paths are rooted at the target/ directory. Use “manifest.json”, “catalog.json”, or “run_results.json” to download dbt-generated artifacts for the run.

  • account_id (Optional[int]) – Optional. The ID of a dbt Cloud account.

  • step (Optional[int]) – Optional. The index of the Step in the Run to query for artifacts. The first step in the run has the index 1. If the step parameter is omitted, artifacts for the last step in the run will be returned.

  • output_file_name (Optional[str]) – Optional. The desired file name for the download artifact file. Defaults to <run_id>_<path> (e.g. “728368_run_results.json”).

template_fields = ['dbt_cloud_conn_id', 'run_id', 'path', 'account_id', 'output_file_name'][source]
execute(context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?