airflow.providers.google.cloud.hooks.datapipeline

This module contains a Google Data Pipelines Hook.

Module Contents

Classes

DataPipelineHook

Hook for Google Data Pipelines.

Attributes

DEFAULT_DATAPIPELINE_LOCATION

airflow.providers.google.cloud.hooks.datapipeline.DEFAULT_DATAPIPELINE_LOCATION = 'us-central1'[source]
class airflow.providers.google.cloud.hooks.datapipeline.DataPipelineHook(gcp_conn_id='google_cloud_default', poll_sleep=10, impersonation_chain=None, drain_pipeline=False, cancel_timeout=5 * 60, wait_until_finished=None, expected_terminal_state=None, **kwargs)[source]

Bases: airflow.providers.google.cloud.hooks.dataflow.DataflowHook

Hook for Google Data Pipelines.

get_conn()[source]

Return a Google Cloud Data Pipelines service object.

create_data_pipeline(body, project_id, location=DEFAULT_DATAPIPELINE_LOCATION)[source]

Create a new Data Pipelines instance from the Data Pipelines API.

run_data_pipeline(data_pipeline_name, project_id, location=DEFAULT_DATAPIPELINE_LOCATION)[source]

Run a Data Pipelines Instance using the Data Pipelines API.

static build_parent_name(project_id, location)[source]

Was this entry helpful?