`airflow.providers.google.cloud.links.dataproc`¶

This module contains Google Dataproc links.

Module Contents¶

Classes¶

`DataprocLink`	Helper class for constructing Dataproc resource link.
`DataprocListLink`	Helper class for constructing list of Dataproc resources link.
`DataprocClusterLink`	Helper class for constructing Dataproc Cluster Link.
`DataprocJobLink`	Helper class for constructing Dataproc Job Link.
`DataprocWorkflowLink`	Helper class for constructing Dataproc Workflow Link.
`DataprocWorkflowTemplateLink`	Helper class for constructing Dataproc Workflow Template Link.
`DataprocBatchLink`	Helper class for constructing Dataproc Batch Link.
`DataprocBatchesListLink`	Helper class for constructing Dataproc Batches List Link.

Functions¶

__getattr__(name)

Attributes¶

`DATAPROC_BASE_LINK`
`DATAPROC_JOB_LINK`
`DATAPROC_CLUSTER_LINK`
`DATAPROC_WORKFLOW_TEMPLATE_LINK`
`DATAPROC_WORKFLOW_LINK`
`DATAPROC_BATCH_LINK`
`DATAPROC_BATCHES_LINK`
`DATAPROC_JOB_LINK_DEPRECATED`
`DATAPROC_CLUSTER_LINK_DEPRECATED`

airflow.providers.google.cloud.links.dataproc.__getattr__(name)[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BASE_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_JOB_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_CLUSTER_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_WORKFLOW_TEMPLATE_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_WORKFLOW_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BATCH_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BATCHES_LINK[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_JOB_LINK_DEPRECATED[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_CLUSTER_LINK_DEPRECATED[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocLink[source]¶

Bases: airflow.models.BaseOperatorLink

Helper class for constructing Dataproc resource link.

Warning

This link is pending to deprecate.

name = 'Dataproc resource'[source]¶

key = 'conf'[source]¶

static persist(context, task_instance, url, resource)[source]¶

get_link(operator, *, ti_key)[source]¶

Link to external system.

Note: The old signature of this function was (self, operator, dttm: datetime). That is still supported at runtime but is deprecated.

Parameters

operator (airflow.models.BaseOperator) – The Airflow operator object this link is associated to.
ti_key (airflow.models.taskinstancekey.TaskInstanceKey) – TaskInstance ID to return link for.

Returns

link to external system

Return type

str

__attrs_post_init__()[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocListLink[source]¶

Bases: airflow.models.BaseOperatorLink

Helper class for constructing list of Dataproc resources link.

Warning

This link is deprecated.

name = 'Dataproc resources'[source]¶

key = 'list_conf'[source]¶

static persist(context, task_instance, url)[source]¶

get_link(operator, *, ti_key)[source]¶

Link to external system.

Note: The old signature of this function was (self, operator, dttm: datetime). That is still supported at runtime but is deprecated.

Parameters

operator (airflow.models.BaseOperator) – The Airflow operator object this link is associated to.
ti_key (airflow.models.taskinstancekey.TaskInstanceKey) – TaskInstance ID to return link for.

Returns

link to external system

Return type

str

__attrs_post_init__()[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocClusterLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Cluster Link.

name = 'Dataproc Cluster'[source]¶

key = 'dataproc_cluster'[source]¶

format_str[source]¶

static persist(context, operator, cluster_id, region, project_id)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocJobLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Job Link.

name = 'Dataproc Job'[source]¶

key = 'dataproc_job'[source]¶

format_str[source]¶

static persist(context, operator, job_id, region, project_id)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocWorkflowLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Workflow Link.

name = 'Dataproc Workflow'[source]¶

key = 'dataproc_workflow'[source]¶

format_str[source]¶

static persist(context, operator, workflow_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocWorkflowTemplateLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Workflow Template Link.

name = 'Dataproc Workflow Template'[source]¶

key = 'dataproc_workflow_template'[source]¶

format_str[source]¶

static persist(context, operator, workflow_template_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocBatchLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Batch Link.

name = 'Dataproc Batch'[source]¶

key = 'dataproc_batch'[source]¶

format_str[source]¶

static persist(context, operator, batch_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocBatchesListLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Batches List Link.

name = 'Dataproc Batches List'[source]¶

key = 'dataproc_batches_list'[source]¶

format_str[source]¶

static persist(context, operator, project_id)[source]¶

airflow.providers.google.cloud.links.dataproc¶

Module Contents¶

Classes¶

Functions¶

Attributes¶

`airflow.providers.google.cloud.links.dataproc`¶