`airflow.providers.google.cloud.operators.dataform`¶

Module Contents¶

Classes¶

`DataformCreateCompilationResultOperator`	Creates a new CompilationResult in a given project and location.
`DataformGetCompilationResultOperator`	Fetches a single CompilationResult.
`DataformCreateWorkflowInvocationOperator`	Creates a new WorkflowInvocation in a given Repository.
`DataformGetWorkflowInvocationOperator`	Fetches a single WorkflowInvocation.
`DataformCancelWorkflowInvocationOperator`	Requests cancellation of a running WorkflowInvocation.

class airflow.providers.google.cloud.operators.dataform.DataformCreateCompilationResultOperator(project_id, region, repository_id, compilation_result, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Creates a new CompilationResult in a given project and location.

Parameters

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.
compilation_result (Union[google.cloud.dataform_v1beta1.types.CompilationResult, Dict]) – Required. The compilation result to create.
retry (Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault]) – Designation of what errors, if any, should be retried.
timeout (Optional[float]) – The timeout for this request.
metadata (Sequence[Tuple[str, str]]) – Strings which should be sent along with the request as metadata.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
delegate_to (Optional[str]) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.
impersonation_chain (Optional[Union[str, Sequence[str]]]) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformGetCompilationResultOperator(project_id, region, repository_id, compilation_result_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Fetches a single CompilationResult.

Parameters

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.
compilation_result_id (str) – The Id of the Dataform Compilation Result
retry (Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault]) – Designation of what errors, if any, should be retried.
timeout (Optional[float]) – The timeout for this request.
metadata (Sequence[Tuple[str, str]]) – Strings which should be sent along with the request as metadata.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
delegate_to (Optional[str]) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.
impersonation_chain (Optional[Union[str, Sequence[str]]]) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'compilation_result_id', 'delegate_to', 'impersonation_chain'][source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformCreateWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, asynchronous=False, wait_time=10, *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Creates a new WorkflowInvocation in a given Repository.

Parameters

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.
workflow_invocation (Union[google.cloud.dataform_v1beta1.types.WorkflowInvocation, Dict]) – Required. The workflow invocation resource to create.
retry (Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault]) – Designation of what errors, if any, should be retried.
timeout (Optional[int]) – The timeout for this request.
metadata (Sequence[Tuple[str, str]]) – Strings which should be sent along with the request as metadata.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
delegate_to (Optional[str]) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.
impersonation_chain (Optional[Union[str, Sequence[str]]]) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
asynchronous (bool) – Flag to return workflow_invocation_id from the Dataform API. This is useful for submitting long running workflows and waiting on them asynchronously using the DataformWorkflowInvocationStateSensor
wait_time (int) – Number of seconds between checks

template_fields = ['workflow_invocation', 'delegate_to', 'impersonation_chain'][source]¶

operator_extra_links[source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformGetWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Fetches a single WorkflowInvocation.

Parameters

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.
workflow_invocation_id (str) – the workflow invocation resource’s id.
retry (Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault]) – Designation of what errors, if any, should be retried.
timeout (Optional[float]) – The timeout for this request.
metadata (Sequence[Tuple[str, str]]) – Strings which should be sent along with the request as metadata.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
delegate_to (Optional[str]) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.
impersonation_chain (Optional[Union[str, Sequence[str]]]) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'workflow_invocation_id', 'delegate_to', 'impersonation_chain'][source]¶

operator_extra_links[source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformCancelWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Requests cancellation of a running WorkflowInvocation.

Parameters

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.
workflow_invocation_id (str) – the workflow invocation resource’s id.
retry (Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault]) – Designation of what errors, if any, should be retried.
timeout (Optional[float]) – The timeout for this request.
metadata (Sequence[Tuple[str, str]]) – Strings which should be sent along with the request as metadata.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
delegate_to (Optional[str]) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.
impersonation_chain (Optional[Union[str, Sequence[str]]]) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'workflow_invocation_id', 'delegate_to', 'impersonation_chain'][source]¶

operator_extra_links[source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

airflow.providers.google.cloud.operators.dataform¶

Module Contents¶

Classes¶

`airflow.providers.google.cloud.operators.dataform`¶