`airflow.providers.google.cloud.operators.dataprep`¶

This module contains a Google Dataprep operator.

Module Contents¶

Classes¶

`DataprepGetJobsForJobGroupOperator`	Get information about the batch jobs within a Cloud Dataprep job.
`DataprepGetJobGroupOperator`	Get the specified job group.
`DataprepRunJobGroupOperator`	Create a `jobGroup`, which launches the specified job as the authenticated user.

class airflow.providers.google.cloud.operators.dataprep.DataprepGetJobsForJobGroupOperator(*, dataprep_conn_id='dataprep_default', job_id, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Get information about the batch jobs within a Cloud Dataprep job. API documentation https://clouddataprep.com/documentation/api#section/Overview

See also

For more information on how to use this operator, take a look at the guide: Get Jobs For Job Group

:param job_id The ID of the job that will be requests

template_fields :Sequence[str] = ['job_id'][source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataprep.DataprepGetJobGroupOperator(*, dataprep_conn_id='dataprep_default', job_group_id, embed, include_deleted, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Get the specified job group. A job group is a job that is executed from a specific node in a flow. API documentation https://clouddataprep.com/documentation/api#section/Overview

See also

For more information on how to use this operator, take a look at the guide: Get Job Group

Parameters

job_group_id (int) – The ID of the job that will be requests
embed (str) – Comma-separated list of objects to pull in as part of the response
include_deleted (bool) – if set to “true”, will include deleted objects

template_fields :Sequence[str] = ['job_group_id', 'embed'][source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataprep.DataprepRunJobGroupOperator(*, dataprep_conn_id='dataprep_default', body_request, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Create a jobGroup, which launches the specified job as the authenticated user. This performs the same action as clicking on the Run Job button in the application. To get recipe_id please follow the Dataprep API documentation https://clouddataprep.com/documentation/api#operation/runJobGroup

See also

For more information on how to use this operator, take a look at the guide: Run Job Group

Parameters

dataprep_conn_id (str) – The Dataprep connection ID
body_request (dict) – Passed as the body_request to GoogleDataprepHook’s run_job_group, where it’s the identifier for the recipe to run

template_fields :Sequence[str] = ['body_request'][source]¶

execute(context)[source]¶

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?