airflow.providers.google.cloud.operators.dataprep

This module contains a Google Dataprep operator.

Module Contents

class airflow.providers.google.cloud.operators.dataprep.DataprepGetJobsForJobGroupOperator(*, dataprep_conn_id: str = 'dataprep_default', job_id: int, **kwargs)[source]

Bases: airflow.models.BaseOperator

Get information about the batch jobs within a Cloud Dataprep job. API documentation https://clouddataprep.com/documentation/api#section/Overview

See also

For more information on how to use this operator, take a look at the guide: Get Jobs For Job Group

:param job_id The ID of the job that will be requests :type job_id: int

template_fields = ['job_id'][source]
execute(self, context: dict)[source]
class airflow.providers.google.cloud.operators.dataprep.DataprepGetJobGroupOperator(*, dataprep_conn_id: str = 'dataprep_default', job_group_id: int, embed: str, include_deleted: bool, **kwargs)[source]

Bases: airflow.models.BaseOperator

Get the specified job group. A job group is a job that is executed from a specific node in a flow. API documentation https://clouddataprep.com/documentation/api#section/Overview

See also

For more information on how to use this operator, take a look at the guide: Get Job Group

Parameters

  • job_group_id (int) -- The ID of the job that will be requests

  • embed (string) -- Comma-separated list of objects to pull in as part of the response

  • include_deleted (bool) -- if set to "true", will include deleted objects

template_fields = ['job_group_id', 'embed'][source]
execute(self, context: dict)[source]
class airflow.providers.google.cloud.operators.dataprep.DataprepRunJobGroupOperator(*, dataprep_conn_id: str = 'dataprep_default', body_request: dict, **kwargs)[source]

Bases: airflow.models.BaseOperator

Create a jobGroup, which launches the specified job as the authenticated user. This performs the same action as clicking on the Run Job button in the application. To get recipe_id please follow the Dataprep API documentation https://clouddataprep.com/documentation/api#operation/runJobGroup

See also

For more information on how to use this operator, take a look at the guide: Run Job Group

Parameters

  • dataprep_conn_id (str) -- The Dataprep connection ID

  • body_request (dict) -- Passed as the body_request to GoogleDataprepHook's run_job_group, where it's the identifier for the recipe to run

template_fields = ['body_request'][source]
execute(self, context: None)[source]

Was this entry helpful?