airflow.providers.google.cloud.hooks.cloud_batch

Module Contents

Classes

CloudBatchHook

Hook for the Google Cloud Batch service.

CloudBatchAsyncHook

Async hook for the Google Cloud Batch service.
class airflow.providers.google.cloud.hooks.cloud_batch.CloudBatchHook(gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Hook for the Google Cloud Batch service.

Parameters

  • gcp_conn_id (str) – The connection ID to use when fetching connection info.

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account.

get_conn()[source]

Retrieve connection to GCE Batch.

Returns

Google Batch Service client object.

submit_batch_job(job_name, job, region, project_id=PROVIDE_PROJECT_ID)[source]
delete_job(job_name, region, project_id=PROVIDE_PROJECT_ID)[source]
list_jobs(region, project_id=PROVIDE_PROJECT_ID, filter=None, limit=None)[source]
list_tasks(region, job_name, project_id=PROVIDE_PROJECT_ID, group_name='group0', filter=None, limit=None)[source]
wait_for_job(job_name, polling_period_seconds=10, timeout=None)[source]
get_job(job_name)[source]
class airflow.providers.google.cloud.hooks.cloud_batch.CloudBatchAsyncHook(gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Async hook for the Google Cloud Batch service.

Parameters

  • gcp_conn_id (str) – The connection ID to use when fetching connection info.

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account.

get_conn()[source]

Return connection for the hook.

async get_batch_job(job_name)[source]

Was this entry helpful?