airflow.providers.google.cloud.operators.cloud_composer

Module Contents

Classes

CloudComposerEnvironmentLink

Helper class for constructing Cloud Composer Environment Link

CloudComposerEnvironmentsLink

Helper class for constructing Cloud Composer Environment Link

CloudComposerCreateEnvironmentOperator

Create a new environment.

CloudComposerDeleteEnvironmentOperator

Delete an environment.

CloudComposerGetEnvironmentOperator

Get an existing environment.

CloudComposerListEnvironmentsOperator

List environments.

CloudComposerUpdateEnvironmentOperator

Update an environment.

CloudComposerListImageVersionsOperator

List ImageVersions for provided location.

Attributes

CLOUD_COMPOSER_BASE_LINK

CLOUD_COMPOSER_DETAILS_LINK

CLOUD_COMPOSER_ENVIRONMENTS_LINK

Bases: airflow.models.BaseOperatorLink

Helper class for constructing Cloud Composer Environment Link

name = Cloud Composer Environment[source]
key = composer_conf[source]
static persist(operator_instance, context)[source]

Link to external system.

Parameters

  • operator (airflow.models.BaseOperator) -- airflow operator

  • dttm (datetime.datetime) -- datetime

Returns

link to external system

Return type

str

Bases: airflow.models.BaseOperatorLink

Helper class for constructing Cloud Composer Environment Link

name = Cloud Composer Environment List[source]
key = composer_conf[source]
static persist(operator_instance, context)[source]

Link to external system.

Parameters

  • operator (airflow.models.BaseOperator) -- airflow operator

  • dttm (datetime.datetime) -- datetime

Returns

link to external system

Return type

str

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerCreateEnvironmentOperator(*, project_id, region, environment_id, environment, gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, retry=None, timeout=None, metadata=(), deferrable=False, pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.models.BaseOperator

Create a new environment.

Parameters

  • project_id (str) -- Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) -- Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) -- Required. The ID of the Google Cloud environment that the service belongs to.

  • environment (Union[google.cloud.orchestration.airflow.service_v1.types.Environment, Dict]) -- The environment to create.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • deferrable (bool) -- Run operator in the deferrable mode

  • pooling_period_seconds (int) -- Optional: Control the rate of the poll for the result of deferrable run. By default the trigger will poll every 30 seconds.

template_fields = ['project_id', 'region', 'environment_id', 'environment', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(self, context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerDeleteEnvironmentOperator(*, project_id, region, environment_id, retry=None, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, deferrable=False, pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.models.BaseOperator

Delete an environment.

Parameters

  • project_id (str) -- Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) -- Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) -- Required. The ID of the Google Cloud environment that the service belongs to.

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

  • deferrable (bool) -- Run operator in the deferrable mode

  • pooling_period_seconds (int) -- Optional: Control the rate of the poll for the result of deferrable run. By default the trigger will poll every 30 seconds.

template_fields = ['project_id', 'region', 'environment_id', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(self, context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerGetEnvironmentOperator(*, project_id, region, environment_id, retry=None, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Get an existing environment.

Parameters

  • project_id (str) -- Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) -- Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) -- Required. The ID of the Google Cloud environment that the service belongs to.

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

template_fields = ['project_id', 'region', 'environment_id', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerListEnvironmentsOperator(*, project_id, region, page_size=None, page_token=None, retry=None, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

List environments.

Parameters

  • project_id (str) -- Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) -- Required. The ID of the Google Cloud region that the service belongs to.

  • page_size (Optional[int]) -- The maximum number of environments to return.

  • page_token (Optional[str]) -- The next_page_token value returned from a previous List request, if any.

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

template_fields = ['project_id', 'region', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerUpdateEnvironmentOperator(*, project_id, region, environment_id, environment, update_mask, retry=None, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, deferrable=False, pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.models.BaseOperator

Update an environment.

Parameters

  • project_id (str) -- Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) -- Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) -- Required. The ID of the Google Cloud environment that the service belongs to.

  • environment (Union[Dict, google.cloud.orchestration.airflow.service_v1.types.Environment]) -- A patch environment. Fields specified by the updateMask will be copied from the patch environment into the environment under update.

  • update_mask (Union[Dict, google.protobuf.field_mask_pb2.FieldMask]) -- Required. A comma-separated list of paths, relative to Environment, of fields to update. If a dict is provided, it must be of the same form as the protobuf message FieldMask

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

  • deferrable (bool) -- Run operator in the deferrable mode

  • pooling_period_seconds (int) -- Optional: Control the rate of the poll for the result of deferrable run. By default the trigger will poll every 30 seconds.

template_fields = ['project_id', 'region', 'environment_id', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(self, context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerListImageVersionsOperator(*, project_id, region, page_size=None, page_token=None, include_past_releases=False, retry=None, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, delegate_to=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

List ImageVersions for provided location.

Parameters

  • request -- The request object. List ImageVersions in a project and location.

  • retry (Optional[google.api_core.retry.Retry]) -- Designation of what errors, if any, should be retried.

  • timeout (Optional[float]) -- The timeout for this request.

  • metadata (Sequence[Tuple[str, str]]) -- Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) --

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

template_fields = ['project_id', 'region', 'impersonation_chain'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?