airflow.providers.google.cloud.operators.datacatalog
¶
Module Contents¶
Classes¶
Creates an entry. |
|
Creates an EntryGroup. |
|
Creates a tag on an entry. |
|
Creates a tag template. |
|
Creates a field in a tag template. |
|
Deletes an existing entry. |
|
Deletes an EntryGroup. |
|
Deletes a tag. |
|
Deletes a tag template and all tags using the template. |
|
Deletes a field in a tag template and all uses of that field. |
|
Gets an entry. |
|
Gets an entry group. |
|
Gets a tag template. |
|
Lists the tags on an Entry. |
|
Get an entry by target resource name. |
|
Renames a field in a tag template. |
|
Searches Data Catalog for multiple resources like entries, tags that match a query. |
|
Updates an existing entry. |
|
Updates an existing tag. |
|
Updates a tag template. |
|
Updates a field in a tag template. This method cannot be used to update the field type. |
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateEntryOperator(*, location, entry_group, entry_id, entry, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Creates an entry.
Currently only entries of ‘FILESET’ type can be created.
The newly created entry ID are saved under the
entry_id
key in XCOM.See also
For more information on how to use this operator, take a look at the guide: Creating an entry
- Parameters
location (str) – Required. The location of the entry to create.
entry_group (str) – Required. Entry group ID under which the entry is created.
entry_id (str) – Required. The id of the entry to create.
entry (dict | google.cloud.datacatalog.Entry) – Required. The entry to create. If a dict is provided, it must be of the same form as the protobuf message
Entry
project_id (str) – The ID of the Google Cloud project that owns the entry. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If set to
None
or missing, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry_id', 'entry', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateEntryGroupOperator(*, location, entry_group_id, entry_group, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Creates an EntryGroup.
The newly created entry group ID are saved under the
entry_group_id
key in XCOM.See also
For more information on how to use this operator, take a look at the guide: Creating an entry group
- Parameters
location (str) – Required. The location of the entry group to create.
entry_group_id (str) – Required. The id of the entry group to create. The id must begin with a letter or underscore, contain only English letters, numbers and underscores, and be at most 64 characters.
entry_group (dict | google.cloud.datacatalog.EntryGroup) –
The entry group to create. Defaults to an empty entry group.
If a dict is provided, it must be of the same form as the protobuf message
EntryGroup
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group_id', 'entry_group', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagOperator(*, location, entry_group, entry, tag, template_id=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Creates a tag on an entry.
The newly created tag ID are saved under the
tag_id
key in XCOM.See also
For more information on how to use this operator, take a look at the guide: Creating a tag on an entry
- Parameters
location (str) – Required. The location of the tag to create.
entry_group (str) – Required. Entry group ID under which the tag is created.
entry (str) – Required. Entry group ID under which the tag is created.
tag (dict | google.cloud.datacatalog.Tag) – Required. The tag to create. If a dict is provided, it must be of the same form as the protobuf message
Tag
template_id (str | None) – Required. Template ID used to create tag
project_id (str) – The ID of the Google Cloud project that owns the tag. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry', 'tag', 'template_id', 'project_id', 'retry', 'timeout',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagTemplateOperator(*, location, tag_template_id, tag_template, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Creates a tag template.
The newly created tag template are saved under the
tag_template_id
key in XCOM.See also
For more information on how to use this operator, take a look at the guide: Creating a tag template
- Parameters
location (str) – Required. The location of the tag template to create.
tag_template_id (str) – Required. The id of the tag template to create.
tag_template (dict | google.cloud.datacatalog.TagTemplate) –
Required. The tag template to create.
If a dict is provided, it must be of the same form as the protobuf message
TagTemplate
project_id (str) – The ID of the Google Cloud project that owns the tag template. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template_id', 'tag_template', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagTemplateFieldOperator(*, location, tag_template, tag_template_field_id, tag_template_field, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Creates a field in a tag template.
The newly created tag template field are saved under the
tag_template_field_id
key in XCOM.See also
For more information on how to use this operator, take a look at the guide: Creating a field
- Parameters
location (str) – Required. The location of the tag template field to create.
tag_template (str) – Required. The id of the tag template to create.
tag_template_field_id (str) – Required. The ID of the tag template field to create. Field ids can contain letters (both uppercase and lowercase), numbers (0-9), underscores (_) and dashes (-). Field IDs must be at least 1 character long and at most 128 characters long. Field IDs must also be unique within their template.
tag_template_field (dict | google.cloud.datacatalog.TagTemplateField) –
Required. The tag template field to create.
If a dict is provided, it must be of the same form as the protobuf message
TagTemplateField
project_id (str) – The ID of the Google Cloud project that owns the tag template field. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template', 'tag_template_field_id', 'tag_template_field', 'project_id',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteEntryOperator(*, location, entry_group, entry, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Deletes an existing entry.
See also
For more information on how to use this operator, take a look at the guide: Deleting a entry
- Parameters
location (str) – Required. The location of the entry to delete.
entry_group (str) – Required. Entry group ID for entries that is deleted.
entry (str) – Entry ID that is deleted.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteEntryGroupOperator(*, location, entry_group, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Deletes an EntryGroup.
Only entry groups that do not contain entries can be deleted.
See also
For more information on how to use this operator, take a look at the guide: Deleting an entry group
- Parameters
location (str) – Required. The location of the entry group to delete.
entry_group (str) – Entry group ID that is deleted.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'project_id', 'retry', 'timeout', 'metadata', 'gcp_conn_id',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagOperator(*, location, entry_group, entry, tag, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Deletes a tag.
See also
For more information on how to use this operator, take a look at the guide: Deleting a tag
- Parameters
location (str) – Required. The location of the tag to delete.
entry_group (str) – Entry group ID for tag that is deleted.
entry (str) – Entry ID for tag that is deleted.
tag (str) – Identifier for TAG that is deleted.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry', 'tag', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagTemplateOperator(*, location, tag_template, force, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Deletes a tag template and all tags using the template.
See also
For more information on how to use this operator, take a look at the guide: Deleting a tag template
- Parameters
location (str) – Required. The location of the tag template to delete.
tag_template (str) – ID for tag template that is deleted.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.force (bool) – Required. Currently, this field must always be set to
true
. This confirms the deletion of any possible tags using this template.force = false
will be supported in the future.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template', 'force', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagTemplateFieldOperator(*, location, tag_template, field, force, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Deletes a field in a tag template and all uses of that field.
See also
For more information on how to use this operator, take a look at the guide: Deleting a field
- Parameters
location (str) – Required. The location of the tag template to delete.
tag_template (str) – Tag Template ID for tag template field that is deleted.
field (str) – Name of field that is deleted.
force (bool) – Required. This confirms the deletion of this field from any tags using this field.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template', 'field', 'force', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetEntryOperator(*, location, entry_group, entry, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Gets an entry.
See also
For more information on how to use this operator, take a look at the guide: Getting an entry
- Parameters
location (str) – Required. The location of the entry to get.
entry_group (str) – Required. The entry group of the entry to get.
entry (str) – The ID of the entry to get.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetEntryGroupOperator(*, location, entry_group, read_mask, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Gets an entry group.
See also
For more information on how to use this operator, take a look at the guide: Getting an entry group
- Parameters
location (str) – Required. The location of the entry group to get.
entry_group (str) – The ID of the entry group to get.
read_mask (google.protobuf.field_mask_pb2.FieldMask) –
The fields to return. If not set or empty, all fields are returned.
If a dict is provided, it must be of the same form as the protobuf message
FieldMask
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'read_mask', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetTagTemplateOperator(*, location, tag_template, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Gets a tag template.
See also
For more information on how to use this operator, take a look at the guide: Getting a tag template
- Parameters
location (str) – Required. The location of the tag template to get.
tag_template (str) – Required. The ID of the tag template to get.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template', 'project_id', 'retry', 'timeout', 'metadata', 'gcp_conn_id',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogListTagsOperator(*, location, entry_group, entry, page_size=100, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Lists the tags on an Entry.
See also
For more information on how to use this operator, take a look at the guide: Listing tags on an entry
- Parameters
location (str) – Required. The location of the tags to get.
entry_group (str) – Required. The entry group of the tags to get.
entry (str) – Required. The entry of the tags to get.
page_size (int) – The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page. (Default: 100)
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'entry_group', 'entry', 'page_size', 'project_id', 'retry', 'timeout', 'metadata',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogLookupEntryOperator(*, linked_resource=None, sql_resource=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Get an entry by target resource name.
This method allows clients to use the resource name from the source Google Cloud service to get the Data Catalog Entry.
See also
For more information on how to use this operator, take a look at the guide: Getting an entry
- Parameters
linked_resource (str | None) – The full name of the Google Cloud resource the Data Catalog entry represents. See: https://cloud.google.com/apis/design/resource_names#full_resource_name. Full names are case-sensitive.
sql_resource (str | None) – The SQL name of the entry. SQL names are case-sensitive.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('linked_resource', 'sql_resource', 'project_id', 'retry', 'timeout', 'metadata', 'gcp_conn_id',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogRenameTagTemplateFieldOperator(*, location, tag_template, field, new_tag_template_field_id, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Renames a field in a tag template.
See also
For more information on how to use this operator, take a look at the guide: Renaming a field
- Parameters
location (str) – Required. The location of the tag template field to rename.
tag_template (str) – The tag template ID for field that is renamed.
field (str) – Required. The old ID of this tag template field. For example,
my_old_field
.new_tag_template_field_id (str) – Required. The new ID of this tag template field. For example,
my_new_field
.project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('location', 'tag_template', 'field', 'new_tag_template_field_id', 'project_id', 'retry',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogSearchCatalogOperator(*, scope, query, page_size=100, order_by=None, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Searches Data Catalog for multiple resources like entries, tags that match a query.
This does not return the complete resource, only the resource identifier and high level fields. Clients can subsequently call
Get
methods.Note that searches do not have full recall. There may be results that match your query but are not returned, even in subsequent pages of results. These missing results may vary across repeated calls to search. Do not rely on this method if you need to guarantee full recall.
See also
For more information on how to use this operator, take a look at the guide: Search resources
- Parameters
scope (dict | google.cloud.datacatalog.SearchCatalogRequest.Scope) –
Required. The scope of this search request.
If a dict is provided, it must be of the same form as the protobuf message
Scope
query (str) –
Required. The query string in search query syntax. The query must be non-empty.
Query strings can be simple as “x” or more qualified as:
name:x
column:x
description:y
Note: Query tokens need to have a minimum of 3 characters for substring matching to work correctly. See Data Catalog Search Syntax for more information.
page_size (int) – The maximum number of resources contained in the underlying API response. If page streaming is performed per-resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.
order_by (str | None) –
Specifies the ordering of results, currently supported case-sensitive choices are:
relevance
, only supports descendinglast_access_timestamp [asc|desc]
, defaults to descending if not specifiedlast_modified_timestamp [asc|desc]
, defaults to descending if not specified
If not specified, defaults to
relevance
descending.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('scope', 'query', 'page_size', 'order_by', 'retry', 'timeout', 'metadata', 'gcp_conn_id',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateEntryOperator(*, entry, update_mask, location=None, entry_group=None, entry_id=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Updates an existing entry.
See also
For more information on how to use this operator, take a look at the guide: Updating an entry
- Parameters
entry (dict | google.cloud.datacatalog.Entry) –
Required. The updated entry. The “name” field must be set.
If a dict is provided, it must be of the same form as the protobuf message
Entry
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –
The fields to update on the entry. If absent or empty, all modifiable fields are updated.
If a dict is provided, it must be of the same form as the protobuf message
FieldMask
location (str | None) – Required. The location of the entry to update.
entry_group (str | None) – The entry group ID for the entry that is being updated.
entry_id (str | None) – The entry ID that is being updated.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('entry', 'update_mask', 'location', 'entry_group', 'entry_id', 'project_id', 'retry',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagOperator(*, tag, update_mask, location=None, entry_group=None, entry=None, tag_id=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Updates an existing tag.
See also
For more information on how to use this operator, take a look at the guide: Updating a tag
- Parameters
tag (dict | google.cloud.datacatalog.Tag) –
Required. The updated tag. The “name” field must be set.
If a dict is provided, it must be of the same form as the protobuf message
Tag
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –
The fields to update on the Tag. If absent or empty, all modifiable fields are updated. Currently the only modifiable field is the field
fields
.If a dict is provided, it must be of the same form as the protobuf message
FieldMask
location (str | None) – Required. The location of the tag to rename.
entry_group (str | None) – The entry group ID for the tag that is being updated.
entry (str | None) – The entry ID for the tag that is being updated.
tag_id (str | None) – The tag ID that is being updated.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('tag', 'update_mask', 'location', 'entry_group', 'entry', 'tag_id', 'project_id', 'retry',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateOperator(*, tag_template, update_mask, location=None, tag_template_id=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Updates a tag template.
This method cannot be used to update the fields of a template. The tag template fields are represented as separate resources and should be updated using their own create/update/delete methods.
See also
For more information on how to use this operator, take a look at the guide: Updating a tag template
- Parameters
tag_template (dict | google.cloud.datacatalog.TagTemplate) –
Required. The template to update. The “name” field must be set.
If a dict is provided, it must be of the same form as the protobuf message
TagTemplate
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –
The field mask specifies the parts of the template to overwrite.
If absent or empty, all of the allowed fields above will be updated.
If a dict is provided, it must be of the same form as the protobuf message
FieldMask
location (str | None) – Required. The location of the tag template to rename.
tag_template_id (str | None) – Optional. The tag template ID for the entry that is being updated.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('tag_template', 'update_mask', 'location', 'tag_template_id', 'project_id', 'retry', 'timeout',...[source]¶
- class airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateFieldOperator(*, tag_template_field, update_mask, tag_template_field_name=None, location=None, tag_template=None, tag_template_field_id=None, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]¶
Bases:
airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator
Updates a field in a tag template. This method cannot be used to update the field type.
See also
For more information on how to use this operator, take a look at the guide: Updating a field
- Parameters
tag_template_field (dict | google.cloud.datacatalog.TagTemplateField) –
Required. The template to update.
If a dict is provided, it must be of the same form as the protobuf message
TagTemplateField
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –
The field mask specifies the parts of the template to be updated. Allowed fields:
display_name
type.enum_type
If
update_mask
is not set or empty, all of the allowed fields above will be updated.When updating an enum type, the provided values will be merged with the existing values. Therefore, enum values can only be added, existing enum values cannot be deleted nor renamed.
If a dict is provided, it must be of the same form as the protobuf message
FieldMask
tag_template_field_name (str | None) – Optional. The name of the tag template field to rename.
location (str | None) – Optional. The location of the tag to rename.
tag_template (str | None) – Optional. The tag template ID for tag template field to rename.
tag_template_field_id (str | None) – Optional. The ID of tag template field to rename.
project_id (str) – The ID of the Google Cloud project that owns the entry group. If set to
None
or missing, the default project_id from the Google Cloud connection is used.retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If
None
is specified, requests will be retried using a default configuration.timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if
retry
is specified, the timeout applies to each individual attempt.metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – Optional, The connection ID used to connect to Google Cloud. Defaults to ‘google_cloud_default’.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
- template_fields: collections.abc.Sequence[str] = ('tag_template_field', 'update_mask', 'tag_template_field_name', 'location', 'tag_template',...[source]¶