airflow.providers.google.cloud.hooks.cloud_memorystore

Hooks for Cloud Memorystore service.

Module Contents

Classes

CloudMemorystoreHook

Hook for Google Cloud Memorystore APIs.

CloudMemorystoreMemcachedHook

Hook for Google Cloud Memorystore for Memcached service APIs.
class airflow.providers.google.cloud.hooks.cloud_memorystore.CloudMemorystoreHook(gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Hook for Google Cloud Memorystore APIs.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

Parameters

  • gcp_conn_id (str) – The connection ID to use when fetching connection info.

  • impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account.

get_conn()[source]

Retrieve client library object that allow access to Cloud Memorystore service.

create_instance(location, instance_id, instance, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Create a Redis instance based on the specified tier and memory size.

By default, the instance is accessible from the project’s default network.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str) –

    Required. The logical name of the Redis instance in the customer project with the following restrictions:

    • Must contain only lowercase letters, numbers, and hyphens.

    • Must start with a letter.

    • Must be between 1-40 characters.

    • Must end with a number or a letter.

    • Must be unique within the customer project / location

  • instance (dict | google.cloud.redis_v1.Instance) –

    Required. A Redis [Instance] resource

    If a dict is provided, it must be of the same form as the protobuf message Instance

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

delete_instance(location, instance, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Delete a specific Redis instance. Instance stops serving and data is deleted.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Redis instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

export_instance(location, instance, output_config, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Export Redis instance data into a Redis RDB format file in Cloud Storage.

Redis will continue serving during this operation.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Redis instance in the customer project.

  • output_config (dict | google.cloud.redis_v1.OutputConfig) –

    Required. Specify data to be exported.

    If a dict is provided, it must be of the same form as the protobuf message OutputConfig

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

failover_instance(location, instance, data_protection_mode, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Failover of the primary node to current replica node.

Initiates a failover of the primary node to current replica node for a specific STANDARD tier Cloud Memorystore for Redis instance.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Redis instance in the customer project.

  • data_protection_mode (google.cloud.redis_v1.FailoverInstanceRequest.DataProtectionMode) – Optional. Available data protection modes that the user can choose. If it’s unspecified, data protection mode will be LIMITED_DATA_LOSS by default. .DataProtectionMode

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

get_instance(location, instance, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Get the details of a specific Redis instance.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Redis instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

import_instance(location, instance, input_config, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

Import a Redis RDB snapshot file from Cloud Storage into a Redis instance.

Redis may stop serving during this operation. Instance state will be IMPORTING for entire operation. When complete, the instance will contain only data from the imported file.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Redis instance in the customer project.

  • input_config (dict | google.cloud.redis_v1.InputConfig) –

    Required. Specify data to be imported.

    If a dict is provided, it must be of the same form as the protobuf message InputConfig

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

list_instances(location, page_size, project_id=PROVIDE_PROJECT_ID, retry=DEFAULT, timeout=None, metadata=())[source]

List Redis instances owned by a project at the specified location (region) or all locations.

Parameters

  • location (str) –

    The location of the Cloud Memorystore instance (for example europe-west1)

    If it is specified as - (wildcard), then all regions available to the project are queried, and the results are aggregated.

  • page_size (int) – The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

update_instance(update_mask, instance, project_id=PROVIDE_PROJECT_ID, location=None, instance_id=None, retry=DEFAULT, timeout=None, metadata=())[source]

Update the metadata and configuration of a specific Redis instance.

Parameters

  • update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –

    Required. Mask of fields to update. At least one path must be supplied in this field. The elements of the repeated paths field may only include these fields from Instance:

    • displayName

    • labels

    • memorySizeGb

    • redisConfig

    If a dict is provided, it must be of the same form as the protobuf message FieldMask

  • instance (dict | google.cloud.redis_v1.Instance) –

    Required. Update description. Only fields specified in update_mask are updated.

    If a dict is provided, it must be of the same form as the protobuf message Instance

  • location (str | None) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str | None) – The logical name of the Redis instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

class airflow.providers.google.cloud.hooks.cloud_memorystore.CloudMemorystoreMemcachedHook(gcp_conn_id='google_cloud_default', impersonation_chain=None)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Hook for Google Cloud Memorystore for Memcached service APIs.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

Parameters

  • gcp_conn_id (str) – The connection ID to use when fetching connection info.

  • impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account.

get_conn()[source]

Retrieve client library object that allow access to Cloud Memorystore Memcached service.

apply_parameters(node_ids, apply_all, project_id, location, instance_id, retry=DEFAULT, timeout=None, metadata=())[source]

Will update current set of Parameters to the set of specified nodes of the Memcached Instance.

Parameters

  • node_ids (Sequence[str]) – Nodes to which we should apply the instance-level parameter group.

  • apply_all (bool) – Whether to apply instance-level parameter group to all nodes. If set to true, will explicitly restrict users from specifying any nodes, and apply parameter group updates to all nodes within the instance.

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str) – The logical name of the Memcached instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

create_instance(location, instance_id, instance, project_id, retry=DEFAULT, timeout=None, metadata=())[source]

Create a Memcached instance based on the specified tier and memory size.

By default, the instance is accessible from the project’s default network.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str) –

    Required. The logical name of the Memcached instance in the customer project with the following restrictions:

    • Must contain only lowercase letters, numbers, and hyphens.

    • Must start with a letter.

    • Must be between 1-40 characters.

    • Must end with a number or a letter.

    • Must be unique within the customer project / location

  • instance (dict | google.cloud.memcache_v1beta2.types.cloud_memcache.Instance) –

    Required. A Memcached [Instance] resource

    If a dict is provided, it must be of the same form as the protobuf message Instance

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

delete_instance(location, instance, project_id, retry=DEFAULT, timeout=None, metadata=())[source]

Delete a specific Memcached instance. Instance stops serving and data is deleted.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Memcached instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

get_instance(location, instance, project_id, retry=DEFAULT, timeout=None, metadata=())[source]

Get the details of a specific Memcached instance.

Parameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance (str) – The logical name of the Memcached instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

list_instances(location, project_id, retry=DEFAULT, timeout=None, metadata=())[source]

List Memcached instances owned by a project at the specified location (region) or all locations.

Parameters

  • location (str) –

    The location of the Cloud Memorystore instance (for example europe-west1)

    If it is specified as - (wildcard), then all regions available to the project are queried, and the results are aggregated.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

update_instance(update_mask, instance, project_id, location=None, instance_id=None, retry=DEFAULT, timeout=None, metadata=())[source]

Update the metadata and configuration of a specific Memcached instance.

Parameters

  • update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) –

    Required. Mask of fields to update. At least one path must be supplied in this field. The elements of the repeated paths field may only include these fields from Instance:

    • displayName

    If a dict is provided, it must be of the same form as the protobuf message FieldMask) Union[dict, google.protobuf.field_mask_pb2.FieldMask]

  • instance (dict | google.cloud.memcache_v1beta2.types.cloud_memcache.Instance) –

    Required. Update description. Only fields specified in update_mask are updated.

    If a dict is provided, it must be of the same form as the protobuf message Instance

  • location (str | None) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str | None) – The logical name of the Memcached instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

update_parameters(update_mask, parameters, project_id, location, instance_id, retry=DEFAULT, timeout=None, metadata=())[source]

Update the defined Memcached Parameters for an existing Instance.

This method only stages the parameters, it must be followed by apply_parameters to apply the parameters to nodes of the Memcached Instance.

Parameters

  • update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) – Required. Mask of fields to update. If a dict is provided, it must be of the same form as the protobuf message FieldMask

  • parameters (dict | google.cloud.memcache_v1beta2.types.cloud_memcache.MemcacheParameters) – The parameters to apply to the instance. If a dict is provided, it must be of the same form as the protobuf message MemcacheParameters

  • location (str) – The location of the Cloud Memorystore instance (for example europe-west1)

  • instance_id (str) – The logical name of the Memcached instance in the customer project.

  • project_id (str) – Project ID of the project that contains the instance. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.

Was this entry helpful?