airflow.providers.google.cloud.hooks.bigtable

This module contains a Google Cloud Bigtable Hook.

Module Contents

class airflow.providers.google.cloud.hooks.bigtable.BigtableHook(gcp_conn_id: str = 'google_cloud_default', delegate_to: Optional[str] = None, impersonation_chain: Optional[Union[str, Sequence[str]]] = None)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Hook for Google Cloud Bigtable APIs.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

get_instance(self, instance_id: str, project_id: str)[source]

Retrieves and returns the specified Cloud Bigtable instance if it exists. Otherwise, returns None.

Parameters

  • instance_id (str) -- The ID of the Cloud Bigtable instance.

  • project_id (str) -- Optional, Google Cloud project ID where the BigTable exists. If set to None or missing, the default project_id from the Google Cloud connection is used.

delete_instance(self, instance_id: str, project_id: str)[source]

Deletes the specified Cloud Bigtable instance. Raises google.api_core.exceptions.NotFound if the Cloud Bigtable instance does not exist.

Parameters

  • project_id (str) -- Optional, Google Cloud project ID where the BigTable exists. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • instance_id (str) -- The ID of the Cloud Bigtable instance.

create_instance(self, instance_id: str, main_cluster_id: str, main_cluster_zone: str, project_id: str, replica_clusters: Optional[List[Dict[str, str]]] = None, replica_cluster_id: Optional[str] = None, replica_cluster_zone: Optional[str] = None, instance_display_name: Optional[str] = None, instance_type: enums.Instance.Type = enums.Instance.Type.TYPE_UNSPECIFIED, instance_labels: Optional[Dict] = None, cluster_nodes: Optional[int] = None, cluster_storage_type: enums.StorageType = enums.StorageType.STORAGE_TYPE_UNSPECIFIED, timeout: Optional[float] = None)[source]

Creates new instance.

Parameters

  • instance_id (str) -- The ID for the new instance.

  • main_cluster_id (str) -- The ID for main cluster for the new instance.

  • main_cluster_zone (str) -- The zone for main cluster. See https://cloud.google.com/bigtable/docs/locations for more details.

  • project_id (str) -- Optional, Google Cloud project ID where the BigTable exists. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • replica_clusters (List[Dict[str, str]]) -- (optional) A list of replica clusters for the new instance. Each cluster dictionary contains an id and a zone. Example: [{"id": "replica-1", "zone": "us-west1-a"}]

  • replica_cluster_id (str) -- (deprecated) The ID for replica cluster for the new instance.

  • replica_cluster_zone (str) -- (deprecated) The zone for replica cluster.

  • instance_type (enums.Instance.Type) -- (optional) The type of the instance.

  • instance_display_name (str) -- (optional) Human-readable name of the instance. Defaults to instance_id.

  • instance_labels (dict) -- (optional) Dictionary of labels to associate with the instance.

  • cluster_nodes (int) -- (optional) Number of nodes for cluster.

  • cluster_storage_type (enums.StorageType) -- (optional) The type of storage.

  • timeout (int) -- (optional) timeout (in seconds) for instance creation. If None is not specified, Operator will wait indefinitely.

update_instance(self, instance_id: str, project_id: str, instance_display_name: Optional[str] = None, instance_type: Optional[Union[enums.Instance.Type, enum.IntEnum]] = None, instance_labels: Optional[Dict] = None, timeout: Optional[float] = None)[source]

Update an existing instance.

Parameters

  • instance_id (str) -- The ID for the existing instance.

  • project_id (str) -- Optional, Google Cloud project ID where the BigTable exists. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • instance_display_name (str) -- (optional) Human-readable name of the instance.

  • instance_type (enums.Instance.Type or enum.IntEnum) -- (optional) The type of the instance.

  • instance_labels (dict) -- (optional) Dictionary of labels to associate with the instance.

  • timeout (int) -- (optional) timeout (in seconds) for instance update. If None is not specified, Operator will wait indefinitely.

static create_table(instance: Instance, table_id: str, initial_split_keys: Optional[List] = None, column_families: Optional[Dict[str, GarbageCollectionRule]] = None)[source]

Creates the specified Cloud Bigtable table. Raises google.api_core.exceptions.AlreadyExists if the table exists.

Parameters

  • instance (Instance) -- The Cloud Bigtable instance that owns the table.

  • table_id (str) -- The ID of the table to create in Cloud Bigtable.

  • initial_split_keys (list) -- (Optional) A list of row keys in bytes to use to initially split the table.

  • column_families (dict) -- (Optional) A map of columns to create. The key is the column_id str, and the value is a google.cloud.bigtable.column_family.GarbageCollectionRule.

delete_table(self, instance_id: str, table_id: str, project_id: str)[source]

Deletes the specified table in Cloud Bigtable. Raises google.api_core.exceptions.NotFound if the table does not exist.

Parameters

  • instance_id (str) -- The ID of the Cloud Bigtable instance.

  • table_id (str) -- The ID of the table in Cloud Bigtable.

  • project_id (str) -- Optional, Google Cloud project ID where the BigTable exists. If set to None or missing, the default project_id from the Google Cloud connection is used.

static update_cluster(instance: Instance, cluster_id: str, nodes: int)[source]

Updates number of nodes in the specified Cloud Bigtable cluster. Raises google.api_core.exceptions.NotFound if the cluster does not exist.

Parameters

  • instance (Instance) -- The Cloud Bigtable instance that owns the cluster.

  • cluster_id (str) -- The ID of the cluster.

  • nodes (int) -- The desired number of nodes.

static get_column_families_for_table(instance: Instance, table_id: str)[source]

Fetches Column Families for the specified table in Cloud Bigtable.

Parameters

  • instance (Instance) -- The Cloud Bigtable instance that owns the table.

  • table_id (str) -- The ID of the table in Cloud Bigtable to fetch Column Families from.

static get_cluster_states_for_table(instance: Instance, table_id: str)[source]

Fetches Cluster States for the specified table in Cloud Bigtable. Raises google.api_core.exceptions.NotFound if the table does not exist.

Parameters

  • instance (Instance) -- The Cloud Bigtable instance that owns the table.

  • table_id (str) -- The ID of the table in Cloud Bigtable to fetch Cluster States from.

Was this entry helpful?