airflow.contrib.hooks.gcp_bigtable_hook

Module Contents

class airflow.contrib.hooks.gcp_bigtable_hook.BigtableHook(gcp_conn_id='google_cloud_default', delegate_to=None)[source]

Bases: airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook

Hook for Google Cloud Bigtable APIs.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

_client[source]
_get_client(self, project_id)[source]
get_instance(self, instance_id, project_id=None)[source]

Retrieves and returns the specified Cloud Bigtable instance if it exists. Otherwise, returns None.

Parameters

  • instance_id (str) – The ID of the Cloud Bigtable instance.

  • project_id (str) – Optional, Google Cloud Platform project ID where the BigTable exists. If set to None or missing, the default project_id from the GCP connection is used.

delete_instance(self, instance_id, project_id=None)[source]

Deletes the specified Cloud Bigtable instance. Raises google.api_core.exceptions.NotFound if the Cloud Bigtable instance does not exist.

Parameters

  • project_id (str) – Optional, Google Cloud Platform project ID where the BigTable exists. If set to None or missing, the default project_id from the GCP connection is used.

  • instance_id (str) – The ID of the Cloud Bigtable instance.

create_instance(self, instance_id, main_cluster_id, main_cluster_zone, project_id=None, replica_cluster_id=None, replica_cluster_zone=None, instance_display_name=None, instance_type=enums.Instance.Type.TYPE_UNSPECIFIED, instance_labels=None, cluster_nodes=None, cluster_storage_type=enums.StorageType.STORAGE_TYPE_UNSPECIFIED, timeout=None)[source]

Creates new instance.

Parameters

  • instance_id (str) – The ID for the new instance.

  • main_cluster_id (str) – The ID for main cluster for the new instance.

  • main_cluster_zone (str) – The zone for main cluster. See https://cloud.google.com/bigtable/docs/locations for more details.

  • project_id (str) – Optional, Google Cloud Platform project ID where the BigTable exists. If set to None or missing, the default project_id from the GCP connection is used.

  • replica_cluster_id (str) – (optional) The ID for replica cluster for the new instance.

  • replica_cluster_zone (str) – (optional) The zone for replica cluster.

  • instance_type (enums.Instance.Type) – (optional) The type of the instance.

  • instance_display_name (str) – (optional) Human-readable name of the instance. Defaults to instance_id.

  • instance_labels (dict) – (optional) Dictionary of labels to associate with the instance.

  • cluster_nodes (int) – (optional) Number of nodes for cluster.

  • cluster_storage_type (enums.StorageType) – (optional) The type of storage.

  • timeout (int) – (optional) timeout (in seconds) for instance creation. If None is not specified, Operator will wait indefinitely.

static create_table(instance, table_id, initial_split_keys=None, column_families=None)[source]

Creates the specified Cloud Bigtable table. Raises google.api_core.exceptions.AlreadyExists if the table exists.

Parameters

  • instance (Instance) – The Cloud Bigtable instance that owns the table.

  • table_id (str) – The ID of the table to create in Cloud Bigtable.

  • initial_split_keys (list) – (Optional) A list of row keys in bytes to use to initially split the table.

  • column_families (dict) – (Optional) A map of columns to create. The key is the column_id str, and the value is a google.cloud.bigtable.column_family.GarbageCollectionRule.

delete_table(self, instance_id, table_id, project_id=None)[source]

Deletes the specified table in Cloud Bigtable. Raises google.api_core.exceptions.NotFound if the table does not exist.

Parameters

  • instance_id (str) – The ID of the Cloud Bigtable instance.

  • table_id (str) – The ID of the table in Cloud Bigtable.

  • project_id (str) – Optional, Google Cloud Platform project ID where the BigTable exists. If set to None or missing, the default project_id from the GCP connection is used.

static update_cluster(instance, cluster_id, nodes)[source]

Updates number of nodes in the specified Cloud Bigtable cluster. Raises google.api_core.exceptions.NotFound if the cluster does not exist.

Parameters

  • instance (Instance) – The Cloud Bigtable instance that owns the cluster.

  • cluster_id (str) – The ID of the cluster.

  • nodes (int) – The desired number of nodes.

static get_column_families_for_table(instance, table_id)[source]

Fetches Column Families for the specified table in Cloud Bigtable.

Parameters

  • instance (Instance) – The Cloud Bigtable instance that owns the table.

  • table_id (str) – The ID of the table in Cloud Bigtable to fetch Column Families from.

static get_cluster_states_for_table(instance, table_id)[source]

Fetches Cluster States for the specified table in Cloud Bigtable. Raises google.api_core.exceptions.NotFound if the table does not exist.

Parameters

  • instance (Instance) – The Cloud Bigtable instance that owns the table.

  • table_id (str) – The ID of the table in Cloud Bigtable to fetch Cluster States from.