airflow.contrib.hooks.gcp_container_hook

Module Contents

airflow.contrib.hooks.gcp_container_hook.OPERATIONAL_POLL_INTERVAL = 15

class airflow.contrib.hooks.gcp_container_hook.GKEClusterHook(gcp_conn_id='google_cloud_default', delegate_to=None, location=None)

Bases:airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook

get_client(self)

static _dict_to_proto(py_dict, proto)

Converts a python dictionary to the proto supplied

Parameters

• py_dict (dict) – The dictionary to convert

• proto (protobuf) – The proto object to merge with dictionary

Returns

A parsed python dictionary in provided proto format

Raises

ParseError: On JSON parsing problems.

wait_for_operation(self, operation, project_id=None)

Given an operation, continuously fetches the status from Google Cloud until either completion or an error occurring

Parameters

• operation (google.cloud.container_V1.gapic.enums.Operation) – The Operation to wait for

• project_id (str) – Google Cloud Platform project ID

Returns

A new, updated operation fetched from Google Cloud

get_operation(self, operation_name, project_id=None)

Fetches the operation from Google Cloud

Parameters

• operation_name (str) – Name of operation to fetch

• project_id (str) – Google Cloud Platform project ID

Returns

The new, updated operation from Google Cloud

static _append_label(cluster_proto, key, val)

Append labels to provided Cluster Protobuf

Labels must fit the regex [a-z]([-a-z0-9]*[a-z0-9])? (current

airflow version string follows semantic versioning spec: x.y.z).

Parameters

• cluster_proto (google.cloud.container_v1.types.Cluster) – The proto to append resource_label airflow version to

• key (str) – The key label

• val (str) –

Returns

The cluster proto updated with new label

delete_cluster(self, name, project_id=None, retry=DEFAULT, timeout=DEFAULT)

Deletes the cluster, including the Kubernetes endpoint and all worker nodes. Firewalls and routes that were configured during cluster creation are also deleted. Other Google Compute Engine resources that might be in use by the cluster (e.g. load balancer resources) will not be deleted if they weren’t present at the initial create time.

Parameters

• name (str) – The name of the cluster to delete

• project_id (str) – Google Cloud Platform project ID

• retry (google.api_core.retry.Retry) – Retry object used to determine when/if to retry requests. If None is specified, requests will not be retried.

• timeout (float) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

Returns

The full url to the delete operation if successful, else None

create_cluster(self, cluster, project_id=None, retry=DEFAULT, timeout=DEFAULT)

Creates a cluster, consisting of the specified number and type of Google Compute Engine instances.

Parameters

• cluster (dict or google.cloud.container_v1.types.Cluster) – A Cluster protobuf or dict. If dict is provided, it must be of the same form as the protobuf message google.cloud.container_v1.types.Cluster

• project_id (str) – Google Cloud Platform project ID

• retry (google.api_core.retry.Retry) – A retry object (google.api_core.retry.Retry) used to retry requests. If None is specified, requests will not be retried.

• timeout (float) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

Returns

The full url to the new, or existing, cluster

Raises

ParseError: On JSON parsing problems when trying to convert dict AirflowException: cluster is not dict type nor Cluster proto type

get_cluster(self, name, project_id=None, retry=DEFAULT, timeout=DEFAULT)

Gets details of specified cluster

Parameters

• name (str) – The name of the cluster to retrieve

• project_id (str) – Google Cloud Platform project ID

• retry (google.api_core.retry.Retry) – A retry object used to retry requests. If None is specified, requests will not be retried.

• timeout (float) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

Returns

google.cloud.container_v1.types.Cluster