`airflow.contrib.operators.gcp_container_operator`¶

Module Contents¶

class airflow.contrib.operators.gcp_container_operator.GKEClusterDeleteOperator(project_id, name, location, gcp_conn_id='google_cloud_default', api_version='v2', *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Deletes the cluster, including the Kubernetes endpoint and all worker nodes.

To delete a certain cluster, you must specify the project_id, the name of the cluster, the location that the cluster is in, and the task_id.

Operator Creation:

operator = GKEClusterDeleteOperator(
            task_id='cluster_delete',
            project_id='my-project',
            location='cluster-location'
            name='cluster-name')

See also

For more detail about deleting clusters have a look at the reference: https://google-cloud-python.readthedocs.io/en/latest/container/gapic/v1/api.html#google.cloud.container_v1.ClusterManagerClient.delete_cluster

Parameters

project_id (str) – The Google Developers Console [project ID or project number]
name (str) – The name of the resource to delete, in this case cluster name
location (str) – The name of the Google Compute Engine zone in which the cluster resides.
gcp_conn_id (str) – The connection ID to use connecting to Google Cloud Platform.
api_version (str) – The api version to use

template_fields = ['project_id', 'gcp_conn_id', 'name', 'location', 'api_version'][source]¶

_check_input(self)[source]¶

execute(self, context)[source]¶

class airflow.contrib.operators.gcp_container_operator.GKEClusterCreateOperator(project_id, location, body=None, gcp_conn_id='google_cloud_default', api_version='v2', *args, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Create a Google Kubernetes Engine Cluster of specified dimensions The operator will wait until the cluster is created.

The minimum required to define a cluster to create is:

dict() ::

cluster_def = {‘name’: ‘my-cluster-name’,: ‘initial_node_count’: 1}

or

Cluster proto ::

from google.cloud.container_v1.types import Cluster

cluster_def = Cluster(name=’my-cluster-name’, initial_node_count=1)

Operator Creation:

operator = GKEClusterCreateOperator(
            task_id='cluster_create',
            project_id='my-project',
            location='my-location'
            body=cluster_def)

See also

For more detail on about creating clusters have a look at the reference: google.cloud.container_v1.types.Cluster

Parameters

project_id (str) – The Google Developers Console [project ID or project number]
location (str) – The name of the Google Compute Engine zone in which the cluster resides.
body (dict or google.cloud.container_v1.types.Cluster) – The Cluster definition to create, can be protobuf or python dict, if dict it must match protobuf message Cluster
gcp_conn_id (str) – The connection ID to use connecting to Google Cloud Platform.
api_version (str) – The api version to use

template_fields = ['project_id', 'gcp_conn_id', 'location', 'api_version', 'body'][source]¶

_check_input(self)[source]¶

execute(self, context)[source]¶

airflow.contrib.operators.gcp_container_operator.KUBE_CONFIG_ENV_VAR = KUBECONFIG[source]¶

class airflow.contrib.operators.gcp_container_operator.GKEPodOperator(project_id, location, cluster_name, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]¶

Bases: airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator

Executes a task in a Kubernetes pod in the specified Google Kubernetes Engine cluster

This Operator assumes that the system has gcloud installed and either has working default application credentials or has configured a connection id with a service account.

The minimum required to define a cluster to create are the variables task_id, project_id, location, cluster_name, name, namespace, and image

Operator Creation:

operator = GKEPodOperator(task_id='pod_op',
                          project_id='my-project',
                          location='us-central1-a',
                          cluster_name='my-cluster-name',
                          name='task-name',
                          namespace='default',
                          image='perl')

airflow.contrib.operators.gcp_container_operator¶

Module Contents¶

`airflow.contrib.operators.gcp_container_operator`¶