• Home
• airflow.providers.amazon
• airflow.providers.amazon.aws
• airflow.providers.amazon.aws.operators
• airflow.providers.amazon.aws.operators.eks

airflow.providers.amazon.aws.operators.eks

This module contains Amazon EKS operators.

Module Contents

Classes

EksCreateClusterOperator	Creates an Amazon EKS Cluster control plane.
EksCreateNodegroupOperator	Creates an Amazon EKS managed node group for an existing Amazon EKS Cluster.
EksCreateFargateProfileOperator	Creates an AWS Fargate profile for an Amazon EKS cluster.
EksDeleteClusterOperator	Deletes the Amazon EKS Cluster control plane and all nodegroups attached to it.
EksDeleteNodegroupOperator	Deletes an Amazon EKS managed node group from an Amazon EKS Cluster.
EksDeleteFargateProfileOperator	Deletes an AWS Fargate profile from an Amazon EKS Cluster.
EksPodOperator	Executes a task in a Kubernetes pod on the specified Amazon EKS Cluster.
EKSCreateClusterOperator	This operator is deprecated.
EKSCreateNodegroupOperator	This operator is deprecated.
EKSCreateFargateProfileOperator	This operator is deprecated.
EKSDeleteClusterOperator	This operator is deprecated.
EKSDeleteNodegroupOperator	This operator is deprecated.
EKSDeleteFargateProfileOperator	This operator is deprecated.
EKSPodOperator	This operator is deprecated.

Attributes

CHECK_INTERVAL_SECONDS
TIMEOUT_SECONDS
DEFAULT_COMPUTE_TYPE
DEFAULT_CONN_ID
DEFAULT_FARGATE_PROFILE_NAME
DEFAULT_NAMESPACE_NAME
DEFAULT_NODEGROUP_NAME
DEFAULT_POD_NAME
ABORT_MSG
CAN_NOT_DELETE_MSG
MISSING_ARN_MSG
SUCCESS_MSG
SUPPORTED_COMPUTE_VALUES
NODEGROUP_FULL_NAME
FARGATE_FULL_NAME

airflow.providers.amazon.aws.operators.eks.CHECK_INTERVAL_SECONDS = 15

airflow.providers.amazon.aws.operators.eks.TIMEOUT_SECONDS

airflow.providers.amazon.aws.operators.eks.DEFAULT_COMPUTE_TYPE = nodegroup

airflow.providers.amazon.aws.operators.eks.DEFAULT_CONN_ID = aws_default

airflow.providers.amazon.aws.operators.eks.DEFAULT_FARGATE_PROFILE_NAME = profile

airflow.providers.amazon.aws.operators.eks.DEFAULT_NAMESPACE_NAME = default

airflow.providers.amazon.aws.operators.eks.DEFAULT_NODEGROUP_NAME = nodegroup

airflow.providers.amazon.aws.operators.eks.DEFAULT_POD_NAME = pod

airflow.providers.amazon.aws.operators.eks.ABORT_MSG = {compute} are still active after the allocated time limit.  Aborting.

airflow.providers.amazon.aws.operators.eks.CAN_NOT_DELETE_MSG = A cluster can not be deleted with attached {compute}.  Deleting {count} {compute}.

airflow.providers.amazon.aws.operators.eks.MISSING_ARN_MSG = Creating an {compute} requires {requirement} to be passed in.

airflow.providers.amazon.aws.operators.eks.SUCCESS_MSG = No {compute} remain, deleting cluster.

airflow.providers.amazon.aws.operators.eks.SUPPORTED_COMPUTE_VALUES

airflow.providers.amazon.aws.operators.eks.NODEGROUP_FULL_NAME = Amazon EKS managed node groups

airflow.providers.amazon.aws.operators.eks.FARGATE_FULL_NAME = AWS Fargate profiles

class airflow.providers.amazon.aws.operators.eks.EksCreateClusterOperator(cluster_name: str, cluster_role_arn: str, resources_vpc_config: Dict, compute: Optional[str] = DEFAULT_COMPUTE_TYPE, nodegroup_name: Optional[str] = DEFAULT_NODEGROUP_NAME, nodegroup_role_arn: Optional[str] = None, fargate_profile_name: Optional[str] = DEFAULT_FARGATE_PROFILE_NAME, fargate_pod_execution_role_arn: Optional[str] = None, fargate_selectors: Optional[List] = None, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Creates an Amazon EKS Cluster control plane.

Optionally, can also create the supporting compute architecture:

• If argument 'compute' is provided with a value of 'nodegroup', will also

  attempt to create an Amazon EKS Managed Nodegroup for the cluster. See EksCreateNodegroupOperator documentation for requirements.

• If argument 'compute' is provided with a value of 'fargate', will also attempt to create an AWS

  Fargate profile for the cluster. See EksCreateFargateProfileOperator documentation for requirements.

See also

For more information on how to use this operator, take a look at the guide: Create an Amazon EKS Cluster

Parameters

• cluster_name (str) -- The unique name to give to your Amazon EKS Cluster. (templated)

• cluster_role_arn (str) -- The Amazon Resource Name (ARN) of the IAM role that provides permissions for the Kubernetes control plane to make calls to AWS API operations on your behalf. (templated)

• resources_vpc_config (Dict) -- The VPC configuration used by the cluster control plane. (templated)

• compute (str) -- The type of compute architecture to generate along with the cluster. (templated) Defaults to 'nodegroup' to generate an EKS Managed Nodegroup.

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

If compute is assigned the value of 'nodegroup', the following are required:

Parameters

• nodegroup_name (str) -- The unique name to give your Amazon EKS managed node group. (templated)

• nodegroup_role_arn (str) -- The Amazon Resource Name (ARN) of the IAM role to associate with the Amazon EKS managed node group. (templated)

If compute is assigned the value of 'fargate', the following are required:

Parameters

• fargate_profile_name (str) -- The unique name to give your AWS Fargate profile. (templated)

• fargate_pod_execution_role_arn -- The Amazon Resource Name (ARN) of the pod execution role to use for pods that match the selectors in the AWS Fargate profile. (templated)

• selectors (List) -- The selectors to match for pods to use this AWS Fargate profile. (templated)

template_fields :Sequence[str] = ['cluster_name', 'cluster_role_arn', 'resources_vpc_config', 'compute', 'nodegroup_name',...

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EksCreateNodegroupOperator(cluster_name: str, nodegroup_subnets: List[str], nodegroup_role_arn: str, nodegroup_name: Optional[str] = DEFAULT_NODEGROUP_NAME, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Creates an Amazon EKS managed node group for an existing Amazon EKS Cluster.

See also

For more information on how to use this operator, take a look at the guide: Create an Amazon EKS Managed NodeGroup

Parameters

• cluster_name (str) -- The name of the Amazon EKS Cluster to create the managed nodegroup in. (templated)

• nodegroup_name (str) -- The unique name to give your managed nodegroup. (templated)

• nodegroup_subnets (List[str]) -- The subnets to use for the Auto Scaling group that is created for the managed nodegroup. (templated)

• nodegroup_role_arn (str) -- The Amazon Resource Name (ARN) of the IAM role to associate with the managed nodegroup. (templated)

• aws_conn_id (str :param region: Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

template_fields :Sequence[str] = ['cluster_name', 'nodegroup_subnets', 'nodegroup_role_arn', 'nodegroup_name', 'aws_conn_id', 'region']

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EksCreateFargateProfileOperator(cluster_name: str, pod_execution_role_arn: str, selectors: List, fargate_profile_name: Optional[str] = DEFAULT_FARGATE_PROFILE_NAME, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Creates an AWS Fargate profile for an Amazon EKS cluster.

See also

For more information on how to use this operator, take a look at the guide: Create an AWS Fargate Profile

Parameters

• cluster_name (str) -- The name of the Amazon EKS cluster to apply the AWS Fargate profile to. (templated)

• pod_execution_role_arn (str) -- The Amazon Resource Name (ARN) of the pod execution role to use for pods that match the selectors in the AWS Fargate profile. (templated)

• selectors (List) -- The selectors to match for pods to use this AWS Fargate profile. (templated)

• fargate_profile_name (str) -- The unique name to give your AWS Fargate profile. (templated)

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

template_fields :Sequence[str] = ['cluster_name', 'pod_execution_role_arn', 'selectors', 'fargate_profile_name', 'aws_conn_id', 'region']

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EksDeleteClusterOperator(cluster_name: str, force_delete_compute: bool = False, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Deletes the Amazon EKS Cluster control plane and all nodegroups attached to it.

See also

For more information on how to use this operator, take a look at the guide: Delete an Amazon EKS Cluster

Parameters

• cluster_name (str) -- The name of the Amazon EKS Cluster to delete. (templated)

• force_delete_compute (bool) -- If True, will delete any attached resources. (templated) Defaults to False.

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

template_fields :Sequence[str] = ['cluster_name', 'force_delete_compute', 'aws_conn_id', 'region']

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

delete_any_nodegroups(self, eks_hook) → None

Deletes all Amazon EKS managed node groups for a provided Amazon EKS Cluster.

Amazon EKS managed node groups can be deleted in parallel, so we can send all of the delete commands in bulk and move on once the count of nodegroups is zero.

delete_any_fargate_profiles(self, eks_hook) → None

Deletes all EKS Fargate profiles for a provided Amazon EKS Cluster.

EKS Fargate profiles must be deleted one at a time, so we must wait for one to be deleted before sending the next delete command.

class airflow.providers.amazon.aws.operators.eks.EksDeleteNodegroupOperator(cluster_name: str, nodegroup_name: str, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Deletes an Amazon EKS managed node group from an Amazon EKS Cluster.

See also

For more information on how to use this operator, take a look at the guide: Delete an Amazon EKS Managed Nodegroup

Parameters

• cluster_name (str) -- The name of the Amazon EKS Cluster associated with your nodegroup. (templated)

• nodegroup_name (str) -- The name of the nodegroup to delete. (templated)

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

template_fields :Sequence[str] = ['cluster_name', 'nodegroup_name', 'aws_conn_id', 'region']

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EksDeleteFargateProfileOperator(cluster_name: str, fargate_profile_name: str, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, **kwargs)

Bases: airflow.models.BaseOperator

Deletes an AWS Fargate profile from an Amazon EKS Cluster.

See also

For more information on how to use this operator, take a look at the guide: Delete an AWS Fargate Profile

Parameters

• cluster_name (str) -- The name of the Amazon EKS cluster associated with your Fargate profile. (templated)

• fargate_profile_name (str) -- The name of the AWS Fargate profile to delete. (templated)

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

template_fields :Sequence[str] = ['cluster_name', 'fargate_profile_name', 'aws_conn_id', 'region']

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EksPodOperator(cluster_name: str, in_cluster: bool = False, namespace: str = DEFAULT_NAMESPACE_NAME, pod_context: Optional[str] = None, pod_name: Optional[str] = None, pod_username: Optional[str] = None, aws_conn_id: str = DEFAULT_CONN_ID, region: Optional[str] = None, is_delete_operator_pod: Optional[bool] = None, **kwargs)

Bases: airflow.providers.cncf.kubernetes.operators.kubernetes_pod.KubernetesPodOperator

Executes a task in a Kubernetes pod on the specified Amazon EKS Cluster.

See also

For more information on how to use this operator, take a look at the guide: Perform a Task on an Amazon EKS Cluster

Parameters

• cluster_name (str) -- The name of the Amazon EKS Cluster to execute the task on. (templated)

• cluster_role_arn (str) -- The Amazon Resource Name (ARN) of the IAM role that provides permissions for the Kubernetes control plane to make calls to AWS API operations on your behalf. (templated)

• in_cluster (bool) -- If True, look for config inside the cluster; if False look for a local file path.

• namespace (str) -- The namespace in which to execute the pod. (templated)

• pod_name (str) -- The unique name to give the pod. (templated)

• aws_profile -- The named profile containing the credentials for the AWS CLI tool to use.

• aws_profile -- str

• region (str) -- Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

• aws_conn_id (str) -- The Airflow connection used for AWS credentials. (templated) If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then the default boto3 configuration would be used (and must be maintained on each worker node).

• is_delete_operator_pod (bool) -- What to do when the pod reaches its final state, or the execution is interrupted. If True, delete the pod; if False, leave the pod. Current default is False, but this will be changed in the next major release of this provider.

template_fields :Sequence[str]

execute(self, context: airflow.utils.context.Context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.eks.EKSCreateClusterOperator(*args, **kwargs)

Bases: EksCreateClusterOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksCreateClusterOperator.

class airflow.providers.amazon.aws.operators.eks.EKSCreateNodegroupOperator(*args, **kwargs)

Bases: EksCreateNodegroupOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksCreateNodegroupOperator.

class airflow.providers.amazon.aws.operators.eks.EKSCreateFargateProfileOperator(*args, **kwargs)

Bases: EksCreateFargateProfileOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksCreateFargateProfileOperator.

class airflow.providers.amazon.aws.operators.eks.EKSDeleteClusterOperator(*args, **kwargs)

Bases: EksDeleteClusterOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksDeleteClusterOperator.

class airflow.providers.amazon.aws.operators.eks.EKSDeleteNodegroupOperator(*args, **kwargs)

Bases: EksDeleteNodegroupOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksDeleteNodegroupOperator.

class airflow.providers.amazon.aws.operators.eks.EKSDeleteFargateProfileOperator(*args, **kwargs)

Bases: EksDeleteFargateProfileOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksDeleteFargateProfileOperator.

class airflow.providers.amazon.aws.operators.eks.EKSPodOperator(*args, **kwargs)

Bases: EksPodOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.eks.EksPodOperator.

Previous Next

Was this entry helpful?