airflow.providers.amazon.aws.hooks.base_aws

This module contains Base AWS Hook.

See also

For more information on how to use this hook, take a look at the guide: Amazon Web Services Connection

Module Contents

Classes

BaseSessionFactory

Base AWS Session Factory class to handle boto3 session creation.

AwsGenericHook

Interact with AWS.

AwsBaseHook

Interact with AWS.

Functions

resolve_session_factory()

Resolves custom SessionFactory class

Attributes

BaseAwsConnection

SessionFactory

airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection[source]
class airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory(conn, region_name, config)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Base AWS Session Factory class to handle boto3 session creation. It can handle most of the AWS supported authentication methods.

User can also derive from this class to have full control of boto3 session creation or to support custom federation.

See also

Session Factory

create_session(self)[source]

Create AWS session.

class airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]

Bases: airflow.hooks.base.BaseHook, Generic[BaseAwsConnection]

Interact with AWS. This class is a thin wrapper around the boto3 python library.

Parameters
  • aws_conn_id (Optional[str]) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • verify (Union[bool, str, None]) – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • region_name (Optional[str]) – AWS region_name. If not specified then the default boto3 behaviour is used.

  • client_type (Optional[str]) – boto3.client client_type. Eg ‘s3’, ‘emr’ etc

  • resource_type (Optional[str]) – boto3.resource resource_type. Eg ‘dynamodb’ etc

  • config (Optional[botocore.config.Config]) – Configuration for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

conn_name_attr = aws_conn_id[source]
default_conn_name = aws_default[source]
conn_type = aws[source]
hook_name = Amazon Web Services[source]
get_client_type(self, client_type=None, region_name=None, config=None)[source]

Get the underlying boto3 client using boto3 session

get_resource_type(self, resource_type=None, region_name=None, config=None)[source]

Get the underlying boto3 resource using boto3 session

conn(self)[source]

Get the underlying boto3 client/resource (cached)

Returns

boto3.client or boto3.resource

Return type

Union[boto3.client, boto3.resource]

conn_client_meta(self)[source]
property conn_region_name(self)[source]
property conn_partition(self)[source]
get_conn(self)[source]

Get the underlying boto3 client/resource (cached)

Implemented so that caching works as intended. It exists for compatibility with subclasses that rely on a super().get_conn() method.

Returns

boto3.client or boto3.resource

Return type

Union[boto3.client, boto3.resource]

get_session(self, region_name=None)[source]

Get the underlying boto3.session.

get_credentials(self, region_name=None)[source]

Get the underlying botocore.Credentials object.

This contains the following authentication attributes: access_key, secret_key and token.

expand_role(self, role, region_name=None)[source]

If the IAM role is a role name, get the Amazon Resource Name (ARN) for the role. If IAM role is already an IAM role ARN, no change is made.

Parameters
  • role (str) – IAM role name or ARN

  • region_name (Optional[str]) – Optional region name to get credentials for

Returns

IAM role ARN

Return type

str

static retry(should_retry)[source]

A decorator that provides a mechanism to repeat requests in response to exceeding a temporary quote limit.

test_connection(self)[source]

Tests the AWS connection by call AWS STS (Security Token Service) GetCallerIdentity API.

class airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]

Bases: AwsGenericHook[Union[boto3.client, boto3.resource]]

Interact with AWS. This class is a thin wrapper around the boto3 python library with basic conn annotation.

See also

AwsGenericHook

airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory()[source]

Resolves custom SessionFactory class

airflow.providers.amazon.aws.hooks.base_aws.SessionFactory[source]

Was this entry helpful?