`airflow.providers.amazon.aws.hooks.base_aws`¶

This module contains Base AWS Hook.

Module Contents¶

Classes¶

`BaseSessionFactory`	Base AWS Session Factory class to handle boto3 session creation.
`AwsGenericHook`	Interact with AWS.
`AwsBaseHook`	Interact with AWS.

Functions¶

resolve_session_factory()

Resolves custom SessionFactory class

Attributes¶

`BaseAwsConnection`
`SessionFactory`

airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection[source]¶

class airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory(conn, region_name=None, config=None)[source]¶

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Base AWS Session Factory class to handle boto3 session creation. It can handle most of the AWS supported authentication methods.

User can also derive from this class to have full control of boto3 session creation or to support custom federation.

See also

Session Factory

conn()[source]¶

Cached AWS Connection Wrapper.

basic_session()[source]¶

Cached property with basic boto3.session.Session.

property extra_config[source]¶

AWS Connection extra_config.

property region_name[source]¶

AWS Region Name read-only property.

property config[source]¶

Configuration for botocore client read-only property.

property role_arn[source]¶

Assume Role ARN from AWS Connection

create_session()[source]¶

Create boto3 Session from connection config.

class airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]¶

Bases: airflow.hooks.base.BaseHook, Generic[BaseAwsConnection]

Interact with AWS. This class is a thin wrapper around the boto3 python library.

Parameters

aws_conn_id (Optional[str]) -- The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).
verify (Optional[Union[bool, str]]) -- Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
region_name (Optional[str]) -- AWS region_name. If not specified then the default boto3 behaviour is used.
client_type (Optional[str]) -- boto3.client client_type. Eg 's3', 'emr' etc
resource_type (Optional[str]) -- boto3.resource resource_type. Eg 'dynamodb' etc
config (Optional[botocore.config.Config]) -- Configuration for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

conn_name_attr = aws_conn_id[source]¶

default_conn_name = aws_default[source]¶

conn_type = aws[source]¶

hook_name = Amazon Web Services[source]¶

conn_config()[source]¶

Get the Airflow Connection object and wrap it in helper (cached).

property region_name[source]¶

AWS Region Name read-only property.

property config[source]¶

Configuration for botocore client read-only property.

property verify[source]¶

Verify or not SSL certificates boto3 client/resource read-only property.

get_session(region_name=None)[source]¶

Get the underlying boto3.session.Session(region_name=region_name).

get_client_type(region_name=None, config=None)[source]¶

Get the underlying boto3 client using boto3 session

get_resource_type(region_name=None, config=None)[source]¶

Get the underlying boto3 resource using boto3 session

conn()[source]¶

Get the underlying boto3 client/resource (cached)

Returns: boto3.client or boto3.resource
Return type: Union[boto3.client, boto3.resource]

conn_client_meta()[source]¶

Get botocore client metadata from Hook connection (cached).

property conn_region_name[source]¶

Get actual AWS Region Name from Hook connection (cached).

property conn_partition[source]¶

Get associated AWS Region Partition from Hook connection (cached).

get_conn()[source]¶

Get the underlying boto3 client/resource (cached)

Implemented so that caching works as intended. It exists for compatibility with subclasses that rely on a super().get_conn() method.

Returns: boto3.client or boto3.resource
Return type: Union[boto3.client, boto3.resource]

get_credentials(region_name=None)[source]¶

Get the underlying botocore.Credentials object.

This contains the following authentication attributes: access_key, secret_key and token. By use this method also secret_key and token will mask in tasks logs.

expand_role(role, region_name=None)[source]¶

If the IAM role is a role name, get the Amazon Resource Name (ARN) for the role. If IAM role is already an IAM role ARN, no change is made.

Parameters

role (str) -- IAM role name or ARN
region_name (Optional[str]) -- Optional region name to get credentials for

Returns

IAM role ARN

Return type

str

static retry(should_retry)[source]¶

A decorator that provides a mechanism to repeat requests in response to exceeding a temporary quote limit.

static get_ui_field_behaviour()[source]¶

Returns custom UI field behaviour for AWS Connection.

test_connection()[source]¶: Tests the AWS connection by call AWS STS (Security Token Service) GetCallerIdentity API.

See also

https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html

class airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]¶

Bases: AwsGenericHook[Union[boto3.client, boto3.resource]]

Interact with AWS. This class is a thin wrapper around the boto3 python library with basic conn annotation.

airflow.providers.amazon.aws.hooks.base_aws¶