AWS: Amazon Web Services

Airflow has support for Amazon Web Services.

All hooks are based on airflow.providers.amazon.aws.hooks.base_aws.

Services

These integrations allow you to perform various operations within the Amazon Web Services.

Amazon CloudWatch Logs

Hooks

airflow.providers.amazon.aws.hooks.logs.

Provider

apache-airflow-providers-amazon

Product documentation

Amazon CloudWatch Logs

Amazon ECS

Operators

airflow.providers.amazon.aws.operators.ecs.

Provider

apache-airflow-providers-amazon

Product documentation

Amazon ECS

Amazon Kinesis Data Firehose

Hooks

airflow.providers.amazon.aws.hooks.kinesis.

Provider

apache-airflow-providers-amazon

Product documentation

Amazon Kinesis Data Firehose

Amazon Simple Email Service (SES)

Hooks

airflow.providers.amazon.aws.hooks.ses.

Provider

apache-airflow-providers-amazon

Product documentation

Amazon Simple Email Service (SES)

Transfers

These integrations allow you to copy data from/to Amazon Web Services.

Amazon DynamoDB to Amazon Simple Storage Service (S3)

Source product documentation

Amazon DynamoDB

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.dynamodb_to_s3

Provider

apache-airflow-providers-amazon

Google Cloud Storage (GCS) to Amazon Simple Storage Service (S3)

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.gcs_to_s3

Provider

apache-airflow-providers-amazon

Amazon Glacier to Google Cloud Storage (GCS)

Source product documentation

Amazon Glacier

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Amazon Glacier Transfer Operator

Python API

airflow.providers.amazon.aws.transfers.glacier_to_gcs

Provider

apache-airflow-providers-amazon

Google to Amazon Simple Storage Service (S3)

Source product documentation

Google

Target product documentation

Amazon Simple Storage Service (S3)

Operator guide

Google API To S3 Transfer

Python API

airflow.providers.amazon.aws.transfers.google_api_to_s3

Provider

apache-airflow-providers-amazon

Apache Hive to Amazon DynamoDB

Source product documentation

Apache Hive

Target product documentation

Amazon DynamoDB

Python API

airflow.providers.amazon.aws.transfers.hive_to_dynamodb

Provider

apache-airflow-providers-amazon

Internet Message Access Protocol (IMAP) to Amazon Simple Storage Service (S3)

Source product documentation

Internet Message Access Protocol (IMAP)

Target product documentation

Amazon Simple Storage Service (S3)

Operator guide

Imap Attachment To S3 Operator

Python API

airflow.providers.amazon.aws.transfers.imap_attachment_to_s3

Provider

apache-airflow-providers-amazon

MongoDB to Amazon Simple Storage Service (S3)

Source product documentation

MongoDB

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.mongo_to_s3

Provider

apache-airflow-providers-amazon

MySQL to Amazon Simple Storage Service (S3)

Source product documentation

MySQL

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.mysql_to_s3

Provider

apache-airflow-providers-amazon

Amazon Redshift to Amazon Simple Storage Service (S3)

Source product documentation

Amazon Redshift

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.redshift_to_s3

Provider

apache-airflow-providers-amazon

Amazon Simple Storage Service (S3) to Amazon Redshift

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

Amazon Redshift

Operator guide

S3 To Redshift Transfer Operator

Python API

airflow.providers.amazon.aws.transfers.s3_to_redshift

Provider

apache-airflow-providers-amazon

Amazon Simple Storage Service (S3) to SSH File Transfer Protocol (SFTP)

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

SSH File Transfer Protocol (SFTP)

Operator guide

Amazon S3 to SFTP Transfer Operator

Python API

airflow.providers.amazon.aws.transfers.s3_to_sftp

Provider

apache-airflow-providers-amazon

SSH File Transfer Protocol (SFTP) to Amazon Simple Storage Service (S3)

Source product documentation

SSH File Transfer Protocol (SFTP)

Target product documentation

Amazon Simple Storage Service (S3)

Operator guide

Amazon SFTP to S3 Transfer Operator

Python API

airflow.providers.amazon.aws.transfers.sftp_to_s3

Provider

apache-airflow-providers-amazon

Amazon Simple Storage Service (S3) to File Transfer Protocol (FTP)

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

File Transfer Protocol (FTP)

Python API

airflow.providers.amazon.aws.transfers.s3_to_ftp

Provider

apache-airflow-providers-amazon

Exasol to Amazon Simple Storage Service (S3)

Source product documentation

Exasol

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.exasol_to_s3

Provider

apache-airflow-providers-amazon

File Transfer Protocol (FTP) to Amazon Simple Storage Service (S3)

Source product documentation

File Transfer Protocol (FTP)

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.ftp_to_s3

Provider

apache-airflow-providers-amazon

Salesforce to Amazon Simple Storage Service (S3)

Source product documentation

Salesforce

Target product documentation

Amazon Simple Storage Service (S3)

Operator guide

Salesforce To S3 Operator

Python API

airflow.providers.amazon.aws.transfers.salesforce_to_s3

Provider

apache-airflow-providers-amazon

Local to Amazon Simple Storage Service (S3)

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.local_to_s3

Provider

apache-airflow-providers-amazon

Amazon Simple Storage Service (S3) to Apache Hive

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

Apache Hive

Python API

airflow.providers.apache.hive.transfers.s3_to_hive

Provider

apache-airflow-providers-apache-hive

Amazon Simple Storage Service (S3) to Google Cloud Storage (GCS)

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Transfer Data from Amazon S3 to Google Cloud Storage

Python API

airflow.providers.google.cloud.transfers.s3_to_gcs

Provider

apache-airflow-providers-google

Amazon Simple Storage Service (S3) to MySQL

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

MySQL

Python API

airflow.providers.mysql.transfers.s3_to_mysql

Provider

apache-airflow-providers-mysql

Amazon Simple Storage Service (S3) to Snowflake

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

Snowflake

Operator guide

S3ToSnowflakeOperator

Python API

airflow.providers.snowflake.transfers.s3_to_snowflake

Provider

apache-airflow-providers-snowflake

Was this entry helpful?