AWS: Amazon Web Services¶
Airflow has support for Amazon Web Services.
All hooks are based on airflow.providers.amazon.aws.hooks.base_aws
.
Service operators and hooks¶
These integrations allow you to perform various operations within the Amazon Web Services.
Amazon Athena¶
- Operators
- Hooks
- Provider
- Product documentation
Amazon CloudFormation¶
- Operators
- Hooks
- Provider
- Product documentation
Amazon CloudWatch Logs¶
- Hooks
- Provider
- Product documentation
Amazon DataSync¶
- Operators
- Hooks
- Guides
- Provider
- Product documentation
Amazon DynamoDB¶
- Hooks
airflow.providers.amazon.aws.hooks.dynamodb
,airflow.providers.amazon.aws.hooks.aws_dynamodb
.- Provider
- Product documentation
Amazon EC2¶
- Operators
airflow.providers.amazon.aws.operators.ec2_start_instance
,airflow.providers.amazon.aws.operators.ec2_stop_instance
.- Hooks
- Provider
- Product documentation
Amazon ECS¶
- Operators
- Provider
- Product documentation
Amazon ElastiCache¶
- Hooks
airflow.providers.amazon.aws.hooks.elasticache_replication_group
.- Provider
- Product documentation
Amazon EMR¶
- Operators
airflow.providers.amazon.aws.operators.emr_add_steps
,airflow.providers.amazon.aws.operators.emr_create_job_flow
,airflow.providers.amazon.aws.operators.emr_modify_cluster
,airflow.providers.amazon.aws.operators.emr_terminate_job_flow
.- Hooks
- Guides
- Provider
- Product documentation
Amazon Glacier¶
- Operators
- Hooks
- Guides
- Provider
- Product documentation
Amazon Kinesis Data Firehose¶
- Hooks
- Provider
- Product documentation
Amazon Redshift¶
- Hooks
- Provider
- Product documentation
Amazon SageMaker¶
- Operators
airflow.providers.amazon.aws.operators.sagemaker_base
,airflow.providers.amazon.aws.operators.sagemaker_endpoint
,airflow.providers.amazon.aws.operators.sagemaker_endpoint_config
,airflow.providers.amazon.aws.operators.sagemaker_model
,airflow.providers.amazon.aws.operators.sagemaker_processing
,airflow.providers.amazon.aws.operators.sagemaker_training
,airflow.providers.amazon.aws.operators.sagemaker_transform
,airflow.providers.amazon.aws.operators.sagemaker_tuning
.- Hooks
- Provider
- Product documentation
Amazon SecretsManager¶
- Hooks
- Provider
- Product documentation
Amazon Simple Email Service (SES)¶
- Hooks
- Provider
- Product documentation
Amazon Simple Notification Service (SNS)¶
- Operators
- Hooks
- Provider
- Product documentation
Amazon Simple Queue Service (SQS)¶
- Operators
- Hooks
- Provider
- Product documentation
Amazon Simple Storage Service (S3)¶
- Operators
airflow.providers.amazon.aws.operators.s3_bucket
,airflow.providers.amazon.aws.operators.s3_bucket_tagging
,airflow.providers.amazon.aws.operators.s3_copy_object
,airflow.providers.amazon.aws.operators.s3_delete_objects
,airflow.providers.amazon.aws.operators.s3_file_transform
,airflow.providers.amazon.aws.operators.s3_list
.- Hooks
- Guides
- Provider
- Product documentation
Amazon Web Services¶
- Hooks
- Provider
- Product documentation
AWS Batch¶
- Operators
- Hooks
airflow.providers.amazon.aws.hooks.batch_client
,airflow.providers.amazon.aws.hooks.batch_waiters
.- Provider
- Product documentation
AWS Glue¶
AWS Lambda¶
- Hooks
- Provider
- Product documentation
AWS Step Functions¶
Transfer operators and hooks¶
These integrations allow you to copy data from/to Amazon Web Services.
Amazon DynamoDB to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Google Cloud Storage (GCS) to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Glacier to Google Cloud Storage (GCS)¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
- Provider
Google to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
- Provider
Apache Hive to Amazon DynamoDB¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Internet Message Access Protocol (IMAP) to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
airflow.providers.amazon.aws.transfers.imap_attachment_to_s3
- Provider
MongoDB to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
MySQL to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Redshift to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Simple Storage Service (S3) to Amazon Redshift¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
- Provider
Amazon Simple Storage Service (S3) to SSH File Transfer Protocol (SFTP)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
SSH File Transfer Protocol (SFTP) to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Simple Storage Service (S3) to File Transfer Protocol (FTP)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Exasol to Amazon Simple Storage Service (S3)¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Simple Storage Service (S3) to Apache Hive¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Simple Storage Service (S3) to Google Cloud Storage (GCS)¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
- Provider
Amazon Simple Storage Service (S3) to MySQL¶
- Source product documentation
- Target product documentation
- Python API
- Provider
Amazon Simple Storage Service (S3) to Snowflake¶
- Source product documentation
- Target product documentation
- Operator guide
- Python API
- Provider