FTP to Amazon S3¶

Use the FTPToS3Operator transfer to copy data from a FTP server to an Amazon Simple Storage Service (S3) file.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation of Airflow®
Setup Connection.

Operators¶

FTP to Amazon S3 transfer operator¶

This operator copies data from a FTP server to an Amazon S3 file.

To get more information about this operator visit: FTPToS3Operator

Example usage:

tests/system/amazon/aws/example_ftp_to_s3.py

ftp_to_s3_task = FTPToS3Operator(
    task_id="ftp_to_s3_task",
    ftp_path="/tmp/ftp_path",
    s3_bucket=s3_bucket,
    s3_key=s3_key,
    replace=True,
)

Reference¶

AWS boto3 library documentation for Amazon S3