SFTP to Amazon S3¶

Use the SFTPToS3Operator transfer to copy the data from a SFTP server to an Amazon Simple Storage Service (S3) file.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

Operators¶

SFTP to Amazon S3 transfer operator¶

This operator loads data from a SFTP server to an Amazon S3 file.

To get more information about this operator visit: SFTPToS3Operator

Example usage:

airflow/providers/amazon/aws/example_dags/example_sftp_to_s3.py[source]

create_sftp_to_s3_job = SFTPToS3Operator(
    task_id="create_sftp_to_s3_job",
    sftp_path="/tmp/sftp_path",
    s3_bucket=S3_BUCKET,
    s3_key=S3_KEY,
)

Reference¶

AWS boto3 library documentation for Amazon S3