SFTP to Amazon S3¶

Use the SFTPToS3Operator transfer to copy the data from a SFTP server to an Amazon Simple Storage Service (S3) file. For more information about the service visits Amazon Transfer for SFTP API documentation.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

Operators¶

SFTP to Amazon S3 transfer operator¶

This operator loads data from a SFTP server to an Amazon S3 file.

To get more information about this operator visit: SFTPToS3Operator

Example usage:

airflow/providers/amazon/aws/example_dags/example_sftp_to_s3.py[source]

create_sftp_to_s3_job = SFTPToS3Operator(
    task_id="create_sftp_to_s3_job",
    sftp_path="/tmp/sftp_path",
    s3_bucket=S3_BUCKET,
    s3_key=S3_KEY,
)

Reference¶

AWS boto3 library documentation for Amazon S3