IO Operators¶
These operators perform various operations on a filesystem or object storage.
Transfer a file¶
Use the FileTransferOperator
to copy a file from one
location to another. Parameters of the operator are:
src
- source path as a str or ObjectStoragePathdst
- destination path as a str or ObjectStoragePathsrc_conn_id
- source connection id (default:None
)dst_conn_id
- destination connection id (default:None
)overwrite
- overwrite destination (default:False
)
If the src
and the dst
are both on the same object storage, copy will be performed in the object storage.
Otherwise the data will be streamed from the source to the destination.
The example below shows how to instantiate the FileTransferOperator task.
transfer = FileTransferOperator(src=temp_file_path, dst=AWS_BUCKET, task_id="transfer")