airflow.providers.sftp.operators.sftp¶

This module contains SFTP operator.

Classes¶

`SFTPOperation`	Operation that can be used with SFTP.
`SFTPOperator`	SFTPOperator for transferring files from remote host to local or vice a versa.

Module Contents¶

class airflow.providers.sftp.operators.sftp.SFTPOperation[source]¶

Operation that can be used with SFTP.

PUT = 'put'[source]¶

GET = 'get'[source]¶

DELETE = 'delete'[source]¶

class airflow.providers.sftp.operators.sftp.SFTPOperator(*, sftp_hook=None, ssh_conn_id=None, remote_host=None, local_filepath=None, remote_filepath, operation=SFTPOperation.PUT, confirm=True, create_intermediate_dirs=False, concurrency=1, prefetch=True, **kwargs)[source]¶

Bases: airflow.providers.common.compat.sdk.BaseOperator

SFTPOperator for transferring files from remote host to local or vice a versa.

This operator uses sftp_hook to open sftp transport channel that serve as basis for file transfer.

Parameters:

ssh_conn_id (str | None) – ssh connection id from airflow Connections.
sftp_hook (airflow.providers.sftp.hooks.sftp.SFTPHook | None) – predefined SFTPHook to use Either sftp_hook or ssh_conn_id needs to be provided.
remote_host (str | None) – remote host to connect (templated) Nullable. If provided, it will replace the remote_host which was defined in sftp_hook or predefined in the connection of ssh_conn_id.
local_filepath (str | list[str] | None) – local file path or list of local file paths to get or put. (templated)
remote_filepath (str | list[str]) – remote file path or list of remote file paths to get, put, or delete. (templated)
operation (str) – specify operation ‘get’, ‘put’, or ‘delete’, defaults to put
confirm (bool) – specify if the SFTP operation should be confirmed, defaults to True
create_intermediate_dirs (bool) –
create missing intermediate directories when copying from remote to local and vice-versa. Default is False.

Example: The following task would copy file.txt to the remote host at /tmp/tmp1/tmp2/ while creating tmp,``tmp1`` and tmp2 if they don’t exist. If the parameter is not passed it would error as the directory does not exist.
```
put_file = SFTPOperator(
    task_id="test_sftp",
    ssh_conn_id="ssh_default",
    local_filepath="/tmp/file.txt",
    remote_filepath="/tmp/tmp1/tmp2/file.txt",
    operation="put",
    create_intermediate_dirs=True,
    dag=dag,
)
```
concurrency (int) – Number of threads when transferring directories. Each thread opens a new SFTP connection. This parameter is used only when transferring directories, not individual files. (Default is 1)
prefetch (bool) – controls whether prefetch is performed (default: True)

template_fields: collections.abc.Sequence[str] = ('local_filepath', 'remote_filepath', 'remote_host')[source]¶

sftp_hook = None[source]¶

ssh_conn_id = None[source]¶

remote_host = None[source]¶

operation = 'put'[source]¶

confirm = True[source]¶

create_intermediate_dirs = False[source]¶

local_filepath = None[source]¶

remote_filepath[source]¶

concurrency = 1[source]¶

prefetch = True[source]¶

execute(context)[source]¶

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

get_openlineage_facets_on_start()[source]¶

Return OpenLineage datasets.

Dataset will have the following structure:: input: file://<local_host>/path output: file://<remote_host>:<remote_port>/path.