airflow.providers.sftp.hooks.sftp

This module contains SFTP hook.

Module Contents

Classes

SFTPHook

This hook is inherited from SSH hook. Please refer to SSH hook for the input
class airflow.providers.sftp.hooks.sftp.SFTPHook(ssh_conn_id='sftp_default', ssh_hook=None, *args, **kwargs)[source]

Bases: airflow.providers.ssh.hooks.ssh.SSHHook

This hook is inherited from SSH hook. Please refer to SSH hook for the input arguments.

Interact with SFTP.

Pitfalls:

  • In contrast with FTPHook describe_directory only returns size, type and modify. It doesn’t return unix.owner, unix.mode, perm, unix.group and unique.

  • retrieve_file and store_file only take a local full path and not a

    buffer.

  • If no mode is passed to create_directory it will be created with 777 permissions.

Errors that may occur throughout but should be handled downstream.

For consistency reasons with SSHHook, the preferred parameter is “ssh_conn_id”.

Parameters

  • ssh_conn_id (str | None) – The sftp connection id

  • ssh_hook (SSHHook | None) – Optional SSH hook (included to support passing of an SSH hook to the SFTP operator)

conn_name_attr = 'ssh_conn_id'[source]
default_conn_name = 'sftp_default'[source]
conn_type = 'sftp'[source]
hook_name = 'SFTP'[source]
static get_ui_field_behaviour()[source]

Returns custom field behaviour

get_conn()[source]

Opens an SFTP connection to the remote host

close_conn()[source]

Closes the SFTP connection

describe_directory(path)[source]

Returns a dictionary of {filename: {attributes}} for all files on the remote system (where the MLSD command is supported).

Parameters

path (str) – full path to the remote directory

list_directory(path)[source]

Returns a list of files on the remote system.

Parameters

path (str) – full path to the remote directory to list

mkdir(path, mode=511)[source]

Creates a directory on the remote system. The default mode is 0777, but on some systems, the current umask value is first masked out.

Parameters

  • path (str) – full path to the remote directory to create

  • mode (int) – int permissions of octal mode for directory

isdir(path)[source]

Checks if the path provided is a directory or not.

Parameters

path (str) – full path to the remote directory to check

isfile(path)[source]

Checks if the path provided is a file or not.

Parameters

path (str) – full path to the remote file to check

create_directory(path, mode=511)[source]

Creates a directory on the remote system. The default mode is 0777, but on some systems, the current umask value is first masked out.

Parameters

  • path (str) – full path to the remote directory to create

  • mode (int) – int permissions of octal mode for directory

delete_directory(path)[source]

Deletes a directory on the remote system.

Parameters

path (str) – full path to the remote directory to delete

retrieve_file(remote_full_path, local_full_path)[source]

Transfers the remote file to a local location. If local_full_path is a string path, the file will be put at that location

Parameters

  • remote_full_path (str) – full path to the remote file

  • local_full_path (str) – full path to the local file

store_file(remote_full_path, local_full_path, confirm=True)[source]

Transfers a local file to the remote location. If local_full_path_or_buffer is a string path, the file will be read from that location

Parameters

  • remote_full_path (str) – full path to the remote file

  • local_full_path (str) – full path to the local file

delete_file(path)[source]

Removes a file on the FTP Server

Parameters

path (str) – full path to the remote file

get_mod_time(path)[source]

Returns modification time.

Parameters

path (str) – full path to the remote file

path_exists(path)[source]

Returns True if a remote entity exists

Parameters

path (str) – full path to the remote file or directory

walktree(path, fcallback, dcallback, ucallback, recurse=True)[source]

Recursively descend, depth first, the directory tree rooted at path, calling discrete callback functions for each regular file, directory and unknown file type.

Parameters

  • path (str) – root of remote directory to descend, use ‘.’ to start at pwd

  • fcallback (callable) – callback function to invoke for a regular file. (form: func(str))

  • dcallback (callable) – callback function to invoke for a directory. (form: func(str))

  • ucallback (callable) – callback function to invoke for an unknown file type. (form: func(str))

  • recurse (bool) – Default: True - should it recurse

Returns

None

Return type

None

get_tree_map(path, prefix=None, delimiter=None)[source]

Return tuple with recursive lists of files, directories and unknown paths from given path. It is possible to filter results by giving prefix and/or delimiter parameters.

Parameters

  • path (str) – path from which tree will be built

  • prefix (str | None) – if set paths will be added if start with prefix

  • delimiter (str | None) – if set paths will be added if end with delimiter

Returns

tuple with list of files, dirs and unknown items

Return type

tuple[list[str], list[str], list[str]]

test_connection()[source]

Test the SFTP connection by calling path with directory

get_file_by_pattern(path, fnmatch_pattern)[source]

Returning the first matching file based on the given fnmatch type pattern

Parameters

  • path – path to be checked

  • fnmatch_pattern – The pattern that will be matched with fnmatch

Returns

string containing the first found file, or an empty string if none matched

Return type

str

get_files_by_pattern(path, fnmatch_pattern)[source]

Returning the list of matching files based on the given fnmatch type pattern

Parameters

  • path – path to be checked

  • fnmatch_pattern – The pattern that will be matched with fnmatch

Returns

list of string containing the found files, or an empty list if none matched

Return type

list[str]

Was this entry helpful?