airflow.providers.ftp.hooks.ftp
¶
Module Contents¶
Classes¶
Interact with FTP. |
|
Interact with FTPS. |
- class airflow.providers.ftp.hooks.ftp.FTPHook(ftp_conn_id=default_conn_name)[source]¶
Bases:
airflow.hooks.base.BaseHook
Interact with FTP.
Errors that may occur throughout but should be handled downstream. You can specify mode for data transfers in the extra field of your connection as
{"passive": "true"}
.- Parameters
ftp_conn_id (str) -- The ftp connection id reference.
- close_conn(self)[source]¶
Closes the connection. An error will occur if the connection wasn't ever opened.
- describe_directory(self, path)[source]¶
Returns a dictionary of {filename: {attributes}} for all files on the remote system (where the MLSD command is supported).
- Parameters
path (str) -- full path to the remote directory
- list_directory(self, path)[source]¶
Returns a list of files on the remote system.
- Parameters
path (str) -- full path to the remote directory to list
- create_directory(self, path)[source]¶
Creates a directory on the remote system.
- Parameters
path (str) -- full path to the remote directory to create
- delete_directory(self, path)[source]¶
Deletes a directory on the remote system.
- Parameters
path (str) -- full path to the remote directory to delete
- retrieve_file(self, remote_full_path, local_full_path_or_buffer, callback=None)[source]¶
Transfers the remote file to a local location.
If local_full_path_or_buffer is a string path, the file will be put at that location; if it is a file-like buffer, the file will be written to the buffer but not closed.
- Parameters
remote_full_path -- full path to the remote file
local_full_path_or_buffer -- full path to the local file or a file-like buffer
callback -- callback which is called each time a block of data is read. if you do not use a callback, these blocks will be written to the file or buffer passed in. if you do pass in a callback, note that writing to a file or buffer will need to be handled inside the callback. [default: output_handle.write()]
hook = FTPHook(ftp_conn_id="my_conn") remote_path = "/path/to/remote/file" local_path = "/path/to/local/file" # with a custom callback (in this case displaying progress on each read) def print_progress(percent_progress): self.log.info("Percent Downloaded: %s%%" % percent_progress) total_downloaded = 0 total_file_size = hook.get_size(remote_path) output_handle = open(local_path, "wb") def write_to_file_with_progress(data): total_downloaded += len(data) output_handle.write(data) percent_progress = (total_downloaded / total_file_size) * 100 print_progress(percent_progress) hook.retrieve_file(remote_path, None, callback=write_to_file_with_progress) # without a custom callback data is written to the local_path hook.retrieve_file(remote_path, local_path)
- store_file(self, remote_full_path, local_full_path_or_buffer)[source]¶
Transfers a local file to the remote location.
If local_full_path_or_buffer is a string path, the file will be read from that location; if it is a file-like buffer, the file will be read from the buffer but not closed.
- Parameters
remote_full_path (str) -- full path to the remote file
local_full_path_or_buffer (Any) -- full path to the local file or a file-like buffer
- delete_file(self, path)[source]¶
Removes a file on the FTP Server.
- Parameters
path (str) -- full path to the remote file