airflow.providers.sftp.sensors.sftp

This module contains SFTP sensor.

Classes

SFTPSensor

Waits for a file or directory to be present on SFTP.

Module Contents

class airflow.providers.sftp.sensors.sftp.SFTPSensor(*, path, file_pattern='', newer_than=None, sftp_conn_id='sftp_default', python_callable=None, op_args=None, op_kwargs=None, use_managed_conn=True, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.providers.common.compat.sdk.BaseSensorOperator

Waits for a file or directory to be present on SFTP.

Parameters:

  • path (str) – Remote file or directory path

  • file_pattern (str) – The pattern that will be used to match the file (fnmatch format)

  • sftp_conn_id (str) – The connection to run the sensor against

  • newer_than (datetime.datetime | str | None) – DateTime for which the file or file path should be newer than, comparison is inclusive

  • python_callable (collections.abc.Callable | None) – Optional callable that will be called after files are found. The callable will receive the found files list in op_kwargs['files_found'] if op_kwargs is provided and not empty. The return value of the callable will be stored in XCom along with the files_found list.

  • op_args (list | None) – A list of positional arguments that will get unpacked when calling your callable (templated). Only used when python_callable is provided.

  • op_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments that will get unpacked in your callable (templated). If provided and not empty, the files_found list will be automatically added to this dictionary. Only used when python_callable is provided.

  • deferrable (bool) – If waiting for completion, whether to defer the task until done, default is False.

template_fields: collections.abc.Sequence[str] = ('path', 'file_pattern', 'newer_than')[source]
path[source]
file_pattern = ''[source]
hook: airflow.providers.sftp.hooks.sftp.SFTPHook | None = None[source]
sftp_conn_id = 'sftp_default'[source]
newer_than: datetime.datetime | str | None = None[source]
use_managed_conn = True[source]
python_callable: collections.abc.Callable | None = None[source]
op_args = [][source]
op_kwargs[source]
deferrable[source]
poke(context)[source]

Override when deriving this class.

execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]

Execute callback when the trigger fires; returns immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

Was this entry helpful?