airflow.providers.microsoft.azure.sensors.wasb

Module Contents

Classes

WasbBlobSensor

Waits for a blob to arrive on Azure Blob Storage.

WasbBlobAsyncSensor

Poll asynchronously for the existence of a blob in a WASB container.

WasbPrefixSensor

Wait for blobs matching a prefix to arrive on Azure Blob Storage.
class airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor(*, container_name, blob_name, wasb_conn_id='wasb_default', check_options=None, public_read=False, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Waits for a blob to arrive on Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • wasb_conn_id (str) – Reference to the wasb connection.

  • check_options (dict | None) – Optional keyword arguments that WasbHook.check_for_blob() takes.

  • deferrable (bool) – Run sensor in the deferrable mode.

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

template_fields: Sequence[str] = ('container_name', 'blob_name')[source]
poke(context)[source]

Override when deriving this class.

execute(context)[source]

Poll for state of the job run.

In deferrable mode, the polling is deferred to the triggerer. Otherwise the sensor waits synchronously.

execute_complete(context, event)[source]

Return immediately - callback for when the trigger fires.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.microsoft.azure.sensors.wasb.WasbBlobAsyncSensor(**kwargs)[source]

Bases: WasbBlobSensor

Poll asynchronously for the existence of a blob in a WASB container.

This class is deprecated and will be removed in a future release.

Please use airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor and set deferrable attribute to True instead.

Parameters

  • container_name – name of the container in which the blob should be searched for

  • blob_name – name of the blob to check existence for

  • wasb_conn_id – the connection identifier for connecting to Azure WASB

  • poke_interval – polling period in seconds to check for the status

  • public_read – whether an anonymous public read access should be used. Default is False

  • timeout – Time, in seconds before the task times out and fails.

class airflow.providers.microsoft.azure.sensors.wasb.WasbPrefixSensor(*, container_name, prefix, wasb_conn_id='wasb_default', check_options=None, public_read=False, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Wait for blobs matching a prefix to arrive on Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • prefix (str) – Prefix of the blob.

  • wasb_conn_id (str) – Reference to the wasb connection.

  • check_options (dict | None) – Optional keyword arguments that WasbHook.check_for_prefix() takes.

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

  • deferrable (bool) – Run operator in the deferrable mode.

template_fields: Sequence[str] = ('container_name', 'prefix')[source]
poke(context)[source]

Override when deriving this class.

execute(context)[source]

Poll for state of the job run.

In deferrable mode, the polling is deferred to the triggerer. Otherwise the sensor waits synchronously.

execute_complete(context, event)[source]

Return immediately - callback for when the trigger fires.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

Was this entry helpful?