airflow.providers.microsoft.azure.sensors.wasb

Module Contents

Classes

WasbBlobSensor

Waits for a blob to arrive on Azure Blob Storage.

WasbBlobAsyncSensor

Polls asynchronously for the existence of a blob in a WASB container.

WasbPrefixSensor

Waits for blobs matching a prefix to arrive on Azure Blob Storage.
class airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor(*, container_name, blob_name, wasb_conn_id='wasb_default', check_options=None, public_read=False, deferrable=False, **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Waits for a blob to arrive on Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • wasb_conn_id (str) – Reference to the wasb connection.

  • check_options (dict | None) – Optional keyword arguments that WasbHook.check_for_blob() takes.

  • deferrable (bool) – Run sensor in the deferrable mode.

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

template_fields: Sequence[str] = ('container_name', 'blob_name')[source]
poke(context)[source]

Function defined by the sensors while deriving this class should override.

execute(context)[source]

Defers trigger class to poll for state of the job run until it reaches a failure state or success state

execute_complete(context, event)[source]

Callback for when the trigger fires - returns immediately. Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.microsoft.azure.sensors.wasb.WasbBlobAsyncSensor(**kwargs)[source]

Bases: WasbBlobSensor

Polls asynchronously for the existence of a blob in a WASB container.

Parameters

  • container_name – name of the container in which the blob should be searched for

  • blob_name – name of the blob to check existence for

  • wasb_conn_id – the connection identifier for connecting to Azure WASB

  • poke_interval – polling period in seconds to check for the status

  • public_read – whether an anonymous public read access should be used. Default is False

  • timeout – Time, in seconds before the task times out and fails.

class airflow.providers.microsoft.azure.sensors.wasb.WasbPrefixSensor(*, container_name, prefix, wasb_conn_id='wasb_default', check_options=None, deferrable=False, **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Waits for blobs matching a prefix to arrive on Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • prefix (str) – Prefix of the blob.

  • wasb_conn_id (str) – Reference to the wasb connection.

  • check_options (dict | None) – Optional keyword arguments that WasbHook.check_for_prefix() takes.

template_fields: Sequence[str] = ('container_name', 'prefix')[source]
poke(context)[source]

Function defined by the sensors while deriving this class should override.

execute(context)[source]

Defers trigger class to poll for state of the job run until it reaches a failure state or success state

execute_complete(context, event)[source]

Callback for when the trigger fires - returns immediately. Relies on trigger to throw an exception, otherwise it assumes execution was successful.

Was this entry helpful?