airflow.contrib.hooks.wasb_hook

Module Contents

class airflow.contrib.hooks.wasb_hook.WasbHook(wasb_conn_id='wasb_default')[source]

Bases:airflow.hooks.base_hook.BaseHook

Interacts with Azure Blob Storage through the wasb:// protocol.

Additional options passed in the ‘extra’ field of the connection will be passed to the BlockBlockService() constructor. For example, authenticate using a SAS token by adding {“sas_token”: “YOUR_TOKEN”}.

Parameters

wasb_conn_id (str) – Reference to the wasb connection.

get_conn(self)[source]

Return the BlockBlobService object.

check_for_blob(self, container_name, blob_name, **kwargs)[source]

Check if a blob exists on Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.exists() takes.

Returns

True if the blob exists, False otherwise.

Return type

bool

check_for_prefix(self, container_name, prefix, **kwargs)[source]

Check if a prefix exists on Azure Blob storage.

Parameters

  • container_name (str) – Name of the container.

  • prefix (str) – Prefix of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.list_blobs() takes.

Returns

True if blobs matching the prefix exist, False otherwise.

Return type

bool

load_file(self, file_path, container_name, blob_name, **kwargs)[source]

Upload a file to Azure Blob Storage.

Parameters

  • file_path (str) – Path to the file to load.

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

load_string(self, string_data, container_name, blob_name, **kwargs)[source]

Upload a string to Azure Blob Storage.

Parameters

  • string_data (str) – String to load.

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.create_blob_from_text() takes.

get_file(self, file_path, container_name, blob_name, **kwargs)[source]

Download a file from Azure Blob Storage.

Parameters

  • file_path (str) – Path to the file to download.

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

read_file(self, container_name, blob_name, **kwargs)[source]

Read a file from Azure Blob Storage and return as a string.

Parameters

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

delete_file(self, container_name, blob_name, is_prefix=False, ignore_if_missing=False, **kwargs)[source]

Delete a file from Azure Blob Storage.

Parameters

  • container_name (str) – Name of the container.

  • blob_name (str) – Name of the blob.

  • is_prefix (bool) – If blob_name is a prefix, delete all matching files

  • ignore_if_missing (bool) – if True, then return success even if the blob does not exist.

  • kwargs (object) – Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.