`airflow.providers.microsoft.azure.log.wasb_task_handler`¶

Module Contents¶

Classes¶

WasbTaskHandler

WasbTaskHandler is a python log handler that handles and reads

class airflow.providers.microsoft.azure.log.wasb_task_handler.WasbTaskHandler(base_log_folder: str, wasb_log_folder: str, wasb_container: str, filename_template: str, delete_local_copy: str)[source]¶

Bases: airflow.utils.log.file_task_handler.FileTaskHandler, airflow.utils.log.logging_mixin.LoggingMixin

WasbTaskHandler is a python log handler that handles and reads task instance logs. It extends airflow FileTaskHandler and uploads to and reads from Wasb remote storage.

hook(self)[source]¶: Returns WasbHook.

set_context(self, ti) → None[source]¶

Provide task_instance context to airflow task handler.

Parameters: ti -- task instance object

close(self) → None[source]¶: Close and upload local log file to remote storage Wasb.

wasb_log_exists(self, remote_log_location: str) → bool[source]¶

Check if remote_log_location exists in remote storage

Parameters: remote_log_location -- log's location in remote storage
Returns: True if location exists else False

wasb_read(self, remote_log_location: str, return_error: bool = False)[source]¶

Returns the log found at the remote_log_location. Returns '' if no logs are found or there is an error.

Parameters

remote_log_location (str (path)) -- the log's location in remote storage
return_error (bool) -- if True, returns a string error message if an error occurs. Otherwise returns '' when an error occurs.

wasb_write(self, log: str, remote_log_location: str, append: bool = True) → None[source]¶

Writes the log to the remote_log_location. Fails silently if no hook was created.

Parameters

log (str) -- the log to write to the remote_log_location
remote_log_location (str (path)) -- the log's location in remote storage
append (bool) -- if False, any existing log file is overwritten. If True, the new log is appended to any existing logs.

airflow.providers.microsoft.azure.log.wasb_task_handler¶

Module Contents¶

Classes¶

`airflow.providers.microsoft.azure.log.wasb_task_handler`¶