airflow.providers.alibaba.cloud.log.oss_task_handler

Module Contents

Classes

OSSTaskHandler

OSSTaskHandler is a python log handler that handles and reads

class airflow.providers.alibaba.cloud.log.oss_task_handler.OSSTaskHandler(base_log_folder, oss_log_folder, filename_template=None)[source]

Bases: airflow.utils.log.file_task_handler.FileTaskHandler, airflow.utils.log.logging_mixin.LoggingMixin

OSSTaskHandler is a python log handler that handles and reads task instance logs. It extends airflow FileTaskHandler and uploads to and reads from OSS remote storage.

hook(self)[source]
set_context(self, ti)[source]

This function is used to set the context of the handler

close(self)[source]

Close and upload local log file to remote storage OSS.

oss_log_exists(self, remote_log_location)[source]

Check if remote_log_location exists in remote storage

Parameters

remote_log_location -- log's location in remote storage

Returns

True if location exists else False

oss_read(self, remote_log_location, return_error=False)[source]

Returns the log found at the remote_log_location. Returns '' if no logs are found or there is an error.

Parameters
  • remote_log_location -- the log's location in remote storage

  • return_error -- if True, returns a string error message if an error occurs. Otherwise returns '' when an error occurs.

oss_write(self, log, remote_log_location, append=True)[source]

Writes the log to the remote_log_location. Fails silently if no hook was created.

Parameters
  • log -- the log to write to the remote_log_location

  • remote_log_location -- the log's location in remote storage

  • append -- if False, any existing log file is overwritten. If True, the new log is appended to any existing logs.

Was this entry helpful?