airflow.providers.amazon.aws.log.s3_task_handler
¶
Module Contents¶
-
class
airflow.providers.amazon.aws.log.s3_task_handler.
S3TaskHandler
(base_log_folder: str, s3_log_folder: str, filename_template: str)[source]¶ Bases:
airflow.utils.log.file_task_handler.FileTaskHandler
,airflow.utils.log.logging_mixin.LoggingMixin
S3TaskHandler is a python log handler that handles and reads task instance logs. It extends airflow FileTaskHandler and uploads to and reads from S3 remote storage.
-
_read
(self, ti, try_number, metadata=None)[source]¶ Read logs of given task instance and try_number from S3 remote storage. If failed, read the log from task instance host machine.
- Parameters
ti -- task instance object
try_number -- task instance try_number to read logs from
metadata -- log metadata, can be used for steaming log reading and auto-tailing.
-
s3_log_exists
(self, remote_log_location: str)[source]¶ Check if remote_log_location exists in remote storage
- Parameters
remote_log_location (str) -- log's location in remote storage
- Returns
True if location exists else False
-
s3_read
(self, remote_log_location: str, return_error: bool = False)[source]¶ Returns the log found at the remote_log_location. Returns '' if no logs are found or there is an error.
-