airflow.providers.amazon.aws.log.cloudwatch_task_handler¶

Classes¶

`CloudWatchRemoteLogIO`	Convenience super-class to have a logger configured with the class name.
`CloudwatchTaskHandler`	CloudwatchTaskHandler is a python log handler that handles and reads task instance logs.

Functions¶

`json_serialize_legacy`(value)	JSON serializer replicating legacy watchtower behavior.
`json_serialize`(value)	JSON serializer replicating current watchtower behavior.

Module Contents¶

airflow.providers.amazon.aws.log.cloudwatch_task_handler.json_serialize_legacy(value)[source]¶

JSON serializer replicating legacy watchtower behavior.

The legacy watchtower@2.0.1 json serializer function that serialized datetime objects as ISO format and all other non-JSON-serializable to null.

Parameters:: value (Any) – the object to serialize
Returns:: string representation of value if it is an instance of datetime or None otherwise
Return type:: str | None

airflow.providers.amazon.aws.log.cloudwatch_task_handler.json_serialize(value)[source]¶

JSON serializer replicating current watchtower behavior.

This provides customers with an accessible import, airflow.providers.amazon.aws.log.cloudwatch_task_handler.json_serialize

Parameters:: value (Any) – the object to serialize
Returns:: string representation of value
Return type:: str | None

class airflow.providers.amazon.aws.log.cloudwatch_task_handler.CloudWatchRemoteLogIO(context=None)[source]¶

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Convenience super-class to have a logger configured with the class name.

base_log_folder: pathlib.Path[source]¶

remote_base: str = ''[source]¶

delete_local_copy: bool = True[source]¶

log_group_arn: str[source]¶

log_stream_name: str = ''[source]¶

log_group: str[source]¶

region_name: str[source]¶

property hook[source]¶: Returns AwsLogsHook.

property handler: watchtower.CloudWatchLogHandler[source]¶

property processors: tuple[structlog.typing.Processor, Ellipsis][source]¶

close()[source]¶

upload(path, ti)[source]¶

read(relative_path, ti)[source]¶

get_cloudwatch_logs(stream_name, task_instance)[source]¶

Return all logs from the given log stream.

Parameters:

stream_name (str) – name of the Cloudwatch log stream to get all logs from
task_instance (airflow.sdk.types.RuntimeTaskInstanceProtocol) – the task instance to get logs about

Returns:

string of all logs from the given log stream

class airflow.providers.amazon.aws.log.cloudwatch_task_handler.CloudwatchTaskHandler(base_log_folder, log_group_arn, max_bytes=0, backup_count=0, delay=False, **kwargs)[source]¶

Bases: airflow.utils.log.file_task_handler.FileTaskHandler, airflow.utils.log.logging_mixin.LoggingMixin

CloudwatchTaskHandler is a python log handler that handles and reads task instance logs.

It extends airflow FileTaskHandler and uploads to and reads from Cloudwatch.

Parameters:

base_log_folder (str) – base folder to store logs locally
log_group_arn (str) – ARN of the Cloudwatch log group for remote log storage with format arn:aws:logs:{region name}:{account id}:log-group:{group name}

trigger_should_wrap = True[source]¶

handler = None[source]¶

log_group[source]¶

region_name[source]¶

closed = False[source]¶

io[source]¶

property hook[source]¶: Returns AwsLogsHook.

set_context(ti, *, identifier=None)[source]¶

Provide task_instance context to airflow task handler.

Generally speaking returns None. But if attr maintain_propagate has been set to propagate, then returns sentinel MAINTAIN_PROPAGATE. This has the effect of overriding the default behavior to set propagate to False whenever set_context is called. At time of writing, this functionality is only used in unit testing.

Parameters:

ti (airflow.models.taskinstance.TaskInstance) – task instance object
identifier (str | None) – if set, adds suffix to log file. For use when relaying exceptional messages to task logs from a context other than task or trigger run

close()[source]¶: Close the handler responsible for the upload of the local log file to Cloudwatch.