airflow.providers.google.cloud.log.stackdriver_task_handler

Handler that integrates with Stackdriver

Module Contents

Classes

StackdriverTaskHandler

Handler that directly makes Stackdriver logging API calls.

Attributes

DEFAULT_LOGGER_NAME

airflow.providers.google.cloud.log.stackdriver_task_handler.DEFAULT_LOGGER_NAME = airflow[source]
class airflow.providers.google.cloud.log.stackdriver_task_handler.StackdriverTaskHandler(gcp_key_path=None, scopes=_DEFAULT_SCOPESS, name=DEFAULT_LOGGER_NAME, transport=BackgroundThreadTransport, resource=_GLOBAL_RESOURCE, labels=None)[source]

Bases: logging.Handler

Handler that directly makes Stackdriver logging API calls.

This is a Python standard logging handler using that can be used to route Python standard logging messages directly to the Stackdriver Logging API.

It can also be used to save logs for executing tasks. To do this, you should set as a handler with the name “tasks”. In this case, it will also be used to read the log for display in Web UI.

This handler supports both an asynchronous and synchronous transport.

Parameters

  • gcp_key_path (Optional[str]) – Path to Google Cloud Credential JSON file. If omitted, authorization based on the Application Default Credentials will be used.

  • scopes (Optional[Collection[str]]) – OAuth scopes for the credentials,

  • name (str) – the name of the custom log in Stackdriver Logging. Defaults to ‘airflow’. The name of the Python logger will be represented in the python_logger field.

  • transport (Type[google.cloud.logging.handlers.transports.Transport]) – Class for creating new transport objects. It should extend from the base google.cloud.logging.handlers.Transport type and implement :meth`google.cloud.logging.handlers.Transport.send`. Defaults to google.cloud.logging.handlers.BackgroundThreadTransport. The other option is google.cloud.logging.handlers.SyncTransport.

  • resource (google.cloud.logging.Resource) – (Optional) Monitored resource of the entry, defaults to the global resource type.

  • labels (Optional[Dict[str, str]]) – (Optional) Mapping of labels for the entry.

LABEL_TASK_ID = task_id[source]
LABEL_DAG_ID = dag_id[source]
LABEL_EXECUTION_DATE = execution_date[source]
LABEL_TRY_NUMBER = try_number[source]
LOG_VIEWER_BASE_URL = https://console.cloud.google.com/logs/viewer[source]
LOG_NAME = Google Stackdriver[source]
emit(self, record)[source]

Actually log the specified logging record.

Parameters

record (logging.LogRecord) – The record to be logged.

set_context(self, task_instance)[source]

Configures the logger to add information with information about the current task

Parameters

task_instance (airflow.models.TaskInstance) – Currently executed task

read(self, task_instance, try_number=None, metadata=None)[source]

Read logs of given task instance from Stackdriver logging.

Parameters

  • task_instance (airflow.models.TaskInstance) – task instance object

  • try_number (Optional[int]) – task instance try_number to read logs from. If None it returns all logs

  • metadata (Optional[Dict]) – log metadata. It is used for steaming log reading and auto-tailing.

Returns

a tuple of ( list of (one element tuple with two element tuple - hostname and logs) and list of metadata)

Return type

Tuple[List[Tuple[Tuple[str, str]]], List[Dict[str, str]]]

property log_name(self)[source]

Return log name.

get_external_log_url(self, task_instance, try_number)[source]

Creates an address for an external log collecting service. :param task_instance: task instance object :param try_number: task instance try_number to read logs from. :return: URL to the external log collection service :rtype: str

close(self)[source]

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

Was this entry helpful?