airflow.providers.google.cloud.log.stackdriver_task_handler

Handler that integrates with Stackdriver

Module Contents

airflow.providers.google.cloud.log.stackdriver_task_handler.DEFAULT_LOGGER_NAME = airflow[source]
class airflow.providers.google.cloud.log.stackdriver_task_handler.StackdriverTaskHandler(gcp_key_path: Optional[str] = None, scopes: Optional[Collection[str]] = _DEFAULT_SCOPESS, name: str = DEFAULT_LOGGER_NAME, transport: Type[Transport] = BackgroundThreadTransport, resource: Resource = _GLOBAL_RESOURCE, labels: Optional[Dict[str, str]] = None)[source]

Bases: logging.Handler

Handler that directly makes Stackdriver logging API calls.

This is a Python standard logging handler using that can be used to route Python standard logging messages directly to the Stackdriver Logging API.

It can also be used to save logs for executing tasks. To do this, you should set as a handler with the name “tasks”. In this case, it will also be used to read the log for display in Web UI.

This handler supports both an asynchronous and synchronous transport.

Parameters

  • gcp_key_path (str) – Path to Google Cloud Credential JSON file. If omitted, authorization based on the Application Default Credentials will be used.

  • scopes (Sequence[str]) – OAuth scopes for the credentials,

  • name (str) – the name of the custom log in Stackdriver Logging. Defaults to ‘airflow’. The name of the Python logger will be represented in the python_logger field.

  • transport (type) – Class for creating new transport objects. It should extend from the base google.cloud.logging.handlers.Transport type and implement :meth`google.cloud.logging.handlers.Transport.send`. Defaults to google.cloud.logging.handlers.BackgroundThreadTransport. The other option is google.cloud.logging.handlers.SyncTransport.

  • resource (Resource) – (Optional) Monitored resource of the entry, defaults to the global resource type.

  • labels (dict) – (Optional) Mapping of labels for the entry.

LABEL_TASK_ID = task_id[source]
LABEL_DAG_ID = dag_id[source]
LABEL_EXECUTION_DATE = execution_date[source]
LABEL_TRY_NUMBER = try_number[source]
LOG_VIEWER_BASE_URL = https://console.cloud.google.com/logs/viewer[source]
LOG_NAME = Google Stackdriver[source]
log_name[source]

Return log name.

emit(self, record: logging.LogRecord)[source]

Actually log the specified logging record.

Parameters

record (logging.LogRecord) – The record to be logged.

set_context(self, task_instance: TaskInstance)[source]

Configures the logger to add information with information about the current task

Parameters

task_instance (airflow.models.TaskInstance) – Currently executed task

read(self, task_instance: TaskInstance, try_number: Optional[int] = None, metadata: Optional[Dict] = None)[source]

Read logs of given task instance from Stackdriver logging.

Parameters

  • task_instance (airflow.models.TaskInstance) – task instance object

  • try_number (Optional[int]) – task instance try_number to read logs from. If None it returns all logs

  • metadata (Dict) – log metadata. It is used for steaming log reading and auto-tailing.

Returns

a tuple of ( list of (one element tuple with two element tuple - hostname and logs) and list of metadata)

Return type

Tuple[List[Tuple[Tuple[str, str]]], List[Dict[str, str]]]

get_external_log_url(self, task_instance: TaskInstance, try_number: int)[source]

Creates an address for an external log collecting service. :param task_instance: task instance object :type: task_instance: TaskInstance :param try_number: task instance try_number to read logs from. :type try_number: Optional[int] :return: URL to the external log collection service :rtype: str

close(self)[source]

Was this entry helpful?