airflow.hooks.druid_hook

Module Contents

class airflow.hooks.druid_hook.DruidHook(druid_ingest_conn_id='druid_ingest_default', timeout=1, max_ingestion_time=None)[source]

Bases: airflow.hooks.base_hook.BaseHook

Connection to Druid overlord for ingestion

To connect to a Druid cluster that is secured with the druid-basic-security extension, add the username and password to the druid ingestion connection.

Parameters

  • druid_ingest_conn_id (str) – The connection id to the Druid overlord machine which accepts index jobs

  • timeout (int) – The interval between polling the Druid job for the status of the ingestion job. Must be greater than or equal to 1

  • max_ingestion_time (int) – The maximum ingestion time before assuming the job failed

get_conn_url(self)[source]
get_auth(self)[source]

Return username and password from connections tab as requests.auth.HTTPBasicAuth object.

If these details have not been set then returns None.

submit_indexing_job(self, json_index_spec)[source]
class airflow.hooks.druid_hook.DruidDbApiHook(*args, **kwargs)[source]

Bases: airflow.hooks.dbapi_hook.DbApiHook

Interact with Druid broker

This hook is purely for users to query druid broker. For ingestion, please use druidHook.

conn_name_attr = druid_broker_conn_id[source]
default_conn_name = druid_broker_default[source]
supports_autocommit = False[source]
get_conn(self)[source]

Establish a connection to druid broker.

get_uri(self)[source]

Get the connection uri for druid broker.

e.g: druid://localhost:8082/druid/v2/sql/

set_autocommit(self, conn, autocommit)[source]
insert_rows(self, table, rows, target_fields=None, commit_every=1000)[source]

Was this entry helpful?