`airflow.providers.apache.druid.hooks.druid`¶

Module Contents¶

Classes¶

`DruidHook`	Connection to Druid overlord for ingestion
`DruidDbApiHook`	Interact with Druid broker

class airflow.providers.apache.druid.hooks.druid.DruidHook(druid_ingest_conn_id='druid_ingest_default', timeout=1, max_ingestion_time=None)[source]¶

Bases: airflow.hooks.base.BaseHook

Connection to Druid overlord for ingestion

To connect to a Druid cluster that is secured with the druid-basic-security extension, add the username and password to the druid ingestion connection.

Parameters

druid_ingest_conn_id (str) -- The connection id to the Druid overlord machine which accepts index jobs
timeout (int) -- The interval between polling the Druid job for the status of the ingestion job. Must be greater than or equal to 1
max_ingestion_time (Optional[int]) -- The maximum ingestion time before assuming the job failed

get_conn_url(self)[source]¶

Get Druid connection url

get_auth(self)[source]¶

Return username and password from connections tab as requests.auth.HTTPBasicAuth object.

If these details have not been set then returns None.

submit_indexing_job(self, json_index_spec)[source]¶

Submit Druid ingestion job

class airflow.providers.apache.druid.hooks.druid.DruidDbApiHook(*args, schema=None, **kwargs)[source]¶

Bases: airflow.hooks.dbapi.DbApiHook

Interact with Druid broker

This hook is purely for users to query druid broker. For ingestion, please use druidHook.

conn_name_attr = druid_broker_conn_id[source]¶

default_conn_name = druid_broker_default[source]¶

conn_type = druid[source]¶

hook_name = Druid[source]¶

supports_autocommit = False[source]¶

get_conn(self)[source]¶

Establish a connection to druid broker.

get_uri(self)[source]¶

Get the connection uri for druid broker.

e.g: druid://localhost:8082/druid/v2/sql/

abstract set_autocommit(self, conn, autocommit)[source]¶

Sets the autocommit flag on the connection

abstract insert_rows(self, table, rows, target_fields=None, commit_every=1000, replace=False, **kwargs)[source]¶

A generic way to insert a set of tuples into a table, a new transaction is created every commit_every rows

Parameters

table (str) -- Name of the target table
rows (Iterable[Tuple[str]]) -- The rows to insert into the table
target_fields (Optional[Iterable[str]]) -- The names of the columns to fill in the table
commit_every (int) -- The maximum number of rows to insert in one transaction. Set to 0 to insert all rows in one transaction.
replace (bool) -- Whether to replace instead of insert

airflow.providers.apache.druid.hooks.druid¶

Module Contents¶

Classes¶

`airflow.providers.apache.druid.hooks.druid`¶