airflow.providers.apache.drill.hooks.drill

Module Contents

Classes

DrillHook

Interact with Apache Drill via sqlalchemy-drill.
class airflow.providers.apache.drill.hooks.drill.DrillHook(*args, schema=None, **kwargs)[source]

Bases: airflow.hooks.dbapi.DbApiHook

Interact with Apache Drill via sqlalchemy-drill.

You can specify the SQLAlchemy dialect and driver that sqlalchemy-drill will employ to communicate with Drill in the extras field of your connection, e.g. {"dialect_driver": "drill+sadrill"} for communication over Drill's REST API. See the sqlalchemy-drill documentation for descriptions of the supported dialects and drivers.

You can specify the default storage_plugin for the sqlalchemy-drill connection using the extras field e.g. {"storage_plugin": "dfs"}.

conn_name_attr = drill_conn_id[source]
default_conn_name = drill_default[source]
conn_type = drill[source]
hook_name = Drill[source]
supports_autocommit = False[source]
get_conn(self)[source]

Establish a connection to Drillbit.

get_uri(self)[source]

Returns the connection URI

e.g: drill://localhost:8047/dfs

abstract set_autocommit(self, conn, autocommit)[source]

Sets the autocommit flag on the connection

abstract insert_rows(self, table, rows, target_fields=None, commit_every=1000, replace=False, **kwargs)[source]

A generic way to insert a set of tuples into a table, a new transaction is created every commit_every rows

Parameters

  • table (str) -- Name of the target table

  • rows (Iterable[Tuple[str]]) -- The rows to insert into the table

  • target_fields (Optional[Iterable[str]]) -- The names of the columns to fill in the table

  • commit_every (int) -- The maximum number of rows to insert in one transaction. Set to 0 to insert all rows in one transaction.

  • replace (bool) -- Whether to replace instead of insert

Was this entry helpful?