Connect to Apache Drill via SQLExecuteQueryOperator¶

Prerequisite¶

Use SQLExecuteQueryOperator, to execute SQL commands in Drill query engine.

To connect to Drill, you must configure a Drill Connection and can pass that as conn_id to SQLExecuteQueryOperator.

Warning

Previously, DrillOperator was used to perform this kind of operation. But at the moment DrillOperator is deprecated and will be removed in future versions of the provider. Please consider to switch to SQLExecuteQueryOperator as soon as possible.

SQLExecuteQueryOperator¶

Executes one or more SQL queries on an Apache Drill server. The sql parameter can be templated and be an external .sql file.

Using the operator¶

tests/system/providers/apache/drill/example_drill_dag.py[source]

sql_task = SQLExecuteQueryOperator(
    task_id="json_to_parquet_table",
    sql="""
    drop table if exists dfs.tmp.employee;
    create table dfs.tmp.employee as select * from cp.`employee.json`;
    """,
)

Reference¶

For further information, see the Drill documentation on querying data.