airflow.providers.elasticsearch.utils.sql

Attributes

log

Functions

read_sql_to_polars(client, query[, params, ...])

Execute an Elasticsearch SQL query and return results as a Polars DataFrame.

Module Contents

airflow.providers.elasticsearch.utils.sql.log[source]
airflow.providers.elasticsearch.utils.sql.read_sql_to_polars(client, query, params=None, fetch_size=1000, max_rows=None)[source]

Execute an Elasticsearch SQL query and return results as a Polars DataFrame.

This uses Elasticsearch SQL cursor-based pagination instead of DB-API, as Elasticsearch does not provide a fully compliant DB-API interface.

Parameters:

  • client (elasticsearch.Elasticsearch) – Elasticsearch client

  • query (str) – SQL query string

  • params (collections.abc.Mapping[str, Any] | collections.abc.Iterable | None) – Optional query parameters

  • fetch_size (int) – Number of rows per batch

  • max_rows (int | None) – Optional limit on total rows fetched

Was this entry helpful?