airflow.providers.presto.hooks.presto

Module Contents

Classes

PrestoHook

Interact with Presto through prestodb.

Functions

generate_presto_client_info()

Return json string with dag_id, task_id, execution_date and try_number
airflow.providers.presto.hooks.presto.generate_presto_client_info()[source]

Return json string with dag_id, task_id, execution_date and try_number

exception airflow.providers.presto.hooks.presto.PrestoException[source]

Bases: Exception

Presto exception

class airflow.providers.presto.hooks.presto.PrestoHook(*args, schema=None, log_sql=True, **kwargs)[source]

Bases: airflow.providers.common.sql.hooks.sql.DbApiHook

Interact with Presto through prestodb.

>>> ph = PrestoHook()
>>> sql = "SELECT count(1) AS num FROM airflow.static_babynames"
>>> ph.get_records(sql)
[[340698]]
conn_name_attr = 'presto_conn_id'[source]
default_conn_name = 'presto_default'[source]
conn_type = 'presto'[source]
hook_name = 'Presto'[source]
placeholder = '?'[source]
get_conn()[source]

Returns a connection object

get_isolation_level()[source]

Returns an isolation level

get_records(sql='', parameters=None)[source]

Executes the sql and returns a set of records.

Parameters

  • sql (str | list[str]) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters (Iterable | Mapping | None) – The parameters to render the SQL query with.

get_first(sql='', parameters=None)[source]

Executes the sql and returns the first resulting row.

Parameters

  • sql (str | list[str]) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters (Iterable | Mapping | None) – The parameters to render the SQL query with.

get_pandas_df(sql='', parameters=None, **kwargs)[source]

Executes the sql and returns a pandas dataframe

Parameters

  • sql (str) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters – The parameters to render the SQL query with.

  • kwargs – (optional) passed into pandas.io.sql.read_sql method

run(sql, autocommit=False, parameters=None, handler=None, split_statements=False, return_last=True)[source]

Runs a command or a list of commands. Pass a list of sql statements to the sql parameter to get them to execute sequentially.

The method will return either single query results (typically list of rows) or list of those results where each element in the list are results of one of the queries (typically list of list of rows :D)

For compatibility reasons, the behaviour of the DBAPIHook is somewhat confusing. In some cases, when multiple queries are run, the return value will be an iterable (list) of results – one for each query. However, in other cases, when single query is run, the return value will be the result of that single query without wrapping the results in a list.

The cases when single query results are returned without wrapping them in a list are as follows:

  1. sql is string and return_last is True (regardless what split_statements value is)

  2. sql is string and split_statements is False

In all other cases, the results are wrapped in a list, even if there is only one statement to process. In particular, the return value will be a list of query results in the following circumstances:

  1. when sql is an iterable of string statements (regardless what return_last value is)

  2. when sql is string, split_statements is True and return_last is False

After run is called, you may access the following properties on the hook object:

  • descriptions: an array of cursor descriptions. If return_last is True, this will be

    a one-element array containing the cursor description for the last statement. Otherwise, it will contain the cursor description for each statement executed.

  • last_description: the description for the last statement executed

Note that query result will ONLY be actually returned when a handler is provided; if handler is None, this method will return None.

Handler is a way to process the rows from cursor (Iterator) into a value that is suitable to be returned to XCom and generally fit in memory.

You can use pre-defined handles (fetch_all_handler`, ‘’fetch_one_handler``) or implement your own handler.

Parameters

  • sql (str | Iterable[str]) – the sql statement to be executed (str) or a list of sql statements to execute

  • autocommit (bool) – What to set the connection’s autocommit setting to before executing the query.

  • parameters (Iterable | Mapping | None) – The parameters to render the SQL query with.

  • handler (Callable | None) – The result handler which is called with the result of each statement.

  • split_statements (bool) – Whether to split a single SQL string into statements and run separately

  • return_last (bool) – Whether to return result for only last statement or for all after split

Returns

if handler provided, returns query results (may be list of results depending on params)

Return type

Any | list[Any] | None

insert_rows(table, rows, target_fields=None, commit_every=0, replace=False, **kwargs)[source]

A generic way to insert a set of tuples into a table.

Parameters

  • table (str) – Name of the target table

  • rows (Iterable[tuple]) – The rows to insert into the table

  • target_fields (Iterable[str] | None) – The names of the columns to fill in the table

  • commit_every (int) – The maximum number of rows to insert in one transaction. Set to 0 to insert all rows in one transaction.

  • replace (bool) – Whether to replace instead of insert

Was this entry helpful?