airflow.providers.google.cloud.transfers.presto_to_gcs

Module Contents

class airflow.providers.google.cloud.transfers.presto_to_gcs._PrestoToGCSPrestoCursorAdapter(cursor: PrestoCursor)[source]

An adapter that adds additional feature to the Presto cursor.

The implementation of cursor in the prestodb library is not sufficient. The following changes have been made:

  • The poke mechanism for row. You can look at the next row without consuming it.

  • The description attribute is available before reading the first row. Thanks to the poke mechanism.

  • the iterator interface has been implemented.

A detailed description of the class methods is available in PEP-249.

description[source]

This read-only attribute is a sequence of 7-item sequences.

Each of these sequences contains information describing one result column:

  • name

  • type_code

  • display_size

  • internal_size

  • precision

  • scale

  • null_ok

The first two items (name and type_code) are mandatory, the other five are optional and are set to None if no meaningful values can be provided.

rowcount[source]

The read-only attribute specifies the number of rows

close(self)[source]

Close the cursor now

execute(self, *args, **kwargs)[source]

Prepare and execute a database operation (query or command).

executemany(self, *args, **kwargs)[source]

Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters.

peekone(self)[source]

Return the next row without consuming it.

fetchone(self)[source]

Fetch the next row of a query result set, returning a single sequence, or None when no more data is available.

fetchmany(self, size=None)[source]

Fetch the next set of rows of a query result, returning a sequence of sequences (e.g. a list of tuples). An empty sequence is returned when no more rows are available.

__next__(self)[source]

Return the next row from the currently executing SQL statement using the same semantics as .fetchone(). A StopIteration exception is raised when the result set is exhausted. :return:

__iter__(self)[source]

Return self to make cursors compatible to the iteration protocol

class airflow.providers.google.cloud.transfers.presto_to_gcs.PrestoToGCSOperator(*, presto_conn_id: str = 'presto_default', **kwargs)[source]

Bases: airflow.providers.google.cloud.transfers.sql_to_gcs.BaseSQLToGCSOperator

Copy data from PrestoDB to Google Cloud Storage in JSON or CSV format.

Parameters

presto_conn_id (str) -- Reference to a specific Presto hook.

ui_color = #a0e08c[source]
type_map[source]
query(self)[source]

Queries presto and returns a cursor to the results.

field_to_bigquery(self, field)[source]

Convert presto field type to BigQuery field type.

convert_type(self, value, schema_type)[source]

Do nothing. Presto uses JSON on the transport layer, so types are simple.

Parameters

  • value (Any) -- Presto column value

  • schema_type (str) -- BigQuery data type

Was this entry helpful?