`airflow.providers.google.cloud.transfers.postgres_to_gcs`¶

PostgreSQL to GCS operator.

Module Contents¶

class airflow.providers.google.cloud.transfers.postgres_to_gcs._PostgresServerSideCursorDecorator(cursor)[source]¶

Inspired by _PrestoToGCSPrestoCursorAdapter to keep this consistent.

Decorator for allowing description to be available for postgres cursor in case server side cursor is used. It doesn’t provide other methods except those needed in BaseSQLToGCSOperator, which is more of a safety feature.

description[source]¶: Fetch first row to initialize cursor description when using server side cursor.

__iter__(self)[source]¶

__next__(self)[source]¶

class airflow.providers.google.cloud.transfers.postgres_to_gcs.PostgresToGCSOperator(*, postgres_conn_id='postgres_default', use_server_side_cursor=False, cursor_itersize=2000, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.transfers.sql_to_gcs.BaseSQLToGCSOperator

Copy data from Postgres to Google Cloud Storage in JSON or CSV format.

Parameters

postgres_conn_id (str) – Reference to a specific Postgres hook.
use_server_side_cursor (bool) – If server-side cursor should be used for querying postgres. For detailed info, check https://www.psycopg.org/docs/usage.html#server-side-cursors
cursor_itersize (int) – How many records are fetched at a time in case of server-side cursor.

ui_color = #a0e08c[source]¶

type_map[source]¶

_unique_name(self)[source]¶

query(self)[source]¶: Queries Postgres and returns a cursor to the results.

field_to_bigquery(self, field)[source]¶

convert_type(self, value, schema_type)[source]¶: Takes a value from Postgres, and converts it to a value that’s safe for JSON/Google Cloud Storage/BigQuery. Dates are converted to UTC seconds. Decimals are converted to floats. Times are converted to seconds.

airflow.providers.google.cloud.transfers.postgres_to_gcs¶

Module Contents¶

`airflow.providers.google.cloud.transfers.postgres_to_gcs`¶