:mod:`airflow.contrib.operators.postgres_to_gcs_operator`
=========================================================

.. py:module:: airflow.contrib.operators.postgres_to_gcs_operator







Module Contents
---------------






.. data:: PY3
   

   









.. py:class:: PostgresToGoogleCloudStorageOperator(sql, bucket, filename, schema_filename=None, approx_max_file_size_bytes=1900000000, postgres_conn_id='postgres_default', google_cloud_storage_conn_id='google_cloud_default', delegate_to=None, parameters=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Copy data from Postgres to Google Cloud Storage in JSON format.


   

   

   .. attribute:: template_fields
      :annotation: = ['sql', 'bucket', 'filename', 'schema_filename', 'parameters'] 

      



   

   

   .. attribute:: template_ext
      :annotation: = ['.sql'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #a0e08c 

      



   

   

   

   .. method:: execute(self, context)

      



   

   .. method:: _query_postgres(self)

      
      Queries Postgres and returns a cursor to the results.

      



   

   .. method:: _write_local_data_files(self, cursor)

      
      Takes a cursor, and writes results to a local file.

      :return: A dictionary where keys are filenames to be used as object
          names in GCS, and values are file handles to local files that
          contain the data for the GCS objects.

      



   

   .. method:: _write_local_schema_file(self, cursor)

      
      Takes a cursor, and writes the BigQuery schema for the results to a
      local file system.

      :return: A dictionary where key is a filename to be used as an object
          name in GCS, and values are file handles to local files that
          contains the BigQuery schema fields in .json format.

      



   

   .. method:: _upload_to_gcs(self, files_to_upload)

      
      Upload all of the file splits (and optionally the schema .json file) to
      Google Cloud Storage.

      



   

   .. classmethod:: convert_types(cls, value)

      
      Takes a value from Postgres, and converts it to a value that's safe for
      JSON/Google Cloud Storage/BigQuery. Dates are converted to UTC seconds.
      Decimals are converted to floats. Times are converted to seconds.

      



   

   .. classmethod:: type_map(cls, postgres_type)

      
      Helper function that maps from Postgres fields to BigQuery fields. Used
      when a schema_filename is set.