MySQL to GCS operator.

Module Contents

class*, mysql_conn_id='mysql_default', ensure_utc=False, **kwargs)[source]


Copy data from MySQL to Google Cloud Storage in JSON or CSV format.

See also

For more information on how to use this operator, take a look at the guide: MySQLToGCSOperator

  • mysql_conn_id (str) -- Reference to mysql connection id.

  • ensure_utc (bool) -- Ensure TIMESTAMP columns exported as UTC. If set to False, TIMESTAMP columns will be exported using the MySQL server's default timezone.

ui_color = #a0e08c[source]

Queries mysql and returns a cursor to the results.

field_to_bigquery(self, field)[source]
convert_type(self, value, schema_type: str)[source]

Takes a value from MySQLdb, and converts it to a value that's safe for JSON/Google Cloud Storage/BigQuery.

  • Datetimes are converted to UTC seconds.

  • Decimals are converted to floats.

  • Dates are converted to ISO formatted string if given schema_type is DATE, or UTC seconds otherwise.

  • Binary type fields are converted to integer if given schema_type is INTEGER, or encoded with base64 otherwise. Imported BYTES data must be base64-encoded according to BigQuery documentation:

  • value (Any) -- MySQLdb column value

  • schema_type (str) -- BigQuery data type

Was this entry helpful?