airflow.providers.mysql.transfers.s3_to_mysql

Module Contents

class airflow.providers.mysql.transfers.s3_to_mysql.S3ToMySqlOperator(*, s3_source_key: str, mysql_table: str, mysql_duplicate_key_handling: str = 'IGNORE', mysql_extra_options: Optional[str] = None, aws_conn_id: str = 'aws_default', mysql_conn_id: str = 'mysql_default', **kwargs)[source]

Bases: airflow.models.BaseOperator

Loads a file from S3 into a MySQL table.

Parameters

  • s3_source_key (str) – The path to the file (S3 key) that will be loaded into MySQL.

  • mysql_table (str) – The MySQL table into where the data will be sent.

  • mysql_duplicate_key_handling (str) –

    Specify what should happen to duplicate data. You can choose either IGNORE or REPLACE.

    See also

    https://dev.mysql.com/doc/refman/8.0/en/load-data.html#load-data-duplicate-key-handling

  • mysql_extra_options (Optional[str]) – MySQL options to specify exactly how to load the data.

  • aws_conn_id (str) – The S3 connection that contains the credentials to the S3 Bucket.

  • mysql_conn_id (str) – The MySQL connection that contains the credentials to the MySQL data base.

template_fields = ['s3_source_key', 'mysql_table'][source]
template_ext = [][source]
ui_color = #f4a460[source]
execute(self, context: Dict)[source]

Executes the transfer operation from S3 to MySQL.

Parameters

context (dict) – The context that is being provided when executing.

Was this entry helpful?