airflow.operators.generic_transfer

Module Contents

class airflow.operators.generic_transfer.GenericTransfer(*, sql: str, destination_table: str, source_conn_id: str, destination_conn_id: str, preoperator: Optional[Union[str, List[str]]] = None, insert_args: Optional[dict] = None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Moves data from a connection to another, assuming that they both provide the required methods in their respective hooks. The source hook needs to expose a get_records method, and the destination a insert_rows method.

This is meant to be used on small-ish datasets that fit in memory.

Parameters

  • sql (str) -- SQL query to execute against the source database. (templated)

  • destination_table (str) -- target table. (templated)

  • source_conn_id (str) -- source connection

  • destination_conn_id (str) -- source connection

  • preoperator (str or list[str]) -- sql statement or list of statements to be executed prior to loading the data. (templated)

  • insert_args (dict) -- extra params for insert_rows method.

template_fields = ['sql', 'destination_table', 'preoperator'][source]
template_ext = ['.sql', '.hql'][source]
template_fields_renderers[source]
ui_color = #b0f07c[source]
execute(self, context)[source]

Was this entry helpful?