airflow.providers.airbyte.operators.airbyte

Module Contents

Classes

AirbyteTriggerSyncOperator

Submits a job to an Airbyte server to run a integration process between your source and destination.
class airflow.providers.airbyte.operators.airbyte.AirbyteTriggerSyncOperator(connection_id, airbyte_conn_id='airbyte_default', asynchronous=False, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), api_version='v1', api_type='config', wait_seconds=3, timeout=3600, **kwargs)[source]

Bases: airflow.models.BaseOperator

Submits a job to an Airbyte server to run a integration process between your source and destination.

See also

For more information on how to use this operator, take a look at the guide: AirbyteTriggerSyncOperator

Parameters

  • airbyte_conn_id (str) – Optional. The name of the Airflow connection to get connection information for Airbyte. Defaults to “airbyte_default”.

  • connection_id (str) – Required. The Airbyte ConnectionId UUID between a source and destination.

  • asynchronous (bool) – Optional. Flag to get job_id after submitting the job to the Airbyte API. This is useful for submitting long running jobs and waiting on them asynchronously using the AirbyteJobSensor. Defaults to False.

  • deferrable (bool) – Run operator in the deferrable mode.

  • api_version (str) – Optional. Airbyte API version. Defaults to “v1”.

  • api_type (Literal[config, cloud]) – Optional. The type of Airbyte API to use. Either “config” or “cloud”. Defaults to “config”.

  • wait_seconds (float) – Optional. Number of seconds between checks. Only used when asynchronous is False. Defaults to 3 seconds.

  • timeout (float) – Optional. The amount of time, in seconds, to wait for the request to complete. Only used when asynchronous is False. Defaults to 3600 seconds (or 1 hour).

template_fields: Sequence[str] = ('connection_id',)[source]
ui_color = '#6C51FD'[source]
execute(context)[source]

Create Airbyte Job and wait to finish.

execute_complete(context, event=None)[source]

Invoke this callback when the trigger fires; return immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

on_kill()[source]

Cancel the job if task is cancelled.

Was this entry helpful?