AirbyteTriggerSyncOperator¶

Use the AirbyteTriggerSyncOperator to trigger an existing ConnectionId sync job in Airbyte.

Warning

This operator triggers a synchronization job in Airbyte. If triggered again, this operator does not guarantee idempotency. You must be aware of the source (database, API, etc) you are updating/sync and the method applied to perform the operation in Airbyte.

Using the Operator¶

The AirbyteTriggerSyncOperator requires the connection_id this is the uuid identifier create in Airbyte between a source and destination synchronization job. Use the airbyte_conn_id parameter to specify the Airbyte connection to use to connect to your account.

You can trigger a synchronization job in Airflow in two ways with the Operator. The first one is a synchronous process. This will trigger the Airbyte job and the Operator manage the status of the job. Another way is use the flag async = True so the Operator only trigger the job and return the job_id that should be pass to the AirbyteSensor.

An example using the synchronous way:

tests/system/providers/airbyte/example_airbyte_trigger_job.py[source]

    sync_source_destination = AirbyteTriggerSyncOperator(
        task_id="airbyte_sync_source_dest_example",
        connection_id=CONN_ID,
    )

An example using the async way:

tests/system/providers/airbyte/example_airbyte_trigger_job.py[source]

    async_source_destination = AirbyteTriggerSyncOperator(
        task_id="airbyte_async_source_dest_example",
        connection_id=CONN_ID,
        asynchronous=True,
    )

    airbyte_sensor = AirbyteJobSensor(
        task_id="airbyte_sensor_source_dest_example",
        airbyte_job_id=async_source_destination.output,
    )