AWS Database Migration Service Operators

Prerequisite Tasks

To use these operators, you must do a few things:

Overview

Airflow to AWS Database Migration Service (DMS) integration provides several operators to create and interact with DMS replication tasks.

One example_dag is provided which showcases some of these operators in action.

  • example_dms_full_load_task.py

Create replication task, wait for it completion and delete it.

Purpose

This example dag example_dms_full_load_task.py uses DmsCreateTaskOperator, DmsStartTaskOperator, DmsTaskCompletedSensor, DmsDeleteTaskOperator to create replication task, start it, wait for it to be completed, and then delete it.

Defining tasks

In the following code we create a new replication task, start it, wait for it to be completed and then delete it.

airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.pyView Source

    create_task = DmsCreateTaskOperator(
        task_id='create_task',
        replication_task_id=REPLICATION_TASK_ID,
        source_endpoint_arn=SOURCE_ENDPOINT_ARN,
        target_endpoint_arn=TARGET_ENDPOINT_ARN,
        replication_instance_arn=REPLICATION_INSTANCE_ARN,
        table_mappings=TABLE_MAPPINGS,
    )

airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.pyView Source

    start_task = DmsStartTaskOperator(
        task_id='start_task',
        replication_task_arn=create_task.output,
    )

airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.pyView Source

    wait_for_completion = DmsTaskCompletedSensor(
        task_id='wait_for_completion',
        replication_task_arn=create_task.output,
    )

airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.pyView Source

    delete_task = DmsDeleteTaskOperator(
        task_id='delete_task',
        replication_task_arn=create_task.output,
    )

Reference

For further information, look at:

Was this entry helpful?