Amazon DynamoDB¶

Amazon DynamoDB Amazon DynamoDB is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation of Apache Airflow®
Setup Connection.

Sensors¶

Wait on Amazon DynamoDB item attribute value match¶

Use the DynamoDBValueSensor to wait for the presence of a matching DynamoDB item’s attribute/value pair.

Wait for a Single Attribute Value Match:¶

This example shows how to use DynamoDBValueSensor to wait for a specific attribute/value pair in a DynamoDB item.

tests/system/providers/amazon/aws/example_dynamodb.py[source]

dynamodb_sensor = DynamoDBValueSensor(
    task_id="waiting_for_dynamodb_item_value",
    table_name=table_name,
    partition_key_name=PK_ATTRIBUTE_NAME,
    partition_key_value="Test",
    sort_key_name=SK_ATTRIBUTE_NAME,
    sort_key_value="2022-07-12T11:11:25-0400",
    attribute_name="Value",
    attribute_value="Testing",
)

Wait for Any Value from a List of Attribute Values:¶

In this example, the sensor waits for the DynamoDB item to have an attribute that matches any value from a provided list.

tests/system/providers/amazon/aws/example_dynamodb.py[source]

dynamodb_sensor_any_value = DynamoDBValueSensor(
    task_id="waiting_for_dynamodb_item_any_value",
    table_name=table_name,
    partition_key_name=PK_ATTRIBUTE_NAME,
    partition_key_value="Test",
    sort_key_name=SK_ATTRIBUTE_NAME,
    sort_key_value="2022-07-12T11:11:25-0400",
    attribute_name="Value",
    attribute_value=["Foo", "Testing", "Bar"],
)

Reference¶

AWS boto3 library documentation for DynamoDB