Apache Hive to Amazon DynamoDB

Use the HiveToDynamoDBOperator transfer to copy the contents of an existing Apache Hive table to an existing Amazon DynamoDB table.

Prerequisite Tasks

To use these operators, you must do a few things:

Operators

Apache Hive to Amazon DynamoDB transfer operator

This operator replicates records from an Apache Hive table to an Amazon DynamoDB table. The user must specify an HQL query to use as filtering criteria.

To get more information visit: HiveToDynamoDBOperator

Example usage:

airflow/providers/amazon/aws/example_dags/example_hive_to_dynamodb.py[source]

backup_to_dynamodb = HiveToDynamoDBOperator(
    task_id="backup_to_dynamodb",
    hiveserver2_conn_id=HIVE_CONNECTION_ID,
    sql=HIVE_SQL,
    table_name=DYNAMODB_TABLE_NAME,
    table_keys=[DYNAMODB_TABLE_HASH_KEY],
)

Was this entry helpful?