Amazon Redshift Data

Amazon Redshift manages all the work of setting up, operating, and scaling a data warehouse: provisioning capacity, monitoring and backing up the cluster, and applying patches and upgrades to the Amazon Redshift engine. You can focus on using your data to acquire new insights for your business and customers.

Prerequisite Tasks

To use these operators, you must do a few things:

Operators

Execute a statement on an Amazon Redshift cluster

Use the RedshiftDataOperator to execute statements against an Amazon Redshift cluster.

This differs from RedshiftSQLOperator in that it allows users to query and retrieve data via the AWS API and avoid the necessity of a Postgres connection.

airflow/providers/amazon/aws/example_dags/example_redshift_data_execute_sql.py[source]

task_query = RedshiftDataOperator(
    task_id='redshift_query',
    cluster_identifier=REDSHIFT_CLUSTER_IDENTIFIER,
    database=REDSHIFT_DATABASE,
    db_user=REDSHIFT_DATABASE_USER,
    sql=REDSHIFT_QUERY,
    poll_interval=POLL_INTERVAL,
    await_result=True,
)

Was this entry helpful?