Amazon Redshift¶

Amazon Redshift manages all the work of setting up, operating, and scaling a data warehouse: provisioning capacity, monitoring and backing up the cluster, and applying patches and upgrades to the Amazon Redshift engine. You can focus on using your data to acquire new insights for your business and customers.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

Operators¶

Execute a SQL query¶

RedshiftSQLOperator executes a SQL query against an Amazon Redshift cluster using a Postgres connection.

To execute a SQL query against an Amazon Redshift cluster without using a Postgres connection, please check RedshiftDataOperator.

airflow/providers/amazon/aws/example_dags/example_redshift_sql.py[source]

task_select_data = RedshiftSQLOperator(
    task_id='task_get_all_table_data', sql="""CREATE TABLE more_fruit AS SELECT * FROM fruit;"""
)

RedshiftSQLOperator supports the parameters attribute which allows us to dynamically pass parameters into SQL statements.

airflow/providers/amazon/aws/example_dags/example_redshift_sql.py[source]

task_select_filtered_data = RedshiftSQLOperator(
    task_id='task_get_filtered_table_data',
    sql="""CREATE TABLE filtered_fruit AS SELECT * FROM fruit WHERE color = '{{ params.color }}';""",
    params={'color': 'Red'},
)

Amazon Redshift¶

Prerequisite Tasks¶

Operators¶

Execute a SQL query¶

Reference¶