Amazon Athena¶

Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon Simple Storage Service (S3) using standard SQL. Athena is serverless, so there is no infrastructure to setup or manage, and you pay only for the queries you run. To get started, simply point to your data in S3, define the schema, and start querying using standard SQL.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

Operators¶

Run a query in Amazon Athena¶

Use the AthenaOperator to run a query in Amazon Athena.

In the following example, we query an existing Athena table and send the results to an existing Amazon S3 bucket. For more examples of how to use this operator, please see the Sample DAG.

tests/system/providers/amazon/aws/example_athena.py[source]

read_table = AthenaOperator(
    task_id='read_table',
    query=query_read_table,
    database=athena_database,
    output_location=f's3://{s3_bucket}/',
)

Sensors¶

Wait on Amazon Athena query results¶

Use the AthenaSensor to wait for the results of a query in Amazon Athena.

tests/system/providers/amazon/aws/example_athena.py[source]

await_query = AthenaSensor(
    task_id='await_query',
    query_execution_id=read_table.output,
)

Reference¶

AWS boto3 library documentation for Athena