AWS Step Functions¶

AWS Step Functions makes it easy to coordinate the components of distributed applications as a series of steps in a visual workflow. You can quickly build and run state machines to execute the steps of your application in a reliable and scalable fashion.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

Operators¶

Start an AWS Step Functions state machine execution¶

To start a new AWS Step Functions state machine execution you can use StepFunctionStartExecutionOperator.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

start_execution = StepFunctionStartExecutionOperator(
    task_id='start_execution', state_machine_arn=STEP_FUNCTIONS_STATE_MACHINE_ARN
)

Get an AWS Step Functions execution output¶

To fetch the output from an AWS Step Function state machine execution you can use StepFunctionGetExecutionOutputOperator.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

get_execution_output = StepFunctionGetExecutionOutputOperator(
    task_id='get_execution_output', execution_arn=start_execution.output
)

Sensors¶

Wait on an AWS Step Functions state machine execution state¶

To wait on the state of an AWS Step Function state machine execution until it reaches a terminal state you can use StepFunctionExecutionSensor.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

wait_for_execution = StepFunctionExecutionSensor(
    task_id='wait_for_execution', execution_arn=start_execution.output
)

References¶

AWS boto3 library documentation for Step Functions