AWS Step Functions Operators¶

AWS Step Functions makes it easy to coordinate the components of distributed applications as a series of steps in a visual workflow. You can quickly build and run state machines to execute the steps of your application in a reliable and scalable fashion.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation
Setup Connection.

AWS Step Functions Start Execution Operator¶

To start a new AWS Step Functions State Machine execution use StepFunctionStartExecutionOperator.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

start_execution = StepFunctionStartExecutionOperator(
    task_id='start_execution', state_machine_arn=STEP_FUNCTIONS_STATE_MACHINE_ARN
)

AWS Step Functions Execution Sensor¶

To wait on the state of an AWS Step Function State Machine execution until it reaches a terminal state you can use StepFunctionExecutionSensor.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

wait_for_execution = StepFunctionExecutionSensor(
    task_id='wait_for_execution', execution_arn=start_execution.output
)

AWS Step Functions Get Execution Output Operator¶

To fetch the output from an AWS Step Function State Machine execution you can use StepFunctionGetExecutionOutputOperator.

airflow/providers/amazon/aws/example_dags/example_step_functions.py[source]

get_execution_output = StepFunctionGetExecutionOutputOperator(
    task_id='get_execution_output', execution_arn=start_execution.output
)

References¶

For further information, look at:

Boto3 Library Documentation for Step Functions