Qubole is an open, simple, and secure data lake platform for machine learning, streaming and adhoc analytics. Qubole delivers a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds.
Airflow provides operators to execute tasks (commands) on QDS and perform checks against Qubole Commands.
To performs checks against Qubole Commands you can use
Operator expects a command that will be executed on QDS.
By default, each value on first row of the result of this Qubole Command is evaluated using python
If any of the values return
False, the check is failed and errors out.