Content¶
Guides
References
Resources
Commits
apache-airflow-providers-apache-spark¶
Provider package¶
This is a provider package for apache.spark provider. All classes for this provider package
are in airflow.providers.apache.spark python package.
Installation¶
Note
On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
pip install --upgrade pip==20.2.4 or, in case you use Pip 20.3, you need to add option
--use-deprecated legacy-resolver to your pip install command.
You can install this package on top of an existing airflow 2.* installation via
pip install apache-airflow-providers-apache-spark