apache-airflow-providers-edge3¶
apache-airflow-providers-edge3 package¶
Handle edge workers on remote sites via HTTP(s) connection and orchestrates work over distributed sites.
When tasks need to be executed on remote sites where the connection need to pass through firewalls or other network restrictions, the Edge Worker can be deployed. The Edge Worker is a lightweight process with reduced dependencies. The worker only needs to be able to communicate with the central Airflow site via HTTPS.
In the central Airflow site the EdgeExecutor is used to orchestrate the work. The EdgeExecutor is a custom executor which is used to schedule tasks on the edge workers. The EdgeExecutor can co-exist with other executors (for example CeleryExecutor or KubernetesExecutor) in the same Airflow site.
Additional REST API endpoints are provided to distribute tasks and manage the edge workers. The endpoints are provided by the API server.
Release: 1.1.0
Provider package¶
This package is for the edge3 provider.
All classes for this package are included in the airflow.providers.edge3 python package.
Installation¶
You can install this package on top of an existing Airflow 2 installation via
pip install apache-airflow-providers-edge3.
For the minimum Airflow version supported, see Requirements below.
Requirements¶
The minimum Apache Airflow version supported by this provider distribution is 2.10.0.
PIP package |
Version required |
|---|---|
|
|
|
|
|
|
|
|
Cross provider package dependencies¶
Those are dependencies that might be needed in order to use all the features of the package. You need to install the specified provider distributions in order to use them.
You can install such cross-provider dependencies when installing from PyPI. For example:
pip install apache-airflow-providers-edge3[fab]
Dependent package |
Extra |
|---|---|
|
Downloading official packages¶
You can download officially released packages and verify their checksums and signatures from the Official Apache Download site