airflow.providers.common.sql.datafusion.base

Classes

ObjectStorageProvider

Abstract base class for object storage providers.

FormatHandler

Abstract base class for format handlers.

Module Contents

class airflow.providers.common.sql.datafusion.base.ObjectStorageProvider(context=None)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin, abc.ABC

Abstract base class for object storage providers.

property get_storage_type: airflow.providers.common.sql.config.StorageType[source]
Abstractmethod:

Return storage type handled by this provider (e.g., ‘s3’, ‘gcs’, ‘local’).

abstract create_object_store(path, connection_config=None)[source]

Create and return a DataFusion object store instance.

abstract get_scheme()[source]

Return URL scheme for this storage type (e.g., ‘s3://’, ‘gs://’).

get_bucket(path)[source]

Extract the bucket name from the given path.

class airflow.providers.common.sql.datafusion.base.FormatHandler(datasource_config)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin, abc.ABC

Abstract base class for format handlers.

datasource_config[source]
property get_format: airflow.providers.common.sql.config.FormatType[source]
Abstractmethod:

Return file format type.

abstract register_data_source_format(ctx)[source]

Register data source format.

Was this entry helpful?