airflow.providers.microsoft.azure.operators.adls_list

Module Contents

class airflow.providers.microsoft.azure.operators.adls_list.AzureDataLakeStorageListOperator(*, path: str, azure_data_lake_conn_id: str = 'azure_data_lake_default', **kwargs)[source]

Bases: airflow.models.BaseOperator

List all files from the specified path

This operator returns a python list with the names of files which can be used by

xcom in the downstream tasks.

Parameters

  • path (str) -- The Azure Data Lake path to find the objects. Supports glob strings (templated)

  • azure_data_lake_conn_id (str) -- The connection ID to use when connecting to Azure Data Lake Storage.

Example:

The following Operator would list all the Parquet files from folder/output/ folder in the specified ADLS account

adls_files = AzureDataLakeStorageListOperator(
    task_id='adls_files',
    path='folder/output/*.parquet',
    azure_data_lake_conn_id='azure_data_lake_default'
)
template_fields :Sequence[str] = ['path'][source]
ui_color = #901dd2[source]
execute(self, context: dict)[source]

Was this entry helpful?