airflow.providers.apache.hdfs.hooks.hdfs

Hook for HDFS operations

Module Contents

Classes

HDFSHook

Interact with HDFS. This class is a wrapper around the snakebite library.

Attributes

snakebite_loaded

airflow.providers.apache.hdfs.hooks.hdfs.snakebite_loaded = True[source]
exception airflow.providers.apache.hdfs.hooks.hdfs.HDFSHookException[source]

Bases: airflow.exceptions.AirflowException

Exception specific for HDFS

class airflow.providers.apache.hdfs.hooks.hdfs.HDFSHook(hdfs_conn_id='hdfs_default', proxy_user=None, autoconfig=False)[source]

Bases: airflow.hooks.base.BaseHook

Interact with HDFS. This class is a wrapper around the snakebite library.

Parameters

  • hdfs_conn_id (str) – Connection id to fetch connection info

  • proxy_user (str | None) – effective user for HDFS operations

  • autoconfig (bool) – use snakebite’s automatically configured client

conn_name_attr = 'hdfs_conn_id'[source]
default_conn_name = 'hdfs_default'[source]
conn_type = 'hdfs'[source]
hook_name = 'HDFS'[source]
get_conn()[source]

Returns a snakebite HDFSClient object.

Was this entry helpful?