airflow.hooks.hdfs_hook

Module Contents

airflow.hooks.hdfs_hook.snakebite_loaded = True

exception airflow.hooks.hdfs_hook.HDFSHookException

Bases: airflow.exceptions.AirflowException

class airflow.hooks.hdfs_hook.HDFSHook(hdfs_conn_id='hdfs_default', proxy_user=None, autoconfig=False)

Bases: airflow.hooks.base_hook.BaseHook

Interact with HDFS. This class is a wrapper around the snakebite library.

Parameters

• hdfs_conn_id (str) – Connection id to fetch connection info

• proxy_user (str) – effective user for HDFS operations

• autoconfig (bool) – use snakebite’s automatically configured client

get_conn(self)

Returns a snakebite HDFSClient object.