airflow.hooks.pig_hook

Module Contents

class airflow.hooks.pig_hook.PigCliHook(pig_cli_conn_id='pig_cli_default')[source]

Bases: airflow.hooks.base_hook.BaseHook

Simple wrapper around the pig CLI.

Note that you can also set default pig CLI properties using the pig_properties to be used in your connection as in {"pig_properties": "-Dpig.tmpfilecompression=true"}

run_cli(self, pig, pig_opts=None, verbose=True)[source]

Run an pig script using the pig cli

>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;", pig_opts="-x mapreduce")
>>> ("hdfs://" in result)
True
kill(self)[source]

Was this entry helpful?