airflow.providers.apache.pig.hooks.pig

Module Contents

Classes

PigCliHook

Simple wrapper around the pig CLI.
class airflow.providers.apache.pig.hooks.pig.PigCliHook(pig_cli_conn_id=default_conn_name, pig_properties=None)[source]

Bases: airflow.hooks.base.BaseHook

Simple wrapper around the pig CLI.

Parameters

  • pig_cli_conn_id (str) – Connection id used by the hook

  • pig_properties (list[str] | None) – additional properties added after pig cli command as list of strings.

conn_name_attr = 'pig_cli_conn_id'[source]
default_conn_name = 'pig_cli_default'[source]
conn_type = 'pig_cli'[source]
hook_name = 'Pig Client Wrapper'[source]
run_cli(pig, pig_opts=None, verbose=True)[source]

Run a pig script using the pig cli.

>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;", pig_opts="-x mapreduce")
>>> ("hdfs://" in result)
True
kill()[source]

Kill Pig job.

Was this entry helpful?