airflow.operators.pig_operator

Module Contents

class airflow.operators.pig_operator.PigOperator(pig, pig_cli_conn_id='pig_cli_default', pigparams_jinja_translate=False, pig_opts=None, *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Executes pig script.

Parameters

  • pig (str) – the pig latin script to be executed. (templated)

  • pig_cli_conn_id (str) – reference to the Hive database

  • pigparams_jinja_translate (bool) – when True, pig params-type templating ${var} gets translated into jinja-type templating {{ var }}. Note that you may want to use this along with the DAG(user_defined_macros=myargs) parameter. View the DAG object documentation for more details.

  • pig_opts (str) – pig options, such as: -x tez, -useHCatalog, …

template_fields = ['pig'][source]
template_ext = ['.pig', '.piglatin'][source]
ui_color = #f0e4ec[source]
get_hook(self)[source]
prepare_template(self)[source]
execute(self, context)[source]
on_kill(self)[source]