airflow.providers.amazon.aws.operators.emr_create_job_flow

Module Contents

Bases: airflow.models.BaseOperatorLink

Operator link for EmrCreateJobFlowOperator. It allows users to access the EMR Cluster

name = EMR Cluster[source]

Get link to EMR cluster.

Parameters

  • operator -- operator

  • dttm -- datetime

Returns

url link

class airflow.providers.amazon.aws.operators.emr_create_job_flow.EmrCreateJobFlowOperator(*, aws_conn_id: str = 'aws_default', emr_conn_id: str = 'emr_default', job_flow_overrides: Optional[Union[str, Dict[str, Any]]] = None, region_name: Optional[str] = None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Creates an EMR JobFlow, reading the config from the EMR connection. A dictionary of JobFlow overrides can be passed that override the config from the connection.

Parameters

  • aws_conn_id (str) -- aws connection to uses

  • emr_conn_id (str) -- emr connection to use

  • job_flow_overrides (dict|str) -- boto3 style arguments or reference to an arguments file (must be '.json') to override emr_connection extra. (templated)

  • region_name (Optional[str]) -- Region named passed to EmrHook

template_fields = ['job_flow_overrides'][source]
template_ext = ['.json'][source]
template_fields_renderers[source]
ui_color = #f9c915[source]
execute(self, context: Dict[str, Any])[source]

Was this entry helpful?