airflow.providers.amazon.aws.operators.sagemaker_processing

Module Contents

class airflow.providers.amazon.aws.operators.sagemaker_processing.SageMakerProcessingOperator(*, config: dict, aws_conn_id: str, wait_for_completion: bool = True, print_log: bool = True, check_interval: int = 30, max_ingestion_time: Optional[int] = None, action_if_job_exists: str = 'increment', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.sagemaker_base.SageMakerBaseOperator

Initiate a SageMaker processing job.

This operator returns The ARN of the processing job created in Amazon SageMaker.

Parameters

  • config (dict) --

    The configuration necessary to start a processing job (templated).

    For details of the configuration parameter see SageMaker.Client.create_processing_job()

  • aws_conn_id (str) -- The AWS connection ID to use.

  • wait_for_completion (bool) -- If wait is set to True, the time interval, in seconds, that the operation waits to check the status of the processing job.

  • print_log (bool) -- if the operator should print the cloudwatch log during processing

  • check_interval (int) -- if wait is set to be true, this is the time interval in seconds which the operator will check the status of the processing job

  • max_ingestion_time (int) -- If wait is set to True, the operation fails if the processing job doesn't finish within max_ingestion_time seconds. If you set this parameter to None, the operation does not timeout.

  • action_if_job_exists (str) -- Behaviour if the job name already exists. Possible options are "increment" (default) and "fail".

expand_role(self)[source]
execute(self, context)[source]

Was this entry helpful?