• Home
• airflow.providers.amazon
• airflow.providers.amazon.aws
• airflow.providers.amazon.aws.sensors
• airflow.providers.amazon.aws.sensors.sagemaker

airflow.providers.amazon.aws.sensors.sagemaker

Module Contents

Classes

SageMakerBaseSensor	Contains general sensor behavior for SageMaker.
SageMakerEndpointSensor	Asks for the state of the endpoint state until it reaches a
SageMakerTransformSensor	Asks for the state of the transform state until it reaches a
SageMakerTuningSensor	Asks for the state of the tuning state until it reaches a terminal
SageMakerTrainingSensor	Asks for the state of the training state until it reaches a

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerBaseSensor(*, aws_conn_id: str = 'aws_default', **kwargs)

Bases: airflow.sensors.base.BaseSensorOperator

Contains general sensor behavior for SageMaker.

Subclasses should implement get_sagemaker_response() and state_from_response() methods. Subclasses should also implement NON_TERMINAL_STATES and FAILED_STATE methods.

ui_color = #ededed

get_hook(self) → airflow.providers.amazon.aws.hooks.sagemaker.SageMakerHook

Get SageMakerHook.

poke(self, context: airflow.utils.context.Context)

Function that the sensors defined while deriving this class should override.

abstract non_terminal_states(self) → Set[str]

Placeholder for returning states with should not terminate.

abstract failed_states(self) → Set[str]

Placeholder for returning states with are considered failed.

abstract get_sagemaker_response(self) → Optional[dict]

Placeholder for checking status of a SageMaker task.

get_failed_reason_from_response(self, response: dict) → str

Placeholder for extracting the reason for failure from an AWS response.

abstract state_from_response(self, response: dict) → str

Placeholder for extracting the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerEndpointSensor(*, endpoint_name, **kwargs)

Bases: SageMakerBaseSensor

Asks for the state of the endpoint state until it reaches a terminal state. If it fails the sensor errors, the task fails.

Parameters

job_name (str) -- job_name of the endpoint instance to check the state of

template_fields :Sequence[str] = ['endpoint_name']

template_ext :Sequence[str] = []

non_terminal_states(self)

Placeholder for returning states with should not terminate.

failed_states(self)

Placeholder for returning states with are considered failed.

get_sagemaker_response(self)

Placeholder for checking status of a SageMaker task.

get_failed_reason_from_response(self, response)

Placeholder for extracting the reason for failure from an AWS response.

state_from_response(self, response)

Placeholder for extracting the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTransformSensor(*, job_name: str, **kwargs)

Bases: SageMakerBaseSensor

Asks for the state of the transform state until it reaches a terminal state. The sensor will error if the job errors, throwing a AirflowException containing the failure reason.

:param job_name: job_name of the transform job instance to check the state of

template_fields :Sequence[str] = ['job_name']

template_ext :Sequence[str] = []

non_terminal_states(self)

Placeholder for returning states with should not terminate.

failed_states(self)

Placeholder for returning states with are considered failed.

get_sagemaker_response(self)

Placeholder for checking status of a SageMaker task.

get_failed_reason_from_response(self, response)

Placeholder for extracting the reason for failure from an AWS response.

state_from_response(self, response)

Placeholder for extracting the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTuningSensor(*, job_name: str, **kwargs)

Bases: SageMakerBaseSensor

Asks for the state of the tuning state until it reaches a terminal state. The sensor will error if the job errors, throwing a AirflowException containing the failure reason.

:param job_name: job_name of the tuning instance to check the state of :type job_name: str

template_fields :Sequence[str] = ['job_name']

template_ext :Sequence[str] = []

non_terminal_states(self)

Placeholder for returning states with should not terminate.

failed_states(self)

Placeholder for returning states with are considered failed.

get_sagemaker_response(self)

Placeholder for checking status of a SageMaker task.

get_failed_reason_from_response(self, response)

Placeholder for extracting the reason for failure from an AWS response.

state_from_response(self, response)

Placeholder for extracting the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTrainingSensor(*, job_name, print_log=True, **kwargs)

Bases: SageMakerBaseSensor

Asks for the state of the training state until it reaches a terminal state. If it fails the sensor errors, failing the task.

Parameters

• job_name (str) -- name of the SageMaker training job to check the state of

• print_log (bool) -- if the operator should print the cloudwatch log

template_fields :Sequence[str] = ['job_name']

template_ext :Sequence[str] = []

init_log_resource(self, hook: airflow.providers.amazon.aws.hooks.sagemaker.SageMakerHook) → None

Set tailing LogState for associated training job.

non_terminal_states(self)

Placeholder for returning states with should not terminate.

failed_states(self)

Placeholder for returning states with are considered failed.

get_sagemaker_response(self)

Placeholder for checking status of a SageMaker task.

get_failed_reason_from_response(self, response)

Placeholder for extracting the reason for failure from an AWS response.

state_from_response(self, response)

Placeholder for extracting the state from an AWS response.

Previous Next

Was this entry helpful?