• Home
• airflow.providers.amazon
• airflow.providers.amazon.aws
• airflow.providers.amazon.aws.sensors
• airflow.providers.amazon.aws.sensors.sagemaker

airflow.providers.amazon.aws.sensors.sagemaker

Classes

SageMakerBaseSensor	Contains general sensor behavior for SageMaker.
SageMakerEndpointSensor	Poll the endpoint state until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerTransformSensor	Poll the transform job until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerTuningSensor	Poll the tuning state until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerTrainingSensor	Poll the training job until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerPipelineSensor	Poll the pipeline until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerAutoMLSensor	Poll the auto ML job until it reaches a terminal state; raise AirflowException with the failure reason.
SageMakerProcessingSensor	Poll the processing job until it reaches a terminal state; raise AirflowException with the failure reason.

Module Contents

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerBaseSensor(*, resource_type='job', **kwargs)

Bases: airflow.providers.amazon.aws.sensors.base_aws.AwsBaseSensor[airflow.providers.amazon.aws.hooks.sagemaker.SageMakerHook]

Contains general sensor behavior for SageMaker.

Subclasses should implement get_sagemaker_response() and state_from_response() methods. Subclasses should also implement NON_TERMINAL_STATES and FAILED_STATE methods.

aws_hook_class

ui_color = '#ededed'

resource_type = 'job'

poke(context)

Override when deriving this class.

abstract non_terminal_states()

Return states with should not terminate.

abstract failed_states()

Return states with are considered failed.

abstract get_sagemaker_response()

Check status of a SageMaker task.

get_failed_reason_from_response(response)

Extract the reason for failure from an AWS response.

abstract state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerEndpointSensor(*, endpoint_name, **kwargs)

Bases: SageMakerBaseSensor

Poll the endpoint state until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker endpoint state

Parameters:

endpoint_name – Name of the endpoint instance to watch.

template_fields: collections.abc.Sequence[str]

template_ext: collections.abc.Sequence[str] = ()

endpoint_name

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

get_failed_reason_from_response(response)

Extract the reason for failure from an AWS response.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTransformSensor(*, job_name, **kwargs)

Bases: SageMakerBaseSensor

Poll the transform job until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker transform job state

Parameters:

job_name (str) – Name of the transform job to watch.

template_fields: collections.abc.Sequence[str]

template_ext: collections.abc.Sequence[str] = ()

job_name

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

get_failed_reason_from_response(response)

Extract the reason for failure from an AWS response.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTuningSensor(*, job_name, **kwargs)

Bases: SageMakerBaseSensor

Poll the tuning state until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker tuning job state

Parameters:

job_name (str) – Name of the tuning instance to watch.

template_fields: collections.abc.Sequence[str]

template_ext: collections.abc.Sequence[str] = ()

job_name

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

get_failed_reason_from_response(response)

Extract the reason for failure from an AWS response.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTrainingSensor(*, job_name, print_log=True, **kwargs)

Bases: SageMakerBaseSensor

Poll the training job until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker training job state

Parameters:

• job_name – Name of the training job to watch.

• print_log – Prints the cloudwatch log if True; Defaults to True.

template_fields: collections.abc.Sequence[str]

template_ext: collections.abc.Sequence[str] = ()

job_name

print_log = True

positions

stream_names = []

instance_count: int | None = None

state: int | None = None

last_description = None

last_describe_job_call = None

log_resource_inited = False

init_log_resource(hook)

Set tailing LogState for associated training job.

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

get_failed_reason_from_response(response)

Extract the reason for failure from an AWS response.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerPipelineSensor(*, pipeline_exec_arn, verbose=True, **kwargs)

Bases: SageMakerBaseSensor

Poll the pipeline until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker pipeline execution state

Parameters:

• pipeline_exec_arn (str) – ARN of the pipeline to watch.

• verbose (bool) – Whether to print steps details while waiting for completion. Defaults to true, consider turning off for pipelines that have thousands of steps.

template_fields: collections.abc.Sequence[str]

pipeline_exec_arn

verbose = True

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerAutoMLSensor(*, job_name, **kwargs)

Bases: SageMakerBaseSensor

Poll the auto ML job until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker AutoML experiment state

Parameters:

job_name (str) – unique name of the AutoML job to watch.

template_fields: collections.abc.Sequence[str]

job_name

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

state_from_response(response)

Extract the state from an AWS response.

class airflow.providers.amazon.aws.sensors.sagemaker.SageMakerProcessingSensor(*, job_name, **kwargs)

Bases: SageMakerBaseSensor

Poll the processing job until it reaches a terminal state; raise AirflowException with the failure reason.

See also

For more information on how to use this sensor, take a look at the guide: Wait on an Amazon SageMaker processing job state

Parameters:

job_name (str) – Name of the processing job to watch.

template_fields: collections.abc.Sequence[str]

template_ext: collections.abc.Sequence[str] = ()

job_name

non_terminal_states()

Return states with should not terminate.

failed_states()

Return states with are considered failed.

get_sagemaker_response()

Check status of a SageMaker task.

state_from_response(response)

Extract the state from an AWS response.

Previous Next

Was this entry helpful?