Module Contents



Execute a command inside a Singularity container

class airflow.providers.singularity.operators.singularity.SingularityOperator(*, image, command, start_command=None, environment=None, pull_folder=None, working_dir=None, force_pull=False, volumes=None, options=None, auto_remove=False, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a command inside a Singularity container

Singularity has more seamless connection to the host than Docker, so no special binds are needed to ensure binding content in the user $HOME and temporary directories. If the user needs custom binds, this can be done with --volumes

  • image (str) -- Singularity image or URI from which to create the container.

  • auto_remove (Optional[bool]) -- Delete the container when the process exits. The default is False.

  • command (Union[str, ast.AST]) -- Command to be run in the container. (templated)

  • start_command (Optional[Union[str, List[str]]]) -- Start command to pass to the container instance.

  • environment (Optional[Dict[str, Any]]) -- Environment variables to set in the container. (templated)

  • working_dir (Optional[str]) -- Set a working directory for the instance.

  • force_pull (Optional[bool]) -- Pull the image on every run. Default is False.

  • volumes (Optional[List[str]]) -- List of volumes to mount into the container, e.g. ['/host/path:/container/path', '/host/path2:/container/path2'].

  • options (Optional[List[str]]) -- Other flags (list) to provide to the instance start.

  • working_dir -- Working directory to set on the container (equivalent to the -w switch the docker client).

template_fields :Sequence[str] = ['command', 'environment'][source]
template_ext :Sequence[str] = ['.sh', '.bash'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.


Override this method to cleanup subprocesses when a task instance gets killed. Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up or it will leave ghost processes behind.

Was this entry helpful?