Apptainer

How NowWhat Apptainer containers, modules, wrappers, overlays, and GPU images work.

This page explains how Apptainer-based modules behave on NowWhat from a user point of view.

The goal is to help you use these modules correctly in interactive sessions and in SLURM jobs, especially when you need Python or R packages on top of the base container.

What These Modules Provide

An Apptainer module exposes software through ordinary shell commands, but those commands actually run inside a container image managed by the platform.

Typical examples are:

apptainer-python/3.11
apptainer-R/4.5
apptainer-jupyter/latest
apptainer-pytorch/25.03

After you load one of these modules, you usually work with standard commands such as:

python
pip
R
Rscript
jupyter
jupyter-lab
rstudio
bash
nvcc

In most cases you do not need to write apptainer exec ... yourself. The module adds wrapper commands to your PATH, and those wrappers start the corresponding command inside the correct container.

Basic Usage Pattern

Load only the module you need:

module purge
module load apptainer-python/3.11
python --version

The same pattern applies in job scripts:

module purge
module load apptainer-R/4.5
R --version

This is the recommended interface. Treat the module as the entry point, not the container image path.

What Happens When You Load the Module

When you load an Apptainer module, the modulefile mainly does three things:

it prepends a wrapper directory to PATH
it exports a few container-related environment variables
for Python and R images, it sets the user package locations expected inside the container

This design keeps the module lightweight. The actual runtime logic lives in the wrapper scripts that the module exposes.

Wrapper Commands and `sapp`

Each module can expose one or more commands.

Examples:

apptainer-python exposes python, pip, and sapp
apptainer-R exposes R, Rscript, rstudio, and sapp
apptainer-cuda exposes bash, nvcc, nvidia-smi, and sapp

The sapp command is the generic entry point for running an arbitrary command inside the container:

module purge
module load apptainer-python/3.11
sapp python -c "import sys; print(sys.version)"

Use sapp when:

the module does not expose the exact command you want as a dedicated wrapper
you want to run a custom command directly inside the container

Use the dedicated wrappers such as python, R, jupyter, or nvcc when available, because they are clearer and easier to read in scripts.

Utility Commands

NowWhat also provides apptainer-utils for helper commands such as export-container. See Apptainer Utils.

The Container Image Variable

Each module also exports an environment variable pointing to the underlying .sif image.

The variable name is derived from the module name by uppercasing it and replacing - with _.

Examples:

apptainer-python -> APPTAINER_PYTHON
apptainer-R -> APPTAINER_R
apptainer-rstudio-server -> APPTAINER_RSTUDIO_SERVER

Example:

module purge
module load apptainer-python/3.11
echo "$APPTAINER_PYTHON"
apptainer exec "$APPTAINER_PYTHON" python --version

You usually do not need this variable for normal work, but it is useful when you need a manual apptainer exec for debugging or advanced workflows.

Persistent User Overlay

For many Apptainer modules, the base image is read-only but each user also gets a personal writable overlay image stored in home:

$HOME/containers/<module>/<version>.img

Important points:

the overlay is created automatically the first time you run a wrapper from that module
it is a sparse image with a default size of 4 GB
it is private to your account
it persists across sessions and jobs

This is the mechanism that lets Python and R packages survive after installation. You are not modifying the shared base image; you are adding your own layer on top of it.

How Concurrency Works

Overlay access is intentionally conservative.

By default, wrappers run with:

HPC_CONTAINER_OVERLAY_MODE=auto

In auto mode:

the first running instance that gets the lock mounts the overlay read-write
a concurrent instance that cannot get that lock runs without the overlay

That second behavior is important: it does not see your personal overlay contents. It only sees the shared base container.

In practice this means:

if you installed user packages and want them to be visible, avoid launching many simultaneous sessions from the same module unless you understand the consequences
concurrent read-only commands may show a different environment from the one seen by the instance that holds the overlay lock

You can change the runtime behavior explicitly:

export HPC_CONTAINER_OVERLAY_MODE=rw
module load apptainer-python/3.11
python

Modes:

auto: default, best general-purpose behavior
rw: require read-write overlay access; if the overlay is busy, the command fails
ro: mount the overlay read-only; useful when you want consistent visibility of already-installed content without allowing writes

ro and rw must be set before launching the wrapped command.

Python Modules

Python-based modules set up a persistent user installation area inside the container and configure pip to use it.

When you load a Python container module, the modulefile sets:

PIP_USER=1
PYTHONUSERBASE=/srv/python
APPTAINERENV_PYTHONUSERBASE=/srv/python
PIP_DISABLE_PIP_VERSION_CHECK=1

It also sets PIP_CACHE_DIR:

$SCRATCH/.pip-cache if SCRATCH is defined
otherwise $HOME/.cache/pip

The practical consequence is that this works as expected:

module purge
module load apptainer-python/3.11
pip install --user numpy
python -c "import numpy; print(numpy.__version__)"

Notes:

pip install --user writes into your personal overlay-backed area inside the container
the installed packages remain available in later sessions
if another process already holds the overlay in read-write mode, pip install --user ... fails instead of silently running without write access

This is a good safety property: package installation either gets writable access or stops.

R Modules

R-based modules configure a user library path inside the container and preserve it through the overlay.

When you load an R container module, the modulefile sets:

R_LIBS_USER=/srv/R/library
APPTAINERENV_R_LIBS_USER=/srv/R/library
APPTAINERENV_R_LIBS=/srv/R/library

This means user-installed R packages are expected to live in that container-side path, backed by your overlay.

Typical usage:

module purge
module load apptainer-R/4.5
R

Then inside R:

install.packages("data.table")
library(data.table)

The installed package remains in your personal overlay for future runs of the same module version.

As with Python, concurrent sessions matter: if a second instance runs without the overlay, it will not see packages that only exist in your personal layer.

GPU-Enabled Containers

Several modules are configured to pass GPU support automatically when their wrappers start the container.

For those modules, the wrapper uses apptainer exec --nv ... instead of plain apptainer exec ....

This is relevant for modules such as:

apptainer-pytorch
apptainer-tensorflow
apptainer-cuda
the Python and R images, which are also configured to allow GPU passthrough when used on GPU nodes

What this means for users:

on a GPU node, the wrapped command can access NVIDIA devices without extra wrapper logic from you
on a non-GPU node, the module still loads, but GPU-dependent workloads obviously cannot use devices that are not present

You still need to request GPU resources through SLURM when your workload requires them.

Shared Project Bind Mount

All wrapper commands bind /projects from the host into the container at the same path:

/projects -> /projects

This means paths under /projects remain stable when you move between host-side shell usage and containerized commands.

If your workflow depends on shared project data, use that path consistently.

Prompt and Container Identity

The wrappers also export container identity variables into the runtime environment:

CONTAINER_NAME
CONTAINER_VERSION
CONTAINER_KEY

They also set matching APPTAINERENV_* variables so the values are visible inside the container.

Interactive shells started through the wrappers get a prompt marker similar to:

[apptainer-python/3.11] [...]

This is only a convenience feature, but it helps you notice when you are operating inside a container-backed environment.

Jupyter and RStudio Wrappers

Some modules expose service-style commands such as jupyter, jupyter-lab, or rstudio.

These are still container-backed wrappers. The main user-facing rule does not change:

start them on compute nodes, not on the login node
load the module first
then run the exported command normally

For RStudio specifically, the wrapper supplies sensible defaults if you do not pass them explicitly:

bind address defaults to 0.0.0.0
port defaults to 8788

You can still override them with arguments such as:

rstudio --ip 0.0.0.0 --port 8787

For complete service workflows, see the dedicated Jupyter and RStudio Server pages.

Recommended User Practices

start with module purge
load exactly one version of the Apptainer module you need
use the exported wrapper commands rather than hand-written apptainer exec ...
install Python packages with pip install --user
install R packages through the usual R mechanisms and let R_LIBS_USER handle placement
avoid concurrent sessions from the same module when you depend on packages stored in your personal overlay
run notebook servers, RStudio, and GPU workloads only inside appropriate SLURM allocations

Troubleshooting Expectations

If a command works after module load but does not see a package you previously installed, the most likely causes are:

you loaded a different module version
the package was installed in your personal overlay for another module
the current process is running without the overlay because another instance already holds the read-write lock

If you need stable visibility of already-installed content across multiple readers, try a read-only overlay run:

export HPC_CONTAINER_OVERLAY_MODE=ro
module purge
module load apptainer-python/3.11
python -c "import numpy"

If you need to install or update packages, use a single writable session instead:

export HPC_CONTAINER_OVERLAY_MODE=rw
module purge
module load apptainer-python/3.11
pip install --user numpy

If a required module or wrapper command is missing, or if container behavior looks inconsistent across nodes, contact nowwhat@stat.unipd.it.

Current Catalogue

Containers lists all Apptainer modules and image paths.
Container Variables documents exported container variables.
Apptainer Utils documents helper commands such as export-container.