My First Project on NowWhat

Create a first NowWhat project, run an R job with SLURM, and start optional interactive sessions.

This tutorial shows how to organize a small project on NowWhat and run a simple R script through SLURM. It also includes optional examples for downloading OSF data and starting interactive RStudio Server or Jupyter sessions.

The source files for this tutorial are available in the dedicated tutorial folder.

What You Will Do

By the end of this tutorial, you will have:

  • created a project directory in your home folder
  • copied the tutorial scripts into the project
  • submitted a batch job with sbatch
  • generated a PDF plot from an R script
  • checked the SLURM logs and job output

Prerequisites

  • a NowWhat account
  • SSH access to nowwhat.stat.unipd.it
  • basic familiarity with the shell
  • the tutorial repository cloned on NowWhat

Get the Tutorial Files

Connect to NowWhat:

ssh your_username@nowwhat.stat.unipd.it

Clone the tutorials repository:

git clone https://nowwhat.stat.unipd.it/gitea/NowWhat/tutorials.git
cd tutorials/01-my-first-project

If you already cloned the repository, update it instead:

cd tutorials
git pull
cd 01-my-first-project

Inspect the Tutorial Layout

The tutorial contains:

01-my-first-project/
├── README.md
├── info.tsv
├── src/
│   └── example.R
├── scripts/
│   ├── download_osf.sh
│   ├── my_job.sbatch
│   ├── rpy-jupyter.sbatch
│   └── rstudio.sbatch
├── python-notebook.ipynb
└── r-notebook.ipynb

The main batch example is scripts/my_job.sbatch. It loads R and runs src/example.R, which creates a Gaussian density plot and saves it as ~/my_project/outputs/gaussian_plot.pdf.

Create Your Project Directory

Create a clean project area in your home directory:

mkdir -p ~/my_project/{src,scripts,logs,outputs,data}

Copy the tutorial files into it:

cp src/example.R ~/my_project/src/
cp scripts/*.sbatch ~/my_project/scripts/
cp scripts/download_osf.sh ~/my_project/scripts/
cp info.tsv ~/my_project/

Move into your project:

cd ~/my_project

Adapt the Batch Script

Open the batch script:

nano scripts/my_job.sbatch

Update these fields before submitting:

  • replace /home/rceccaroni/my_project with your own project path, for example /home/your_username/my_project
  • replace email@unipd.it with your email address, or remove the mail directives if you do not need email notifications
  • adjust --mem, --time, and --cpus-per-task if your real workload needs more resources

The important execution block is:

module purge
module load r
 
Rscript $HOME/my_project/src/example.R

This keeps the job reproducible: the module environment is reset, R is loaded explicitly, and the script is executed from your project directory.

Submit the Job

Submit the job from the login node:

sbatch scripts/my_job.sbatch

SLURM will print a job id:

Submitted batch job 123456

Check the queue:

squeue -u $USER

When the job finishes, inspect the logs:

ls logs
cat logs/my_job_123456.out
cat logs/my_job_123456.err

Replace 123456 with your actual job id.

Check the Result

The R script writes the output plot here:

ls outputs

Expected output:

gaussian_plot.pdf

You can copy the file to your local machine with scp:

scp your_username@nowwhat.stat.unipd.it:~/my_project/outputs/gaussian_plot.pdf .

Optional: Download OSF Data

The tutorial includes scripts/download_osf.sh, which downloads a directory from OSF project 5n4q3 into /projects/shared/osf.io/5n4q3.

Load the utility environment:

module purge
module load micromamba

Run the downloader with one of the dataset directory names listed in info.tsv:

bash scripts/download_osf.sh CosMx1k_MouseBrain2

The script normalizes the remote path and fetches all files under that OSF directory.

Optional: Start RStudio Server

The tutorial includes scripts/rstudio.sbatch as a batch template for RStudio Server.

Before using it, edit the same hard-coded values as in the batch job:

  • output and error log paths
  • email address
  • port number, if needed

Then submit:

sbatch scripts/rstudio.sbatch

When the job starts, identify the compute node from the log or with:

squeue -u $USER

Create an SSH tunnel from your local machine. Replace wn01, 8787, and your_username with the compute node, port, and username you are using:

ssh -N -L 8787:wn01:8787 your_username@nowwhat.stat.unipd.it

Open:

http://127.0.0.1:8787

For the full interactive workflow, see RStudio Server.

Optional: Start Jupyter

The tutorial also includes scripts/rpy-jupyter.sbatch, which starts JupyterLab inside the nw-rpy-ml environment:

micromamba run -n nw-rpy-ml jupyter lab $HOME/my_project --ip 0.0.0.0 --port 8888

Submit the job:

sbatch scripts/rpy-jupyter.sbatch

Then open a tunnel from your local machine:

ssh -N -L 8888:wn01:8888 your_username@nowwhat.stat.unipd.it

Open the local Jupyter URL and use the token printed by Jupyter:

http://127.0.0.1:8888

For the full interactive workflow, see Jupyter.

Clean Up

When you are done, cancel any still-running interactive jobs:

scancel JOB_ID

Remove temporary outputs only when you no longer need them:

rm -r ~/my_project/outputs