File Transfer

Use the login node for file transfer:

nowwhat.stat.unipd.it

The practical storage rule is:

• keep scripts, code, notebooks, configuration, and lightweight project files in your home directory
• keep datasets in /projects/shared
• copy large outputs back to your workstation only when you need them locally

Choose the Destination First

Before copying files, decide what kind of file you are moving.

Do not put large datasets in your home directory.

Create Remote Folders

Create the folders you need before copying files:

ssh your_username@nowwhat.stat.unipd.it
mkdir -p ~/my-project/scritps
mkdir -p /projects/shared/my-dataset

If you cannot create a folder under /projects/shared, contact nowwhat@stat.unipd.it and ask for the appropriate shared location.

Copy Scripts to Your Home Directory

Copy a single job script:

scp run-analysis.slurm your_username@nowwhat.stat.unipd.it:~/my-project/scripts/

Copy a local scripts folder:

scp -r scripts/ your_username@nowwhat.stat.unipd.it:~/my-project/

For repeated updates, prefer rsync because it copies only changes:

rsync -av scripts/ your_username@nowwhat.stat.unipd.it:~/my-project/scripts/

Copy Datasets to /projects/shared

Copy one dataset directory:

rsync -av dataset/ your_username@nowwhat.stat.unipd.it:/projects/shared/my-dataset/

Copy a compressed dataset archive:

scp dataset.tar.gz your_username@nowwhat.stat.unipd.it:/projects/shared/my-dataset/

Then unpack it on NowWhat:

ssh your_username@nowwhat.stat.unipd.it
cd /projects/shared/my-dataset
tar -xzf dataset.tar.gz

For large datasets, prefer rsync because interrupted transfers can be resumed:

rsync -av --partial --progress dataset/ your_username@nowwhat.stat.unipd.it:/projects/shared/my-dataset/

Copy Results Back to Your Workstation

Copy one result file from NowWhat to your current local directory:

scp your_username@nowwhat.stat.unipd.it:~/my-project/results/summary.csv .

Copy a results folder:

rsync -av your_username@nowwhat.stat.unipd.it:~/my-project/results/ ./results/

Copy shared results from /projects/shared:

rsync -av your_username@nowwhat.stat.unipd.it:/projects/shared/my-project/results/ ./results/

Download a Dataset from the Internet

ssh your_username@nowwhat.stat.unipd.it
mkdir -p /projects/shared/my-dataset
cd /projects/shared/my-dataset
wget https://example.org/data/file.csv

For large downloads, please contact nowwhat@stat.unipd.it

Check Before Submitting Jobs

Before submitting a job, verify that the paths used by your script match where you copied the files:

ssh your_username@nowwhat.stat.unipd.it
ls ~/my-project/scripts
ls /projects/shared/my-dataset

Inside job scripts, refer to datasets with explicit /projects/shared/... paths and keep script paths under your home project directory.