Storage

Writable areas, project storage, and practical storage rules for NowWhat.

This page explains where to place code, datasets, and shared material. Correct placement matters because poor storage choices are one of the easiest ways to damage performance, clutter shared space, or confuse collaborators.

Writable Areas

Users can write files in:

  • their home directory
  • the /projects directory

At present, the /projects area includes the shared subdirectory shared, which is accessible to all NowWhat users.

If you need a project area shared only with a subset of users, contact nowwhat@stat.unipd.it.

Home Directory

Use your home directory for:

  • source code
  • shell scripts
  • configuration files
  • software environments
  • notebooks

This is usually the best place for active development work and project structure.

Projects Directory

Use /projects/shared for:

  • large datasets
  • shared datasets
  • files used by multiple users
  • data that would be awkward to duplicate in several home directories

The practical rule is simple:

  • keep code and lightweight project structure in your home directory
  • keep datasets in /projects/shared

Suggested Project Layout

A maintainable layout often looks like this:

~/my-project/
├── configs/
├── env/
├── logs/
├── scripts/
├── src/
└── README.md

With input data stored separately, for example:

/projects/shared/my-dataset/

Storage Rules That Prevent Trouble

  • do not treat the login node as a long-term scratchpad
  • do not duplicate large datasets without a reason
  • do not leave abandoned logs, checkpoints, and failed outputs scattered across directories
  • use explicit project names instead of mysterious folders such as test2-final-final
  • verify paths inside job scripts before submitting production workloads

Before Running a Job

Check these points:

  1. input paths exist
  2. output paths are writable
  3. logs will be written to a directory you can find later

Shared Data Requests

If your group needs:

  • a dedicated shared directory
  • a better data layout for collaboration
  • help deciding where a large dataset should live

write to nowwhat@stat.unipd.it.