Note -
...
As of July 1st, 2016
...
, files on scratch filesystems are subject to deletion 60 days after they
...
were created.
User
...
Scratch Space
Each compute node has its own local scratch filesystem. Users may read from and write to this using their own exclusive directory at /localscratch/Users/<HawkID>.
In addition to local storage, each HPC cluster has its own large, shared filesystem mounted across all its nodes via NFS. Analogously, users can read from and write to this using their own exclusive directory at /nfsscratch/Users/<HawkID>.
| Anchor | ||||
|---|---|---|---|---|
|
As of July 1 2016Scratch filesystems are a shared resource available for the convenience of all users. Therefore, files on scratch filesystem these filesystems are subject to deletion beginning after a certain lifespan as specified by the HPC policy committee. As of July 1 2016, this lifespan is 60 days after they were first being written there. Home account storage and purchased storage are *not* subject to this policy. A . On /nfsscratch, a file's age is the time elapsed since its creation timestamp ("crtime"), which is tracked on the fileserver. An automated cleanup process will run periodically on the server to delete files whose crtime is older than the policy's allowed lifespan.
Home account storage and purchased storage are *not* subject to this policy.
| Note |
|---|
Duplicating files to circumvent the scratch cleanup process is against policy. Please move your results to stable storage in a timely manner. |
Please contact hpc-sysadmins@iowa.uiowa.edu if you need assistance with this.
File Timestamps
Note that this crtime is distinct from the other timestamps on a file:
- modification time (mtime): This is the time the contents of the file were last modified, for example, by editing it. The modification time can be seen with
ls -l file
- change time (ctime): This is the time the metadata of the file was last changed. An example of this would be moving a file to a different directory. The change time can be seen with
ls -lc file
- access time (atime): This is the time the contents of the file were last accessed
...
- ; for example, by viewing with 'less'. The access time can be seen with
ls -lu file - creation time (crtime): This is the time the contents of the file were first written to the filesystem. This attribute is part of the underlying ZFS filesystem and is not
...
- accessible via NFS or standard Linux
...
- utilities.
The cleanup script will run periodically on the server to delete files whose crtime is older than the policy. It is possible for all of these timestamps to be different for the a single file. Most archive utilities will maintain the first 3 timestamps, either by default or optionally. This includes using archive mode ('-a') with either 'cp' or 'rsync'. However, note that an archive a file's crtime is not affected by archive utilities at all over NFS.
| Note |
|---|
Duplicating files to update crtimes solely to circumvent the scratch cleanup process is against policy. |
...
Local or Shared Scratch?
- Multiple jobs might be running on your job's node. These jobs can compete for local storage I/O, causing contention independent of /nfsscratch. Only a job with exclusive access to a node can expect the full performance potential of the node's local storage.
- A parallel job running on multiple nodes typically shouldn't use filesystems local to any of its nodes. Even if you're writing your own MPI instead of using an off-the-shelf application, you can expect better performance collating results via message passing and then writing your result to the shared filesystem.
- If your job places partial results on /localscratch and then dies, you won't have access to these anywhere else and they will be difficult to recover.
- As always, please test a few jobs if you are unsure.