Note - Scratch cleaning policy will be changing on July 1st, 2016. The new policy will be to delete files 60 days after they are created.
User scratch space
All users have /nfsscratch/Users/HawkID available to them via our ZFS/NFS file systems. It is set to be read/write/executable for the user only. This user space is located on /nfsscratch/$Username on all nodes. There is also a local scratch space available (/localscratch/$USER) on all nodes if local disk scratch is needed. Note that using /localscratch is not usually the best choice. While there is no potential for network contention it is only a single disk on a node. If you are running a single job on a node, and have exclusive access to the node, then /localscratch should perform well. However, if you have multiple jobs on a node accessing /localscratch then the disk IO contention will be high and job performance will suffer. If the IO load is light then /localscratch may still be a viable option but heavy IO load could create a disk bottleneck. If you are not sure then test a job or small set of jobs and compare the performance of /localscratch vs. /nfsscratch. If the job is a parallel job and all nodes need access to the files then /localscratch is not an option as the files are only available to a single node.
Cleanup policy
As of July 1 2016, files on scratch filesystem are subject to deletion beginning 60 days after they were first written there. Home account storage and purchased storage are *not* subject to this policy. A file's age is the time elapsed since its creation timestamp ("crtime"), which is tracked on the fileserver. Note that this is distinct from the other timestamps on a file:
modification time (mtime): This is the time the contents of the file were last modified, for example, by editing it. The modification time can be seen with
ls -l file
change time (ctime): This is the time the metadata of the file was last changed. An example of this would be moving a file to a different directory. The change time can be seen with
ls -lc file
access time (atime): This is the time the contents of the file were last accessed, for example, by viewing with 'less'. The access time can be seen with
ls -lu file
creation time (crtime): This is the time the contents of the file were first written to the filesystem. This attribute is part of the underlying ZFS filesystem and is not visible to an NFS-mounted filesystem, nor accessible via standard Linux tools.
The cleanup script will run periodically on the server to delete files whose crtime is older than the policy. It is possible for all of these timestamps to be different for the file. Most archive utilities will maintain the first 3 timestamps, either by default or optionally. This includes using archive mode ('-a') with either 'cp' or 'rsync'. However, note that an archive file's crtime is not affected by archive utilities at all over NFS.
Duplicating files to update crtimes solely to circumvent the scratch cleanup process is against policy.
Please contact hpc-sysadmins@iowa.uiowa.edu if you need assistance with this.