...
All users have /nfsscratch/Users/HawkID available to them via our ZFS/NFS file systems. It is set to be read/write/executable for the user only. This user space is located on /nfsscratch/$Username on all nodes. There is also a local scratch space available (/localscratch/$USER) on all nodes if local disk scratch is needed. Note that using /localscratch is not usually the best choice. While there is no potential for network contention it is only a single disk on a node. If you are running a single job on a node, and have exclusive access to the node, then /localscratch should perform well. However, if you have multiple jobs on a node accessing /localscratch then the disk IO contention will be high and job performance will suffer. If the IO load is light then /localscratch may still be a viable option but heavy IO load could create a disk bottleneck. If you are not sure then test a job or small set of jobs and compare the performance of /localscratch vs. /nfsscratch. If the job is a parallel job and all nodes need access to the files then /localscratch is not an option as the files are only available to a single node. There is a cleaning policy in effect on all scratch volumes where, if the files have not been accessed in 30 days or more, they will be deleted.
...