File Storage in IDAS
- Giang Rudderham
In this page:
Storage Options
The following file storage systems can be used with IDAS:
Storage system | Description | How to connect to your IDAS session - Research users | How to connect to your IDAS session - Classroom users | Max storage capacity | Are data backed up? | Fees | Use with sensitive data? |
|---|---|---|---|---|---|---|---|
IDAS home directory | Automatically granted to IDAS users | Automatically available when logging in to IDAS |
| No | No charge |
| |
Argon (HPC) home directory | If a user has an Argon account, they can connect their Argon home directory to their IDAS session. | Select Yes in Mount Argon Home when logging in to IDAS | Please contact research-computing@uiowa.edu to discuss this option for a class in IDAS | 1 TB For more information, please see this support article. | Please see this support article. | No charge | |
Research users - If a user has an LSS share, they can connect their LSS share to their IDAS session. Classroom users - An instructor can request a shared storage space for their course. This storage space only lasts for the duration of the course. | Enter the name of your LSS share(s) in the LSS Shared field when logging in to IDAS. In some cases, you may need to enter the name of the LSS share in the LSS Dedicated field instead. Please contact research-computing@uiowa.edu if you have questions. | If an instructor requested a shared storage space for the class, it will automatically be available to students the class when they log in to IDAS. |
| ||||
If a user has an IHDR data enclave share, they can connect their IHDR data enclave share to their IDAS session. | Enter the name of your IHDR data enclave share(s) in the LSS Dedicated field when logging in to IDAS | N/A | Please see the IHDR data enclave service page and this article on Research Data Storage Options | ||||
The rest of this article describes the IDAS home directory system.
IDAS Home Directory
When a user is granted an IDAS account, they will automatically be granted an IDAS home directory. The user can use their IDAS home directory to store datasets, scripts, and other files.
Check your IDAS home directory usage
Each IDAS research user gets up to 100 GB for storage in their home directory.
Each IDAS classroom user can get 10-100 GB in their home directory. The course instructor specifies the storage capacity when they request IDAS for their class at the beginning of the semester: Requesting An IDAS Account
To check how much of your IDAS home directory capacity you have used:
1. Log in to IDAS from https://notebooks.hpc.uiowa.edu/
2. Start a Terminal session - Pick one of the options below:
a) Option 1: If you are using JupyterLab, click the Terminal tile under Other in your JupyterLab page:
b) Option 2: If you are using Jupyter Notebook, click New in the upper right section of the Jupyter Hub page, then click Terminal:
c) Option 3: If you are using RStudio, click on the Terminal tab, which is next to the "Console" tab in RStudio:
3. In Terminal, type df -h $HOME. In the example below, 26GB out of 100GB has been used and 75GB is available:
hawkid@jupyter-notebook-research-hawkid:~$ df -h $HOME
Filesystem Size Used Avail Use% Mounted on
/dev/rbd64 100G 26G 75G 26% /home/hawkidWhen your home directory is full
Method 1: Using the Terminal
If your home directory is full, you can use commands in the Terminal to list and remove any files.
Please note: this will permanently delete the file or folder. Please back up your data as needed before you delete them from IDAS. Follow the steps here to download data from IDAS to your computer before deleting your data.
# list all files including hidden files
ls -al
# get the top largest 10 files and directories in your IDAS home drive:
du -shx $HOME/.[^.]* * | sort -rh | head
# delete files you no longer need
rm file1.txt file2.txt
# delete a directory and all of its contents
rm -r directory
Method 2: Using JupyterLab
You can also use the JupyterLab interface to remove any files you don't need.
Right-click on a file or folder that you want to delete. Then click Delete. A dialogue will ask you to confirm that you want to permanently delete that file or folder:
Please note: this will permanently delete the file or folder. Please back up your data as needed before you delete them from IDAS. Follow the steps here to download data from IDAS to your computer before deleting your data.
Method 3: Using RStudio
In the Files panel in RStudio, check the boxes in front of the files or folders that you want to delete, then click Delete. You will be asked to confirm that you want to permanently delete the selected files.
Please note: this will permanently delete the file or folder. Please back up your data as needed before you delete them from IDAS. Follow the steps here to download data from IDAS to your computer before deleting your data.
Contact
If you have any questions or comments, please contact research-computing@uiowa.edu.