In this page:

Jupyter Notebook for Python, R, and Julia

The IDAS interface

• After creating an IDAS instance for research or class use, you will see your IDAS home page. If this is your first time accessing IDAS, your home page will be empty.
• You can create Jupyter notebooks, files, and folders in IDAS. (See details below.) Click any file or folder to open that file or folder. 
• In order to navigate back to your IDAS home page, click the Jupyter symbol on the top left corner: .

Uploading files

• You can upload your own Jupyter notebooks and data files to IDAS. After creating an IDAS instance for research or class use, in the top right corner of your IDAS home page, click Upload.
  
  
• Select the files to upload in your local computer. Once the upload is done, you should see the uploaded files in your directory. 

Creating a new Jupyter notebook

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

• In the top right corner of your IDAS home page, click New.

• Under Notebook, choose Python, R, or Julia in order to create a notebook in that language. For example, if you had created an instance with Python, you can create a new Jupyter notebook with Python:

  
  
    If you are creating a research instance, note that Python, R, and Julia each have their own container image. For example, in order to use Julia, please select one of the three options for Julia under IDAS Image in the Spawner Options form.

  

Managing Jupyter notebooks

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

• In your IDAS home page, check the box next to a Jupyter notebook. If the notebook is not currently running, you will have the option to duplicate, rename, move, download, view, edit, or delete that notebook.
  

• If the notebook is currently running, you will have the options to duplicate, shutdown, view, edit, or delete that notebook.
  
  
  
• Below is a summary of these options:
  • Duplicate: will make a copy of the notebook, stored in the same folder.
  • Rename: will rename the notebook. Enter the new name in the pop-up box, then click Rename.
  • Move: will move the notebook to a destination of your choice. In the pop-up box, enter the destination directory path, then click Move.
  • Download: will download a copy of the notebook to your local computer.
  • View: will open the notebook and allow editing the notebook, including adding cells and running codes.
  • Edit: will allow editing the source code in the notebook.
  • Shutdown: will shut down a notebook that is running.
  • Delete (the red trash can icon): will permanently delete the notebook.

  The path to your IDAS home directory is /home/<your_HawkID>. In the example below, the Jupyter notebook will be moved to the "demo" folder inside the IDAS home directory.

  

Creating new files and folders

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.
• In the top right corner of your IDAS home page, click New.
• Choose Text File to create a new .txt file, or Folder to create a new Folder. The text file can later be renamed to another extension, such as .py.
  

Managing files and folders

File options

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

• In your IDAS home page, check the box next to a file. You will be able to see the options available for that file:

  

• Below is a summary of the options for a .txt file:
  • Duplicate: will make a copy of the file, stored in the same folder.
  • Rename: will rename the file. Enter the new name in the pop-up box, then click Rename.
  • Move: will move the file to a destination of your choice. In the pop-up box, enter the destination directory path, then click Move.
  • Download: will download a copy of the file to your local computer.
  • View: will open the file for viewing only. No editing.
  • Edit: will allow editing the file.

  • Delete (the red trash can icon): will permanently delete the file.

Folder options

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

• In your IDAS home page, check the box next to a folder. You will have the option to rename, move, or delete that folder.

  

• To rename a folder, check the box next to that folder, click Rename, enter the new folder name in the pop-up box, and click Rename:
  
  
  

• To move a folder, check the box next to that folder, click Move, enter a new directory path in the pop-up box, and click Move. In the example below, the folder will be moved to the "demo" folder inside the IDAS home directory.
  
  
  

  The path to your IDAS home directory is /home/<your_HawkID>.

• To delete a folder, check the box next to that folder, and then click the red trash can icon. You will be asked to confirm that you want to permanently delete the folder. Click Delete to confirm the action.
  
  
  

Working with Terminal in IDAS

The Terminal allows you to perform more tasks in IDAS, including checking system information, managing files and folders, and installing packages. Below are a few common tasks that may be helpful in IDAS.

Creating a new Terminal session

• First, create an IDAS instance with Python, R, or Julia. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

• In the top right corner of your IDAS home page, click New.

• Click Terminal in order to start a Terminal session.
  
  
  

Checking information using Terminal

The following commands can be typed after the prompt symbol $ in Terminal:

• pwd: prints the current working directory. If you are in your IDAS home diretory, the directory will be /home/<your_HawkID>.

• ls -a: lists the files and directories in your current directory. The -a option lists all files, including hidden files that start with ., such as .bashrc.

• cd <destination>: change directory to a destination directory. For example,

  • cd demo: changes to the demo directory

  • cd: (with no argument) returns to your home directory 

• jupyter notebook --version: prints the version of Jupyter Notebook. Similarly,  you can type python --version or conda --version.

RStudio for R

Creating and saving R scripts

• After creating an IDAS instance for research or class use, you will see the RStudio interface. 
• To create a new (blank) R script, click File on the menu bar at the top of your RStudio screen, New File, and then R Script. 
  
  
  
  
• To save an R script, click File on the menu bar, and then Save As... 
  
  
• In the pop-up box, enter a name for your script in File name. Click to select a destination for your file. (Click Home to go to your home directory anytime). Click Save. 
  

Managing files and folders

• In the lower right quadrant of your screen, the Files tab shows your files and folders. If this is your first time using RStudio, your home directory will only have a folder called "R".
  
  
• Click the Home button to go to your home directory anytime.
  
  
  
• Under the Files tab, there are a number of options to manage your files and folders.
  
  
  
• Below is a summary of these options:
  • New Folder: create a new folder in your current directory. Enter the folder name, and then click OK.
  • Upload: upload files from your computer to your current directory in IDAS. Select the files in your local computer, and then click OK. 
    • To upload multiple files or a directory, create a zip file, which will be automatically uncompressed after upload.
  • Delete: permanently delete a file or folder. Check the box next to a file or folder to select it, click Delete, then Yes.
  • Rename: rename a file or folder. Check the box next to a file or folder to select it, then click Rename. Enter the new name in the pop-up box, then click OK.
  • Copy: make a copy of a file or folder. Check the box next to a file or folder to select it, click More, then Copy, enter a name for the new file or folder, then click OK.
  • Copy To: make a copy of a file or folder and save the copy to a directory. Check the box next to a file or folder to select it, click More, then Copy To, enter a name for the new file or folder, select a directory, then click Save.
  • Move: move the file or folder to a directory of your choice. Check the box next to a file or folder to select it, click More, then Move. In the pop-up box, choose a directory, then click Choose.
  • Export: download files and folders (see below)
  • Set As Working Directory: navigate to a directory of your choice, then click this option to choose the current directory as the working directory.
  • Go To Working Directory: click this option to go to the current working directory.
  • Show Hidden Files: toggle this option to show or hide hidden files, e.g. files that start with a "."

Downloading files and folders

• • To download a file, check the box next to it. Click More, then Export. Enter a name for the downloaded file, then click Download.
  • To download a folder, check the box next to it. Click More, then Export. Enter a name for the downloaded file, then click Download. The folder will be downloaded as a .zip file.
  • To download multiple files and folders, check the boxes next to those files and folders. Click More, then Export. Enter a name for the downloaded file, then click Download. They will be downloaded as a .zip file.
    

Switching to JupyterLab from RStudio

To access the JupyterLab page and use your Jupyter notebooks, from your RStudio session, change the end of the URL to "lab".

Specifically for research users (fill in your HawkID):

Downloading Data from IDAS to Your Computer

There are a couple of options to get data from your IDAS account to your computer:

1. Compress the files you want to download, and then download the compressed file using the web interface of IDAS (see the section below)

2. For IDAS research users:

• If you have an Argon account or Large Scale Storage (LSS) shares, you can mount them when you spawn an IDAS research instance. This option allows you to access data in your Argon home or LSS shares while you work in IDAS. You can also move files from your IDAS home drive to your Argon home or LSS shares.
• For more information about how to mount Argon home and LSS shares in IDAS, please see step 6 in Accessing IDAS for Research Use. 

Downloading Data Using the Web Interface

Follow the steps below to download data from your IDAS home drive to your computer, using the web interface of IDAS:

1. Log in to the IDAS research instance if you have a research account. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

2. Start a Terminal session:

• Option 1: If you are using JupyterLab, click the "Terminal" tile under "Other" in your Jupyter Lab page:

• Option 2: If you are using Jupyter Notebook, click "New" in the upper right section of the Jupyter Hub tree page, then choose "Terminal:

• Option 3: If you are using RStudio, click on the "Terminal" tab, which is next to the "Console" tab in RStudio:

3. In Terminal, use "zip" to compress the file(s) and/or directories that you want to download. This can take a few minutes, especially if you are compressing a lot of files. Here are some examples for using "zip":

# compress all files in the "mydir" directory
zip -r backup.zip mydir

# compress a specific file in the current directory
zip backup.zip myfile.csv

• In the "zip" command, the "-r" option means including all files in a directory.
• For more information about the "zip" command, please type in Terminal: "zip --help"

4. After the commands finish running in Terminal, a file called "backup.zip" will be available in your current directory.

• If you are using JupyterLab, you can find "backup.zip" in the left-side panel of JupyterLab. Note: the location of "backup.zip" in your IDAS home drive will depend on how you use the "zip" command in step 3 above. The below is only an example screenshot:

• If you are using Jupyter Notebook, you can find the "backup.zip" file in your Jupyter Hub tree page, like in this example screenshot:

• If you are using RStudio, you can find the "backup.zip" file in the "Files" tab in RStudio, like in this example screenshot:

5. Download the "backup.zip" file to your computer:

• In JupyterLab, right-click on the "backup.zip" file and click "Download":

• In Jupyter Notebook, check the box in front of the "backup.zip" file and click "Download":

• In RStudio, check the box in front of the "backup.zip" file and click "More", then "Export":

6. Then you can use a free software such as 7-zip to extract the files in "backup.zip" to a location on your computer.

IDAS Home Drives

Check your home drive usage

On IDAS, each research user gets 100GB for storage in their home drive. To check how much you have used:

1. Log in to the IDAS research instance if you have a research account. If you are a student in a class that uses IDAS, follow the instructions here to access your class instance.

2. Start a Terminal session - Pick one of the options below:

a) Option 1: If you are using JupyterLab, click the "Terminal" tile under "Other" in your Jupyter Lab page:

b) Option 2: If you are using Jupyter Notebook, click "New" in the upper right section of the Jupyter Hub tree page, then choose "Terminal:

c) Option 3: If you are using RStudio, click on the "Terminal" tab, which is next to the "Console" tab in RStudio:

3. In Terminal, type df -h $HOME. In the example below, 26GB out of 100GB has been used and 75GB is available:          

hawkid@jupyter-notebook-research-hawkid:~$ df -h $HOME
Filesystem      Size  Used Avail Use% Mounted on
/dev/rbd64      100G   26G   75G  26% /home/hawkid

When your home drive is full

Method 1: Using Terminal - If your home drive is full, you can use Terminal to list and remove any files. 

# list all files including hidden files
ls -al

# get the top largest 10 files and directories in your IDAS home drive:
du -shx $HOME/.[^.]* * | sort -rh | head

# delete files you no longer need
rm file1.txt file2.txt

# delete a directory and all of its contents
rm -r directory

Method 2: Using JupyterLab - You can also use the JupyterLab interface to remove any files you don't need:

1. Go to the JupyterLab page - please fill in your HawkID: 

2. Right-click on a file or folder that you want to delete. Then click "Delete". A dialogue will ask you to confirm that you want to permanently delete that file or folder:

Shutting down an IDAS instance 

1. Save any scripts or files you need to save. 

2.  Research users: Go to https://idas.uiowa.edu/research/hub/home

• Click the red button Stop My Server.
  
  
  
• Wait a few seconds for the blue button to change to Start My Server.

• Click Start My Server to start the server again if you want to continue using IDAS. Otherwise, if you are done with IDAS, close your browser tab. 

Contact

If you have any questions or comments, please contact research-computing@uiowa.edu.