Jupyter notebook

Although Argon is unsuitable for providing a full JupyterHub installation, you can run a Jupyter Notebook on Argon and connect a web browser to it. This connection requires creating a network path, consisting of at least one SSH tunnel, between the system running your notebook and the system running the web browser. Thus to use a browser running on your local computer, one end of the tunnel needs to be on your computer.

The required SSH software and configuration are typically present on Linux and Mac workstations/notebooks by default, so the instructions on this page describing tunnel configuration should function there without additional configuration. A variation commonly used on Windows systems is also shown. The instructions for selecting a Python environment and running Jupyter are specific to the Argon HPC system.

1. Generate script for running Jupyter Notebook:

   $ mkdir $HOME/bin
   $ echo -e '#!/bin/bash\njupyter notebook --ip $(hostname -f) --no-browser --port=8080' > $HOME/bin/notebook.sh

   In this step, you create a folder containing your private scripts including notebook.sh. In the second line, you will generate the script for running Jupyter Notebook. In this example, you set the port number to 8080, which can be replaced with any four-digit number.
   
   The notebook.sh will have the lines below:

   #!/bin/bash
   jupyter notebook --ip $(hostname -f) --no-browser --port=8080

   You can reuse this script later as well and modify the port number if needed.
   

2. Convert the script to be executable

   $ chmod +x $HOME/bin/notebook.sh

3. Load required modules

   $ module load python
   $ module load py-jupyter

$ module spider python

$ module load numpy
$ module load matplotlib

3. Start a Jupyter notebook on the compute nodes using the SGE scheduler

   $ qrsh -q UI -N Jupyter -pe smp 2 -cwd $HOME/bin/notebook.sh

   In this example, it requests UI queue with 2 slots (-pe smp 2) and the name of the job is Jupyter. The -cwd flag indicates that the Jupyter Notebook will access the current directory. You can modify the queue, number of slots, and the Job name.
   
   If it successfully launches the Jupyter Notebook on the compute node, it will print out the logs like below:

   

   In this example, it runs on argon-itf-bx47-36.hpc node with forwarding the port to 8080. The token should be memorized to access the notebook in your local browser (the characters in the blue box above).
   
   The port number is essentially a public address that anyone can use, even unintentionally (by typo, for example). The token is like a password; it only applies to the specific notebook running there (the one you launched just now), and it prints only in your specific terminal.

4. We will next set up an SSH tunnel from a port on your local machine to the remote machine. This will allow your browser to connect to the port on your local machine, and SSH will transfer the traffic between your system and the remote system to communicate with your notebook instance. By default, any computer should be able to refer to itself using the name "localhost", and this example follows that standard convention.
   
   Ordinarily, all Web browsers try connecting to the standard Web port by default. But your notebook on the remote system is connected to its unique port there, so we'll specify a parameter to alter the destination port to match the port reported in the previous step.
   
   The port on your system is independent of the port on the remote system running your notebook, but you may find it convenient to use the same port number. Doing so will let you use the URL presented by some versions of Jupyter Notebook without modification, such as in the example above.
   
   Open another terminal session on your local workstation or laptop and run the following command:

   $ ssh -N -L localhost:8080:argon-itf-bx47-36.hpc:8080 HawkID@argon.hpc.uiowa.edu

   That will configure a tunnel, from Local entry port 8080 on "localhost" (your system), to argon-itf-bx47-36.hpc node, with destination port 8080 (your notebook's port) through the login node. The destination compute node and the port should match what you set in the previous steps.

5. Open a browser on your local computer and go to http://localhost:8080. This may require you to enter a token for secure access. You will need to copy the token (in step 3) and paste it.
   

$ jupyter notebook list

$ qstat -u your_HawkID

Additional Tip

The process above should work on Linux and Mac systems without modification, but the process for Windows systems can vary.

In the left pane, expand "SSH" and select "Tunnels":

To create your new forwarded connection, enter the local port as the "Source" port, then the hostname and port number as the "Destination", localhost:8889 (in the example above it was 8080):

Click "Add" so that the new configuration is displayed as an entry listed in the "Forwarded Ports" box as shown here:

Click on the "Session" tab on the left side of the window:

Then open a normal SSH connection to your chosen login node:

The remaining steps and usage should be the same regardless of your tunnel.

After port forwarding is set up and running on your system, copy the URL reported when you launched your Jupyter Notebook, which resembles this:

http://localhost:8889/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Paste it into the browser running on your local machine, modify it if your chosen version of Jupyter is reported in a different form, and hit enter. That should establish a session connected to your Jupyter notebook server program, allowing you to interact with the notebook normally.

When you are finished using the notebook,

1. Control-C in the local session with the ssh port forwarding to stop forwarding

2. Control-C in the shell session on the remote node where your jupyter notebook is running