Content Comparison

While we do not have an installation of Jupyterhub it is possible to set up a single Jupyter notebook running on the HPC system. This is a relatively straight forward process on Linux and Mac workstations/laptops as these systems have terminal programs built in. The Argon system has jupyter installed in the python stacks so the information here is currently only relevant Although Argon is unsuitable for providing a full JupyterHub installation, you can run a Jupyter Notebook on Argon and connect a web browser to it. This connection requires creating a network path, consisting of at least one SSH tunnel, between the system running your notebook and the system running the web browser. Thus to use a browser running on your local computer, one end of the tunnel needs to be on your computer.

The required SSH software and configuration are typically present on Linux and Mac workstations/notebooks by default, so the instructions on this page describing tunnel configuration should function there without additional configuration. A variation commonly used on Windows systems is also shown. The instructions for selecting a Python environment and running Jupyter are specific to the Argon HPC system.

There are two login nodes and normally you will wind up on one or the other based on a round robin scheme. For a jupyter notebook connection though you will have to connect to a specific login node. So, for Argon, pick one of the following login nodes to use:

• argon-login-1.hpc.uiowa.edu
• argon-login-2.hpc.uiowa.edu

The example below will use argon-login-1 but either one will work. First login to the HPC system via the selected login node.

ssh argon-login-1.hpc.uiowa.edu

Once on the login node you must select the version of python that you want to use. Run the following command to see the available python environment modules.

module -tr avail ^python

Load the selected python environment module.

module load python/2.7.13

Now, start a jupyter notebook, but without a browser. The goal is to be able to use the browser on your local machine.

jupyter notebook --no-browser

The default port is 8888 but note that if there is anything else using that port, such as another jupyter notebook, then a different port will be used. The above command will automatically try a different port and will report that. The output will look something like the following:

...

The important bit of information from the above for the next step is the port that was selected. That can be found immediately after localhost. In the above the port number used is 8889.

...

1. Generate script for running Jupyter Notebook:

   Code Block
   $ mkdir $HOME/bin $ echo -e '#!/bin/bash\njupyter notebook --ip $(hostname -f) --no-browser --port=8080' > $HOME/bin/notebook.sh

   In this step, you create a folder containing your private scripts including notebook.sh. In the second line, you will generate the script for running Jupyter Notebook. In this example, you set the port number to 8080, which can be replaced with any four-digit number.
   
   The notebook.sh will have the lines below:

   Code Block
   #!/bin/bash jupyter notebook --ip $(hostname -f) --no-browser --port=8080

   You can reuse this script later as well and modify the port number if needed.
   

2. Convert the script to be executable

   Code Block
   $ chmod +x $HOME/bin/notebook.sh

3. Load required modules

   Code Block
   $ module load python $ module load py-jupyter

$ module spider python

$ module load py-numpy
$ module load py-matplotlib

$ module -r avail ^py--

-------------------------------------------------------------------- /opt/ssoft/modules/2021.1/packages --------------------------------------------------------------------
   py-absl-py/0.10.0_gcc-9.3.0                     py-jupyter-console/6.1.0_gcc-9.3.0           (L)      py-pyyaml/5.1.2_gcc-9.3.0
   py-agate-dbf/0.2.1_gcc-9.3.0                    py-jupyter-core/4.6.3_gcc-9.3.0              (L)      py-pyzmq/18.1.0_gcc-9.3.0                      (L)
   py-agate-excel/0.2.3_gcc-9.3.0                  py-jupyter/1.0.0_gcc-9.3.0                   (L)      py-qtconsole/4.5.1_gcc-9.3.0                   (L)
   py-agate-sql/0.5.4_gcc-9.3.0                    py-jupyterlab-pygments/0.1.1_gcc-9.3.0       (L)      py-rdflib/5.0.0_gcc-9.3.0
   py-agate/1.6.1_gcc-9.3.0                        py-keras-applications/1.0.8_gcc-9.3.0                 py-reindent/3.5.1_gcc-9.3.0
   py-alabaster/0.7.12_gcc-9.3.0                   py-keras-preprocessing/1.1.2_gcc-9.3.0                py-reportlab/3.4.0_gcc-9.3.0
   py-argon2-cffi/20.1.0_gcc-9.3.0          (L)    py-keras/2.2.4_gcc-9.3.0                              py-requests/2.24.0_gcc-9.3.0
   ...

3. Start a Jupyter notebook on the compute nodes using the SGE scheduler

   Code Block
   $ qrsh -q UI -N Jupyter -pe smp 2 -cwd $HOME/bin/notebook.sh

   In this example, it requests UI queue with 2 slots (-pe smp 2) and the name of the job is Jupyter. The -cwd flag indicates that the Jupyter Notebook will access the current directory. You can modify the queue, number of slots, and the Job name.
   
   If it successfully launches the Jupyter Notebook on the compute node, it will print out the logs like below:

   Image Added

   In this example, it runs on argon-itf-bx47-36.hpc node with forwarding the port to 8080. The token should be memorized to access the notebook in your local browser (the characters in the blue box above).
   
   The port number is essentially a public address that anyone can use, even unintentionally (by typo, for example). The token is like a password; it only applies to the specific notebook running there (the one you launched just now), and it prints only in your specific terminal.

...

4. We will next set up

...

4. an SSH tunnel from a port on your local

...

4. machine to the remote machine. This will allow your browser to connect to the port on your local machine, and SSH will transfer the traffic between your system and the remote system to communicate with your notebook instance. By default, any computer should be able to refer to itself using the name "localhost", and this example follows that standard convention.
   
   Ordinarily, all Web browsers try connecting to the standard Web port by default. But your notebook on the remote system is connected to its unique port there, so we'll specify a parameter to alter the destination port to match the port reported in the previous step.
   
   The port on your system is independent of the port on the remote system running your notebook, but you may find it convenient to use the same port number.

...

4. Doing so will let you use the URL presented by

...

4. some versions of Jupyter Notebook without modification, such as in the example above.
   
   Open another terminal session on your local

...

4. workstation or laptop and run the following command:

...

4. Code Block
   $ ssh -N -L localhost:8080:argon-itf-

...

4. bx47-36.hpc:8080 HawkID@argon.hpc.uiowa.edu

...

4. That will

...

4. configure a tunnel, from Local entry port 8080 on "localhost" (your system), to argon-itf-bx47-36.hpc node, with destination port 8080 (your notebook's port) through the login node. The destination compute node and the port should match what you set in the previous steps.

...

5. Open a browser on your local computer and go to http://localhost:8080. This may require you to enter a token for secure access. You will need to copy the token (in step 3) and paste it.
   

$ jupyter notebook list

$ qstat -u your_HawkID

Additional Tip

The process above should work on Linux and Mac systems without modification, but the process for Windows systems can vary.

In the left pane, expand "SSH" and select "Tunnels":

...

To create your new forwarded connection, enter the local port as the "Source" port, then the hostname and port number as the "Destination", localhost:8889 (in the example above it was 8080):

...

Click "Add" so that the new configuration is displayed as an entry listed in the "Forwarded Ports" box as shown here:

...

Click on the "Session" tab on the left side of the window:

...

Then open a normal SSH connection to your chosen login node:

...

The remaining steps and usage should be the same regardless of your tunnel.

After port forwarding is set up and running on your system, copy the URL reported when you launched your Jupyter Notebook, which resembles this:

http://localhost:8889/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Paste it into the browser of running on your local machine, modify it if your chosen version of Jupyter is reported in a different form, and hit enter. That should then open up establish a session connected to the jupyter notebook server. your Jupyter notebook server program, allowing you to interact with the notebook normally.

When you are finished using the notebook,

1. Control-C in the local session with the ssh port forwarding to stop forwarding

...

2. Control-C in the

...

1. shell session on the remote

...

1. node where

...

1. your jupyter notebook

...

1. is running