While we do not have an installation of Jupyterhub it is possible to set up a single Jupyter notebook running on the HPC system. This is a relatively straight forward process on Linux and Mac workstations/laptops as these systems have terminal programs built in. The Argon system has jupyter installed in the python stacks so the information here is currently only relevant to the Argon HPC system.
There are two login nodes and normally you will wind up on one or the other based on a round robin scheme. For a jupyter notebook connection though you will have to connect to a specific login node. So, for Argon, pick one of the following login nodes to useAlthough Argon is unsuitable for providing a full JupyterHub installation, you can run a Jupyter notebook on Argon and connect a web browser to it. This connection requires creating a network path, consisting of at least one SSH tunnel, between the system running your notebook and the system running the web browser. Thus to use a browser running on your local computer, one end of the tunnel needs to be on your computer.
The required SSH software and configuration are typically present on Linux and Mac workstations/notebooks by default, so the instructions on this page describing tunnel configuration should function there without additional configuration. A variation commonly used on Windows systems is also shown. The instructions for selecting a Python environment and running jupyter are specific to the Argon HPC system.
For most Argon use, it's typical to connect to "argon.hpc.uiowa.edu", resulting in a login session on whichever of Argon's login nodes least recently served a new connection. However, to configure the tunnel for your jupyter notebook connection, you must ensure you consistently connect to the same login node in multiple steps. Therefore, for each connection you should use the "true" name one of Argon's login nodes instead of "argon.hpc.uiowa.edu":
- argon-login-1.hpc.uiowa.edu
- argon-login-2.hpc.uiowa.edu
...
- argon-login-3.hpc.uiowa.edu
- argon-login-4.hpc.uiowa.edu
The example here assumes shows how to connect using the browser on your local computer, and uses argon-login-1 but either one will work. First login to the HPC system via the selected login node..hpc.uiowa.edu only as a specific example.
Note |
---|
For simplicity in illustrating tunnel setup, this particular example runs an instance of jupyter notebook on a login node. This arrangement is only suitable for visualizing data sets of modest size without additional processing. More generally, you should run jupyter notebook in a qlogin session on a compute node, in which case the simplest arrangement is to log into an Argon login node using FastX and use Firefox there. More elaborate arrangements are possible using more complex tunnel setup than illustrated here. |
First, connect to your chosen login node:
No Format |
---|
ssh argon-login-1.hpc.uiowa.edu |
Once In your session on the login node, you must select the version of python that you want to use. Run the following command to see the available python environment modules.
...
Now, start a jupyter notebook, but without a browser . The goal is to be able to use the (because you will use a browser on your local machine.):
No Format |
---|
jupyter notebook --no-browser |
The default port is 8888 but note that if there is anything else using that port, This makes launches a notebook instance, makes it available at a particular port on the system running, and prints the port in the terminal. It tries to use port 8888 first, but it if something else (such as another jupyter notebook, then a different port will be used. The above command will automatically try a different port and will report that) is already using a port, then it tries higher ports in sequence until it finds the next one available. The output will look something like the following:
Panel |
---|
[I 10:26:27.563 NotebookApp] The port 8888 is already in use, trying another port. |
Warning |
---|
Do not share the token with anyone. |
...
For the next step, note the reported port, namely the number between "localhost:
" and "/
". In this example, it's 8889
.
Info |
---|
You can start the jupyter notebook in a screen or tmux session so that you can detach the terminal. See Screen and Tmux for more information. |
Open another terminal session on your local workstation or laptop. The port number is essentially a public address which anyone can use, even unintentionally (by typo, for example). The token is like a password; it only applies to the specific notebook running there (the one you launched just now), and it prints only in your specific terminal.
Warning |
---|
Do not share the token with anyone. |
We will next set up a tunnel over ssh so that connections to an SSH tunnel from a port on your local machine are forwarded to a port on the remote machine. In this case, the port Jupyter is using on the login node. The port that you use locally is independent of the remote port but it typically makes sense to the remote machine. This will allow your browser to connect to the port on your local machine, and SSH will transfer the traffic between your system and the remote system to communicate with your notebook instance. By default, any computer should be able to refer to itself using the name "localhost", and this example follows that standard convention.
Ordinarily, all Web browsers try connecting to the standard Web port by default. But your notebook on the remote system is connected to its own unique port there, so we'll specify a parameter to alter the destination port to match the port reported in the previous step.
The port on your system is independent of the port on the remote system running your notebook, but you may find it convenient to use the same port number. That will also allow a simple copy/paste of Doing so will let you use the URL presented by some versions of jupyter notebook to work. In the newly opened without modification, such as in the example above.
Open another terminal session on your local machine, workstation or laptop and run the following command:
No Format |
---|
ssh argon-login-1.hpc.uiowa.edu-N -L 8889:localhost:8889 -Nargon-login-1.hpc.uiowa.edu |
That will then forward traffic on the local port 8889 to the remote port 8889configure a tunnel, from Local entry port 8889 on "localhost" (your system), to "argon-login-1.hpc.uiowa.edu", with destination port 8889 (your notebook's port).
Info |
---|
You could background the port forwarded connection by can specify additional flags to the ssh command to further customize its behavior. These are beyond the scope of this demonstration. For example, using the |
The process above should work on Linux and Mac systems without modification, but the process for Windows systems can vary.
Info |
---|
Windows users can use Putty instead of the "ssh" command to create the SSH tunnel as well by configuring their client like below. Just skip past this section if you aren't using Putty. |
In the left pane, expand "SSH" and select "Tunnels":
Then To create your new forwarded connection, enter the local port as the "Source" port and , then the hostname and port number in this example localhost:8889as the "Destination", localhost:8889 as in the example above:
Click add and you will see the information move the "Add" so that the new configuration is displayed as an entry listed in the "Forwarded Ports" box .as shown here:
At this point click Click on the "sessionSession" tab on the left side of the window:
and then SSH to the Then open a normal SSH connection to your chosen login node.:
Once the
The remaining steps and usage should be the same regardless of your tunnel.
After port forwarding is setup then copy/paste the URL that was output by jupyter notebook earlier.set up and running on your system, copy the URL reported when you launched your jupyter notebook, which resembles this:
http://localhost:8889/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Paste it into the browser of running on your local machine, modify it if your chosen version of jupyter reported in a different form, and hit enter. That should then open up establish a session connected to the your jupyter notebook server program, allowing you to interact with the notebook normally.
When you are finished using the notebook,
Control-C
in the local session with the ssh port forwarding to stop forwarding.Control-C
in the ssh shell session on the remote login node where the your jupyter notebook was startedis running
Info |
---|
Please remember that computationally intensive |
...
tasks should not be run on login nodes |
...
. Such tasks typically aren't well suited to interactive, Jupyter notebook-style computation in the first place, and should be formulated as non-interactive jobs which you then submit to Argon so that it can run them on compute node resources on your behalf. |