In On this page:
| Table of Contents |
|---|
...
We strongly encourage using virtual environments to manage your projects. You can create a Python virtual environment for each of your projects. If you make changes to one of your Python virtual environments, your other environments are not affected. For this reason, virtual environments are helpful especially if you have multiple ongoing projects that require different versions of Python packages.
...
Aside: You can also use conda to manage packages and environments for any programming language. Please see the Conda Wiki article for more information about using conda in IDAS. For more general information about conda, see the Conda User Guide.
Create a Python virtual environment
1. First, create an IDAS instance with Python. If you are a student in a class that uses Python in IDAS, follow the instructions here
...
to access your class instance.
2. In
...
JupyterLab, click the Terminal tile under Other to start a Terminal session.
...
...
3. In Terminal, create a virtual environment called
...
projectName
...
inside the directory
...
virtenvs
...
. This will create the
...
virtenvs
...
directory if it
...
doesn't exist.
| Code Block |
|---|
python3 -m venv $HOME/virtenvs/projectName |
...
Additional notes:
You will probably have several virtual environments for unrelated projects
...
; that’s why we created the
...
virtenvs
...
directory to organize them. You can create another (separate) virtual environment in the future like so:
...
| Code Block |
|---|
python3 -m venv $HOME/virtenvs/another-project |
...
In some cases, you might want your virtual environment to have access to the packages provided by IDAS. You can include the
--system-site-packagesflag in the command like so:
| Code Block |
|---|
python3 -m venv --system-site-packages $HOME/virtenvs/projectName |
This way, the packages that are installed at the system level in IDAS, outside of the virtual environment, will be available inside the virtual environment, and you don’t have to install those packages inside of the virtual environment. This can save you time and efforts if you want to use the system-level packages that were already installed in IDAS.
On the other hand, if you want packages in your virtual environment to be separated from the system-level packages in IDAS, you can omit the --system-site-packages flag. This can be helpful if, for example, you want to use a package with a different version than the version provided by IDAS.
There are other flags (options) available with the venv command. For more information, please see the official Python documentation.
4. Once this environment is created, we can activate it.
| Code Block |
|---|
source $HOME/virtenvs/projectName/bin/activate |
...
5. The command prompt in your Terminal will change to indicate the active environment. It will look like the following:
| Code Block |
|---|
(projectName) hawkid@jupyter-notebookhawkid@idas-research-hawkid:~$ |
...
6. Before installing packages in this virtual environment, it's helpful to ensure
...
pip
...
,
...
setuptools
...
, and
...
wheel
...
are up to date:
| Code Block |
|---|
python -m pip install -U pip setuptools wheel |
7. After that, there are several ways to install packages in this virtual environment:
a) You can specify the names of the packages that you want to install:
| Code Block |
|---|
python -m pip install -U package1 package2 package3 |
The
...
-U
...
option upgrades all specified packages to the newest available version. Omit this option if you don't want to upgrade packages. For more options with
...
pip install
...
, see the pip documentation.
b) If you have a requirement file that contains a list of packages that you want to install, you can use:
| Code Block |
|---|
python -m pip install -r requirements.txt |
8. If you want to use
...
JupyterLab with this virtual environment, we can install a kernel. First, install the IPython kernel:
| Code Block |
|---|
python -m pip install -U ipykernel |
9. Now install a kernel in this environment:
| Code Block |
|---|
python -m ipykernel install --user --name projectName --display-name "Python Project Name" |
The value for
...
--name
...
is used by Jupyter internally. Any existing kernel with the same
...
--name
...
value will be overwritten.
...
The --display-name
...
will be displayed in the Notebook menu in the
...
JupyterLab page.
10. Go back to the Jupyter Hub tree page. Press F5 to reload it. Under New / JupyterLab Launcher page by pressing Ctrl+Shift+L (Windows) or Cmd+Shift+L (Mac). Under Notebook, a new option for your kernel will now be available. In this example screenshot, the "Python 3.8Project Vis" kernel was just installed and now became available to use.
Click on that new option will to start a notebook using notebook. In that notebook, you can use the packages that you installed in the Python virtual environment you just created.
...
Other useful commands to manage your Python virtual environments
The following commands may be useful to manage your Python virtual environments. Type the following commands in Terminal. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial. For more information about pip, the Python package manager, see the pip documentation.
Re-enter a virtual environment that was previously created:
Code Block source $HOME/virtenvs/projectName/bin/activate
Leave the virtual environment once you are finished working with it:
Code Block deactivate
Install packages in the virtual environment:
Code Block # first activate the virtual environment source $HOME/virtenvs/projectName/bin/activate # then install packages. # The "-U" option upgrades all specified packages to the newest available version. # Omit this option-U if you don't want to upgrade packages. python -m pip install -U package1 package2 package3
List all packages installed in the virtual environment:
Option 1: using pip list
| Code Block |
|---|
# first activate the virtual environment
source $HOME/virtenvs/projectName/bin/activate
# list all installed packages
python -m pip list |
See the pip documentation on "pip list" for more information.
Option 2: using pip freeze
Using pip freeze will produce a similar list of installed packages as using pip list. However, the output of pip freeze follows a format that can be used with pip install. For example, you can use pip freeze and pip install together to create a requirement file and quickly install packages into a new virtual environment like so:
| Code Block |
|---|
# activate the virtual environment project1
# suppose you want to get a list of installed packages from project1
source $HOME/virtenvs/project1/bin/activate
# put the list of all installed packages into a requirements file
python -m pip freeze > requirements.txt
# activate the virtual environment project2
source $HOME/virtenvs/project2/bin/activate
# install packages into project2 using the requirements.txt file
python -m pip install -r requirements.txt |
See the pip documentation on “pip freeze” for more information.
Uninstall packages in the virtual environment:
Code Block # first activate the virtual environment source $HOME/virtenvs/projectName/bin/activate # then uninstall package python -m pip uninstall packageName
See the pip documentation on "pip uninstall" for more information.
Contact
If you have any questions or comments, please contact research-computing@uiowa.edu.