Python Virtual Environments
- Giang Rudderham
- Cody Johnson
On this page:
We strongly encourage using virtual environments to manage your projects. You can create a Python virtual environment for each of your projects. If you make changes to one of your Python virtual environments, your other environments are not affected. For this reason, virtual environments are helpful especially if you have multiple ongoing projects that require different versions of Python packages.
This article illustrates creating a Python virtual environment in IDAS. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial.
Aside: You can also use conda to manage packages and environments for any programming language. Please see the Conda Wiki article for more information about using conda in IDAS. For more general information about conda, see the Conda User Guide.
Create a Python virtual environment
1. First, create an IDAS instance with Python. If you are a student in a class that uses Python in IDAS, follow the instructions here to access your class instance.
2. In JupyterLab, click the Terminal tile under Other to start a Terminal session.
3. In Terminal, create a virtual environment called projectName inside the directory virtenvs. This will create the virtenvs directory if it doesn't exist.
python3 -m venv $HOME/virtenvs/projectName
Additional notes:
You will probably have several virtual environments for unrelated projects; that’s why we created the
virtenvsdirectory to organize them. You can create another (separate) virtual environment in the future like so:
python3 -m venv $HOME/virtenvs/another-project
In some cases, you might want your virtual environment to have access to the packages provided by IDAS. You can include the
--system-site-packagesflag in the command like so:
python3 -m venv --system-site-packages $HOME/virtenvs/projectNameThis way, the packages that are installed at the system level in IDAS, outside of the virtual environment, will be available inside the virtual environment, and you don’t have to install those packages inside of the virtual environment. This can save you time and efforts if you want to use the system-level packages that were already installed in IDAS.
On the other hand, if you want packages in your virtual environment to be separated from the system-level packages in IDAS, you can omit the --system-site-packages flag. This can be helpful if, for example, you want to use a package with a different version than the version provided by IDAS.
There are other flags (options) available with the venv command. For more information, please see the official Python documentation.
4. Once this environment is created, we can activate it.
5. The command prompt in your Terminal will change to indicate the active environment. It will look like the following:
6. Before installing packages in this virtual environment, it's helpful to ensure pip, setuptools, and wheel are up to date:
7. After that, there are several ways to install packages in this virtual environment:
a) You can specify the names of the packages that you want to install:
The -U option upgrades all specified packages to the newest available version. Omit this option if you don't want to upgrade packages. For more options with pip install, see the pip documentation.
b) If you have a requirement file that contains a list of packages that you want to install, you can use:
8. If you want to use JupyterLab with this virtual environment, we can install a kernel. First, install the IPython kernel:
9. Now install a kernel in this environment:
The value for --name is used by Jupyter internally. Any existing kernel with the same --name value will be overwritten. The --display-name will be displayed in the Notebook menu in the JupyterLab page.
10. Go back to the JupyterLab Launcher page by pressing Ctrl+Shift+L (Windows) or Cmd+Shift+L (Mac). Under Notebook, a new option for your kernel will now be available. In this example screenshot, the "Python Project Vis" kernel was just installed and now became available to use.
Click on that new option to start a notebook. In that notebook, you can use the packages that you installed in the Python virtual environment you just created.
Other useful commands to manage your Python virtual environments
The following commands may be useful to manage your Python virtual environments. Type the following commands in Terminal. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial. For more information about pip, the Python package manager, see the pip documentation.
Re-enter a virtual environment that was previously created:
Leave the virtual environment once you are finished working with it:
Install packages in the virtual environment:
List all packages installed in the virtual environment:
Option 1: using pip list
See the pip documentation on "pip list" for more information.
Option 2: using pip freeze
Using pip freeze will produce a similar list of installed packages as using pip list. However, the output of pip freeze follows a format that can be used with pip install. For example, you can use pip freeze and pip install together to create a requirement file and quickly install packages into a new virtual environment like so:
See the pip documentation on “pip freeze” for more information.
Uninstall packages in the virtual environment:
See the pip documentation on "pip uninstall" for more information.
Contact
If you have any questions or comments, please contact research-computing@uiowa.edu.