On this page:
We strongly encourage using virtual environments to manage your projects. You can create a Python virtual environment for each of your projects. If you make changes to one of your Python virtual environments, your other environments are not affected. For this reason, virtual environments are helpful especially if you have multiple ongoing projects that require different versions of Python packages.
This article illustrates creating a Python virtual environment in IDAS. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial.
Aside: You can also use conda to manage packages and environments for any programming language. Please see the Conda Wiki article for more information about using conda in IDAS. For more general information about conda, see the Conda User Guide.
Create a Python virtual environment
1. First, create an IDAS instance with Python. If you are a student in a class that uses Python in IDAS, follow the instructions here to access your class instance.
2. In JupyterLab, click the Terminal tile under Other to start a Terminal session.
3. In Terminal, create a virtual environment called projectName inside the directory virtenvs. This will create the virtenvs directory if it doesn't exist.
python3 -m venv $HOME/virtenvs/projectName
a) You will probably have several virtual environments for unrelated projects; that’s why we created the virtenvs directory to organize them. You can create another (separate) virtual environment in the future like so:
python3 -m venv $HOME/virtenvs/another-project
b) In some cases, you might want your virtual environment to have access to the packages provided by IDAS. You can include the --system-site-packages flag in the command like so:
python3 -m venv --system-site-packages $HOME/virtenvs/projectName
This way, the packages that are installed at the system level in IDAS, outside of the virtual environment, will be available inside the virtual environment, and you don’t have to install those packages inside of the virtual environment. This can save you time and efforts if you want to use the system-level packages that were already installed in IDAS.
On the other hand, if you want packages in your virtual environment to be separated from the system-level packages in IDAS, you can omit the --system-site-packages flag. This can be helpful if, for example, you want to use a different package version than what is available in IDAS.
There are other flags (options) available with the venv command. For more information, please see the official Python documentation.
4. Once this environment is created, we can activate it.
source $HOME/virtenvs/projectName/bin/activate
5. The command prompt in your Terminal will change to indicate the active environment. It will look like the following:
(projectName) hawkid@idas-research-hawkid:~$
6. Before installing packages in this virtual environment, it's helpful to ensure pip, setuptools, and wheel are up to date:
python -m pip install -U pip setuptools wheel
7. After that, there are several ways to install packages in this virtual environment:
a) You can specify the names of the packages that you want to install:
python -m pip install -U package1 package2 package3
The -U option upgrades all specified packages to the newest available version. Omit this option if you don't want to upgrade packages. For more options with pip install, see the pip documentation.
b) If you have a requirement file that contains a list of packages that you want to install, you can use:
python -m pip install -r requirements.txt
8. If you want to use JupyterLab with this virtual environment, we can install a kernel. First, install the IPython kernel:
python -m pip install -U ipykernel
9. Now install a kernel in this environment:
python -m ipykernel install --user --name projectName --display-name "Project Name"
The value for --name is used by Jupyter internally. Any existing kernel with the same --name value will be overwritten. The --display-name will be displayed in the Notebook menu in the JupyterLab page.
10. Go back to the JupyterLab Launcher page by pressing Ctrl+Shift+L (Windows) or Cmd+Shift+L (Mac). Under Notebook, a new option for your kernel will now be available. In this example screenshot, the "Python Project Vis" kernel was just installed and now became available to use.
Click on that new option to start a notebook. In that notebook, you can use the packages that you installed in the Python virtual environment you just created.
Other useful commands to manage your Python virtual environments
The following commands may be useful to manage your Python virtual environments. Type the following commands in Terminal. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial. For more information about pip, the Python package manager, see the pip documentation.
Re-enter a virtual environment that was previously created:
source $HOME/virtenvs/projectName/bin/activate
Leave the virtual environment once you are finished working with it:
deactivate
Install packages in the virtual environment:
# first activate the virtual environment source $HOME/virtenvs/projectName/bin/activate # then install packages. # The -U option upgrades all specified packages to the newest available version. # Omit -U if you don't want to upgrade packages. python -m pip install -U package1 package2 package3
List all packages installed in the virtual environment:
Option 1: using pip list
# first activate the virtual environment source $HOME/virtenvs/projectName/bin/activate # list all installed packages python -m pip list
See the pip documentation on "pip list" for more information.
Option 2: using pip freeze
Using pip freeze will produce a similar list of installed packages as using pip list. However, the output of pip freeze follows a format that can be used with pip install. For example, you can use pip freeze and pip install together to create a requirement file and quickly install packages into a new virtual environment like so:
# activate the virtual environment project1 # suppose you want to get a list of installed packages from project1 source $HOME/virtenvs/project1/bin/activate # put the list of all installed packages into a requirements file python -m pip freeze > requirements.txt # activate the virtual environment project2 source $HOME/virtenvs/project2/bin/activate # install packages into project2 using the requirements.txt file python -m pip install -r requirements.txt
See the pip documentation on “pip freeze” for more information.
Uninstall packages in the virtual environment:
# first activate the virtual environment source $HOME/virtenvs/projectName/bin/activate # then uninstall package python -m pip uninstall packageName
See the pip documentation on "pip uninstall" for more information.
Contact
If you have any questions or comments, please contact research-computing@uiowa.edu.