On this page:

We strongly encourage using virtual environments to manage your projects. You can create a Python virtual environment for each of your projects. If you make changes to one of your Python virtual environments, your other environments are not affected. For this reason, virtual environments are helpful especially if you have multiple ongoing projects that require different versions of Python packages.

This article illustrates creating a Python virtual environment in IDAS. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial.

Aside: You can also use conda to manage packages and environments for any programming language. Please see the Conda Wiki article for more information about using conda in IDAS. For more general information about conda, see the Conda User Guide.

Create a Python virtual environment 

1. First, create an IDAS instance with Python. If you are a student in a class that uses Python in IDAS, follow the instructions here to access your class instance.

2. In JupyterLab, click the Terminal tile under Other to start a Terminal session.

3. In Terminal, create a virtual environment called projectName inside the directory virtenvs. This will create the virtenvs directory if it doesn't exist.

python3 -m venv $HOME/virtenvs/projectName

python3 -m venv $HOME/virtenvs/another-project

python3 -m venv --system-site-packages $HOME/virtenvs/projectName

4. Once this environment is created, we can activate it.

source $HOME/virtenvs/projectName/bin/activate

5. The command prompt in your Terminal will change to indicate the active environment. It will look like the following:

(projectName) hawkid@idas-research-hawkid:~$

6. Before installing packages in this virtual environment, it's helpful to ensure pip, setuptools, and wheel are up to date:

python -m pip install -U pip setuptools wheel

7. After that, there are several ways to install packages in this virtual environment:

    a) You can specify the names of the packages that you want to install:

python -m pip install -U package1 package2 package3

The -U option upgrades all specified packages to the newest available version. Omit this option if you don't want to upgrade packages. For more options with pip install, see the pip documentation.

    b) If you have a requirement file that contains a list of packages that you want to install, you can use:

python -m pip install -r requirements.txt

8. If you want to use JupyterLab with this virtual environment, we can install a kernel. First, install the IPython kernel:

python -m pip install -U ipykernel

9. Now install a kernel in this environment:

python -m ipykernel install --user --name projectName --display-name "Project Name"

The value for --name is used by Jupyter internally. Any existing kernel with the same --name value will be overwritten. The --display-name will be displayed in the Notebook menu in the JupyterLab page.

10. Go back to the JupyterLab Launcher page by pressing Ctrl+Shift+L (Windows) or Cmd+Shift+L (Mac). Under Notebook, a new option for your kernel will now be available. In this example screenshot, the "Python Project Vis" kernel was just installed and now became available to use.

Click on that new option to start a notebook. In that notebook, you can use the packages that you installed in the Python virtual environment you just created.

Other useful commands to manage your Python virtual environments

The following commands may be useful to manage your Python virtual environments. Type the following commands in Terminal. For more general information about using Python virtual environments, please see the Python Packaging User Guide and the Python Tutorial. For more information about pip, the Python package manager, see the pip documentation.

• Re-enter a virtual environment that was previously created: 

  source $HOME/virtenvs/projectName/bin/activate

• Leave the virtual environment once you are finished working with it:

  deactivate

• Install packages in the virtual environment:

  # first activate the virtual environment
  source $HOME/virtenvs/projectName/bin/activate
  
  # then install packages. 
  # The -U option upgrades all specified packages to the newest available version. 
  # Omit -U if you don't want to upgrade packages.
  
  python -m pip install -U package1 package2 package3

• List all packages installed in the virtual environment:

Option 1: using pip list

# first activate the virtual environment
source $HOME/virtenvs/projectName/bin/activate

# list all installed packages
python -m pip list

See the pip documentation on "pip list" for more information.

Option 2: using pip freeze

Using pip freeze will produce a similar list of installed packages as using pip list. However, the output of pip freeze follows a format that can be used with pip install. For example, you can use pip freeze and pip install together to create a requirement file and quickly install packages into a new virtual environment like so:

# activate the virtual environment project1
# suppose you want to get a list of installed packages from project1
source $HOME/virtenvs/project1/bin/activate

# put the list of all installed packages into a requirements file
python -m pip freeze > requirements.txt

# activate the virtual environment project2
source $HOME/virtenvs/project2/bin/activate

# install packages into project2 using the requirements.txt file
python -m pip install -r requirements.txt

See the pip documentation on “pip freeze” for more information.

• Uninstall packages in the virtual environment:

  # first activate the virtual environment
  source $HOME/virtenvs/projectName/bin/activate
  
  # then uninstall package
  python -m pip uninstall packageName

  See the pip documentation on "pip uninstall" for more information.

Contact

If you have any questions or comments, please contact research-computing@uiowa.edu.