Virtual environments

Virtual environments#

Learning Objectives#

  • Understand the importance of using virtual environments in Python development

  • Differentiate between various tools for managing virtual environments: venv, Conda, Pipenv, and Poetry

  • Create and activate a virtual environment using venv

  • Use Conda to manage environments and packages.

  • Utilize Pipenv and Poetry for dependency management and virtual environments

Why do we need virtual environments?#

Control#

Virtual environments give you control over the version of Python and the versions of installed libraries (modules). Typically this means you will be using the latest versions of Python and libraries for your latest project.

Freedom to update#

By using virtual environments you are free to update to new versions of libraries for your latest project, without breaking earlier projects.

Reliable deployment#

It is far easier to create reproducible and portable software with virtual environments.

System Python!#

Use of Python is so widespread that some operating systems include Python, for a long time this was Python 2.7 though more recently Python 3 (3.8,3.9) has become a popular default. It is never a good idea to change this configuration, except with the usual system upgrade tools.

Why not use virtual machines or containers?#

These have their own place in development and deployment of software. We are not going to cover them here, except to note the following.

If you find yourself having to do complicated or unusual things with virtual environments, or virtual environments do not provide the facilities or security your software requires, then consider alternatives such as Docker.

Yes, but

I have admin rights so I can install libraries and a new Python version for all users.

Just don’t!

venv#

Tutorial https://docs.python.org/3/tutorial/venv.html

python -m venv .venv

source .venv/bin/activate

Though in practice your IDE, e.g. VS Code will typically activate environments for you.

Virtualenv#

Tutorial: https://virtualenv.pypa.io/en/latest/user_guide.html

Virtualenv is a tool that can be used to set up a virtual environment. If it is not already on your machine, you’ll need to install it such as by:

pip install virtualenv

To create a new environment:

virtualenv env_name

To activate and deactivate the environment:

source env_name/bin/activate
deactivate

To install packages into the environment, a great method is to store all dependencies listed in a requirements.txt (rather than directly installing them with pip install package_name). This is because you can save and share this file alongside your repository, and it allows others to easily see and make a copy of the environment you used for your analysis - and for yourself to reproduce that environment, when you return to your code years later!

An example requirements.txt file:

jupyter==1.0.0
pandas==2.2.2

To install the packages from requirements.txt into your environment:

pip install -r requirements.txt

To update your environment (such as if you to have add a new package to the requirements file):

pip install -r requirements.txt --upgrade

To delete your environment, use the command below - but be careful! Do not name your environment with the same name as a folder in your current location. If so, you could accidentally permanently delete that folder rather than your environment…

rm -r env_name

Conda#

Tutorial: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

Conda is another popular tool for environment management in Python. If not already on your machine, follow these installation instructions to install conda.

You can save the dependencies needed for your Python environment using an environment.yml file. Other people can then build an environment with the same dependencies based on that file. Within this file, you can just list the packages needed, or you can include specific versions (if you want people to use the same environment as you).

Example file:

name: shoaib2022
channels:
  - defaults
dependencies:
  - matplotlib=3.3.4
  - pytest=7.4.4
  - pip:
    - pytest-xdist==3.6.1

To create environment from the file:

conda env create --name env_name --file environment.yml

To activate the environment:

conda activate env_name

To see packages in the current environment:

conda list

To see the conda environments on your machine:

conda env list

To update the current environment from a .yml file (such as if you have changed the dependencies or versions listed):

conva env update --file environment.yml --prune

To delete the environment:

conda remove -n env_name --all

Pipenv#

mkdir .venv
pipenv install numpy
pipenv shell

Poetry#

https://python-poetry.org/docs/

pipx install poetry

https://python-poetry.org/docs/basic-usage/